18 KiB

Raw Blame History

title

date

Rust Crates that do What the Go Standard library Does

One of Go's greatest strengths is how batteries-included the standard library is. You can do most of what you need to do with only the standard library. On the other hand, Rust's standard library is severely lacking by comparison. However, the community has capitalized on this and been working on a bunch of batteries that you can include in your rust projects. I'm going to cover a bunch of them in this post in a few sections.

A lot of these are actually used to help make this blog site work!

Logging

Go has logging out of the box with package log. Package log is a very uncontroversial logger. It does what it says it does and with little fuss. However it does not include a lot of niceties like logging levels and context-aware values.

In Rust, we have the log crate which is a very simple interface. It uses the error!, warn!, info!, debug! and trace! macros which correlate to the highest and lowest levers. If you want to use log in a Rust crate, you can add it to your Cargo.toml file like this:

[dependencies]
log = "0.4"

Then you can use it in your Rust code like this:

use log::{error, warn, info, debug, trace};

fn main() {
  trace!("starting main");
  debug!("debug message");
  info!("this is some information");
  warn!("oh no something bad is about to happen");
  error!("oh no it's an error");
}

Wait, where does that log to? I ran that example locally but I didn't see any of the messages anywhere

This is because the log crate doesn't directly log anything anywhere, it is a facade that other packages build off of. pretty_env_logger is a commonly used crate with the log facade. Let's add it to the program and work from there:

[dependencies]
log = "0.4"
pretty_env_logger = "0.4"

Then let's enable it in our code:

use log::{error, warn, info, debug, trace};

fn main() {
  pretty_env_logger::init();

  trace!("starting main");
  debug!("debug message");
  info!("this is some information");
  warn!("oh no something bad is about to happen");
  error!("oh no it's an error");
}

And now let's run it with RUST_LOG=trace:

$ env RUST_LOG=trace cargo run --example logger_test
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `/home/cadey/code/christine.website/target/debug/logger_test`
 TRACE logger_test > starting main
 DEBUG logger_test > debug message
 INFO  logger_test > this is some information
 WARN  logger_test > oh no something bad is about to happen
 ERROR logger_test > oh no it's an error

There are many other consumers of the log crate and implementing a consumer is easy should you want to do more than pretty_env_logger can do on its own. However, I have found that pretty_env_logger does just enough on its own. See its documentation for more information.

Flags

Go's standard library has the flag package out of the box. This package is incredibly basic, but is surprisingly capable in terms of what you can actually do with it. A common thing to do is use flags for configuration or other options, such as here:

package main

import "flag"

var (
	program      = flag.String("p", "", "h program to compile/run")
	outFname     = flag.String("o", "", "if specified, write the webassembly binary created by -p here")
	watFname     = flag.String("o-wat", "", "if specified, write the uncompiled webassembly created by -p here")
	port         = flag.String("port", "", "HTTP port to listen on")
	writeTao     = flag.Bool("koan", false, "if true, print the h koan and then exit")
	writeVersion = flag.Bool("v", false, "if true, print the version of h and then exit")
)

This will make a few package-global variables that will contain the values of the command-line arguments.

In Rust, a commonly used command line parsing package is structopt. It works in a bit of a different way than Go's flag package does though. structopt focuses on loading options into a structure rather than into globally mutable variables.

Something you may notice in Rust-land is that globally mutable state is talked about as if it is something to be avoided. It's not inherently bad, but it does make things more likely to crash at runtime. In most cases, these global variables with package flag are fine, but only if they are ever written to before the program really starts to do what it needs to do. If they are ever written to and read from dynamically at runtime, then you can get into a lot of problems such as race conditions.

Here's a quick example copied from pa'i:

#[derive(Debug, StructOpt)]
#[structopt(
    name = "pa'i",
    about = "A WebAssembly runtime in Rust meeting the Olin ABI."
)]
struct Opt {
    /// Backend
    #[structopt(short, long, default_value = "cranelift")]
    backend: String,


    /// Print syscalls on exit
    #[structopt(short, long)]
    function_log: bool,


    /// Do not cache compiled code?
    #[structopt(short, long)]
    no_cache: bool,


    /// Binary to run
    #[structopt()]
    fname: String,


    /// Main function
    #[structopt(short, long, default_value = "_start")]
    entrypoint: String,


    /// Arguments of the wasm child
    #[structopt()]
    args: Vec<String>,
}

This has the Rust compiler generate the needed argument parsing code for you, so you can just use the values as normal:

fn main() {
  let opt = Opt::from_args();
  debug!("args: {:?}", opt.args);
}

You can even handle subcommands with this, such as in palisade. This package should handle just about everything you'd do with the flag package, but will also work for cases where flag falls apart.

Errors

Go's standard library has the error interface which lets you create a type that describes why functions fail to do what they intend. Rust has the Error trait which lets you also create a type that describes why functions fail to do what they intend.

In my last post I described eyre and the Result type. However, this time we're going to dive into thiserror for making our own error type. Let's add thiserror to our crate:

[dependencies]
thiserror = "1"

And then let's re-implement our DivideByZero error from the last post:

use std::fmt;
use thiserror::Error;

#[derive(Debug, Error)]
struct DivideByZero;

impl fmt::Display for DivideByZero {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(f, "cannot divide by zero")
    }
}

The compiler made our error instance for us! It can even do that for more complicated error types like this one that wraps a lot of other error cases and error types in maj:

#[derive(thiserror::Error, Debug)]
pub enum Error {
    #[error("TLS error: {0:?}")]
    TLS(#[from] TLSError),

    #[error("URL error: {0:?}")]
    URL(#[from] url::ParseError),

    #[error("Invalid DNS name: {0:?}")]
    InvalidDNSName(#[from] webpki::InvalidDNSNameError),

    #[error("IO error: {0:?}")]
    IO(#[from] std::io::Error),

    #[error("Response parsing error: {0:?}")]
    ResponseParse(#[from] crate::ResponseError),

    #[error("Invalid URL scheme {0:?}")]
    InvalidScheme(String),
}

These #[error("whatever")] annotations will show up when the error message is printed. See here for more information on what details you can include here.

Serialization / Deserialization

Go has JSON encoding/decoding in its standard library via package encoding/json. This allows you to define types that can be read from and write to JSON easily. Let's take this simple JSON object representing a comment from some imaginary API as an example:

{
  "id": 31337,
  "author": {
    "id": 420,
    "name": "Cadey"
  },
  "body": "hahaha its is an laughter image",
  "in_reply_to": 31335
}

In Go you could write this as:

type Author struct {
  ID   int    `json:"id"`
  Name string `json:"name"`
}

type Comment struct {
  ID        int    `json:"id"`
  Author    Author `json:"author"`
  Body      string `json:"body"`
  InReplyTo int    `json:"in_reply_to"`
}

Rust does not have this capability out of the box, however there is a fantastic framework available known as serde which works across JSON and every other serialization method that you can think of. Let's add serde and its JSON support to our crate:

[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"

You might notice that the dependency line for serde is different here. Go's JSON package works by using struct tags as metadata, but Rust doesn't have these. We need to use Rust's derive feature instead.

So, to use serde for our comment type, we would write Rust that looks like this:

use serde::{Deserialize, Serialize};

#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct Author {
  pub id: i32,
  pub name: String,
}

#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct Comment {
  pub id: i32,
  pub author: Author,
  pub body: String,
  pub in_reply_to: i32,
}

And then we can load that from JSON using code like this:

fn main() {
  let data = r#"
  {
    "id": 31337,
    "author": {
      "id": 420,
      "name": "Cadey"
    },
    "body": "hahaha its is an laughter image",
    "in_reply_to": 31335
  }
  "#;
  
  let c: Comment = serde_json::from_str(data).expect("json to parse");
  println!("comment: {:#?}", c);
}

And you can use it like this:

$ cargo run --example json
   Compiling xesite v2.0.1 (/home/cadey/code/christine.website)
    Finished dev [unoptimized + debuginfo] target(s) in 0.43s
     Running `target/debug/examples/json`
comment: Comment {
    id: 31337,
    author: Author {
        id: 420,
        name: "Cadey",
    },
    body: "hahaha its is an laughter image",
    in_reply_to: 31335,
}

HTTP

Many APIs expose their data over HTTP. Go has the net/http package that acts as a production-grade (Google uses this in production) HTTP client and server. This allows you to get going with new projects very easily. The Rust standard library doesn't have this out of the box, but there are some very convenient crates that can fill in the blanks.

Client

For an HTTP client, we can use reqwest. It can also seamlessly integrate with serde to allow you to parse JSON from HTTP without any issues. Let's add reqwest to our crate as well as tokio to act as an asynchronous runtime:

[dependencies]
reqwest = { version = "0.10", features = ["json"] }
tokio = { version = "0.2", features = ["full"] }

We need tokio because Rust doesn't ship with an asynchronous runtime by default. Go does as a core part of the standard library (and arguably the language), but tokio is about equivalent to most of the important things that the Go runtime handles for you. This omission may seem annoying, but it makes it easy for you to create a custom asynchronous runtime should you need to.

And then let's integrate with that imaginary comment api at https://xena.greedo.xeserv.us/files/comment.json:

use eyre::Result;
use serde::{Deserialize, Serialize};

#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct Author {
    pub id: i32,
    pub name: String,
}

#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct Comment {
    pub id: i32,
    pub author: Author,
    pub body: String,
    pub in_reply_to: i32,
}

#[tokio::main]
async fn main() -> Result<()> {
  let c: Comment = reqwest::get("https://xena.greedo.xeserv.us/files/comment.json")
      .await?
      .json()
      .await?;
  println!("comment: {:#?}", c);
  
  Ok(())
}

And then let's run this:

$ cargo run --example http
   Compiling xesite v2.0.1 (/home/cadey/code/christine.website)
    Finished dev [unoptimized + debuginfo] target(s) in 2.20s
     Running `target/debug/examples/http`
comment: Comment {
    id: 31337,
    author: Author {
        id: 420,
        name: "Cadey",
    },
    body: "hahaha its is an laughter image",
    in_reply_to: 31335,
}

But what if the response status is not 200?

We can change the code to something like this:

let c: Comment = reqwest::get("https://xena.greedo.xeserv.us/files/comment2.json")
    .await?
    .error_for_status()?
    .json()
    .await?;

And then when we run it we get an error back:

$ cargo run --example http_fail
   Compiling xesite v2.0.1 (/home/cadey/code/christine.website)
    Finished dev [unoptimized + debuginfo] target(s) in 1.84s
     Running `/home/cadey/code/christine.website/target/debug/examples/http_fail`
Error: HTTP status client error (404 Not Found) for url (https://xena.greedo.xeserv.us/files/comment2.json)

This combined with the other features in reqwest give you an very capable HTTP client that does even more than Go's HTTP client does out of the box.

Server

As for HTTP servers though, let's take a look at warp. warp is a HTTP server framework that builds on top of Rust's type system. You can add warp to your dependencies like this:

[dependencies]
warp = "0.2"

Let's take a look at its "Hello, World" example:

use warp::Filter;

#[tokio::main]
async fn main() {
    // GET /hello/warp => 200 OK with body "Hello, warp!"
    let hello = warp::path!("hello" / String)
        .map(|name| format!("Hello, {}!", name));

    warp::serve(hello)
        .run(([127, 0, 0, 1], 3030))
        .await;
}

We can then build up multiple routes with its or pattern:

let hello = warp::path!("hello" / String)
    .map(|name| format!("Hello, {}!", name));
let health = warp::path!(".within" / "health")
    .map(|| "OK");
let routes = hello.or(health);

And even inject other datatypes into your handlers with filters such as in the printer facts API server:

let fact = {
    let facts = pfacts::make();
    warp::any().map(move || facts.clone())
};

let fact_handler = warp::get()
    .and(warp::path("fact"))
    .and(fact.clone())
    .and_then(give_fact);

warp is an extremely capable HTTP server and can work across everything you need for production-grade web apps.

The blog you are looking at right now is powered by warp!

Templating

Go's standard library also includes HTML and plain text templating with its packages html/template and text/template. There are many solutions for templating HTML in Rust, but the one I like the most is ructe. ructe uses Cargo's build.rs feature to generate Rust code for its templates at compile time. This allows your HTML templates to be compiled into the resulting application binary, allowing them to render at ludicrous speeds. To use it, you need to add it to your build-dependencies section of your Cargo.toml:

[build-dependencies]
ructe = { version = "0.12", features = ["warp02"] }

You will also need to add the mime crate to your dependencies because the generated template code will require it at runtime.

[dependencies]
mime = "0.3.0"

Once you've done this, create a new folder named templates in your current working directory. Create a file called hello.rs.html and put the following in it:

@(title: String, message: String)

<html>
  <head>
    <title>@title</title>
  </head>
  <body>
    <h1>@title</h1>
    <p>@message</p>
  </body>
</html>

Now add the following to the bottom of your main.rs file:

include!(concat!(env!("OUT_DIR"), "/templates.rs"));

And then use the template like this:

use warp::{http::Response, Filter, Rejection, Reply};

async fn hello_html(message: String) -> Result<impl Reply, Rejection> {
    Response::builder()
        .html(|o| templates::index_html(o, "Hello".to_string(), message).unwrap().clone()))
}

And hook it up in your main function:

let hello_html_rt = warp::path!("hello" / "html" / String)
    .and_then(hello_html);
    
let routes = hello_html_rt.or(health).or(hello);

For a more comprehensive example, check out the printerfacts server. It also shows how to handle 404 responses and other things like that.

Wow, this covered a lot. I've included most of the example code in the examples folder of this site's GitHub repo. I hope it will help you on your journey in Rust. This is documentation that I wish I had when I was learning Rust.

18 KiB Raw Blame History