--- title: Rust Crates that do What the Go Standard library Does date: 2020-09-27 tags: - rust --- # Rust Crates that do What the Go Standard library Does One of Go's greatest strengths is how batteries-included the standard library is. You can do most of what you need to do with only the standard library. On the other hand, Rust's standard library is severely lacking by comparison. However, the community has capitalized on this and been working on a bunch of batteries that you can include in your rust projects. I'm going to cover a bunch of them in this post in a few sections. [A lot of these are actually used to help make this blog site work!](conversation://Mara/hacker) ## Logging Go has logging out of the box with package [`log`](https://godoc.org/log). Package `log` is a very uncontroversial logger. It does what it says it does and with little fuss. However it does not include a lot of niceties like logging levels and context-aware values. In Rust, we have the [`log`](https://docs.rs/log/) crate which is a very simple interface. It uses the `error!`, `warn!`, `info!`, `debug!` and `trace!` macros which correlate to the highest and lowest levers. If you want to use `log` in a Rust crate, you can add it to your `Cargo.toml` file like this: ```toml [dependencies] log = "0.4" ``` Then you can use it in your Rust code like this: ```rust use log::{error, warn, info, debug, trace}; fn main() { trace!("starting main"); debug!("debug message"); info!("this is some information"); warn!("oh no something bad is about to happen"); error!("oh no it's an error"); } ``` [Wait, where does that log to? I ran that example locally but I didn't see any of the messages anywhere](conversation://Mara/wat) This is because the `log` crate doesn't directly log anything anywhere, it is a facade that other packages build off of. [`pretty_env_logger`](https://docs.rs/pretty_env_logger) is a commonly used crate with the `log` facade. Let's add it to the program and work from there: ```toml [dependencies] log = "0.4" pretty_env_logger = "0.4" ``` Then let's enable it in our code: ```rust use log::{error, warn, info, debug, trace}; fn main() { pretty_env_logger::init(); trace!("starting main"); debug!("debug message"); info!("this is some information"); warn!("oh no something bad is about to happen"); error!("oh no it's an error"); } ``` And now let's run it with `RUST_LOG=trace`: ```console $ env RUST_LOG=trace cargo run --example logger_test Finished dev [unoptimized + debuginfo] target(s) in 0.07s Running `/home/cadey/code/christine.website/target/debug/logger_test` TRACE logger_test > starting main DEBUG logger_test > debug message INFO logger_test > this is some information WARN logger_test > oh no something bad is about to happen ERROR logger_test > oh no it's an error ``` There are [many other](https://docs.rs/log/0.4.11/log/#available-logging-implementations) consumers of the log crate and implementing a consumer is easy should you want to do more than `pretty_env_logger` can do on its own. However, I have found that `pretty_env_logger` does just enough on its own. See its documentation for more information. ## Flags Go's standard library has the [`flag`](https://godoc.org/flag) package out of the box. This package is incredibly basic, but is surprisingly capable in terms of what you can actually do with it. A common thing to do is use flags for configuration or other options, such as [here](https://github.com/Xe/hlang/blob/44bb74efa6f124ca05483a527c0e735ce0fca143/main.go#L15-L22): ```go package main import "flag" var ( program = flag.String("p", "", "h program to compile/run") outFname = flag.String("o", "", "if specified, write the webassembly binary created by -p here") watFname = flag.String("o-wat", "", "if specified, write the uncompiled webassembly created by -p here") port = flag.String("port", "", "HTTP port to listen on") writeTao = flag.Bool("koan", false, "if true, print the h koan and then exit") writeVersion = flag.Bool("v", false, "if true, print the version of h and then exit") ) ``` This will make a few package-global variables that will contain the values of the command-line arguments. In Rust, a commonly used command line parsing package is [`structopt`](https://docs.rs/structopt). It works in a bit of a different way than Go's `flag` package does though. `structopt` focuses on loading options into a structure rather than into globally mutable variables. [Something you may notice in Rust-land is that globally mutable state is talked about as if it is something to be avoided. It's not inherently bad, but it does make things more likely to crash at runtime. In most cases, these global variables with package `flag` are fine, but only if they are ever written to before the program really starts to do what it needs to do. If they are ever written to and read from dynamically at runtime, then you can get into a lot of problems such as race conditions.](conversation://Mara/hacker) Here's a quick example copied from [pa'i](https://github.com/Xe/pahi): ```rust #[derive(Debug, StructOpt)] #[structopt( name = "pa'i", about = "A WebAssembly runtime in Rust meeting the Olin ABI." )] struct Opt { /// Backend #[structopt(short, long, default_value = "cranelift")] backend: String, /// Print syscalls on exit #[structopt(short, long)] function_log: bool, /// Do not cache compiled code? #[structopt(short, long)] no_cache: bool, /// Binary to run #[structopt()] fname: String, /// Main function #[structopt(short, long, default_value = "_start")] entrypoint: String, /// Arguments of the wasm child #[structopt()] args: Vec, } ``` This has the Rust compiler generate the needed argument parsing code for you, so you can just use the values as normal: ```rust fn main() { let opt = Opt::from_args(); debug!("args: {:?}", opt.args); } ``` You can even handle subcommands with this, such as in [palisade](https://github.com/lightspeed/palisade/blob/master/src/main.rs). This package should handle just about everything you'd do with the `flag` package, but will also work for cases where `flag` falls apart. ## Errors Go's standard library has the [`error` interface](https://godoc.org/builtin#error) which lets you create a type that describes why functions fail to do what they intend. Rust has the [`Error` trait](https://doc.rust-lang.org/std/error/trait.Error.html) which lets you also create a type that describes why functions fail to do what they intend. In [my last post](https://christine.website/blog/TLDR-rust-2020-09-19) I described [`eyre`](https://docs.rs/eyre) and the Result type. However, this time we're going to dive into [`thiserror`](https://docs.rs/thiserror) for making our own error type. Let's add `thiserror` to our crate: ```toml [dependencies] thiserror = "1" ``` And then let's re-implement our `DivideByZero` error from the last post: ```rust use std::fmt; use thiserror::Error; #[derive(Debug, Error)] struct DivideByZero; impl fmt::Display for DivideByZero { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "cannot divide by zero") } } ``` The compiler made our error instance for us! It can even do that for more complicated error types like this one that wraps a lot of other error cases and error types in [maj](https://tulpa.dev/cadey/maj): ```rust #[derive(thiserror::Error, Debug)] pub enum Error { #[error("TLS error: {0:?}")] TLS(#[from] TLSError), #[error("URL error: {0:?}")] URL(#[from] url::ParseError), #[error("Invalid DNS name: {0:?}")] InvalidDNSName(#[from] webpki::InvalidDNSNameError), #[error("IO error: {0:?}")] IO(#[from] std::io::Error), #[error("Response parsing error: {0:?}")] ResponseParse(#[from] crate::ResponseError), #[error("Invalid URL scheme {0:?}")] InvalidScheme(String), } ``` [These `#[error("whatever")]` annotations will show up when the error message is printed. See here for more information on what details you can include here.](conversation://Mara/hacker) ## Serialization / Deserialization Go has JSON encoding/decoding in its standard library via package [`encoding/json`](https://godoc.org/encoding/json). This allows you to define types that can be read from and write to JSON easily. Let's take this simple JSON object representing a comment from some imaginary API as an example: ```json { "id": 31337, "author": { "id": 420, "name": "Cadey" }, "body": "hahaha its is an laughter image", "in_reply_to": 31335 } ``` In Go you could write this as: ```go type Author struct { ID int `json:"id"` Name string `json:"name"` } type Comment struct { ID int `json:"id"` Author Author `json:"author"` Body string `json:"body"` InReplyTo int `json:"in_reply_to"` } ``` Rust does not have this capability out of the box, however there is a fantastic framework available known as [serde](https://serde.rs/) which works across JSON and every other serialization method that you can think of. Let's add serde and its JSON support to our crate: ```toml [dependencies] serde = { version = "1", features = ["derive"] } serde_json = "1" ``` [You might notice that the dependency line for serde is different here. Go's JSON package works by using struct tags as metadata, but Rust doesn't have these. We need to use Rust's derive feature instead.](conversation://Mara/hacker) So, to use serde for our comment type, we would write Rust that looks like this: ```rust use serde::{Deserialize, Serialize}; #[derive(Clone, Debug, Deserialize, Serialize)] pub struct Author { pub id: i32, pub name: String, } #[derive(Clone, Debug, Deserialize, Serialize)] pub struct Comment { pub id: i32, pub author: Author, pub body: String, pub in_reply_to: i32, } ``` And then we can load that from JSON using code like this: ```rust fn main() { let data = r#" { "id": 31337, "author": { "id": 420, "name": "Cadey" }, "body": "hahaha its is an laughter image", "in_reply_to": 31335 } "#; let c: Comment = serde_json::from_str(data).expect("json to parse"); println!("comment: {:#?}", c); } ``` And you can use it like this: ```console $ cargo run --example json Compiling xesite v2.0.1 (/home/cadey/code/christine.website) Finished dev [unoptimized + debuginfo] target(s) in 0.43s Running `target/debug/examples/json` comment: Comment { id: 31337, author: Author { id: 420, name: "Cadey", }, body: "hahaha its is an laughter image", in_reply_to: 31335, } ``` ## HTTP Many APIs expose their data over HTTP. Go has the [`net/http`](https://godoc.org/net/http) package that acts as a production-grade (Google uses this in production) HTTP client and server. This allows you to get going with new projects very easily. The Rust standard library doesn't have this out of the box, but there are some very convenient crates that can fill in the blanks. ### Client For an HTTP client, we can use [`reqwest`](https://docs.rs/reqwest). It can also seamlessly integrate with serde to allow you to parse JSON from HTTP without any issues. Let's add reqwest to our crate as well as [`tokio`](https://tokio.rs) to act as an asynchronous runtime: ```toml [dependencies] reqwest = { version = "0.10", features = ["json"] } tokio = { version = "0.2", features = ["full"] } ``` [We need `tokio` because Rust doesn't ship with an asynchronous runtime by default. Go does as a core part of the standard library (and arguably the language), but `tokio` is about equivalent to most of the important things that the Go runtime handles for you. This omission may seem annoying, but it makes it easy for you to create a custom asynchronous runtime should you need to.](conversation://Mara/hacker) And then let's integrate with that imaginary comment api at [https://xena.greedo.xeserv.us/files/comment.json](https://xena.greedo.xeserv.us/files/comment.json): ```rust use eyre::Result; use serde::{Deserialize, Serialize}; #[derive(Clone, Debug, Deserialize, Serialize)] pub struct Author { pub id: i32, pub name: String, } #[derive(Clone, Debug, Deserialize, Serialize)] pub struct Comment { pub id: i32, pub author: Author, pub body: String, pub in_reply_to: i32, } #[tokio::main] async fn main() -> Result<()> { let c: Comment = reqwest::get("https://xena.greedo.xeserv.us/files/comment.json") .await? .json() .await?; println!("comment: {:#?}", c); Ok(()) } ``` And then let's run this: ```console $ cargo run --example http Compiling xesite v2.0.1 (/home/cadey/code/christine.website) Finished dev [unoptimized + debuginfo] target(s) in 2.20s Running `target/debug/examples/http` comment: Comment { id: 31337, author: Author { id: 420, name: "Cadey", }, body: "hahaha its is an laughter image", in_reply_to: 31335, } ``` [But what if the response status is not 200?](conversation://Mara/hmm) We can change the code to something like this: ```rust let c: Comment = reqwest::get("https://xena.greedo.xeserv.us/files/comment2.json") .await? .error_for_status()? .json() .await?; ``` And then when we run it we get an error back: ```console $ cargo run --example http_fail Compiling xesite v2.0.1 (/home/cadey/code/christine.website) Finished dev [unoptimized + debuginfo] target(s) in 1.84s Running `/home/cadey/code/christine.website/target/debug/examples/http_fail` Error: HTTP status client error (404 Not Found) for url (https://xena.greedo.xeserv.us/files/comment2.json) ``` This combined with the other features in `reqwest` give you an very capable HTTP client that does even more than Go's HTTP client does out of the box. ### Server As for HTTP servers though, let's take a look at [`warp`](https://docs.rs/warp). `warp` is a HTTP server framework that builds on top of Rust's type system. You can add warp to your dependencies like this: ```toml [dependencies] warp = "0.2" ``` Let's take a look at its ["Hello, World" example](https://github.com/seanmonstar/warp/blob/master/examples/hello.rs): ```rust use warp::Filter; #[tokio::main] async fn main() { // GET /hello/warp => 200 OK with body "Hello, warp!" let hello = warp::path!("hello" / String) .map(|name| format!("Hello, {}!", name)); warp::serve(hello) .run(([127, 0, 0, 1], 3030)) .await; } ``` We can then build up multiple routes with its `or` pattern: ``` let hello = warp::path!("hello" / String) .map(|name| format!("Hello, {}!", name)); let health = warp::path!(".within" / "health") .map(|| "OK"); let routes = hello.or(health); ``` And even inject other datatypes into your handlers with filters such as in the [printer facts API server](https://tulpa.dev/cadey/printerfacts/src/branch/main/src/main.rs): ``` let fact = { let facts = pfacts::make(); warp::any().map(move || facts.clone()) }; let fact_handler = warp::get() .and(warp::path("fact")) .and(fact.clone()) .and_then(give_fact); ``` `warp` is an extremely capable HTTP server and can work across everything you need for production-grade web apps. [The blog you are looking at right now is powered by warp!](conversation://Mara/hacker) ## Templating Go's standard library also includes HTML and plain text templating with its packages [`html/template`](https://godoc.org/html/template) and [`text/template`](https://godoc.org/text/template). There are many solutions for templating HTML in Rust, but the one I like the most is [`ructe`](https://docs.rs/ructe). `ructe` uses Cargo's [build.rs](https://doc.rust-lang.org/cargo/reference/build-scripts.html) feature to generate Rust code for its templates at compile time. This allows your HTML templates to be compiled into the resulting application binary, allowing them to render at ludicrous speeds. To use it, you need to add it to your `build-dependencies` section of your `Cargo.toml`: ```toml [build-dependencies] ructe = { version = "0.12", features = ["warp02"] } ``` You will also need to add the [`mime`](https://docs.rs/mime) crate to your dependencies because the generated template code will require it at runtime. ```toml [dependencies] mime = "0.3.0" ``` Once you've done this, create a new folder named `templates` in your current working directory. Create a file called `hello.rs.html` and put the following in it: ```html @(title: String, message: String) @title

@title

@message

``` Now add the following to the bottom of your `main.rs` file: ```rust include!(concat!(env!("OUT_DIR"), "/templates.rs")); ``` And then use the template like this: ```rust use warp::{http::Response, Filter, Rejection, Reply}; async fn hello_html(message: String) -> Result { Response::builder() .html(|o| templates::index_html(o, "Hello".to_string(), message).unwrap().clone())) } ``` And hook it up in your main function: ```rust let hello_html_rt = warp::path!("hello" / "html" / String) .and_then(hello_html); let routes = hello_html_rt.or(health).or(hello); ``` For a more comprehensive example, check out the [printerfacts server](https://tulpa.dev/cadey/printerfacts). It also shows how to handle 404 responses and other things like that. --- Wow, this covered a lot. I've included most of the example code in the [`examples`](https://github.com/Xe/site/tree/master/examples) folder of [this site's GitHub repo](https://github.com/Xe/site). I hope it will help you on your journey in Rust. This is documentation that I wish I had when I was learning Rust.