575 lines
14 KiB
Markdown
575 lines
14 KiB
Markdown
---
|
|
title: "How I Start: Nix"
|
|
date: 2020-03-08
|
|
series: howto
|
|
tags:
|
|
- nix
|
|
- rust
|
|
---
|
|
|
|
# How I Start: Nix
|
|
|
|
[Nix][nix] is a tool that helps people create reproducible builds. This means that
|
|
given a known input, you can get the same output on other machines. Let's build
|
|
and deploy a small Rust service with Nix. This will not require the Rust compiler
|
|
to be installed with [rustup][rustup] or similar.
|
|
|
|
[nix]: https://nixos.org/nix/
|
|
[rustup]: https://rustup.rs
|
|
|
|
- Setting up your environment
|
|
- A new project
|
|
- Setting up the Rust compiler
|
|
- Serving HTTP
|
|
- A simple package build
|
|
- Shipping it in a docker image
|
|
|
|
## Setting up your environment
|
|
|
|
The first step is to install Nix. If you are using a Linux machine, run this
|
|
script:
|
|
|
|
```console
|
|
$ curl https://nixos.org/nix/install | sh
|
|
```
|
|
|
|
This will prompt you for more information as it goes on, so be sure to follow
|
|
the instructions carefully. Once it is done, close and re-open your shell. After
|
|
you have done this, `nix-env` should exist in your shell. Try to run it:
|
|
|
|
```console
|
|
$ nix-env
|
|
error: no operation specified
|
|
Try 'nix-env --help' for more information.
|
|
```
|
|
|
|
Let's install a few other tools to help us with development. First, let's
|
|
install [lorri][lorri] to help us manage our development shell:
|
|
|
|
[lorri]: https://github.com/target/lorri
|
|
|
|
```
|
|
$ nix-env --install --file https://github.com/target/lorri/archive/master.tar.gz
|
|
```
|
|
|
|
This will automatically download and build lorri for your system based on the
|
|
latest possible version. Once that is done, open another shell window (the lorri
|
|
docs include ways to do this more persistently, but this will work for now) and run:
|
|
|
|
```console
|
|
$ lorri daemon
|
|
```
|
|
|
|
Now go back to your main shell window and install [direnv][direnv]:
|
|
|
|
[direnv]: https://direnv.net
|
|
|
|
```console
|
|
$ nix-env --install direnv
|
|
```
|
|
|
|
Next, follow the [shell setup][direnvsetup] needed for your shell. I personally
|
|
use `fish` with [oh my fish][omf], so I would run this:
|
|
|
|
[direnvsetup]: https://direnv.net/docs/hook.html
|
|
[omf]: https://github.com/oh-my-fish/oh-my-fish
|
|
|
|
```console
|
|
$ omf install direnv
|
|
```
|
|
|
|
Finally, let's install [niv][niv] to help us handle dependencies for the
|
|
project. This will allow us to make sure that our builds pin _everything_ to a
|
|
specific set of versions, including operating system packages.
|
|
|
|
[niv]: https://github.com/nmattia/niv
|
|
|
|
```console
|
|
$ nix-env --install niv
|
|
```
|
|
|
|
Now that we have all of the tools we will need installed, let's create the
|
|
project.
|
|
|
|
# A new project
|
|
|
|
Go to your favorite place to put code and make a new folder. I personally prefer
|
|
`~/code`, so I will be using that here:
|
|
|
|
```console
|
|
$ cd ~/code
|
|
$ mkdir helloworld
|
|
$ cd helloworld
|
|
```
|
|
|
|
Let's set up the basic skeleton of the project. First, initialize niv:
|
|
|
|
```console
|
|
$ niv init
|
|
```
|
|
|
|
This will add the latest versions of `niv` itself and the packages used for the
|
|
system to `nix/sources.json`. This will allow us to pin exact versions so the
|
|
environment is as predictable as possible. Sometimes the versions of software in
|
|
the pinned nixpkgs are too old. If this happens, you can update to the
|
|
"unstable" branch of nixpkgs with this command:
|
|
|
|
```console
|
|
$ niv update nixpkgs -b nixpkgs-unstable
|
|
```
|
|
|
|
Next, set up lorri using `lorri init`:
|
|
|
|
```console
|
|
$ lorri init
|
|
```
|
|
|
|
This will create `shell.nix` and `.envrc`. `shell.nix` will be where we define
|
|
the development environment for this service. `.envrc` is used to tell direnv
|
|
what it needs to do. Let's try and activate the `.envrc`:
|
|
|
|
```console
|
|
$ cd .
|
|
direnv: error /home/cadey/code/helloworld/.envrc is blocked. Run `direnv allow`
|
|
to approve its content
|
|
```
|
|
|
|
Let's review its content:
|
|
|
|
```console
|
|
$ cat .envrc
|
|
eval "$(lorri direnv)"
|
|
```
|
|
|
|
This seems reasonable, so approve it with `direnv allow` like the error message
|
|
suggests:
|
|
|
|
```console
|
|
$ direnv allow
|
|
```
|
|
|
|
Now let's customize the `shell.nix` file to use our pinned version of nixpkgs.
|
|
Currently, it looks something like this:
|
|
|
|
```nix
|
|
# shell.nix
|
|
let
|
|
pkgs = import <nixpkgs> {};
|
|
in
|
|
pkgs.mkShell {
|
|
buildInputs = [
|
|
pkgs.hello
|
|
];
|
|
}
|
|
```
|
|
|
|
This currently imports nixpkgs from the system-level version of it. This means
|
|
that different systems could have different versions of nixpkgs on it, and that
|
|
could make the `shell.nix` file hard to reproduce between machines. Let's import
|
|
the pinned version of nixpkgs that niv created:
|
|
|
|
```nix
|
|
# shell.nix
|
|
let
|
|
sources = import ./nix/sources.nix;
|
|
pkgs = import sources.nixpkgs {};
|
|
in
|
|
pkgs.mkShell {
|
|
buildInputs = [
|
|
pkgs.hello
|
|
];
|
|
}
|
|
```
|
|
|
|
And then let's test it with `lorri shell`:
|
|
|
|
```console
|
|
$ lorri shell
|
|
lorri: building environment........ done
|
|
(lorri) $
|
|
```
|
|
|
|
And let's see if `hello` is available inside the shell:
|
|
|
|
```console
|
|
(lorri) $ hello
|
|
Hello, world!
|
|
```
|
|
|
|
You can set environment variables inside the `shell.nix` file. Do so like this:
|
|
|
|
```nix
|
|
# shell.nix
|
|
let
|
|
sources = import ./nix/sources.nix;
|
|
pkgs = import sources.nixpkgs {};
|
|
in
|
|
pkgs.mkShell {
|
|
buildInputs = [
|
|
pkgs.hello
|
|
];
|
|
|
|
# Environment variables
|
|
HELLO="world";
|
|
}
|
|
```
|
|
|
|
Wait a moment for lorri to finish rebuilding the development environment and
|
|
then let's see if the environment variable shows up:
|
|
|
|
```console
|
|
$ cd .
|
|
direnv: loading ~/code/helloworld/.envrc
|
|
<output snipped>
|
|
$ echo $HELLO
|
|
world
|
|
```
|
|
|
|
Now that we have the basics of the environment set up, lets install the Rust
|
|
compiler.
|
|
|
|
# Setting up the Rust compiler
|
|
|
|
First, add [nixpkgs-mozilla][nixpkgsmoz] to niv:
|
|
|
|
[nixpkgsmoz]: https://github.com/mozilla/nixpkgs-mozilla
|
|
|
|
```console
|
|
$ niv add mozilla/nixpkgs-mozilla
|
|
```
|
|
|
|
Then create `nix/rust.nix` in your repo:
|
|
|
|
```nix
|
|
# nix/rust.nix
|
|
{ sources ? import ./sources.nix }:
|
|
|
|
let
|
|
pkgs =
|
|
import sources.nixpkgs { overlays = [ (import sources.nixpkgs-mozilla) ]; };
|
|
channel = "nightly";
|
|
date = "2020-03-08";
|
|
targets = [ ];
|
|
chan = pkgs.rustChannelOfTargets channel date targets;
|
|
in chan
|
|
```
|
|
|
|
This creates a nix function that takes in the pre-imported list of sources,
|
|
creates a copy of nixpkgs with Rust at the nightly version `2020-03-08` overlaid
|
|
into it, and exposes the rust package out of it. Let's add this to `shell.nix`:
|
|
|
|
```nix
|
|
# shell.nix
|
|
let
|
|
sources = import ./nix/sources.nix;
|
|
rust = import ./nix/rust.nix { inherit sources; };
|
|
pkgs = import sources.nixpkgs { };
|
|
in
|
|
pkgs.mkShell {
|
|
buildInputs = [
|
|
rust
|
|
];
|
|
}
|
|
```
|
|
|
|
Then ask lorri to recreate the development environment. This may take a bit to
|
|
run because it's setting up everything the Rust compiler requires to run.
|
|
|
|
```console
|
|
$ lorri shell
|
|
(lorri) $
|
|
```
|
|
|
|
Let's see what version of Rust is installed:
|
|
|
|
```console
|
|
(lorri) $ rustc --version
|
|
rustc 1.43.0-nightly (823ff8cf1 2020-03-07)
|
|
```
|
|
|
|
This is exactly what we expect. Rust nightly versions get released with the
|
|
date of the previous day in them. To be extra sure, let's see what the shell
|
|
thinks `rustc` resolves to:
|
|
|
|
```console
|
|
(lorri) $ which rustc
|
|
/nix/store/w6zk1zijfwrnjm6xyfmrgbxb6dvvn6di-rust-1.43.0-nightly-2020-03-07-823ff8cf1/bin/rustc
|
|
```
|
|
|
|
And now exit that shell and reload direnv:
|
|
|
|
```console
|
|
(lorri) $ exit
|
|
$ cd .
|
|
direnv: loading ~/code/helloworld/.envrc
|
|
$ which rustc
|
|
/nix/store/w6zk1zijfwrnjm6xyfmrgbxb6dvvn6di-rust-1.43.0-nightly-2020-03-07-823ff8cf1/bin/rustc
|
|
```
|
|
|
|
And now we have Rust installed at an arbitrary nightly version for _that project
|
|
only_. This will work on other machines too. Now that we have our development
|
|
environment set up, let's serve HTTP.
|
|
|
|
## Serving HTTP
|
|
|
|
[Rocket][rocket] is a popular web framework for Rust programs. Let's use that to
|
|
create a small "hello, world" server. We will need to do the following:
|
|
|
|
[rocket]: https://rocket.rs
|
|
|
|
- Create the new Rust project
|
|
- Add Rocket as a dependency
|
|
- Write our "hello world" route
|
|
- Test a build of the service with `cargo build`
|
|
|
|
### Create the new Rust project
|
|
|
|
Create the new Rust project with `cargo init`:
|
|
|
|
```console
|
|
$ cargo init --vcs git .
|
|
Created binary (application) package
|
|
```
|
|
|
|
This will create the directory `src` and a file named `Cargo.toml`. Rust code
|
|
goes in `src` and the `Cargo.toml` file configures dependencies. Adding the
|
|
`--vcs git` flag also has cargo create a [gitignore][gitignore] file so that the
|
|
target folder isn't tracked by git.
|
|
|
|
[gitignore]: https://git-scm.com/docs/gitignore
|
|
|
|
### Add Rocket as a dependency
|
|
|
|
Open `Cargo.toml` and add the following to it:
|
|
|
|
```toml
|
|
[dependencies]
|
|
rocket = "0.4.3"
|
|
```
|
|
|
|
Then download/build Rocket with `cargo build`:
|
|
|
|
```console
|
|
$ cargo build
|
|
```
|
|
|
|
This will download all of the dependencies you need and precompile Rocket, and it
|
|
will help speed up later builds.
|
|
|
|
### Write our "hello world" route
|
|
|
|
Now put the following in `src/main.rs`:
|
|
|
|
```rust
|
|
#![feature(proc_macro_hygiene, decl_macro)] // language features needed by Rocket
|
|
|
|
// Import the rocket macros
|
|
#[macro_use]
|
|
extern crate rocket;
|
|
|
|
// Create route / that returns "Hello, world!"
|
|
#[get("/")]
|
|
fn index() -> &'static str {
|
|
"Hello, world!"
|
|
}
|
|
|
|
fn main() {
|
|
rocket::ignite().mount("/", routes![index]).launch();
|
|
}
|
|
```
|
|
|
|
### Test a build
|
|
|
|
Rerun `cargo build`:
|
|
|
|
```console
|
|
$ cargo build
|
|
```
|
|
|
|
This will create the binary at `target/debug/helloworld`. Let's run it locally
|
|
and see if it works:
|
|
|
|
```console
|
|
$ ./target/debug/helloworld &
|
|
$ curl http://127.0.0.1:8000
|
|
Hello, world!
|
|
$ fg
|
|
<press control-c>
|
|
```
|
|
|
|
The HTTP service works. We have a binary that is created with the Rust compiler
|
|
Nix installed.
|
|
|
|
## A simple package build
|
|
|
|
Now that we have the HTTP service working, let's put it inside a nix package. We
|
|
will need to use [naersk][naersk] to do this. Add naersk to your project with
|
|
niv:
|
|
|
|
[naersk]: https://github.com/nmattia/naersk
|
|
|
|
```console
|
|
$ niv add nmattia/naersk
|
|
```
|
|
|
|
Now let's create `helloworld.nix`:
|
|
|
|
```
|
|
# import niv sources and the pinned nixpkgs
|
|
{ sources ? import ./nix/sources.nix, pkgs ? import sources.nixpkgs { }}:
|
|
let
|
|
# import rust compiler
|
|
rust = import ./nix/rust.nix { inherit sources; };
|
|
|
|
# configure naersk to use our pinned rust compiler
|
|
naersk = pkgs.callPackage sources.naersk {
|
|
rustc = rust;
|
|
cargo = rust;
|
|
};
|
|
|
|
# tell nix-build to ignore the `target` directory
|
|
src = builtins.filterSource
|
|
(path: type: type != "directory" || builtins.baseNameOf path != "target")
|
|
./.;
|
|
in naersk.buildPackage {
|
|
inherit src;
|
|
remapPathPrefix =
|
|
true; # remove nix store references for a smaller output package
|
|
}
|
|
```
|
|
|
|
And then build it with `nix-build`:
|
|
|
|
```console
|
|
$ nix-build helloworld.nix
|
|
```
|
|
|
|
This can take a bit to run, but it will do the following things:
|
|
|
|
- Download naersk
|
|
- Download every Rust crate your HTTP service depends on into the Nix store
|
|
- Run your program's tests
|
|
- Build your dependencies into a Nix package
|
|
- Build your program with those dependencies
|
|
- Place a link to the result at `./result`
|
|
|
|
Once it is done, let's take a look at the result:
|
|
|
|
```console
|
|
$ du -hs ./result/bin/helloworld
|
|
2.1M ./result/bin/helloworld
|
|
|
|
$ ldd ./result/bin/helloworld
|
|
linux-vdso.so.1 (0x00007fffae080000)
|
|
libdl.so.2 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libdl.so.2 (0x0
|
|
0007f3a01666000)
|
|
librt.so.1 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/librt.so.1 (0x0
|
|
0007f3a0165c000)
|
|
libpthread.so.0 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libpthread
|
|
.so.0 (0x00007f3a0163b000)
|
|
libgcc_s.so.1 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libgcc_s.so.
|
|
1 (0x00007f3a013f5000)
|
|
libc.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libc.so.6 (0x000
|
|
07f3a0123f000)
|
|
/nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/ld-linux-x86-64.so.2 => /lib6
|
|
4/ld-linux-x86-64.so.2 (0x00007f3a0160b000)
|
|
libm.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libm.so.6 (0x000
|
|
07f3a010a9000)
|
|
```
|
|
|
|
This means that the Nix build created a 2.1 megabyte binary that only depends on
|
|
[glibc][glibc], the implementation of the C language standard library that Nix
|
|
prefers.
|
|
|
|
[glibc]: https://www.gnu.org/software/libc/
|
|
|
|
For repo cleanliness, add the `result` link to the [gitignore][gitignore]:
|
|
|
|
```console
|
|
$ echo 'result*' >> .gitignore
|
|
```
|
|
|
|
## Shipping it in a Docker image
|
|
|
|
Now that we have a package built, let's ship it in a docker image. nixpkgs
|
|
provides [dockerTools][dockertools] which helps us create docker images out of
|
|
Nix packages. Let's create `default.nix` with the following contents:
|
|
|
|
[dockertools]: https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools
|
|
|
|
```nix
|
|
{ system ? builtins.currentSystem }:
|
|
|
|
let
|
|
sources = import ./nix/sources.nix;
|
|
pkgs = import sources.nixpkgs { };
|
|
helloworld = import ./helloworld.nix { inherit sources pkgs; };
|
|
|
|
name = "xena/helloworld";
|
|
tag = "latest";
|
|
|
|
in pkgs.dockerTools.buildLayeredImage {
|
|
inherit name tag;
|
|
contents = [ helloworld ];
|
|
|
|
config = {
|
|
Cmd = [ "/bin/helloworld" ];
|
|
Env = [ "ROCKET_PORT=5000" ];
|
|
WorkingDir = "/";
|
|
};
|
|
}
|
|
```
|
|
|
|
And then build it with `nix-build`:
|
|
|
|
```console
|
|
$ nix-build default.nix
|
|
```
|
|
|
|
This will create a tarball containing the docker image information as the result
|
|
of the Nix build. Load it into docker using `docker load`:
|
|
|
|
```console
|
|
$ docker load -i result
|
|
```
|
|
|
|
And then run it using `docker run`:
|
|
|
|
```console
|
|
$ docker run --rm -itp 52340:5000 xena/helloworld
|
|
```
|
|
|
|
Now test it using curl:
|
|
|
|
```console
|
|
$ curl http://127.0.0.1:52340
|
|
Hello, world!
|
|
```
|
|
|
|
And now you have a docker image you can run wherever you want. The
|
|
`buildLayeredImage` function used in `default.nix` also makes Nix put each
|
|
dependency of the package into its own docker layer. This makes new versions of
|
|
your program very efficient to upgrade on your clusters, realistically this
|
|
reduces the amount of data needed for new versions of the program down to what
|
|
changed. If nothing but some resources in their own package were changed, only
|
|
those packages get downloaded.
|
|
|
|
This is how I start a new project with Nix. I put all of the code described in
|
|
this post in [this GitHub repo][helloworldrepo] in case it helps. Have fun and
|
|
be well.
|
|
|
|
[helloworldrepo]: https://github.com/Xe/helloworld
|
|
|
|
---
|
|
|
|
For some "extra credit" tasks, try and see if you can do the following:
|
|
|
|
- Use the version of [niv][niv] that niv pinned
|
|
- Customize the environment of the container by following the [Rocket
|
|
configuration documentation](https://rocket.rs/v0.4/guide/configuration/)
|
|
- Add some more routes to the program
|
|
- Read the [Nix
|
|
documentation](https://nixos.org/nix/manual/#chap-writing-nix-expressions) and
|
|
learn more about writing Nix expressions
|
|
- Configure your editor/IDE to use the `direnv` path
|