How I Start: Nix (#124)
* blog: how i start with Nix * blog/howistartnix: updates * blog/howistartnix: i words good * blog/howistartnix: words better * blog/howistartnix: link to dockerTools * blog/howistartnix: nl -> ul * blog/howistartnix: link better * blog/howistartnix: more word variety
This commit is contained in:
GitHub
parent
1da6129332
commit
fa36afbb6c
|
|
@ -0,0 +1,574 @@
|
|||
---
|
||||
title: "How I Start: Nix"
|
||||
date: 2020-03-08
|
||||
series: howto
|
||||
tags:
|
||||
- nix
|
||||
- rust
|
||||
---
|
||||
|
||||
# How I Start: Nix
|
||||
|
||||
[Nix][nix] is a tool that helps people create reproducible builds. This means that
|
||||
given the a known input, you can get the same output on other machines.Let's
|
||||
build and deploy a small Rust service with Nix. This will not require the Rust
|
||||
compiler to be installed with [rustup][rustup] or similar.
|
||||
|
||||
[nix]: https://nixos.org/nix/
|
||||
[rustup]: https://rustup.rs
|
||||
|
||||
- Setting up your environment
|
||||
- A new project
|
||||
- Setting up the Rust compiler
|
||||
- Serving HTTP
|
||||
- A simple package build
|
||||
- Shipping it in a docker image
|
||||
|
||||
## Setting up your environment
|
||||
|
||||
The first step is to install Nix. If you are using a Linux machine, run this
|
||||
script:
|
||||
|
||||
```console
|
||||
$ curl https://nixos.org/nix/install | sh
|
||||
```
|
||||
|
||||
This will prompt you for more information as it goes on, so be sure to follow
|
||||
the instructions carefully. Once it is done, close and re-open your shell. After
|
||||
you have done this, `nix-env` should exist in your shell. Try to run it:
|
||||
|
||||
```console
|
||||
$ nix-env
|
||||
error: no operation specified
|
||||
Try 'nix-env --help' for more information.
|
||||
```
|
||||
|
||||
Let's install a few other tools to help us with development. First, let's
|
||||
install [lorri][lorri] to help us manage our development shell:
|
||||
|
||||
[lorri]: https://github.com/target/lorri
|
||||
|
||||
```
|
||||
$ nix-env --install --file https://github.com/target/lorri/archive/master.tar.gz
|
||||
```
|
||||
|
||||
This will automatically download and build lorri for your system based on the
|
||||
latest possible version. Once that is done, open another shell window (the lorri
|
||||
docs include ways to do this more persistently, but this will work for now) and run:
|
||||
|
||||
```console
|
||||
$ lorri daemon
|
||||
```
|
||||
|
||||
Now go back to your main shell window and install [direnv][direnv]:
|
||||
|
||||
[direnv]: https://direnv.net
|
||||
|
||||
```console
|
||||
$ nix-env --install direnv
|
||||
```
|
||||
|
||||
Next, follow the [shell setup][direnvsetup] needed for your shell. I personally
|
||||
use `fish` with [oh my fish][omf], so I would run this:
|
||||
|
||||
[direnvsetup]: https://direnv.net/docs/hook.html
|
||||
[omf]: https://github.com/oh-my-fish/oh-my-fish
|
||||
|
||||
```console
|
||||
$ omf install direnv
|
||||
```
|
||||
|
||||
Finally, let's install [niv][niv] to help us handle dependencies for the
|
||||
project. This will allow us to make sure that our builds pin _everything_ to a
|
||||
specific set of versions, including operating system packages.
|
||||
|
||||
[niv]: https://github.com/nmattia/niv
|
||||
|
||||
```console
|
||||
$ nix-env --install niv
|
||||
```
|
||||
|
||||
Now that we have all of the tools we will need installed, let's create the
|
||||
project.
|
||||
|
||||
# A new project
|
||||
|
||||
Go to your favorite place to put code and make a new folder. I personally prefer
|
||||
`~/code`, so I will be using that here:
|
||||
|
||||
```console
|
||||
$ cd ~/code
|
||||
$ mkdir helloworld
|
||||
$ cd helloworld
|
||||
```
|
||||
|
||||
Let's set up the basic skeleton of the project. First, initialize niv:
|
||||
|
||||
```console
|
||||
$ niv init
|
||||
```
|
||||
|
||||
This will add the latest versions of `niv` itself and the packages used for the
|
||||
system to `nix/sources.json`. This will allow us to pin exact versions so the
|
||||
environment is as predictable as possible. Sometimes the versions of software in
|
||||
the pinned nixpkgs are too old. If this happens, you can update to the
|
||||
"unstable" branch of nixpkgs with this command:
|
||||
|
||||
```console
|
||||
$ niv update nixpkgs -b nixpkgs-unstable
|
||||
```
|
||||
|
||||
Next, set up lorri using `lorri init`:
|
||||
|
||||
```console
|
||||
$ lorri init
|
||||
```
|
||||
|
||||
This will create `shell.nix` and `.envrc`. `shell.nix` will be where we define
|
||||
the development environment for this service. `.envrc` is used to tell direnv
|
||||
what it needs to do. Let's try and activate the `.envrc`:
|
||||
|
||||
```console
|
||||
$ cd .
|
||||
direnv: error /home/cadey/code/helloworld/.envrc is blocked. Run `direnv allow`
|
||||
to approve its content
|
||||
```
|
||||
|
||||
Let's review its content:
|
||||
|
||||
```console
|
||||
$ cat .envrc
|
||||
eval "$(lorri direnv)"
|
||||
```
|
||||
|
||||
This seems reasonable, so approve it with `direnv allow` like the error message
|
||||
suggests:
|
||||
|
||||
```console
|
||||
$ direnv allow
|
||||
```
|
||||
|
||||
Now let's customize the `shell.nix` file to use our pinned version of nixpkgs.
|
||||
Currently, it looks something like this:
|
||||
|
||||
```nix
|
||||
# shell.nix
|
||||
let
|
||||
pkgs = import <nixpkgs> {};
|
||||
in
|
||||
pkgs.mkShell {
|
||||
buildInputs = [
|
||||
pkgs.hello
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
This currently imports nixpkgs from the system-level version of it. This means
|
||||
that different systems could have different versions of nixpkgs on it, and that
|
||||
could make the `shell.nix` file hard to reproduce between machines. Let's import
|
||||
the pinned version of nixpkgs that niv created:
|
||||
|
||||
```nix
|
||||
# shell.nix
|
||||
let
|
||||
sources = import ./nix/sources.nix;
|
||||
pkgs = import sources.nixpkgs {};
|
||||
in
|
||||
pkgs.mkShell {
|
||||
buildInputs = [
|
||||
pkgs.hello
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
And then let's test it with `lorri shell`:
|
||||
|
||||
```console
|
||||
$ lorri shell
|
||||
lorri: building environment........ done
|
||||
(lorri) $
|
||||
```
|
||||
|
||||
And let's see if `hello` is available inside the shell:
|
||||
|
||||
```console
|
||||
(lorri) $ hello
|
||||
Hello, world!
|
||||
```
|
||||
|
||||
You can set environment variables inside the `shell.nix` file. Do so like this:
|
||||
|
||||
```nix
|
||||
# shell.nix
|
||||
let
|
||||
sources = import ./nix/sources.nix;
|
||||
pkgs = import sources.nixpkgs {};
|
||||
in
|
||||
pkgs.mkShell {
|
||||
buildInputs = [
|
||||
pkgs.hello
|
||||
];
|
||||
|
||||
# Environment variables
|
||||
HELLO="world";
|
||||
}
|
||||
```
|
||||
|
||||
Wait a moment for lorri to finish rebuilding the development environment and
|
||||
then let's see if the environment variable shows up:
|
||||
|
||||
```console
|
||||
$ cd .
|
||||
direnv: loading ~/code/helloworld/.envrc
|
||||
<output snipped>
|
||||
$ echo $HELLO
|
||||
world
|
||||
```
|
||||
|
||||
Now that we have the basics of the environment set up, lets install the Rust
|
||||
compiler.
|
||||
|
||||
# Setting up the Rust compiler
|
||||
|
||||
First, add [nixpkgs-mozilla][nixpkgsmoz] to niv:
|
||||
|
||||
[nixpkgsmoz]: https://github.com/mozilla/nixpkgs-mozilla
|
||||
|
||||
```console
|
||||
$ niv add mozilla/nixpkgs-mozilla
|
||||
```
|
||||
|
||||
Then create `nix/rust.nix` in your repo:
|
||||
|
||||
```nix
|
||||
# nix/rust.nix
|
||||
{ sources ? import ./sources.nix }:
|
||||
|
||||
let
|
||||
pkgs =
|
||||
import sources.nixpkgs { overlays = [ (import sources.nixpkgs-mozilla) ]; };
|
||||
channel = "nightly";
|
||||
date = "2020-03-08";
|
||||
targets = [ ];
|
||||
chan = pkgs.rustChannelOfTargets channel date targets;
|
||||
in chan
|
||||
```
|
||||
|
||||
This creates a nix function that takes in the pre-imported list of sources,
|
||||
creates a copy of nixpkgs with Rust at the nightly version `2020-03-08` overlaid
|
||||
into it, and exposes the rust package out of it. Let's add this to `shell.nix`:
|
||||
|
||||
```nix
|
||||
# shell.nix
|
||||
let
|
||||
sources = import ./nix/sources.nix;
|
||||
rust = import ./nix/rust.nix { inherit sources; };
|
||||
pkgs = import sources.nixpkgs { };
|
||||
in
|
||||
pkgs.mkShell {
|
||||
buildInputs = [
|
||||
rust
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
Then ask lorri to recreate the development environment. This may take a bit to
|
||||
run because it's setting up everything the Rust compiler requires to run.
|
||||
|
||||
```console
|
||||
$ lorri shell
|
||||
(lorri) $
|
||||
```
|
||||
|
||||
Let's see what version of Rust is installed:
|
||||
|
||||
```console
|
||||
(lorri) $ rustc --version
|
||||
rustc 1.43.0-nightly (823ff8cf1 2020-03-07)
|
||||
```
|
||||
|
||||
This is exactly what we expect. Rust nightly versions get released with the
|
||||
date of the previous day in them. To be extra sure, let's see what the shell
|
||||
thinks `rustc` resolves to:
|
||||
|
||||
```console
|
||||
(lorri) $ which rustc
|
||||
/nix/store/w6zk1zijfwrnjm6xyfmrgbxb6dvvn6di-rust-1.43.0-nightly-2020-03-07-823ff8cf1/bin/rustc
|
||||
```
|
||||
|
||||
And now exit that shell and reload direnv:
|
||||
|
||||
```console
|
||||
(lorri) $ exit
|
||||
$ cd .
|
||||
direnv: loading ~/code/helloworld/.envrc
|
||||
$ which rustc
|
||||
/nix/store/w6zk1zijfwrnjm6xyfmrgbxb6dvvn6di-rust-1.43.0-nightly-2020-03-07-823ff8cf1/bin/rustc
|
||||
```
|
||||
|
||||
And now we have Rust installed at an arbitrary nightly version for _that project
|
||||
only_. This will work on other machines too. Now that we have our development
|
||||
environment set up, let's serve HTTP.
|
||||
|
||||
## Serving HTTP
|
||||
|
||||
[Rocket][rocket] is a popular web framework for Rust programs. Let's use that to
|
||||
create a small "hello, world" server. We will need to do the following:
|
||||
|
||||
[rocket]: https://rocket.rs
|
||||
|
||||
- Create the new Rust project
|
||||
- Add Rocket as a dependency
|
||||
- Write our "hello world" route
|
||||
- Test a build of the service with `cargo build`
|
||||
|
||||
### Create the new Rust project
|
||||
|
||||
Create the new Rust project with `cargo init`:
|
||||
|
||||
```console
|
||||
$ cargo init --vcs git .
|
||||
Created binary (application) package
|
||||
```
|
||||
|
||||
This will create the directory `src` and a file named `Cargo.toml`. Rust code
|
||||
goes in `src` and the `Cargo.toml` file configures dependencies. Adding the
|
||||
`--vcs git` flag also has cargo create a [gitignore][gitignore] file so that the
|
||||
target folder isn't tracked by git.
|
||||
|
||||
[gitignore]: https://git-scm.com/docs/gitignore
|
||||
|
||||
### Add Rocket as a dependency
|
||||
|
||||
Open `Cargo.toml` and add the following to it:
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
rocket = "0.4.3"
|
||||
```
|
||||
|
||||
Then download/build Rocket with `cargo build`:
|
||||
|
||||
```console
|
||||
$ cargo build
|
||||
```
|
||||
|
||||
This will download all of the dependencies you need and precompile Rocket. This
|
||||
will help speed up later builds.
|
||||
|
||||
### Write our "hello world" route
|
||||
|
||||
Now put the following in `src/main.rs`:
|
||||
|
||||
```rust
|
||||
#![feature(proc_macro_hygiene, decl_macro)] // language features needed by Rocket
|
||||
|
||||
// Import the rocket macros
|
||||
#[macro_use]
|
||||
extern crate rocket;
|
||||
|
||||
// Create route / that returns "Hello, world!"
|
||||
#[get("/")]
|
||||
fn index() -> &'static str {
|
||||
"Hello, world!"
|
||||
}
|
||||
|
||||
fn main() {
|
||||
rocket::ignite().mount("/", routes![index]).launch();
|
||||
}
|
||||
```
|
||||
|
||||
### Test a build
|
||||
|
||||
Rerun `cargo build`:
|
||||
|
||||
```console
|
||||
$ cargo build
|
||||
```
|
||||
|
||||
This will create the binary at `target/debug/helloworld`. Let's run it locally
|
||||
and see if it works:
|
||||
|
||||
```console
|
||||
$ ./target/debug/helloworld &
|
||||
$ curl http://127.0.0.1:8000
|
||||
Hello, world!
|
||||
$ fg
|
||||
<press control-c>
|
||||
```
|
||||
|
||||
The HTTP service works. We have a binary that is created with the Rust compiler
|
||||
Nix installed.
|
||||
|
||||
## A simple package build
|
||||
|
||||
Now that we have the HTTP service working, let's put it inside a nix package. We
|
||||
will need to use [naersk][naersk] to do this. Add naersk to your project with
|
||||
niv:
|
||||
|
||||
[naersk]: https://github.com/nmattia/naersk
|
||||
|
||||
```console
|
||||
$ niv add nmattia/naersk
|
||||
```
|
||||
|
||||
Now let's create `helloworld.nix`:
|
||||
|
||||
```
|
||||
# import niv sources and the pinned nixpkgs
|
||||
{ sources ? import ./nix/sources.nix, pkgs ? import sources.nixpkgs { }}:
|
||||
let
|
||||
# import rust compiler
|
||||
rust = import ./nix/rust.nix { inherit sources; };
|
||||
|
||||
# configure naersk to use our pinned rust compiler
|
||||
naersk = pkgs.callPackage sources.naersk {
|
||||
rustc = rust;
|
||||
cargo = rust;
|
||||
};
|
||||
|
||||
# tell nix-build to ignore the `target` directory
|
||||
src = builtins.filterSource
|
||||
(path: type: type != "directory" || builtins.baseNameOf path != "target")
|
||||
./.;
|
||||
in naersk.buildPackage {
|
||||
inherit src;
|
||||
remapPathPrefix =
|
||||
true; # remove nix store references for a smaller output package
|
||||
}
|
||||
```
|
||||
|
||||
And then build it with `nix-build`:
|
||||
|
||||
```console
|
||||
$ nix-build helloworld.nix
|
||||
```
|
||||
|
||||
This can take a bit to run, but it will do the following things:
|
||||
|
||||
- Download naersk
|
||||
- Download every Rust crate your HTTP service depends on into the Nix store
|
||||
- Run your program's tests
|
||||
- Build your dependencies into a Nix package
|
||||
- Build your program with those dependencies
|
||||
- Place a link to the result at `./result`
|
||||
|
||||
Once it is done, let's take a look at the result:
|
||||
|
||||
```console
|
||||
$ du -hs ./result/bin/helloworld
|
||||
2.1M ./result/bin/helloworld
|
||||
|
||||
$ ldd ./result/bin/helloworld
|
||||
linux-vdso.so.1 (0x00007fffae080000)
|
||||
libdl.so.2 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libdl.so.2 (0x0
|
||||
0007f3a01666000)
|
||||
librt.so.1 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/librt.so.1 (0x0
|
||||
0007f3a0165c000)
|
||||
libpthread.so.0 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libpthread
|
||||
.so.0 (0x00007f3a0163b000)
|
||||
libgcc_s.so.1 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libgcc_s.so.
|
||||
1 (0x00007f3a013f5000)
|
||||
libc.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libc.so.6 (0x000
|
||||
07f3a0123f000)
|
||||
/nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/ld-linux-x86-64.so.2 => /lib6
|
||||
4/ld-linux-x86-64.so.2 (0x00007f3a0160b000)
|
||||
libm.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libm.so.6 (0x000
|
||||
07f3a010a9000)
|
||||
```
|
||||
|
||||
This means that the Nix build created a 2.1 megabyte binary that only depends on
|
||||
[glibc][glibc], the implementation of the C language standard library that Nix
|
||||
prefers.
|
||||
|
||||
[glibc]: https://www.gnu.org/software/libc/
|
||||
|
||||
For repo cleanliness, add the `result` link to the [gitignore][gitignore]:
|
||||
|
||||
```console
|
||||
$ echo 'result*' >> .gitignore
|
||||
```
|
||||
|
||||
## Shipping it in a Docker image
|
||||
|
||||
Now that we have a package built, let's ship it in a docker image. nixpkgs
|
||||
provides [dockerTools][dockertools] which helps us create docker images out of
|
||||
Nix packages. Let's create `default.nix` with the following contents:
|
||||
|
||||
[dockertools]: https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools
|
||||
|
||||
```nix
|
||||
{ system ? builtins.currentSystem }:
|
||||
|
||||
let
|
||||
sources = import ./nix/sources.nix;
|
||||
pkgs = import sources.nixpkgs { };
|
||||
helloworld = import ./helloworld.nix { inherit sources pkgs; };
|
||||
|
||||
name = "xena/helloworld";
|
||||
tag = "latest";
|
||||
|
||||
in pkgs.dockerTools.buildLayeredImage {
|
||||
inherit name tag;
|
||||
contents = [ helloworld ];
|
||||
|
||||
config = {
|
||||
Cmd = [ "/bin/helloworld" ];
|
||||
Env = [ "ROCKET_PORT=5000" ];
|
||||
WorkingDir = "/";
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
And then build it with `nix-build`:
|
||||
|
||||
```console
|
||||
$ nix-build default.nix
|
||||
```
|
||||
|
||||
This will create a tarball containing the docker image information as the result
|
||||
of the Nix build. Load it into docker using `docker load`:
|
||||
|
||||
```console
|
||||
$ docker load -i result
|
||||
```
|
||||
|
||||
And then run it using `docker run`:
|
||||
|
||||
```console
|
||||
$ docker run --rm -itp 52340:5000 xena/helloworld
|
||||
```
|
||||
|
||||
Now test it using curl:
|
||||
|
||||
```console
|
||||
$ curl http://127.0.0.1:52340
|
||||
Hello, world!
|
||||
```
|
||||
|
||||
And now you have a docker image you can run wherever you want. The
|
||||
`buildLayeredImage` function used in `default.nix` also makes Nix put each
|
||||
dependency of the package into its own docker layer. This makes new versions of
|
||||
your program very efficient to upgrade on your clusters, realistically this
|
||||
reduces the amount of data needed for new versions of the program down to what
|
||||
changed. If nothing but some resources in their own package were changed, only
|
||||
those packages get downloaded.
|
||||
|
||||
This is how I start a new project with Nix. I put all of the code described in
|
||||
this post in [this GitHub repo][helloworldrepo] in case it helps. Have fun and
|
||||
be well.
|
||||
|
||||
[helloworldrepo]: https://github.com/Xe/helloworld
|
||||
|
||||
---
|
||||
|
||||
For some "extra credit" tasks, try and see if you can do the following:
|
||||
|
||||
- Use the version of [niv][niv] that niv pinned
|
||||
- Customize the environment of the container by following the [Rocket
|
||||
configuration documentation](https://rocket.rs/v0.4/guide/configuration/)
|
||||
- Add some more routes to the program
|
||||
- Read the [Nix
|
||||
documentation](https://nixos.org/nix/manual/#chap-writing-nix-expressions) and
|
||||
learn more about writing Nix expressions
|
||||
- Configure your editor/IDE to use the `direnv` path
|
||||
Loading…
Reference in New Issue