artemis
/
xesite
forked from cadey/xesite
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

How I Start: Nix (#124) 

* blog: how i start with Nix

* blog/howistartnix: updates

* blog/howistartnix: i words good

* blog/howistartnix: words better

* blog/howistartnix: link to dockerTools

* blog/howistartnix: nl -> ul

* blog/howistartnix: link better

* blog/howistartnix: more word variety
This commit is contained in:
Cadey Ratio 2020-03-08 15:55:07 -04:00 committed by GitHub
parent 1da6129332
commit fa36afbb6c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 574 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

574
blog/how-i-start-nix-2020-03-08.markdown Normal file
View File

@ -0,0 +1,574 @@
---
title: "How I Start: Nix"
date: 2020-03-08
series: howto
tags:
- nix
- rust
---
# How I Start: Nix
[Nix][nix] is a tool that helps people create reproducible builds. This means that
given the a known input, you can get the same output on other machines.Let's
build and deploy a small Rust service with Nix. This will not require the Rust
compiler to be installed with [rustup][rustup] or similar.
[nix]: https://nixos.org/nix/
[rustup]: https://rustup.rs
- Setting up your environment
- A new project
- Setting up the Rust compiler
- Serving HTTP
- A simple package build
- Shipping it in a docker image
## Setting up your environment
The first step is to install Nix. If you are using a Linux machine, run this
script:
```console
$ curl https://nixos.org/nix/install | sh
```
This will prompt you for more information as it goes on, so be sure to follow
the instructions carefully. Once it is done, close and re-open your shell. After
you have done this, `nix-env` should exist in your shell. Try to run it:
```console
$ nix-env
error: no operation specified
Try 'nix-env --help' for more information.
```
Let's install a few other tools to help us with development. First, let's
install [lorri][lorri] to help us manage our development shell:
[lorri]: https://github.com/target/lorri
```
$ nix-env --install --file https://github.com/target/lorri/archive/master.tar.gz
```
This will automatically download and build lorri for your system based on the
latest possible version. Once that is done, open another shell window (the lorri
docs include ways to do this more persistently, but this will work for now) and run:
```console
$ lorri daemon
```
Now go back to your main shell window and install [direnv][direnv]:
[direnv]: https://direnv.net
```console
$ nix-env --install direnv
```
Next, follow the [shell setup][direnvsetup] needed for your shell. I personally
use `fish` with [oh my fish][omf], so I would run this:
[direnvsetup]: https://direnv.net/docs/hook.html
[omf]: https://github.com/oh-my-fish/oh-my-fish
```console
$ omf install direnv
```
Finally, let's install [niv][niv] to help us handle dependencies for the
project. This will allow us to make sure that our builds pin _everything_ to a
specific set of versions, including operating system packages.
[niv]: https://github.com/nmattia/niv
```console
$ nix-env --install niv
```
Now that we have all of the tools we will need installed, let's create the
project.
# A new project
Go to your favorite place to put code and make a new folder. I personally prefer
`~/code`, so I will be using that here:
```console
$ cd ~/code
$ mkdir helloworld
$ cd helloworld
```
Let's set up the basic skeleton of the project. First, initialize niv:
```console
$ niv init
```
This will add the latest versions of `niv` itself and the packages used for the
system to `nix/sources.json`. This will allow us to pin exact versions so the
environment is as predictable as possible. Sometimes the versions of software in
the pinned nixpkgs are too old. If this happens, you can update to the
"unstable" branch of nixpkgs with this command:
```console
$ niv update nixpkgs -b nixpkgs-unstable
```
Next, set up lorri using `lorri init`:
```console
$ lorri init
```
This will create `shell.nix` and `.envrc`. `shell.nix` will be where we define
the development environment for this service. `.envrc` is used to tell direnv
what it needs to do. Let's try and activate the `.envrc`:
```console
$ cd .
direnv: error /home/cadey/code/helloworld/.envrc is blocked. Run `direnv allow`
to approve its content
```
Let's review its content:
```console
$ cat .envrc
eval "$(lorri direnv)"
```
This seems reasonable, so approve it with `direnv allow` like the error message
suggests:
```console
$ direnv allow
```
Now let's customize the `shell.nix` file to use our pinned version of nixpkgs.
Currently, it looks something like this:
```nix
# shell.nix
let
pkgs = import <nixpkgs> {};
in
pkgs.mkShell {
buildInputs = [
pkgs.hello
];
}
```
This currently imports nixpkgs from the system-level version of it. This means
that different systems could have different versions of nixpkgs on it, and that
could make the `shell.nix` file hard to reproduce between machines. Let's import
the pinned version of nixpkgs that niv created:
```nix
# shell.nix
let
sources = import ./nix/sources.nix;
pkgs = import sources.nixpkgs {};
in
pkgs.mkShell {
buildInputs = [
pkgs.hello
];
}
```
And then let's test it with `lorri shell`:
```console
$ lorri shell
lorri: building environment........ done
(lorri) $
```
And let's see if `hello` is available inside the shell:
```console
(lorri) $ hello
Hello, world!
```
You can set environment variables inside the `shell.nix` file. Do so like this:
```nix
# shell.nix
let
sources = import ./nix/sources.nix;
pkgs = import sources.nixpkgs {};
in
pkgs.mkShell {
buildInputs = [
pkgs.hello
];
# Environment variables
HELLO="world";
}
```
Wait a moment for lorri to finish rebuilding the development environment and
then let's see if the environment variable shows up:
```console
$ cd .
direnv: loading ~/code/helloworld/.envrc
<output snipped>
$ echo $HELLO
world
```
Now that we have the basics of the environment set up, lets install the Rust
compiler.
# Setting up the Rust compiler
First, add [nixpkgs-mozilla][nixpkgsmoz] to niv:
[nixpkgsmoz]: https://github.com/mozilla/nixpkgs-mozilla
```console
$ niv add mozilla/nixpkgs-mozilla
```
Then create `nix/rust.nix` in your repo:
```nix
# nix/rust.nix
{ sources ? import ./sources.nix }:
let
pkgs =
import sources.nixpkgs { overlays = [ (import sources.nixpkgs-mozilla) ]; };
channel = "nightly";
date = "2020-03-08";
targets = [ ];
chan = pkgs.rustChannelOfTargets channel date targets;
in chan
```
This creates a nix function that takes in the pre-imported list of sources,
creates a copy of nixpkgs with Rust at the nightly version `2020-03-08` overlaid
into it, and exposes the rust package out of it. Let's add this to `shell.nix`:
```nix
# shell.nix
let
sources = import ./nix/sources.nix;
rust = import ./nix/rust.nix { inherit sources; };
pkgs = import sources.nixpkgs { };
in
pkgs.mkShell {
buildInputs = [
rust
];
}
```
Then ask lorri to recreate the development environment. This may take a bit to
run because it's setting up everything the Rust compiler requires to run.
```console
$ lorri shell
(lorri) $
```
Let's see what version of Rust is installed:
```console
(lorri) $ rustc --version
rustc 1.43.0-nightly (823ff8cf1 2020-03-07)
```
This is exactly what we expect. Rust nightly versions get released with the
date of the previous day in them. To be extra sure, let's see what the shell
thinks `rustc` resolves to:
```console
(lorri) $ which rustc
/nix/store/w6zk1zijfwrnjm6xyfmrgbxb6dvvn6di-rust-1.43.0-nightly-2020-03-07-823ff8cf1/bin/rustc
```
And now exit that shell and reload direnv:
```console
(lorri) $ exit
$ cd .
direnv: loading ~/code/helloworld/.envrc
$ which rustc
/nix/store/w6zk1zijfwrnjm6xyfmrgbxb6dvvn6di-rust-1.43.0-nightly-2020-03-07-823ff8cf1/bin/rustc
```
And now we have Rust installed at an arbitrary nightly version for _that project
only_. This will work on other machines too. Now that we have our development
environment set up, let's serve HTTP.
## Serving HTTP
[Rocket][rocket] is a popular web framework for Rust programs. Let's use that to
create a small "hello, world" server. We will need to do the following:
[rocket]: https://rocket.rs
- Create the new Rust project
- Add Rocket as a dependency
- Write our "hello world" route
- Test a build of the service with `cargo build`
### Create the new Rust project
Create the new Rust project with `cargo init`:
```console
$ cargo init --vcs git .
Created binary (application) package
```
This will create the directory `src` and a file named `Cargo.toml`. Rust code
goes in `src` and the `Cargo.toml` file configures dependencies. Adding the
`--vcs git` flag also has cargo create a [gitignore][gitignore] file so that the
target folder isn't tracked by git.
[gitignore]: https://git-scm.com/docs/gitignore
### Add Rocket as a dependency
Open `Cargo.toml` and add the following to it:
```toml
[dependencies]
rocket = "0.4.3"
```
Then download/build Rocket with `cargo build`:
```console
$ cargo build
```
This will download all of the dependencies you need and precompile Rocket. This
will help speed up later builds.
### Write our "hello world" route
Now put the following in `src/main.rs`:
```rust
#![feature(proc_macro_hygiene, decl_macro)] // language features needed by Rocket
// Import the rocket macros
#[macro_use]
extern crate rocket;
// Create route / that returns "Hello, world!"
#[get("/")]
fn index() -> &'static str {
"Hello, world!"
}
fn main() {
rocket::ignite().mount("/", routes![index]).launch();
}
```
### Test a build
Rerun `cargo build`:
```console
$ cargo build
```
This will create the binary at `target/debug/helloworld`. Let's run it locally
and see if it works:
```console
$ ./target/debug/helloworld &
$ curl http://127.0.0.1:8000
Hello, world!
$ fg
<press control-c>
```
The HTTP service works. We have a binary that is created with the Rust compiler
Nix installed.
## A simple package build
Now that we have the HTTP service working, let's put it inside a nix package. We
will need to use [naersk][naersk] to do this. Add naersk to your project with
niv:
[naersk]: https://github.com/nmattia/naersk
```console
$ niv add nmattia/naersk
```
Now let's create `helloworld.nix`:
```
# import niv sources and the pinned nixpkgs
{ sources ? import ./nix/sources.nix, pkgs ? import sources.nixpkgs { }}:
let
# import rust compiler
rust = import ./nix/rust.nix { inherit sources; };
# configure naersk to use our pinned rust compiler
naersk = pkgs.callPackage sources.naersk {
rustc = rust;
cargo = rust;
};
# tell nix-build to ignore the `target` directory
src = builtins.filterSource
(path: type: type != "directory" || builtins.baseNameOf path != "target")
./.;
in naersk.buildPackage {
inherit src;
remapPathPrefix =
true; # remove nix store references for a smaller output package
}
```
And then build it with `nix-build`:
```console
$ nix-build helloworld.nix
```
This can take a bit to run, but it will do the following things:
- Download naersk
- Download every Rust crate your HTTP service depends on into the Nix store
- Run your program's tests
- Build your dependencies into a Nix package
- Build your program with those dependencies
- Place a link to the result at `./result`
Once it is done, let's take a look at the result:
```console
$ du -hs ./result/bin/helloworld
2.1M ./result/bin/helloworld
$ ldd ./result/bin/helloworld
linux-vdso.so.1 (0x00007fffae080000)
libdl.so.2 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libdl.so.2 (0x0
0007f3a01666000)
librt.so.1 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/librt.so.1 (0x0
0007f3a0165c000)
libpthread.so.0 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libpthread
.so.0 (0x00007f3a0163b000)
libgcc_s.so.1 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libgcc_s.so.
1 (0x00007f3a013f5000)
libc.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libc.so.6 (0x000
07f3a0123f000)
/nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/ld-linux-x86-64.so.2 => /lib6
4/ld-linux-x86-64.so.2 (0x00007f3a0160b000)
libm.so.6 => /nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libm.so.6 (0x000
07f3a010a9000)
```
This means that the Nix build created a 2.1 megabyte binary that only depends on
[glibc][glibc], the implementation of the C language standard library that Nix
prefers.
[glibc]: https://www.gnu.org/software/libc/
For repo cleanliness, add the `result` link to the [gitignore][gitignore]:
```console
$ echo 'result*' >> .gitignore
```
## Shipping it in a Docker image
Now that we have a package built, let's ship it in a docker image. nixpkgs
provides [dockerTools][dockertools] which helps us create docker images out of
Nix packages. Let's create `default.nix` with the following contents:
[dockertools]: https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools
```nix
{ system ? builtins.currentSystem }:
let
sources = import ./nix/sources.nix;
pkgs = import sources.nixpkgs { };
helloworld = import ./helloworld.nix { inherit sources pkgs; };
name = "xena/helloworld";
tag = "latest";
in pkgs.dockerTools.buildLayeredImage {
inherit name tag;
contents = [ helloworld ];
config = {
Cmd = [ "/bin/helloworld" ];
Env = [ "ROCKET_PORT=5000" ];
WorkingDir = "/";
};
}
```
And then build it with `nix-build`:
```console
$ nix-build default.nix
```
This will create a tarball containing the docker image information as the result
of the Nix build. Load it into docker using `docker load`:
```console
$ docker load -i result
```
And then run it using `docker run`:
```console
$ docker run --rm -itp 52340:5000 xena/helloworld
```
Now test it using curl:
```console
$ curl http://127.0.0.1:52340
Hello, world!
```
And now you have a docker image you can run wherever you want. The
`buildLayeredImage` function used in `default.nix` also makes Nix put each
dependency of the package into its own docker layer. This makes new versions of
your program very efficient to upgrade on your clusters, realistically this
reduces the amount of data needed for new versions of the program down to what
changed. If nothing but some resources in their own package were changed, only
those packages get downloaded.
This is how I start a new project with Nix. I put all of the code described in
this post in [this GitHub repo][helloworldrepo] in case it helps. Have fun and
be well.
[helloworldrepo]: https://github.com/Xe/helloworld
---
For some "extra credit" tasks, try and see if you can do the following:
- Use the version of [niv][niv] that niv pinned
- Customize the environment of the container by following the [Rocket
configuration documentation](https://rocket.rs/v0.4/guide/configuration/)
- Add some more routes to the program
- Read the [Nix
documentation](https://nixos.org/nix/manual/#chap-writing-nix-expressions) and
learn more about writing Nix expressions
- Configure your editor/IDE to use the `direnv` path