forked from cadey/xesite
blog: add nix flakes post
Signed-off-by: Xe Iaso <me@christine.website>
This commit is contained in:
parent
249676f0cc
commit
b8e4717a5b
|
|
@ -0,0 +1,527 @@
|
|||
---
|
||||
title: "Nix Flakes: an Introduction"
|
||||
date: 2022-02-21
|
||||
tags:
|
||||
- nix
|
||||
- nixos
|
||||
series: nix-flakes
|
||||
author: Twi
|
||||
---
|
||||
|
||||
Nix is a package manager that lets you have a more deterministic view of your
|
||||
software dependencies and build processes. One if its biggest weaknesses out of
|
||||
the box is that there are very few conventions on how projects using Nix should
|
||||
work together. It's like having a build system but also having to configure
|
||||
systems to run software yourself. This could mean copying a NixOS module out of
|
||||
the project's git repo, writing your own or more. In contrast to this, [Nix
|
||||
flakes](https://nixos.wiki/wiki/Flakes) define a set of conventions for how
|
||||
software can be build, run, integrated and deployed without having to rely on
|
||||
external tools such as [Niv](https://github.com/nmattia/niv) or
|
||||
[Lorri](https://github.com/nix-community/lorri) to help you do basic tasks in a
|
||||
timely manner.
|
||||
|
||||
This is going to be a series of posts that will build on eachother. This post
|
||||
will be an introduction to Nix flakes and serve as a "why should I care?" style
|
||||
overview of what you can do with flakes without going into too much detail. Most
|
||||
of these will get separate posts (some more than one post).
|
||||
|
||||
In my opinion, here are some of the big reasons you should care about Nix
|
||||
flakes:
|
||||
|
||||
- Flakes adds project templates to Nix
|
||||
- Flakes define a standard way to say "this is a program you can run"
|
||||
- Flakes consolidate development environments into project configuration
|
||||
- Flakes can pull in dependencies from outside git repos trivially
|
||||
- Flakes can work with people that don't use flakes too
|
||||
- Flakes supports using private git repos
|
||||
- Flakes let you define system configuration alongside your application code
|
||||
- Flakes let you embed the git hash of your configurations repository into
|
||||
machines you deploy
|
||||
|
||||
## Project Templates
|
||||
|
||||
One of the big annoying parts about getting into Nix is that setting up projects
|
||||
isn't totally a defined science. Nix configurations just tend to grow
|
||||
organically and can easily become weird or difficult to understand for people
|
||||
that didn't start the project. Nix flakes helps fix this by doing a few things:
|
||||
|
||||
1. Defining a `flake.nix` as the central "hub" for your project's dependencies,
|
||||
exposed packages, NixOS configuration modules [and
|
||||
more](https://nixos.wiki/wiki/Flakes#Output_schema).
|
||||
2. Shipping a [set of templates](https://github.com/NixOS/templates) so that you
|
||||
can get projects started easily. Think something like
|
||||
[Yeoman](https://yeoman.io) but built directly into Nix. You can also define
|
||||
your own templates in your `flake.nix`.
|
||||
|
||||
As an example that we will use for the rest of this post to help explain it,
|
||||
let's make a Go project with their Go template. First you will need to enable
|
||||
Nix flakes on your machine. If you are using NixOS, add this to your
|
||||
`configuration.nix` file:
|
||||
|
||||
```nix
|
||||
nix = {
|
||||
package = pkgs.nixFlakes;
|
||||
extraOptions = ''
|
||||
experimental-features = nix-command flakes
|
||||
'';
|
||||
};
|
||||
```
|
||||
|
||||
Then rebuild your system and you can continue along with the article.
|
||||
|
||||
If you are not on NixOS, you will need to either edit `~/.config/nix/nix.conf`
|
||||
or `/etc/nix/nix.conf` and add the following line to it:
|
||||
|
||||
```
|
||||
experimental-features = nix-command flakes
|
||||
```
|
||||
|
||||
[You may need to restart the Nix daemon here, but if you are unsure how Nix was
|
||||
set up on that non-NixOS machine feel free to totally restart your
|
||||
computer.](conversation://Mara/hacker)
|
||||
|
||||
Now go to a temporary folder and run these commands to make a folder and create
|
||||
a new flake from a template:
|
||||
|
||||
```console
|
||||
mkdir ~/tmp/go-demo
|
||||
cd ~/tmp/go-demo
|
||||
nix flake new -t templates#go-hello .
|
||||
git init && git add .
|
||||
```
|
||||
|
||||
This will create a few files in the folder:
|
||||
|
||||
```console
|
||||
$ ls
|
||||
flake.lock flake.nix go.mod main.go
|
||||
```
|
||||
|
||||
Then you can look at `flake.nix` to see what's up:
|
||||
|
||||
```nix
|
||||
{
|
||||
description = "A simple Go package";
|
||||
|
||||
# Nixpkgs / NixOS version to use.
|
||||
inputs.nixpkgs.url = "nixpkgs/nixos-21.11";
|
||||
|
||||
outputs = { self, nixpkgs }:
|
||||
let
|
||||
|
||||
# Generate a user-friendly version number.
|
||||
version = builtins.substring 0 8 self.lastModifiedDate;
|
||||
|
||||
# System types to support.
|
||||
supportedSystems = [ "x86_64-linux" "x86_64-darwin" "aarch64-linux" "aarch64-darwin" ];
|
||||
|
||||
# Helper function to generate an attrset '{ x86_64-linux = f "x86_64-linux"; ... }'.
|
||||
forAllSystems = nixpkgs.lib.genAttrs supportedSystems;
|
||||
|
||||
# Nixpkgs instantiated for supported system types.
|
||||
nixpkgsFor = forAllSystems (system: import nixpkgs { inherit system; });
|
||||
|
||||
in
|
||||
{
|
||||
|
||||
# Provide some binary packages for selected system types.
|
||||
packages = forAllSystems (system:
|
||||
let
|
||||
pkgs = nixpkgsFor.${system};
|
||||
in
|
||||
{
|
||||
go-hello = pkgs.buildGoModule {
|
||||
pname = "go-hello";
|
||||
inherit version;
|
||||
# In 'nix develop', we don't need a copy of the source tree
|
||||
# in the Nix store.
|
||||
src = ./.;
|
||||
|
||||
# This hash locks the dependencies of this package. It is
|
||||
# necessary because of how Go requires network access to resolve
|
||||
# VCS. See https://www.tweag.io/blog/2021-03-04-gomod2nix/ for
|
||||
# details. Normally one can build with a fake sha256 and rely on native Go
|
||||
# mechanisms to tell you what the hash should be or determine what
|
||||
# it should be "out-of-band" with other tooling (eg. gomod2nix).
|
||||
# To begin with it is recommended to set this, but one must
|
||||
# remeber to bump this hash when your dependencies change.
|
||||
#vendorSha256 = pkgs.lib.fakeSha256;
|
||||
|
||||
vendorSha256 = "sha256-pQpattmS9VmO3ZIQUFn66az8GSmB4IvYhTTCFn6SUmo=";
|
||||
};
|
||||
});
|
||||
|
||||
# The default package for 'nix build'. This makes sense if the
|
||||
# flake provides only one package or there is a clear "main"
|
||||
# package.
|
||||
defaultPackage = forAllSystems (system: self.packages.${system}.go-hello);
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
This defines a single Go package that is supported on macOS and Linux for 64 bit
|
||||
x86 processors and 64 bit ARM processors.
|
||||
|
||||
[In practice this spread should cover all of the main targets you'll need to
|
||||
care about for local development and cloud
|
||||
deployment.](conversation://Mara/hacker)
|
||||
|
||||
You can then build the flake with `nix build`:
|
||||
|
||||
```console
|
||||
$ nix build
|
||||
```
|
||||
|
||||
And then run it:
|
||||
|
||||
```console
|
||||
$ ./result/bin/go-hello
|
||||
Hello Nix!
|
||||
```
|
||||
|
||||
## Standard Default Package
|
||||
|
||||
Let's take a closer look at the higher level things in the flake:
|
||||
|
||||
```nix
|
||||
{
|
||||
description = "A simple Go package";
|
||||
|
||||
inputs.nixpkgs.url = "nixpkgs/nixos-21.11";
|
||||
|
||||
outputs = { self, nixpkgs }: {
|
||||
packages = { ... };
|
||||
defaultPackage = { ... };
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
[A note: in the rest of this article (and series of articles), when I refer to a
|
||||
"flake output", I am referring to an attribute in the `outputs` attribute of
|
||||
your `flake.nix`. Ditto with "flake input" referring to the `inputs` attribute
|
||||
of your `flake.nix`.](conversation://Cadey/enby)
|
||||
|
||||
When you ran `nix build` earlier, it defaulted to building the package in
|
||||
`defaultPackage`. You can also build the `go-hello` package by running this
|
||||
command:
|
||||
|
||||
```console
|
||||
$ nix build .#go-hello
|
||||
```
|
||||
|
||||
And if you want to build the copy I made for this post:
|
||||
|
||||
```console
|
||||
$ nix build github:Xe/gohello
|
||||
$ ./result/bin/go-hello
|
||||
Hello reader!
|
||||
```
|
||||
|
||||
A standard default package means that you can more easily build software without
|
||||
having to read documentation on what file to build. `nix build` will Just Work™️.
|
||||
|
||||
## Exposing Packages as Applications
|
||||
|
||||
Additionally, you can expose a package as an application. This allows you to
|
||||
simplify that above `nix build` and `./result/bin/go-hello` cycle into a single
|
||||
`nix run` command. Open `flake.nix` in your favorite editor and let's configure
|
||||
`go-hello` to be the default app:
|
||||
|
||||
```nix
|
||||
# below defaultPackage
|
||||
|
||||
defaultApp = forAllSystems (system: {
|
||||
type = "app";
|
||||
program = "${self.packages.${system}.go-hello}/bin/go-hello";
|
||||
});
|
||||
```
|
||||
|
||||
Then you can run it with `nix run`:
|
||||
|
||||
```console
|
||||
$ nix run
|
||||
Hello Nix!
|
||||
```
|
||||
|
||||
Or you can run my copy:
|
||||
|
||||
```console
|
||||
$ nix run github:Xe/gohello/main
|
||||
Hello reader!
|
||||
```
|
||||
|
||||
[What is that extra part of the URL path for? Is that a git
|
||||
branch?](conversation://Mara/hmm)
|
||||
|
||||
[Yes, you can use that syntax to set the git branch that Nix should build from.
|
||||
By default it will use the default branch (typically `main`), but sometimes you
|
||||
need to specify a branch or commit directly.](conversation://Cadey/enby)
|
||||
|
||||
## Development Environment Configuration
|
||||
|
||||
One of Nix's superpowers is the ability to declaratively manage the development
|
||||
environment for a project so that you can be sure that everyone working on the
|
||||
project is using the same tools.
|
||||
|
||||
[I use this with all of my projects to the point that when I am outside of a
|
||||
project folder I do not have any development tools
|
||||
available.](conversation://Cadey/enby)
|
||||
|
||||
Flakes has the ability to specify this using the `devShell` flake output. You
|
||||
can add it to your `flake.nix` using this:
|
||||
|
||||
```nix
|
||||
# after defaultApp
|
||||
|
||||
devShell = forAllSystems (system:
|
||||
let pkgs = nixpkgsFor.${system};
|
||||
in pkgs.mkShell {
|
||||
buildInputs = with pkgs; [ go gopls goimports go-tools ];
|
||||
});
|
||||
```
|
||||
|
||||
[I consider this to be a basic Go development environment. It includes standard
|
||||
tools such as the language server, `goimports` for better formatting and tools
|
||||
like <a href="https://staticcheck.io">staticcheck</a>. If you use staticcheck
|
||||
regularly at work, please consider throwing <a
|
||||
href="https://github.com/users/dominikh/sponsorship">Dominik</a> a couple bucks
|
||||
a month if you find it useful. It helps the project be more
|
||||
self-sustaining.](conversation://Mara/happy)
|
||||
|
||||
Then you can enter the development shell with `nix develop`:
|
||||
|
||||
```
|
||||
$ nix develop
|
||||
|
||||
[cadey@pneuma:~/tmp/gohello]$ go version
|
||||
go version go1.16.9 linux/amd64
|
||||
```
|
||||
|
||||
And then hack at your project all you want. You can send this git repo to a
|
||||
friend and they will have the same setup.
|
||||
|
||||
## External Dependencies
|
||||
|
||||
Now let's talk about inputs. Flake inputs let you add external dependencies to a
|
||||
project. As an example, let's look at the `nixpkgs` input:
|
||||
|
||||
```nix
|
||||
# Nixpkgs / NixOS version to use.
|
||||
inputs.nixpkgs.url = "nixpkgs/nixos-21.11";
|
||||
```
|
||||
|
||||
This defines the release of nixpkgs that should be used for the project. This
|
||||
template defaults to NixOS 21.11's version of nixpkgs, however we can upgrade it
|
||||
to nixos-unstable by changing it to this:
|
||||
|
||||
```nix
|
||||
# Nixpkgs / NixOS version to use.
|
||||
inputs.nixpkgs.url = "nixpkgs/nixos-unstable";
|
||||
```
|
||||
|
||||
Then we can run `nix flake update` and then `nix develop` and see that we are
|
||||
running a newer version of Go:
|
||||
|
||||
```console
|
||||
$ nix flake update
|
||||
warning: updating lock file '/home/cadey/tmp/gohello/flake.lock':
|
||||
• Updated input 'nixpkgs':
|
||||
'github:NixOS/nixpkgs/77aa71f66fd05d9e7b7d1c084865d703a8008ab7' (2022-01-19)
|
||||
→ 'github:NixOS/nixpkgs/2128d0aa28edef51fd8fef38b132ffc0155595df' (2022-02-16)
|
||||
|
||||
$ nix develop
|
||||
|
||||
[cadey@pneuma:~/tmp/gohello]$ go version
|
||||
go version go1.17.7 linux/amd64
|
||||
```
|
||||
|
||||
This also lets you pull in other Nix flakes projects, such as my CSS framework
|
||||
[Xess](https://github.com/Xe/Xess):
|
||||
|
||||
```nix
|
||||
inputs.xess.url = "github:Xe/Xess";
|
||||
inputs.xess.inputs.nixpkgs.follows = "nixpkgs";
|
||||
```
|
||||
|
||||
[Why is that second line needed?](conversation://Mara/hmm)
|
||||
|
||||
[By default when you pull in another project with Nix flakes, it treats that
|
||||
project as an entirely separate universe and only interacts with the outputs of
|
||||
that flake. This means it pulls in its own version of nixpkgs, each dependency
|
||||
it has can pull in that own version of nixpkgs and vice versa ad infinitum. By
|
||||
making Xess' nixpkgs input follows our own one, we are saying "I understand this
|
||||
may be incompatible, but please use this version of nixpkgs instead". This can
|
||||
help larger projects with many inputs (such as a nixos configs repo made by
|
||||
someone with too many throwaway side projects) evaluate and build faster. Nix
|
||||
flakes does have a cached evaluator, but still it helps to avoid the problem in
|
||||
the first place.](conversation://Cadey/enby)
|
||||
|
||||
Or anything you want! A useful library to pull in is
|
||||
[flake-utils](https://github.com/numtide/flake-utils), that can help you
|
||||
simplify your `flake.nix` and get rid of those ugly `forAllSystems` and
|
||||
`nixpkgsFor` functions in the `flake.nix` that this post used by default. For an
|
||||
example of a flake that uses this library, see [this
|
||||
`flake.nix`](https://tulpa.dev/Xe/mara/src/branch/main/flake.nix) from the IRC
|
||||
bot that lives in [`#xeserv`](https://web.libera.chat/#xeserv).
|
||||
|
||||
[Adapting this trivial example to use `flake-utils` is an excellent exercise for
|
||||
the reader!](conversation://Mara/happy)
|
||||
|
||||
## Backwards Compatibility
|
||||
|
||||
Normally you need to enable Nix flakes in your Nix daemon to take advantage of
|
||||
them. This is great for when you can do that, but sometimes you'll need to make
|
||||
things work for people without flakes enabled. This could happen when needing to
|
||||
graft in a Nix flakes project to one without flakes enabled. There is a library
|
||||
called [flake-compat](https://github.com/edolstra/flake-compat) that makes this
|
||||
easy.
|
||||
|
||||
Add the following to your flake inputs:
|
||||
|
||||
```nix
|
||||
inputs.flake-compat = {
|
||||
url = "github:edolstra/flake-compat";
|
||||
flake = false;
|
||||
};
|
||||
```
|
||||
|
||||
And then create `default.nix` with the following contents:
|
||||
|
||||
```nix
|
||||
(import (
|
||||
let
|
||||
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
|
||||
in fetchTarball {
|
||||
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
|
||||
sha256 = lock.nodes.flake-compat.locked.narHash; }
|
||||
) {
|
||||
src = ./.;
|
||||
}).defaultNix
|
||||
```
|
||||
|
||||
And `shell.nix` with the following contents:
|
||||
|
||||
```nix
|
||||
(import (
|
||||
let
|
||||
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
|
||||
in fetchTarball {
|
||||
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
|
||||
sha256 = lock.nodes.flake-compat.locked.narHash; }
|
||||
) {
|
||||
src = ./.;
|
||||
}).shellNix
|
||||
```
|
||||
|
||||
Then you can use `nix-build` and `nix-shell` like you have in other Nix
|
||||
projects.
|
||||
|
||||
## Private Git Repos
|
||||
|
||||
Nix flakes has native support for private git repositories as inputs. This can
|
||||
be useful when trying to build software you don't want to release as open to the
|
||||
world. To use a private repo, your flake input URL should look something like
|
||||
this:
|
||||
|
||||
```
|
||||
ssh+git://git@github.com:user/repo
|
||||
```
|
||||
|
||||
[I'm pretty sure you could use private git repos outside of flakes, however it
|
||||
was never really clear to me _how_ you end up doing
|
||||
it.](conversation://Cadey/coffee)
|
||||
|
||||
## Embed NixOS Modules in Flakes
|
||||
|
||||
The biggest ticket item for me is that it lets you embed NixOS modules in flakes
|
||||
themselves. This lets you define the system configuration for software right
|
||||
next to where the software is defined, thus shipping it as a unit. Using this
|
||||
you can make installing software a matter of adding it to your system's flake,
|
||||
adding the module and then enabling the settings you want to enable.
|
||||
|
||||
As an example, here is the NixOS module for that IRC bot I mentioned:
|
||||
|
||||
```nix
|
||||
nixosModules.bot = { config, lib, ... }: {
|
||||
options.within.services.mara-bot.enable =
|
||||
lib.mkEnableOption "enable Mara bot";
|
||||
|
||||
config = lib.mkIf config.within.services.mara-bot.enable {
|
||||
users.groups.mara-bot = { };
|
||||
|
||||
users.users.mara-bot = {
|
||||
createHome = true;
|
||||
isSystemUser = true;
|
||||
home = "/var/lib/mara-bot";
|
||||
group = "mara-bot";
|
||||
};
|
||||
|
||||
systemd.services.mara-bot = {
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
environment.RUST_LOG = "tower_http=debug,info";
|
||||
unitConfig.ConditionPathExists = "/var/lib/mara-bot/config.yaml";
|
||||
serviceConfig = {
|
||||
User = "mara-bot";
|
||||
Group = "mara-bot";
|
||||
Restart = "always";
|
||||
WorkingDirectory = "/var/lib/mara-bot";
|
||||
ExecStart = "${self.defaultPackage."${system}"}/bin/mara";
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
The key important part here is the `ExecStart` line. It points back to the
|
||||
flake's default package (which is hopefully where the bot's code is defined),
|
||||
and then has systemd manage that.
|
||||
|
||||
I plan to use this to radically simplify my nixos-configs repo. Right now it has
|
||||
a lot of code that is very project-specific and if I can move that into the
|
||||
projects in question, I can eliminate a lot of code out of the core of my
|
||||
configs repo.
|
||||
|
||||
## Embedding Configuration Git Hash into Systems
|
||||
|
||||
Finally, Nix flakes lets you see the configuration version of a system by
|
||||
embedding it at the build step. Normally NixOS lets you see the following
|
||||
information with `nixos-version --json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"nixosVersion": "22.05pre348581.c07b471b52b",
|
||||
"nixpkgsRevision": "c07b471b52be8fbc49a7dc194e9b37a6e19ee04d"
|
||||
}
|
||||
```
|
||||
|
||||
You have the NixOS version and the nixpkgs hash. That doesn't tell you what
|
||||
configuration you are running or anything about it though. However with flakes
|
||||
you can embed the git hash of your configuration into the system config:
|
||||
|
||||
```json
|
||||
{
|
||||
"configurationRevision": "f53891121ce4204f57409cbe9e6fcee3b030a350",
|
||||
"nixosVersion": "22.05.20220210.48d63e9",
|
||||
"nixpkgsRevision": "48d63e924a2666baf37f4f14a18f19347fbd54a2"
|
||||
}
|
||||
```
|
||||
|
||||
This can let you make a URL pointing to the commit in that output:
|
||||
|
||||
```console
|
||||
$ echo "https://tulpa.dev/cadey/nixos-configs/src/commit/$(ssh logos nixos-version --json | jq -r .configurationRevision)"
|
||||
```
|
||||
|
||||
Which will spit out a link to
|
||||
[cadey/nixos-configs@f53891121](https://tulpa.dev/cadey/nixos-configs/src/commit/f53891121ce4204f57409cbe9e6fcee3b030a350).
|
||||
|
||||
I'll cover more on how to do this in the NixOS deployment post.
|
||||
|
||||
---
|
||||
|
||||
There is a lot more to get into with each of these topics. I'm only really
|
||||
giving a very high level overview on them while I learn more and migrate over my
|
||||
NixOS configurations to flakes
|
||||
[piecemeal](https://tulpa.dev/cadey/nixos-configs). This has also given me the
|
||||
opportunity to clean things up and chew out a lot of the fat from my NixOS
|
||||
configurations. More to come when it is ready.
|
||||
Loading…
Reference in New Issue