18 KiB
title | date | tags | series | author | ||
---|---|---|---|---|---|---|
Nix Flakes: an Introduction | 2022-02-21 |
|
nix-flakes | Twi |
Nix is a package manager that lets you have a more deterministic view of your software dependencies and build processes. One if its biggest weaknesses out of the box is that there are very few conventions on how projects using Nix should work together. It's like having a build system but also having to configure systems to run software yourself. This could mean copying a NixOS module out of the project's git repo, writing your own or more. In contrast to this, Nix flakes define a set of conventions for how software can be build, run, integrated and deployed without having to rely on external tools such as Niv or Lorri to help you do basic tasks in a timely manner.
This is going to be a series of posts that will build on eachother. This post will be an introduction to Nix flakes and serve as a "why should I care?" style overview of what you can do with flakes without going into too much detail. Most of these will get separate posts (some more than one post).
In my opinion, here are some of the big reasons you should care about Nix flakes:
- Flakes adds project templates to Nix
- Flakes define a standard way to say "this is a program you can run"
- Flakes consolidate development environments into project configuration
- Flakes can pull in dependencies from outside git repos trivially
- Flakes can work with people that don't use flakes too
- Flakes supports using private git repos
- Flakes let you define system configuration alongside your application code
- Flakes let you embed the git hash of your configurations repository into machines you deploy
Project Templates
One of the big annoying parts about getting into Nix is that setting up projects isn't totally a defined science. Nix configurations just tend to grow organically and can easily become weird or difficult to understand for people that didn't start the project. Nix flakes helps fix this by doing a few things:
- Defining a
flake.nix
as the central "hub" for your project's dependencies, exposed packages, NixOS configuration modules and more. - Shipping a set of templates so that you
can get projects started easily. Think something like
Yeoman but built directly into Nix. You can also define
your own templates in your
flake.nix
.
As an example that we will use for the rest of this post to help explain it,
let's make a Go project with their Go template. First you will need to enable
Nix flakes on your machine. If you are using NixOS, add this to your
configuration.nix
file:
nix = {
package = pkgs.nixFlakes;
extraOptions = ''
experimental-features = nix-command flakes
'';
};
Then rebuild your system and you can continue along with the article.
If you are not on NixOS, you will need to either edit ~/.config/nix/nix.conf
or /etc/nix/nix.conf
and add the following line to it:
experimental-features = nix-command flakes
Now go to a temporary folder and run these commands to make a folder and create a new flake from a template:
mkdir ~/tmp/go-demo
cd ~/tmp/go-demo
nix flake new -t templates#go-hello .
git init && git add .
This will create a few files in the folder:
$ ls
flake.lock flake.nix go.mod main.go
Then you can look at flake.nix
to see what's up:
{
description = "A simple Go package";
# Nixpkgs / NixOS version to use.
inputs.nixpkgs.url = "nixpkgs/nixos-21.11";
outputs = { self, nixpkgs }:
let
# Generate a user-friendly version number.
version = builtins.substring 0 8 self.lastModifiedDate;
# System types to support.
supportedSystems = [ "x86_64-linux" "x86_64-darwin" "aarch64-linux" "aarch64-darwin" ];
# Helper function to generate an attrset '{ x86_64-linux = f "x86_64-linux"; ... }'.
forAllSystems = nixpkgs.lib.genAttrs supportedSystems;
# Nixpkgs instantiated for supported system types.
nixpkgsFor = forAllSystems (system: import nixpkgs { inherit system; });
in
{
# Provide some binary packages for selected system types.
packages = forAllSystems (system:
let
pkgs = nixpkgsFor.${system};
in
{
go-hello = pkgs.buildGoModule {
pname = "go-hello";
inherit version;
# In 'nix develop', we don't need a copy of the source tree
# in the Nix store.
src = ./.;
# This hash locks the dependencies of this package. It is
# necessary because of how Go requires network access to resolve
# VCS. See https://www.tweag.io/blog/2021-03-04-gomod2nix/ for
# details. Normally one can build with a fake sha256 and rely on native Go
# mechanisms to tell you what the hash should be or determine what
# it should be "out-of-band" with other tooling (eg. gomod2nix).
# To begin with it is recommended to set this, but one must
# remeber to bump this hash when your dependencies change.
#vendorSha256 = pkgs.lib.fakeSha256;
vendorSha256 = "sha256-pQpattmS9VmO3ZIQUFn66az8GSmB4IvYhTTCFn6SUmo=";
};
});
# The default package for 'nix build'. This makes sense if the
# flake provides only one package or there is a clear "main"
# package.
defaultPackage = forAllSystems (system: self.packages.${system}.go-hello);
};
}
This defines a single Go package that is supported on macOS and Linux for 64 bit x86 processors and 64 bit ARM processors.
You can then build the flake with nix build
:
$ nix build
And then run it:
$ ./result/bin/go-hello
Hello Nix!
Standard Default Package
Let's take a closer look at the higher level things in the flake:
{
description = "A simple Go package";
inputs.nixpkgs.url = "nixpkgs/nixos-21.11";
outputs = { self, nixpkgs }: {
packages = { ... };
defaultPackage = { ... };
};
}
When you ran nix build
earlier, it defaulted to building the package in
defaultPackage
. You can also build the go-hello
package by running this
command:
$ nix build .#go-hello
And if you want to build the copy I made for this post:
$ nix build github:Xe/gohello
$ ./result/bin/go-hello
Hello reader!
A standard default package means that you can more easily build software without
having to read documentation on what file to build. nix build
will Just Work™️.
Exposing Packages as Applications
Additionally, you can expose a package as an application. This allows you to
simplify that above nix build
and ./result/bin/go-hello
cycle into a single
nix run
command. Open flake.nix
in your favorite editor and let's configure
go-hello
to be the default app:
# below defaultPackage
defaultApp = forAllSystems (system: {
type = "app";
program = "${self.packages.${system}.go-hello}/bin/go-hello";
});
Then you can run it with nix run
:
$ nix run
Hello Nix!
Or you can run my copy:
$ nix run github:Xe/gohello/main
Hello reader!
What is that extra part of the URL path for? Is that a git branch?
Development Environment Configuration
One of Nix's superpowers is the ability to declaratively manage the development environment for a project so that you can be sure that everyone working on the project is using the same tools.
Flakes has the ability to specify this using the devShell
flake output. You
can add it to your flake.nix
using this:
# after defaultApp
devShell = forAllSystems (system:
let pkgs = nixpkgsFor.${system};
in pkgs.mkShell {
buildInputs = with pkgs; [ go gopls goimports go-tools ];
});
I consider this to be a basic Go development environment. It includes standard
tools such as the language server, goimports
for better formatting and tools
like staticcheck. If you use staticcheck
regularly at work, please consider throwing Dominik a couple bucks
a month if you find it useful. It helps the project be more
self-sustaining.
Then you can enter the development shell with nix develop
:
$ nix develop
[cadey@pneuma:~/tmp/gohello]$ go version
go version go1.16.9 linux/amd64
And then hack at your project all you want. You can send this git repo to a friend and they will have the same setup.
External Dependencies
Now let's talk about inputs. Flake inputs let you add external dependencies to a
project. As an example, let's look at the nixpkgs
input:
# Nixpkgs / NixOS version to use.
inputs.nixpkgs.url = "nixpkgs/nixos-21.11";
This defines the release of nixpkgs that should be used for the project. This template defaults to NixOS 21.11's version of nixpkgs, however we can upgrade it to nixos-unstable by changing it to this:
# Nixpkgs / NixOS version to use.
inputs.nixpkgs.url = "nixpkgs/nixos-unstable";
Then we can run nix flake update
and then nix develop
and see that we are
running a newer version of Go:
$ nix flake update
warning: updating lock file '/home/cadey/tmp/gohello/flake.lock':
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/77aa71f66fd05d9e7b7d1c084865d703a8008ab7' (2022-01-19)
→ 'github:NixOS/nixpkgs/2128d0aa28edef51fd8fef38b132ffc0155595df' (2022-02-16)
$ nix develop
[cadey@pneuma:~/tmp/gohello]$ go version
go version go1.17.7 linux/amd64
This also lets you pull in other Nix flakes projects, such as my CSS framework Xess:
inputs.xess.url = "github:Xe/Xess";
inputs.xess.inputs.nixpkgs.follows = "nixpkgs";
Why is that second line needed?
Or anything you want! A useful library to pull in is
flake-utils, that can help you
simplify your flake.nix
and get rid of those ugly forAllSystems
and
nixpkgsFor
functions in the flake.nix
that this post used by default. For an
example of a flake that uses this library, see this
flake.nix
from the IRC
bot that lives in #xeserv
.
Backwards Compatibility
Normally you need to enable Nix flakes in your Nix daemon to take advantage of them. This is great for when you can do that, but sometimes you'll need to make things work for people without flakes enabled. This could happen when needing to graft in a Nix flakes project to one without flakes enabled. There is a library called flake-compat that makes this easy.
Create default.nix
with the following contents:
(import (
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/99f1c2157fba4bfe6211a321fd0ee43199025dbf.tar.gz";
sha256 = "0x2jn3vrawwv9xp15674wjz9pixwjyj3j771izayl962zziivbx2"; }
) {
src = ./.;
}).defaultNix
And shell.nix
with the following contents:
(import (
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/99f1c2157fba4bfe6211a321fd0ee43199025dbf.tar.gz";
sha256 = "0x2jn3vrawwv9xp15674wjz9pixwjyj3j771izayl962zziivbx2"; }
) {
src = ./.;
}).shellNix
Then you can use nix-build
and nix-shell
like you have in other Nix
projects.
Private Git Repos
Nix flakes has native support for private git repositories as inputs. This can be useful when trying to build software you don't want to release as open to the world. To use a private repo, your flake input URL should look something like this:
ssh+git://git@github.com:user/repo
Embed NixOS Modules in Flakes
The biggest ticket item for me is that it lets you embed NixOS modules in flakes themselves. This lets you define the system configuration for software right next to where the software is defined, thus shipping it as a unit. Using this you can make installing software a matter of adding it to your system's flake, adding the module and then enabling the settings you want to enable.
As an example, here is the NixOS module for that IRC bot I mentioned:
nixosModules.bot = { config, lib, ... }: {
options.within.services.mara-bot.enable =
lib.mkEnableOption "enable Mara bot";
config = lib.mkIf config.within.services.mara-bot.enable {
users.groups.mara-bot = { };
users.users.mara-bot = {
createHome = true;
isSystemUser = true;
home = "/var/lib/mara-bot";
group = "mara-bot";
};
systemd.services.mara-bot = {
wantedBy = [ "multi-user.target" ];
environment.RUST_LOG = "tower_http=debug,info";
unitConfig.ConditionPathExists = "/var/lib/mara-bot/config.yaml";
serviceConfig = {
User = "mara-bot";
Group = "mara-bot";
Restart = "always";
WorkingDirectory = "/var/lib/mara-bot";
ExecStart = "${self.defaultPackage."${system}"}/bin/mara";
};
};
};
};
The key important part here is the ExecStart
line. It points back to the
flake's default package (which is hopefully where the bot's code is defined),
and then has systemd manage that.
I plan to use this to radically simplify my nixos-configs repo. Right now it has a lot of code that is very project-specific and if I can move that into the projects in question, I can eliminate a lot of code out of the core of my configs repo.
Embedding Configuration Git Hash into Systems
Finally, Nix flakes lets you see the configuration version of a system by
embedding it at the build step. Normally NixOS lets you see the following
information with nixos-version --json
:
{
"nixosVersion": "22.05pre348581.c07b471b52b",
"nixpkgsRevision": "c07b471b52be8fbc49a7dc194e9b37a6e19ee04d"
}
You have the NixOS version and the nixpkgs hash. That doesn't tell you what configuration you are running or anything about it though. However with flakes you can embed the git hash of your configuration into the system config:
{
"configurationRevision": "f53891121ce4204f57409cbe9e6fcee3b030a350",
"nixosVersion": "22.05.20220210.48d63e9",
"nixpkgsRevision": "48d63e924a2666baf37f4f14a18f19347fbd54a2"
}
This can let you make a URL pointing to the commit in that output:
$ echo "https://tulpa.dev/cadey/nixos-configs/src/commit/$(ssh logos nixos-version --json | jq -r .configurationRevision)"
Which will spit out a link to cadey/nixos-configs@f53891121.
I'll cover more on how to do this in the NixOS deployment post.
There is a lot more to get into with each of these topics. I'm only really giving a very high level overview on them while I learn more and migrate over my NixOS configurations to flakes piecemeal. This has also given me the opportunity to clean things up and chew out a lot of the fat from my NixOS configurations. More to come when it is ready.