10 KiB
title | date | series | tags | ||
---|---|---|---|---|---|
Nixops Services on Your Home Network | 2020-11-09 | howto |
|
Nixops Services on Your Home Network
My homelab has a few NixOS machines. Right now they mostly run services inside Docker, because that has been what I have done for years. This works fine, but persistent state gets annoying*. NixOS has a tool called Nixops that allows you to push configurations to remote machines. I use this for managing my fleet of machines, and today I'm going to show you how to create service deployments with Nixops and push them to your servers.
Pedantically, Docker offers volumes to simplify this, but it is very easy to accidentally delete Docker volumes. Plain disk files like we are going to use today are a bit simpler than docker volumes, and thusly a bit harder to mess up.
Parts of a Service
For this example, let's deploy a chatbot. To make things easier, let's assume the following about this chatbot:
- The chatbot has a git repo somewhere
- The chatbot's git repo has a
default.nix
that builds the service and includes any supporting files it might need - The chatbot reads its configuration from environment variables which may contain secret values (API keys, etc.)
- The chatbot stores any temporary files in its current working directory
- The chatbot is "well-behaved" (for some definition of "well-behaved")
I will also need to assume that you have a git repo (or at least a folder) with all of your configuration similar to mine.
For this example I'm going to use withinbot as the service we will deploy via Nixops. withinbot is a chatbot that I use on my own Discord guild that does a number of vital functions including supplying amusing facts about printers:
<Cadey~> ~printerfact
<Within[BOT]> @Cadey~ Printers, especially older printers, do get cancer. Many
times this disease can be treated successfully
To get your own amusing facts about printers, see here or for using its API, call /fact
. This API has no
practical rate limits, but please don't test that.
Service Definition
We will need to do a few major things for defining this service:
- Add the bot code as a package
- Create a "services" folder for the service modules
- Create a user account for the service
- Set up a systemd unit for the service
- Configure the secrets using Nixops keys
Add the Code as a Package
In order for the program to be installed to the remote system, you need to tell
the system how to import it. There's many ways to do this, but the cheezy way is
to add the packages to
nixpkgs.config.packageOverrides
like this:
nixpkgs.config = {
packageOverrides = pkgs: {
within = {
withinbot = import (builtins.fetchTarball
"https://github.com/Xe/withinbot/archive/main.tar.gz") { };
};
};
};
And now we can access it as pkgs.within.withinbot
in the rest of our config.
In production circumstances you should probably use a fetcher that locks to a specific version using unique URLs and hashing, but this will work enough to get us off the ground in this example.
Create a "services" Folder
In your configuration folder, create a folder that you will use for these
service definitions. I made mine in common/services
. In that folder, create a
default.nix
with the following contents:
{ config, lib, ... }:
{
imports = [ ./withinbot.nix ];
users.groups.within = {};
}
The group listed here is optional, but I find that having a group like that can help you better share resources and files between services.
Now we need a folder for storing secrets. Let's create that under the services folder:
$ mkdir secrets
And let's also add a gitignore file so that we don't accidentally commit these secrets to the repo:
# common/services/secrets/.gitignore
*
Now we can put any secrets we want in the secrets folder without the risk of committing them to the git repo.
Service Manifest
Let's create withinbot.nix
and set it up:
{ config, lib, pkgs, ... }:
with lib; {
options.within.services.withinbot.enable =
mkEnableOption "Activates Withinbot (the furryhole chatbot)";
config = mkIf config.within.services.withinbot.enable {
};
}
This sets up an option called within.services.withinbot.enable
which will only
add the service configuration if that option is set to true
. This will allow
us to define a lot of services that are available, but none of their config will
be active unless they are explicitly enabled.
Now, let's create a user account for the service:
# ...
config = ... {
users.users.withinbot = {
createHome = true;
description = "github.com/Xe/withinbot";
isSystemUser = true;
group = "within";
home = "/srv/within/withinbot";
extraGroups = [ "keys" ];
};
};
# ...
This will create a user named withinbot
with the home directory
/srv/within/withinbot
, the group within
and also in the group keys
so the
withinbot user can read deployment secrets.
Now let's add the deployment secrets to the configuration:
# ...
config = ... {
users.users.withinbot = { ... };
deployment.keys.withinbot = {
text = builtins.readFile ./secrets/withinbot.env;
user = "withinbot";
group = "within";
permissions = "0640";
};
};
# ...
Assuming you have the configuration at ./secrets/withinbot.env
, this will
register the secrets into /run/keys/withinbot
and also create a systemd
oneshot service named withinbot-key
. This allows you to add the secret's
existence as a condition for withinbot to run. However, Nixops puts these keys
in /run
, which by default is mounted using a temporary memory-only filesystem,
meaning these keys will need to be re-added to machines when they are rebooted.
Fortunately, nixops reboot
will automatically add the keys back after the
reboot succeeds.
Now that we have everything else we need, let's add the service configuration:
# ...
config = ... {
users.users.withinbot = { ... };
deployment.keys.withinbot = { ... };
systemd.services.withinbot = {
wantedBy = [ "multi-user.target" ];
after = [ "withinbot-key.service" ];
wants = [ "withinbot-key.service" ];
serviceConfig = {
User = "withinbot";
Group = "within";
Restart = "on-failure"; # automatically restart the bot when it dies
WorkingDirectory = "/srv/within/withinbot";
RestartSec = "30s";
};
script = let withinbot = pkgs.within.withinbot;
in ''
# load the environment variables from /run/keys/withinbot
export $(grep -v '^#' /run/keys/withinbot | xargs)
# service-specific configuration
export CAMPAIGN_FOLDER=${withinbot}/campaigns
# kick off the chatbot
exec ${withinbot}/bin/withinbot
'';
};
};
# ...
This will create the systemd configuration for the service so that it starts on
boot, waits to start until the secrets have been loaded into it, runs withinbot
as its own user and in the within
group, and throttles the service restart so
that it doesn't incur Discord rate limits as easily. This will also put all
withinbot logs in journald, meaning that you can manage and monitor this service
like you would any other systemd service.
Deploying the Service
In your target server's configuration.nix
file, add an import of your services
directory:
{
# ...
imports = [
# ...
/home/cadey/code/nixos-configs/common/services
];
# ...
}
And then enable the withinbot service:
{
# ...
within.services = {
withinbot.enable = true;
};
# ...
}
Make that a block so you can enable multiple services at once like this!
Now you are free to deploy it to your network with nixops deploy
:
$ nixops deploy -d hexagone
And then you can verify the service is up with systemctl status
:
$ nixops ssh -d hexagone chrysalis -- systemctl status withinbot
● withinbot.service
Loaded: loaded (/nix/store/7ab7jzycpcci4f5wjwhjx3al7xy85ka7-unit-withinbot.service/withinbot.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-11-09 09:51:51 EST; 2h 29min ago
Main PID: 12295 (withinbot)
IP: 0B in, 0B out
Tasks: 13 (limit: 4915)
Memory: 7.9M
CPU: 4.456s
CGroup: /system.slice/withinbot.service
└─12295 /nix/store/qpq281hcb1grh4k5fm6ksky6w0981arp-withinbot-0.1.0/bin/withinbot
Nov 09 09:51:51 chrysalis systemd[1]: Started withinbot.service.
This basic template is enough to expand out to anything you would need and is what I am using for my own network. This should be generic enough for most of your needs. Check out the NixOS manual for more examples and things you can do with this. The Nixops manual is also a good read. It can also set up deployments with VirtualBox, libvirtd, AWS, Digital Ocean, and even Google Cloud.
The cloud is the limit! Be well.