279 lines
11 KiB
Markdown
279 lines
11 KiB
Markdown
---
|
|
title: "A Trip into FreeBSD"
|
|
date: 2021-02-13
|
|
tags:
|
|
- freebsd
|
|
---
|
|
|
|
I normally deal with Linux machines. Linux is what I know and it's what I've
|
|
been using since I was in college. A friend of mine has been coaxing me into
|
|
trying out [FreeBSD](https://www.freebsd.org), and I decided to try it out and
|
|
see what it's like. Here's some details about my experience and what I've
|
|
learned.
|
|
|
|
## Hardware
|
|
|
|
I've tried out FreeBSD on the following hardware:
|
|
|
|
- qemu/KVM on amd64
|
|
- Raspberry Pi 4 (4 GB)
|
|
- Raspberry Pi 3B (1 GB)
|
|
|
|
I've had the most luck with the Raspberry Pi 3 though. The KVM machine would
|
|
hang infinitely after the install process waiting for the mail service to do a
|
|
DNS probe of its own hostname (I do not host automagic FQDNS for my vms). I'm
|
|
pretty sure I was doing something wrong there but I wasn't able to figure out
|
|
what I needed to do in order to disable the DNS probe blocking startup.
|
|
|
|
[If you know what we were doing wrong here, please feel free to <a
|
|
href="/contact">contact</a> us with the thing we messed
|
|
up.](conversation://Mara/hacker)
|
|
|
|
After waiting for about 5 minutes I gave up and decided to try out the Raspberry
|
|
Pi 4. The Raspberry Pi 4 is the most powerful arm board I own. It has 4 GB of
|
|
ram and a quad core processor that is way more than sufficient for my needs. I
|
|
was hoping to use FreeBSD on that machine so I could benefit from the hardware
|
|
the most. Following the instructions on [the wiki
|
|
page](https://wiki.freebsd.org/arm/Raspberry%20Pi), I downloaded the 12.2 RPI
|
|
image and flashed it to an SD card using Etcher. I put the SD card in, turned
|
|
the raspi on and then waited for it to show up on the network.
|
|
|
|
Except it never showed up on the network. I ran scans with nmap (specifically
|
|
with the command `sudo nmap -sS -p 22 192.168.0.0/24`) and the IP address never
|
|
showed up. I also didn't see any new MAC addresses on the network, so that lead
|
|
me to believe that the pi was failing to boot. I downloaded an image for 13-BETA
|
|
and followed [this
|
|
guide](https://medium.com/swlh/freebsd-usb-boot-on-raspberry-pi-4-765cb6e75570)
|
|
that claims to make it work on the pi 4, but I got the same issue. The Raspberry
|
|
Pi 4 unfortunately has a micro-HDMI port on it, so I was unable to attach it to
|
|
my monitor to see any error messages. After trying for a while to see if I could
|
|
set up a serial port to get the serial log messages (spoiler: I couldn't), I dug
|
|
up my Pi 3 and stuck the same SD card into it, hooked it up to my monitor,
|
|
attached a spare keyboard to it and booted into FreeBSD first try.
|
|
|
|
## Using FreeBSD
|
|
|
|
FreeBSD is a very down to earth operating system. It also has a
|
|
[handbook](https://docs.freebsd.org/en/books/handbook/) that legitimately
|
|
includes all of the information you need to get up and running. Following the
|
|
handbook, I set a new password, installed the `pkg` tool, set up
|
|
[fish](https://fishshell.com) and then also installed the Go compiler toolchain
|
|
for the hell of it.
|
|
|
|
`pkg` is a very minimal looking package manager. It doesn't have very many
|
|
frills and it is integrated into the system pretty darn well. It looks like it
|
|
prefers putting everything into `/usr/local`, including init scripts and other
|
|
configuration files.
|
|
|
|
This interestingly lets you separate out the concerns of the base system from
|
|
individual machine-local configuration. I am not sure if this also works with
|
|
files like `/etc/resolv.conf` or other system configuration files, but it does
|
|
really give `/usr/local` a reason to exist beyond being a legacy location for
|
|
yolo-installed software that may or may not be able to be upgraded separately.
|
|
|
|
## Custom Services
|
|
|
|
Speaking of services, I wanted to see how hard it would be to get a custom
|
|
service running on a FreeBSD box. At the minimum I would need the following:
|
|
|
|
- The binary built for freebsd/aarch64 and installed to `/usr/local/bin`
|
|
- A user account for that service
|
|
- An init script for that service
|
|
- To enable the init script in `/etc/rc.conf`
|
|
|
|
I decided to do this with a service I made years ago called
|
|
[whatsmyip](https://github.com/Xe/whatsmyip).
|
|
|
|
### Building a Binary
|
|
|
|
Building the service is easy, I just go into the directory and run `go build`.
|
|
Then I get a binary. Running it in another tmux tab, we can see it in action:
|
|
|
|
```console
|
|
$ curl http://[::1]:9090
|
|
::1
|
|
```
|
|
|
|
I can also run the curl command from my macbook:
|
|
|
|
```console
|
|
$ curl http://pai:9090
|
|
100.72.190.5
|
|
```
|
|
|
|
Cool, I've got a working service! Let's install it to `/usr/local/bin`:
|
|
|
|
```console
|
|
$ doas cp ./whatismyip /usr/local/bin
|
|
```
|
|
|
|
[Wait, `doas`? What is `doas`? It looks like it's doing something close to what
|
|
sudo does.](conversation://Mara/hmm)
|
|
|
|
[doas](https://en.wikipedia.org/wiki/Doas) is a program that does most of the
|
|
same things that sudo does, but it's a much smaller codebase. I decided to try
|
|
out doas for this install for no other reason than I thought it would be a cool
|
|
thing to learn. It's actually pretty simple, and I'm going to look at using it
|
|
elsewhere (with an alias for `sudo` -> `doas`).
|
|
|
|
### Service User
|
|
|
|
The handbook says that we use the
|
|
[adduser](https://people.freebsd.org/~blackend/en_US.ISO8859-1/books/handbook/users-synopsis.html)
|
|
command to add users to the system. So, let's run `adduser` to create a
|
|
`whatsmyip` user:
|
|
|
|
```console
|
|
# adduser
|
|
Username: whatsmyip
|
|
Full name: github.com/Xe/whatsmyip
|
|
Uid (Leave empty for default): 666
|
|
Login group [whatsmyip]:
|
|
Login group is whatsmyip. Invite whatsmyip into other groups? []:
|
|
Login class [default]:
|
|
Shell (sh csh tcsh bash rbash git-shell fish nologin) [sh]: sh
|
|
Home directory [/home/whatsmyip]: /var/db/whatsmyip
|
|
Home directory permissions (Leave empty for default):
|
|
Use password-based authentication? [yes]: no
|
|
Lock out the account after creation? [no]: yes
|
|
Username : whatsmyip
|
|
Password : <disabled>
|
|
Full Name : github.com/Xe/whatsmyip
|
|
Uid : 666
|
|
Class :
|
|
Groups : whatsmyip
|
|
Home : /var/db/whatsmyip
|
|
Home Mode :
|
|
Shell : /bin/sh
|
|
Locked : yes
|
|
OK? (yes/no): yes
|
|
adduser: INFO: Successfully added (whatsmyip) to the user database.
|
|
adduser: INFO: Account (whatsmyip) is locked.
|
|
Add another user? (yes/no): no
|
|
Goodbye!
|
|
```
|
|
|
|
It's a bit weird that there's not a flow for creating a "system user" that
|
|
automatically sets the flags that I expect from Linux system administration, but
|
|
I was able to specify the values manually without too much effort.
|
|
|
|
Something interesting is that when I set the user account to `nologin` I
|
|
actually was unable to log in as the user. Usually in Linux you can hack around
|
|
this with `su` flags but FreeBSD doesn't have this escape hatch. Neat.
|
|
|
|
### Init Script
|
|
|
|
Now that I had the service account set up, I need to write an init service that
|
|
will start this program on boot. Following other parts of the handbook I was
|
|
able to get a base script that looks like this:
|
|
|
|
```shell
|
|
#!/bin/sh
|
|
#
|
|
# PROVIDE: whatsmyip
|
|
# REQUIRE: DAEMON
|
|
# KEYWORD: shutdown
|
|
|
|
. /etc/rc.subr
|
|
|
|
name=whatsmyip
|
|
rcvar=whatsmyip_enable
|
|
|
|
command="/usr/sbin/daemon"
|
|
command_args="-S -u whatsmyip -r -f -p /var/run/whatsmyip.pid /usr/local/bin/whatsmyip"
|
|
load_rc_config $name
|
|
|
|
#
|
|
# DO NOT CHANGE THESE DEFAULT VALUES HERE
|
|
# SET THEM IN THE /etc/rc.conf FILE
|
|
#
|
|
whatsmyip_enable=${whatsmyip_enable-"NO"}
|
|
pidfile=${whatsmyip_pidfile-"/var/run/whatsmyip.pid"}
|
|
|
|
run_rc_command "$1"
|
|
```
|
|
|
|
Now I can copy this file to `/usr/local/etc/rc.d/whatsmyip` and then make sure
|
|
it's set to the permissions `0555` with something like:
|
|
|
|
```console
|
|
$ chmod 0555 ./whatsmyip.rc
|
|
$ doas cp ./whatsmyip.rc /usr/local/etc/rc.d/whatsmyip
|
|
```
|
|
|
|
### Enabling The Service
|
|
|
|
Once I had the file in the right place, I enabled the service in `/etc/rc.conf`
|
|
like this:
|
|
|
|
```shell
|
|
# whatsmyip
|
|
whatsmyip_enable="YES"
|
|
```
|
|
|
|
Then I started the service with `service whatsmyip start`, and I was unable to
|
|
start the service. I got this error:
|
|
|
|
```
|
|
Feb 13 20:40:00 pai freebsd[1519]: /usr/local/etc/rc.d/whatsmyip: WARNING: failed to start whatsmyip
|
|
```
|
|
|
|
And no other useful information to help me actually fix the problem. I assume
|
|
there's some weirdness going on with permissions, so let's sidestep the user
|
|
account for now and just run the service as root directly by changing the
|
|
`command_args` in `/usr/local/etc/rc.d/whatsmyip`:
|
|
|
|
```shell
|
|
command_args="-S -r -p /var/run/whatsmyip.pid /usr/local/bin/whatsmyip"
|
|
```
|
|
|
|
Restarting the service, everything works! I can hit that service all I want and
|
|
I get back the IP address that I used to hit that service.
|
|
|
|
## What I Learned
|
|
|
|
FreeBSD has _excellent_ documentation. The people on the documentation team
|
|
really care about making the handbook useful. I wish it went into more detail
|
|
about best practices for making your own services (I had to crib from some other
|
|
service files as well as googling for a minimal template), but overall it gives
|
|
you enough information to get off the ground.
|
|
|
|
FreeBSD is also fairly weird. It's familiar-ish, but it's a very different
|
|
experience. It's also super-minimal. Looking at the output of `ps x`, there's
|
|
only 45 processes running on the system, including kernel threads.
|
|
|
|
```
|
|
root@pai ~# ps x | wc -l
|
|
45
|
|
```
|
|
|
|
The only processes are `init`, `dhclient`, a device manager, `syslog`,
|
|
`tailscaled`, `sshd`, `cron`, `whatsmyip`, `fish` and a few instances of `getty`
|
|
to allow me to log in with an HDMI monitor and keyboard should I need to. That's
|
|
it. That's all that's running. It's only using 96 MB of ram and most of the
|
|
machine's power is left over to me.
|
|
|
|
It's just a shame that FreeBSD support for programming languages is so poor in
|
|
general. Go works fine on it, but Rust doesn't have any pre-built binaries for
|
|
the compiled (and using ports/pkg isn't an option because aarch64 is a tier-2
|
|
architecture in FreeBSD land which means that it's not guaranteed to have
|
|
prebuilt binaries for everything). Compiling Rust from source also really isn't
|
|
an option because I don't have enough ram on my raspi to do that. Go works
|
|
though.
|
|
|
|
I really wonder how this kind of network effect will boil down with more and
|
|
more security libraries like
|
|
[pyca](https://github.com/pyca/cryptography/issues/5771) integrating Rust deeper
|
|
into core security components. It probably means that people are going to have
|
|
to step up and actually do the legwork required to get Rust working on more
|
|
platforms, however it definitely is going to leave some older hardware or less
|
|
commonly used configurations (like aarch64 FreeBSD) in the dust if we aren't
|
|
careful. Maybe this isn't a technical problem, but it is definitely something
|
|
interesting to think about.
|
|
|
|
Overall, FreeBSD is an interesting tool and if I ever have a good use for it in
|
|
my server infrastructure I will definitely give it a solid look. I just wish it
|
|
was as easy to manage a FreeBSD system as it is to manage a NixOS system. A lot
|
|
of my faffing about with `rc.conf` and rc scripts wouldn't have needed to
|
|
happen if that was the case.
|