unix domain socket post
Signed-off-by: Christine Dodrill <me@christine.website>
This commit is contained in:
parent
a6063676d3
commit
09a6c26fd4
|
|
@ -0,0 +1,390 @@
|
|||
---
|
||||
title: Unix Domain Sockets for Serving HTTP in Production
|
||||
date: 2021-04-01
|
||||
series: howto
|
||||
tags:
|
||||
- unix
|
||||
- nginx
|
||||
- devops
|
||||
- nixos
|
||||
- systemd
|
||||
---
|
||||
|
||||
# Unix Domain Sockets for Serving HTTP in Production
|
||||
|
||||
Securing production servers can be a chore. It is a seemingly endless game of
|
||||
balancing risks with convenience and not breaking what you want to do. Small,
|
||||
incremental gains are usually a very good idea however. Today we'll learn how to
|
||||
use [Unix Domain Sockets](https://en.wikipedia.org/wiki/Unix_domain_socket) to
|
||||
host your HTTP services. This allows you to run your services like normal on
|
||||
production machines without there being a risk of people being able to access
|
||||
the raw HTTP port.
|
||||
|
||||
[Wait, what. You're having a _service_ listen on a _file_? Why would you want to
|
||||
do this?](conversation://Mara/hmm?smol)
|
||||
|
||||
Mostly to prevent you from messing up and accidentally exposing your backend
|
||||
port to the internet. Firewall configuration is probably the most "correct" way
|
||||
to solve that concern, however this lets you also take advantage of filesystem
|
||||
permissions to fine-tune access down to the exact users and groups that should
|
||||
have access to the socket. In our case we only want ngnix to access this socket,
|
||||
so we can use filesystem permissions (and a unix group) to ensure this.
|
||||
Attackers can't connect to anything they aren't able to connect to.
|
||||
|
||||
[I see. How do you do this?](conversation://Mara/aha?smol)
|
||||
|
||||
At a high level every file in a unix filesystem has 3 kinds of permissions:
|
||||
user, group and "other". Every file has an owner and a UNIX group associated
|
||||
with it. Here's an example using the
|
||||
[Cargo.toml](https://github.com/Xe/site/blob/main/Cargo.toml) of this website's
|
||||
app server:
|
||||
|
||||
```console
|
||||
$ stat ./Cargo.toml
|
||||
File: ./Cargo.toml
|
||||
Size: 1572 Blocks: 8 IO Block: 4096 regular file
|
||||
Device: 10301h/66305d Inode: 20447261 Links: 1
|
||||
Access: (0644/-rw-r--r--) Uid: ( 1001/ cadey) Gid: ( 100/ users)
|
||||
Access: 2021-04-01 19:48:44.791162535 -0400
|
||||
Modify: 2021-04-01 19:48:44.786162545 -0400
|
||||
Change: 2021-04-01 19:48:44.786162545 -0400
|
||||
Birth: 2021-03-25 09:09:35.490311674 -0400
|
||||
```
|
||||
|
||||
[The <a href="https://man7.org/linux/man-pages/man1/stat.1.html">`stat(1)`</a>
|
||||
command lets you query the filesystem for common types of metadata about a given
|
||||
file.](conversation://Mara/hacker?smol)
|
||||
|
||||
In this case the permissions of this file are `0644`, which is a base-8 (octal)
|
||||
number that describes the permissions for the user, group and others. It breaks
|
||||
up something like this:
|
||||
|
||||
<center><blockquote class="twitter-tweet"><p dir="ltr">unix permissions <a
|
||||
href="https://t.co/2WcL6w44FR">pic.twitter.com/2WcL6w44FR</a></p>— 🔎Julia
|
||||
Evans🔍 (@b0rk) <a
|
||||
href="https://twitter.com/b0rk/status/982641594305273856?ref_src=twsrc%5Etfw">April
|
||||
7, 2018</a></blockquote> <script async
|
||||
src="https://platform.twitter.com/widgets.js" charset="utf-8"></script></center>
|
||||
|
||||
If we wanted to create a socket that only nginx can access, assuming we
|
||||
share a group with nginx we would need a socket with something like `0770` (user
|
||||
and group can read, write and "execute", everyone else gets denied) for its
|
||||
permissions. Then we would need to chuck it somewhere that both the app backend
|
||||
and nginx have access to and finally configure nginx to do this.
|
||||
|
||||
So let's do it! Let's take the venerable [printer facts
|
||||
server](https://tulpa.dev/cadey/printerfacts) server and make it listen on a
|
||||
Unix socket. Right now it uses something like this to listen for requests:
|
||||
|
||||
```rust
|
||||
warp::serve(
|
||||
fact_handler
|
||||
.or(index_handler)
|
||||
.or(files)
|
||||
.or(not_found_handler)
|
||||
.with(warp::log(APPLICATION_NAME)),
|
||||
)
|
||||
.run(([0, 0, 0, 0], port))
|
||||
.await;
|
||||
```
|
||||
|
||||
This configures [warp](https://github.com/seanmonstar/warp/) (the HTTP framework
|
||||
that I'm using for the printer facts server) to listen over TCP on some port.
|
||||
This is hard-coded to listen on `0.0.0.0`, which means that TCP sessions from
|
||||
_any_ network interface can connect to the service. This is _very_ convenient
|
||||
for development, so we are going to want to keep this behaviour in some way.
|
||||
|
||||
Fortunately warp [has an
|
||||
example](https://github.com/seanmonstar/warp/blob/master/examples/unix_socket.rs)
|
||||
for listening on a unix socket. Let's make the service listen on
|
||||
`./printerfacts.sock` so we can make sure that everything still works:
|
||||
|
||||
```rust
|
||||
let server = warp::serve(
|
||||
fact_handler
|
||||
.or(index_handler)
|
||||
.or(files)
|
||||
.or(not_found_handler)
|
||||
.with(warp::log(APPLICATION_NAME)),
|
||||
);
|
||||
|
||||
if let Ok(sockpath) = std::env::var("SOCKPATH") {
|
||||
use tokio::net::UnixListener;
|
||||
use tokio_stream::wrappers::UnixListenerStream;
|
||||
let listener = UnixListener::bind(sockpath).unwrap();
|
||||
let incoming = UnixListenerStream::new(listener);
|
||||
server.run_incoming(incoming).await;
|
||||
} else {
|
||||
server.run(([0, 0, 0, 0], port));
|
||||
}
|
||||
```
|
||||
|
||||
Then we can launch the service with a domain socket using a command like this:
|
||||
|
||||
```console
|
||||
$ env SOCKPATH=./printerfacts.sock cargo run
|
||||
```
|
||||
|
||||
Let's see how the output of `stat(1)` changed compared to when we ran it on a
|
||||
file:
|
||||
|
||||
```console
|
||||
$ stat ./printerfacts.sock
|
||||
File: ./printerfacts.sock
|
||||
Size: 0 Blocks: 0 IO Block: 4096 socket
|
||||
Device: 10301h/66305d Inode: 23858442 Links: 1
|
||||
Access: (0755/srwxr-xr-x) Uid: ( 1001/ cadey) Gid: ( 100/ users)
|
||||
Access: 2021-04-01 21:00:51.558219253 -0400
|
||||
Modify: 2021-04-01 21:00:51.558219253 -0400
|
||||
Change: 2021-04-01 21:00:51.558219253 -0400
|
||||
Birth: 2021-04-01 21:00:51.558219253 -0400
|
||||
```
|
||||
|
||||
`stat(1)` reports that the file is a socket! Let's see if everything still works
|
||||
by using `curl --unix-socket` to connect to the service and retrieve an amusing
|
||||
fact about printers:
|
||||
|
||||
```console
|
||||
$ curl --unix-socket ./printerfacts.sock http://foo/fact
|
||||
The strongest climber among the big printers, a leopard can carry prey twice its
|
||||
weight up a tree.
|
||||
```
|
||||
|
||||
[Why do you have `foo` as the HTTP hostname for the
|
||||
request?](conversation://Mara/hmm?smol)
|
||||
|
||||
Because it doesn't matter! I could have anything there, but foo is fast for me
|
||||
to type. The URL host information usually tells curl where to connect, but the
|
||||
`--unix-socket` flag overrides this logic.
|
||||
|
||||
[Wait, what the heck are printer facts?](conversation://Mara/wat?smol)
|
||||
|
||||
Blame Foone and #infoforcefeed.
|
||||
|
||||
Anyways, let's make the TCP logic a bit more clean in the process. Right now it
|
||||
only listens on IPv4 and it would be nice if it listened on IPv6 too. Let's
|
||||
replace that last `else` body with this:
|
||||
|
||||
```rust
|
||||
} else {
|
||||
server
|
||||
.run((std::net::IpAddr::from_str("::").unwrap(), port))
|
||||
.await;
|
||||
}
|
||||
```
|
||||
|
||||
[`::` is the IPv6 version of `0.0.0.0`, or the <a
|
||||
href="https://tools.ietf.org/html/rfc4291#section-2.5.2">unspecified
|
||||
address</a>. It tells most IP stacks to allow traffic from any network
|
||||
interface.](conversation://Mara/hacker?smol)
|
||||
|
||||
Now let's re-build the printer facts service and re-run it to make sure it still
|
||||
works:
|
||||
|
||||
```console
|
||||
$ env SOCKPATH=./printerfacts.sock cargo run
|
||||
Finished dev [unoptimized + debuginfo] target(s) in 0.04s
|
||||
Running `target/debug/printerfacts`
|
||||
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os {
|
||||
code: 98, kind: AddrInUse, message: "Address already in use" }',
|
||||
src/main.rs:73:53
|
||||
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
|
||||
```
|
||||
|
||||
[Wait, what. Isn't this serving HTTP from a file? Why would it be an address in
|
||||
use error?](conversation://Mara/wat?smol)
|
||||
|
||||
Even though it looks like a file to us humans, it's still a socket under the
|
||||
hood. In this case it means the filename is already in use. Working around this
|
||||
is simple though, all we need to do is-
|
||||
|
||||
[DELETE THIS!](conversation://Numa/delet?smol)
|
||||
|
||||
[Where the hell did you come from?](conversation://Cadey/angy?smol)
|
||||
|
||||
But yes, we do need to delete the socket file if it doesn't already exist. Let's
|
||||
sneak this bit of code in before we listen on the Unix socket:
|
||||
|
||||
```rust
|
||||
if let Ok(sockpath) = std::env::var("SOCKPATH") {
|
||||
let _ = std::fs::remove_file(&sockpath); // nuke the socket
|
||||
let listener = UnixListener::bind(sockpath).unwrap();
|
||||
let incoming = UnixListenerStream::new(listener);
|
||||
server.run_incoming(incoming).await;
|
||||
} else {
|
||||
server
|
||||
.run((std::net::IpAddr::from_str("::").unwrap(), port))
|
||||
.await;
|
||||
}
|
||||
```
|
||||
|
||||
[Didn't you just say "if it doesn't already exist"? Why delete it
|
||||
unconditionally and throw away any errors?](conversation://Mara/hmm?smol)
|
||||
|
||||
Two reasons:
|
||||
1. Statistically if the file doesn't exist and the service can't create it when
|
||||
it binds to that path, you probably have bigger problems and it's probably
|
||||
better for the program to explode there.
|
||||
2. The filename is passed in as an environment variable. If your environment
|
||||
variable is wrong, we can treat this as a fundamental assertion error and
|
||||
blow up when the file fails to bind.
|
||||
|
||||
Let's define this in the [NixOS module for the printerfacts
|
||||
service](https://github.com/Xe/nixos-configs/blob/master/common/services/printerfacts.nix).
|
||||
First we will need to add a configuration option for the socket path:
|
||||
|
||||
```nix
|
||||
let cfg = config.within.services.printerfacts;
|
||||
in {
|
||||
options.within.services.printerfacts = {
|
||||
# ...
|
||||
sockPath = mkOption rec {
|
||||
type = types.str;
|
||||
default = "/tmp/printerfacts.sock";
|
||||
example = default;
|
||||
description = "The unix domain socket that printerfacts should listen on";
|
||||
};
|
||||
};
|
||||
# ...
|
||||
}
|
||||
```
|
||||
|
||||
This creates an option at `cfg.sockPath` that we can pipe through elsewhere,
|
||||
such as the start script for the service:
|
||||
|
||||
```nix
|
||||
# inside
|
||||
script = let site = pkgs.tulpa.dev.cadey.printerfacts;
|
||||
in ''
|
||||
export SOCKPATH=${cfg.sockPath}
|
||||
export DOMAIN=${toString cfg.domain}
|
||||
export RUST_LOG=info
|
||||
cd ${site}
|
||||
exec ${site}/bin/printerfacts
|
||||
'';
|
||||
```
|
||||
|
||||
And then we can go on to setting up nginx. First, let's figure out how to
|
||||
reverse proxy to a unix socket. In nginx configuration land,
|
||||
[`proxy_pass`](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass)
|
||||
is the name of the configuration directive that lets you tell nginx to reverse
|
||||
proxy to somewhere. There's an example with a unix socket! This would let us
|
||||
reverse proxy a unix socket to a TCP port like this:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 127.0.0.1:9000;
|
||||
location / {
|
||||
proxy_pass http://unix:/tmp/printerfacts.sock;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For comparison here's how you'd reverse proxy to a HTTP server running on port
|
||||
`42069`:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 127.0.0.1:9001;
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:42069;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
So, we just need to change where nginx reverse proxies to in the NixOS config.
|
||||
Let's look down at the nginx config for `printerfacts`:
|
||||
|
||||
```nix
|
||||
# ...
|
||||
services.nginx.virtualHosts."${cfg.domain}" = {
|
||||
locations."/" = {
|
||||
proxyPass = "http://127.0.0.1:${toString cfg.port}";
|
||||
proxyWebsockets = true;
|
||||
};
|
||||
forceSSL = cfg.useACME;
|
||||
useACMEHost = "cetacean.club";
|
||||
extraConfig = ''
|
||||
access_log /var/log/nginx/printerfacts.access.log;
|
||||
'';
|
||||
};
|
||||
```
|
||||
|
||||
The `proxyPass` option directly translates to a `proxy_pass` directive, so we
|
||||
can get away with something like this:
|
||||
|
||||
```nix
|
||||
# ...
|
||||
proxyPass = "http://unix:${cfg.sockPath}";
|
||||
```
|
||||
|
||||
And now we can deploy the service and everything should work right? printerfacts
|
||||
provides a unix socket at the given path and then nginx is configured to use
|
||||
that socket to send back printer facts. Let's deploy it and see what happens:
|
||||
|
||||
<center>
|
||||
|
||||
|
||||
|
||||
</center>
|
||||
|
||||
Oh no. Let's see what `journalctl -fu nginx` has to say:
|
||||
|
||||
```console
|
||||
$ journalctl -fu nginx
|
||||
Apr 01 23:29:58 lufta nginx[15396]: 2021/04/01 23:29:58 [crit] 15396#15396: *198
|
||||
connect() to unix:/tmp/printerfacts.sock failed (13: Permission denied) while
|
||||
connecting to upstream, client: lol.no.ip.here, server:
|
||||
printerfacts.cetacean.club, request: "GET / HTTP/2.0", upstream:
|
||||
"http://unix:/tmp/printerfacts.sock:/", host: "printerfacts.cetacean.club"
|
||||
```
|
||||
|
||||
[Wait, what. Isn't `/tmp` guaranteed by the filesystem hierarchy standards to
|
||||
always be readable and writable by any user?](conversation://Mara/wat?smol)
|
||||
|
||||
Normally, yes. However we are running nginx inside systemd, and one of the
|
||||
things you can do with systemd is make `/tmp` isolated for given services. This
|
||||
allows you to prevent a service from being able to exfiltrate data inside
|
||||
`/tmp`. However, this is definitely NOT the behaviour we want in this case.
|
||||
Let's change the systemd unit for nginx to disable this and also make nginx run
|
||||
as the same group as the printerfacts service:
|
||||
|
||||
```nix
|
||||
systemd.services.nginx.serviceConfig = {
|
||||
PrivateTmp = lib.mkForce "false";
|
||||
SupplementaryGroups = "within";
|
||||
};
|
||||
```
|
||||
|
||||
[In NixOS, most of the time if the same option is declared in multiple
|
||||
places it will result in a build error. `lib.mkForce` disables this behaviour
|
||||
and instead "forcibly" sets this value.](convesation://Mara/hacker?smol)
|
||||
|
||||
Now nginx has the same `/tmp` as the printerfacts service, everything will work
|
||||
as we expect. Users are none the wiser that I'm using a domain socket here. I
|
||||
get to have another service not bound to the network and I have moved towards
|
||||
better security on my machine!
|
||||
|
||||
[What about Prometheus? Doesn't it need a direct line of fire to the service to
|
||||
scrape metrics?](conversation://Mara/hmm?smol)
|
||||
|
||||
...Time for some percussive maintenance!
|
||||
|
||||
<center>
|
||||
|
||||
|
||||
|
||||
</center>
|
||||
|
||||
---
|
||||
|
||||
I'm experimenting with a new "smol" mode for the Mara interludes as well as
|
||||
introducing a few more characters to the christine dot website cinematic
|
||||
universe. Please do let me know how this works out for you. I think I have the
|
||||
sizes optimized for mobile usage better, but [contributions to fix my horrible
|
||||
CSS](https://github.com/Xe/site/blob/main/css/shim.css) would really, really,
|
||||
really be appreciated.
|
||||
|
||||
I'm considering moving over all of the Mara interludes to use smol mode. If you
|
||||
have opinions about this please let me know them.
|
||||
|
|
@ -28,3 +28,12 @@ img {
|
|||
min-width: 9rem;
|
||||
max-width: calc(((70rem - 2rem)/6));
|
||||
}
|
||||
|
||||
.conversation-smol {
|
||||
min-width: 6rem;
|
||||
max-width: 6rem;
|
||||
}
|
||||
|
||||
.conversation-chat {
|
||||
align-self: center;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
use crate::templates::Html;
|
||||
use color_eyre::eyre::{Result, WrapErr};
|
||||
use comrak::nodes::{Ast, AstNode, NodeValue};
|
||||
use comrak::{format_html, parse_document, markdown_to_html, Arena, ComrakOptions};
|
||||
use comrak::{format_html, markdown_to_html, parse_document, Arena, ComrakOptions};
|
||||
use std::cell::RefCell;
|
||||
use crate::templates::Html;
|
||||
use url::Url;
|
||||
|
||||
pub fn render(inp: &str) -> Result<String> {
|
||||
|
|
@ -29,6 +29,7 @@ pub fn render(inp: &str) -> Result<String> {
|
|||
if u.scheme() != "conversation" {
|
||||
return Ok(());
|
||||
}
|
||||
let smol = u.query().unwrap_or("").contains("smol");
|
||||
let parent = node.parent().unwrap();
|
||||
node.detach();
|
||||
let mut message = vec![];
|
||||
|
|
@ -41,10 +42,11 @@ pub fn render(inp: &str) -> Result<String> {
|
|||
let name = u.host_str().unwrap_or("Mara");
|
||||
|
||||
let mut html = vec![];
|
||||
crate::templates::mara(&mut html, mood, name, Html(message))?;
|
||||
crate::templates::mara(&mut html, mood, name, Html(message.trim().into()), smol)?;
|
||||
|
||||
let new_node =
|
||||
arena.alloc(AstNode::new(RefCell::new(Ast::new(NodeValue::HtmlInline(html)))));
|
||||
let new_node = arena.alloc(AstNode::new(RefCell::new(Ast::new(
|
||||
NodeValue::HtmlInline(html),
|
||||
))));
|
||||
parent.append(new_node);
|
||||
|
||||
Ok(())
|
||||
|
|
|
|||
|
|
@ -84,6 +84,8 @@
|
|||
|
||||
<p>The art for Mara was drawn by <a href="https://selic.re/">Selicre</a>.</p>
|
||||
|
||||
<p>The art for Cadey was drawn by <a href="https://artzorastudios.weebly.com/">ArtZora Studios</a>.</p>
|
||||
|
||||
<script>
|
||||
// The actual function. Set this as an onclick function for your "Share on Mastodon" button
|
||||
function share_on_mastodon() @{
|
||||
|
|
|
|||
|
|
@ -1,12 +1,15 @@
|
|||
@(mood: &str, character: &str, message: Html<String>)
|
||||
@(mood: &str, character: &str, message: Html<String>, smol: bool)
|
||||
<div class="conversation">
|
||||
<div class="conversation-picture">
|
||||
<div class="conversation-picture @if smol { conversation-smol }">
|
||||
<picture>
|
||||
<source srcset="https://cdn.christine.website/file/christine-static/stickers/@character.to_lowercase()/@(mood).avif" type="image/avif">
|
||||
<source srcset="https://cdn.christine.website/file/christine-static/stickers/@character.to_lowercase()/@(mood).webp" type="image/webp">
|
||||
<img src="https://cdn.christine.website/file/christine-static/stickers/@character.to_lowercase()/@(mood).png" alt="@character is @mood">
|
||||
</picture>
|
||||
</div>
|
||||
@if smol {
|
||||
<div class="conversation-chat"><b><@character></b> @message</div>
|
||||
} else {
|
||||
<div>
|
||||
<p>
|
||||
<b>@character</b>
|
||||
|
|
@ -15,4 +18,5 @@
|
|||
@message
|
||||
</blockquote>
|
||||
</div>
|
||||
}
|
||||
</div>
|
||||
|
|
|
|||
Loading…
Reference in New Issue