diff --git a/static/talks/surreal-horror-pam.pdf b/static/talks/surreal-horror-pam.pdf new file mode 100644 index 0000000..17dae71 Binary files /dev/null and b/static/talks/surreal-horror-pam.pdf differ diff --git a/static/talks/surreal-horror-pam/001.jpeg b/static/talks/surreal-horror-pam/001.jpeg new file mode 100755 index 0000000..dcfe918 Binary files /dev/null and b/static/talks/surreal-horror-pam/001.jpeg differ diff --git a/static/talks/surreal-horror-pam/002.jpeg b/static/talks/surreal-horror-pam/002.jpeg new file mode 100755 index 0000000..3a1450c Binary files /dev/null and b/static/talks/surreal-horror-pam/002.jpeg differ diff --git a/static/talks/surreal-horror-pam/003.jpeg b/static/talks/surreal-horror-pam/003.jpeg new file mode 100755 index 0000000..982ff12 Binary files /dev/null and b/static/talks/surreal-horror-pam/003.jpeg differ diff --git a/static/talks/surreal-horror-pam/004.jpeg b/static/talks/surreal-horror-pam/004.jpeg new file mode 100755 index 0000000..4b58336 Binary files /dev/null and b/static/talks/surreal-horror-pam/004.jpeg differ diff --git a/static/talks/surreal-horror-pam/005.jpeg b/static/talks/surreal-horror-pam/005.jpeg new file mode 100755 index 0000000..7d2f14b Binary files /dev/null and b/static/talks/surreal-horror-pam/005.jpeg differ diff --git a/static/talks/surreal-horror-pam/006.jpeg b/static/talks/surreal-horror-pam/006.jpeg new file mode 100755 index 0000000..43dffe2 Binary files /dev/null and b/static/talks/surreal-horror-pam/006.jpeg differ diff --git a/static/talks/surreal-horror-pam/007.jpeg b/static/talks/surreal-horror-pam/007.jpeg new file mode 100755 index 0000000..2d0ba21 Binary files /dev/null and b/static/talks/surreal-horror-pam/007.jpeg differ diff --git a/static/talks/surreal-horror-pam/008.jpeg b/static/talks/surreal-horror-pam/008.jpeg new file mode 100755 index 0000000..0546261 Binary files /dev/null and b/static/talks/surreal-horror-pam/008.jpeg differ diff --git a/static/talks/surreal-horror-pam/009.jpeg b/static/talks/surreal-horror-pam/009.jpeg new file mode 100755 index 0000000..05ba5c7 Binary files /dev/null and b/static/talks/surreal-horror-pam/009.jpeg differ diff --git a/static/talks/surreal-horror-pam/010.jpeg b/static/talks/surreal-horror-pam/010.jpeg new file mode 100755 index 0000000..7349ea4 Binary files /dev/null and b/static/talks/surreal-horror-pam/010.jpeg differ diff --git a/static/talks/surreal-horror-pam/011.jpeg b/static/talks/surreal-horror-pam/011.jpeg new file mode 100755 index 0000000..4dc6221 Binary files /dev/null and b/static/talks/surreal-horror-pam/011.jpeg differ diff --git a/static/talks/surreal-horror-pam/012.jpeg b/static/talks/surreal-horror-pam/012.jpeg new file mode 100755 index 0000000..a8d6565 Binary files /dev/null and b/static/talks/surreal-horror-pam/012.jpeg differ diff --git a/static/talks/surreal-horror-pam/013.jpeg b/static/talks/surreal-horror-pam/013.jpeg new file mode 100755 index 0000000..a55da46 Binary files /dev/null and b/static/talks/surreal-horror-pam/013.jpeg differ diff --git a/static/talks/surreal-horror-pam/014.jpeg b/static/talks/surreal-horror-pam/014.jpeg new file mode 100755 index 0000000..0ccf09e Binary files /dev/null and b/static/talks/surreal-horror-pam/014.jpeg differ diff --git a/static/talks/surreal-horror-pam/015.jpeg b/static/talks/surreal-horror-pam/015.jpeg new file mode 100755 index 0000000..d87f1e5 Binary files /dev/null and b/static/talks/surreal-horror-pam/015.jpeg differ diff --git a/static/talks/surreal-horror-pam/016.jpeg b/static/talks/surreal-horror-pam/016.jpeg new file mode 100755 index 0000000..05457ef Binary files /dev/null and b/static/talks/surreal-horror-pam/016.jpeg differ diff --git a/static/talks/surreal-horror-pam/017.jpeg b/static/talks/surreal-horror-pam/017.jpeg new file mode 100755 index 0000000..7746121 Binary files /dev/null and b/static/talks/surreal-horror-pam/017.jpeg differ diff --git a/static/talks/surreal-horror-pam/018.jpeg b/static/talks/surreal-horror-pam/018.jpeg new file mode 100755 index 0000000..694a182 Binary files /dev/null and b/static/talks/surreal-horror-pam/018.jpeg differ diff --git a/static/talks/surreal-horror-pam/019.jpeg b/static/talks/surreal-horror-pam/019.jpeg new file mode 100755 index 0000000..6a38166 Binary files /dev/null and b/static/talks/surreal-horror-pam/019.jpeg differ diff --git a/talks/surreal-horror-pam-2021-11-09.markdown b/talks/surreal-horror-pam-2021-11-09.markdown new file mode 100644 index 0000000..a077602 --- /dev/null +++ b/talks/surreal-horror-pam-2021-11-09.markdown @@ -0,0 +1,263 @@ +--- +title: The Surreal Horror of PAM +date: 2021-11-09 +slides_link: /static/talks/surreal-horror-pam.pdf +tags: + - alpinelinux + - pam + - satire +--- + + + +[https://youtu.be/INjCiHUIjgg](https://youtu.be/INjCiHUIjgg) + +--- + +![](/static/talks/surreal-horror-pam/001.jpeg) + +Hi, I’m Xe. You know this because that is what your computer tells you. But how +does it know that? + +This is a partially satirical talk. It is intended to be mostly factually +accurate, however some of the details are stretched for comedic effect. This +talk may contain opinions, none of these opinions are the opinions of my +employer. I hope you enjoy this catharsis. + +You may want to make sure you all are muted, as this is probably going to make +you laugh and I’ll get speech jammed if I hear it. I’m also planning on +publishing this publicly, so please avoid mentioning privileged information in +the Q&A section at the end. + +![](/static/talks/surreal-horror-pam/002.jpeg) + +So before we talk about complicated things, let’s start with the basics. This is +how UNIX systems authenticate. They have some files in /etc/ that are +effectively plaintext databases for usernames, ids, groups and password hashes, +and those are what are used for this legacy authentication flow. + +You start with a login program running as root (such as /bin/login) and then it +gets your username and password. Then it checks /etc/passwd to see if your user +account exists. If it does it grabs the user ID and uses that to look up more +information from /etc/groups to build up your dossier. Then it takes your +password and does some crazy math to it to compare it to the hashed password in +/etc/shadow. If it matches (or if someone forces it to match through malicious +means), then the login program forks a child process, impersonates your user +account based on that dossier from earlier, creates a login shell and finally +sends you off to do whatever it is you want to the poor computer. + +That’s it. That’s how the classic System V authentication stack works. +Technically I’m stretching things a bit as /etc/shadow was a fairly recent +addition (mostly because /etc/passwd is world-readable by design for some arcane +reason that I can’t find on Google), but it’s basically that. There’s a few +steps I’m leaving out for brevity, but they are boring things that only nerds +care about. + +![](/static/talks/surreal-horror-pam/003.jpeg) + +This UNIX authentication model is really that simple. You can explain the high +level details on a single slide that was hastily written at 8 am in about 5 +minutes. However, because of this simplicity it leaves attackers with a small +list of targets to try to trick the computer into mucking with. But it does +work, mostly. + +Some of the huge downsides are that it only works on one machine at the time. +This made sense for when UNIX was created as the model was to have a big ol +mainframe for a company and then have everyone connect to it, but in the +meantime we’ve gone around carrying supercomputers in our watches and always +having a calculator in our pockets (Miss Van Hamme, you should have been more +forward thinking than to insinuate otherwise in my second grade math class!). +Because of this (and the fact that said supercomputer watches also run a full +fledged UNIX kernel), we can’t really rely on a model created for 1970’s +mainframe technology to get the job done in this day and age of hyperconverged +cloud federated femtoservices. + +![](/static/talks/surreal-horror-pam/004.jpeg) + +The last sentence probably set off the beard twitching alarms, so yes there are +some workarounds here. + +You could JUST put the files on a network filesystem. That would make them +immutable to whoever tries to mess with them on an individual machine. However +the peak of network filesystem security on Linux is “don’t get your network +hacked lol”. They won’t add the ability for a kernel mode filesystem driver to +make TLS validated sessions so you can use this thing called cryptography to +secure access to the filesystem and data on the wire. They are busy arguing with +people about how to send plain-text email and the like. You could also put those +files on a CD, set the immutable flag or something, but all that will do is +making changing passwords more expensive, annoying and filled with anger. + +![](/static/talks/surreal-horror-pam/005.jpeg) + +What’s that? I think I hear something coming. + +![](/static/talks/surreal-horror-pam/006.jpeg) + +It’s sshd! Turns out that we do in fact need something more complicated because +we have networks and the cloud and complicated mutifactor auth requirements for +acronym compliance! We can’t really do that with UNIX authentication because it +was designed before such things were even a glimmer in the eye of security +professionals. + +Surely there has to be a better option out there _somewhere_. + +![](/static/talks/surreal-horror-pam/007.jpeg) + +Et voila! C’est le PAM! Turns out someone else a long time ago had the same +problems and somehow got legal to sign off on making it open source! PAM is a +modular system for making authentication and authorization work. + +![](/static/talks/surreal-horror-pam/008.jpeg) + +For reference, authentication and authorization are being split up into two +concepts here (like they are in a lot of the industry). We’re gonna take a page +out of the white hat’s guide to security here and call these concepts +authentication (who you are and how we know who you are) and authorization (can +you _really_ take all the money out of the bank account?). It is a solid 90’s +solution to a 70’s problem and good god it shows. + +![](/static/talks/surreal-horror-pam/009.jpeg) + +PAM was made in the 90’s by this little startup nobody here has heard of called +Sun Microsystems. They had a problem where they had a bunch of machines to apply +complicated authentication rules to (all thanks to those pesky enterprise +contracts) and no way to really do it. Money won this valiant fight between +engineering and sales, so we ended up with PAM. + +![](/static/talks/surreal-horror-pam/010.jpeg) + +So you’re probably wondering something along the lines of “how does this thing +work?”. Carefully, that's how. + +![](/static/talks/surreal-horror-pam/011.jpeg) + +This is a screenshot of a text file (a common thing to do these days) of the +main PAM configuration file in a distribution called Alpine Linux. I’m using +Alpine Linux here because it is the simpler option for getting PAM to work and I +really do not want to spend all day debugging PAM with gdb and strace on Ubuntu +to demonstrate it with that. PAM has a few kinds of modules: + +* authentication, this is not just checking your password, but also making sure + that your account is allowed to be logged into and setting up things like your + preferred login shell +* account, the things that assign a user an account based on the circumstances + of their authentication or validate that somehow (this is also where an LDAP + server would get thrown into the mix if you really hate yourself) +* password, the things that check passwords or do other kinds of validation like + that (if you want to use Google Authenticator TOTP codes, you’d do that here) +* session, these things handle other system errata like making sure the + message-of-the-day (MOTD) is shown when you log in or letting logind know + about the session so it can make a cgroup for you + +All of these modules are implemented as dynamically linked libraries in C (the +HEIGHT of modern programming security, as we all know) and PAM works by loading +all of these files out of /lib/security, throwing them directly into ram and +then executing arbitrary C ABI functions out of them to see what they return. + +Yes, really. I am still surprised that the modules are written in C and not Java +given it was from Sun. + +If you typo this configuration file and don’t have a root session open with your +box, it is even worse than typoing the /etc/sudoers file. Typoing /etc/sudoers +will just make it impossible to use sudo. Your system can limp along in the +meantime or you can directly login as root or something to mend the situation, +but typoing the PAM file will cause glibc to hold your wife and children hostage +until you forcibly reboot the system and hack back into it so you can regain +control. + +![](/static/talks/surreal-horror-pam/012.jpeg) + +How is this relevant to us? Well, I have a bit of a side project going on. I’ve +been trying to write a PAM module that would use Tailscale as its authentication +method. + +When you are on a Tailscale network, you are already past a two-factor auth +trust barrier. If we know who you are, and you are authorized to connect to the +server by its ACLs, why should we subject you to the surreal horror of local +authentication logic in order to let you SSH into the server? We know who you +are. You’re allowed to connect to it, so why stop you? + +![](/static/talks/surreal-horror-pam/013.jpeg) + +The heart of the PAM module I’ve been writing looks like this right now. It sets +up syslog for its log sink (this is really your only good option in PAM land) +with a syslog client, grabs the status of the network from tailscaled, and +finally makes sure that the IP address is in the tailnet. This probably should +be more complicated in the future, I’ve had ideas for sending a TSMP message to +the source machine to prompt you with a “are you sure you want to do that” style +message, but they are just ideas right now. But yes, the rest is a bunch of +random boilerplate code to deal with PAM’s complexities, making sure that the C +ABI functions are exposed correctly and other helpers to grab things from +tailscaled with unix sockets. + +This is written in Rust because I personally believe that writing security +critical components that we would ship with the operating system in C is a +massive disservice to our users. Go also doesn’t really have a good story to do +interoperability with core C system components like this (the Go runtime is +_massive_ and as of writing this post the entire PAM module I’ve written is +smaller than the Go runtime, even with a statically compiled copy of libcurl). + +Plus I also get to use this to point out the little question mark at the end of +the third line of this code blurb. See that question mark? It is an “if err != +nil, return nil, err” statement. It’s handled at compile time and it will even +return the Ok side of the result if there is one. God I understand why Go can’t +have that nice thing but it would take at least 7 lines of code out of my +keyboard firmware if we had that nice thing in Go (not to mention countless +editor macros for other people). + +If you want to peek around the C module part of the PAM project, the QR code +will take you there. + +![](/static/talks/surreal-horror-pam/014.jpeg) + +I’m fairly sure that I can get away with this (I made my appropriate sacrifices +to the demo gods this morning), so let’s try SSHing into a VM on my laptop. If +you are watching the recording of this talk or you are not in the corp tailnet, +that command will not work. However you should see something like this: + +![](/static/talks/surreal-horror-pam/015.jpeg) + +It would be really cool to flesh this out as a full product. I feel this could +really make people’s lives a lot easier. The hard part is going to be making +sure that this absolutely has security experts pour over this to make sure that +this is _actually_ safe. I’m fairly sure that it is safe as it is, but right now +this is an uberhammer that lets you log in as root if you get SSH access to a +system. I would love to have this send a TSMP message to have a GUI prompt +validate that you want to do this as a kind of second factor for authentication, +but even in this limited state I feel it has a lot of value as is. + +![](/static/talks/surreal-horror-pam/016.jpeg) + +Something you may wonder (and something I had to wonder too) is how do you debug +PAM? + +![](/static/talks/surreal-horror-pam/017.jpeg) + +It ain’t easy. I’m currently trying to get this thing to work on Ubuntu and all +of the paths I have taken are fraught with despair. I have luckily not managed +to lock myself out of the system yet, but it is really fighting me. You know +you’re in for a ride when obscure PDFs of ring-binder manuals that have been +poorly maintained tell you to do things that literally do not exist anymore. +I’ve had to use a combination of a debugger and a system call tracing tool to +get anywhere with it. PAM is a surreal horror because the most terrifying part +is that it works and that there’s not really any good other options. + +![](/static/talks/surreal-horror-pam/018.jpeg) + +This is not OpenBSD or Plan 9. This is Linux and macOS. Those exist but we can’t +use them because we are cursed into using PAM. Especially so if we want to do +this on arbitrary customer machines. + +![](/static/talks/surreal-horror-pam/019.jpeg) + +That’s the end of the talk! I want to give special thanks to the council of +elders that I summoned the help of in order to get this far. Without their help +(and at least 800 bing points worth of searching) I would have never been able +to understand this at all. If you have any questions, you can ask them now; just +remember that you probably are still on mute. + +--- + +As a note to people who are reading: if you want my wit, charm and/or smarmy +style to grace your conference of choice, please [get in contact with +me](/contact) and I'll see what I can do to make it happen.