blog: I Miss Heroku's DevEx

Signed-off-by: Xe Iaso <me@christine.website>

blog/heroku-devex-2022-05-12.markdown

@@ -0,0 +1,102 @@
+---
+title: "I Miss Heroku's DevEx"
+date: 2022-05-12
+---
+
+If you've never really experienced it before, it's gonna sound really weird.
+Basically the main way that Heroku worked is that they would set up a git remote
+for each "app" it hosted. Each "app" had its source code in a git repo and a
+"Procfile" that told Heroku what to do with it. So when it came time to deploy
+that app, you'd just `git push heroku main` and then Heroku would just go off
+and build that app and run it _somewhere_ in the cloud. You got back a HTTPS URL
+and then bam you have a website.
+
+The developer experience didn't stop there. Most of how Heroku apps are
+configured are via environment variables, and there were addons that let you
+tell Heroku things like "hi yes I would like one (1) postgres please" and the
+platform would spin up a database somewhere and drop a config variable into the
+app's config. It was magic. Things just worked and it left you free to go do
+what made you money.
+
+Heroku's free tier got me the in I needed to make my career really start. If I
+didn't have something like Heroku in my life I doubt that my career would be the
+same or even I would be the same person I am today. It's really hard to describe
+what having access to a platform that lets you turn ideas into production
+quality code does to your output ability. I even ended up reinventing Heroku a
+few times in my career (working for Deis and later reinventing most of the core
+of Heroku as a project between jobs), but nothing really hit that same level of
+wonder/magic that Heroku did.
+
+I ended up working there and when I did I understood why Heroku had fallen so
+much. Heroku is owned by Salesforce and Salesforce doesn't really understand
+what they had acquired with Heroku. Heroku had resisted integration into the
+larger Salesforce organization and as a result was really really starved for
+headcount. I had to have a come-to-jesus meeting with the CTO of Heroku where I
+spelled out my medical needs and how the insurance that the contracting agency
+they were using was insufficent (showing comparisons between bills for blood
+draws where paying with the insurance ended up costing me more than not using
+it). I got hired and then that was just in time for Salesforce to really start
+pulling Heroku into the fold.
+
+The really great part about working at Heroku was that setting up a new service
+was so easy that the majority of the productionalization checklist was just
+enabling hidden feature flags to lock down the app. I'm surprised that didn't
+get streamlined.
+
+The Heroku I joined no longer exists. I joined Heroku but I left Salesforce. I
+can't blame any of my coworkers from Heroku from fleeing the sinking ship. The
+ship has been sinking for years but the culture of Heroku really stuck around
+long enough that it was hard to realize the shop was sinking.
+
+It can really be seen with how long it's taken Heroku to react to [that one
+horrible security event](https://status.heroku.com/incidents/2413) they've been
+dealing with. Based on what I remember about the internal architecture (it was a
+microservices tire fire unlike you have ever seen, it's part of the inspiration
+that lead me to write [this post](/blog/make-microservices-cluster-2022-01-27))
+and the notes that have been put on the public facing status page, I'm guessing
+that most of Heroku is "legacy" code (IE: nobody on the team that made this
+service works here anymore) at this point. When I was there most of the services
+on my team were "legacy" code that was production-facing, load-bearing and
+overall critical to the company succeeding; but it was built to be reliable
+enough that we could overall ignore it until it was actually falling over. But
+then because of the ways that things were chorded together it could take a very
+long time to actually fix issues because the symptoms were all over the place.
+
+Don't get me wrong, I loved working there but it was mostly for the people. That
+and the ability to say that I helped make Heroku better for the next generation.
+If you've ever used the metrics tab on Heroku, chances are that you've
+encountered my code indirectly. If you've ever done Heroku threshold autoscaling
+or response time alerting, you've dealt with code I helped write. The body of
+Heroku remains but the soul has long since fled.
+
+At the few points of my career that I have tried to reinvent Heroku (be it on my
+own or working for a company doing that), there has mostly been this weird
+realization that in order to have a thing like Heroku exist it really needs to
+be hosted by someone else in the cloud. One of the places I worked for was
+selling self-hosted Heroku on top of CoreOS and fleetd (remember fleetd? that
+was magical) and while it did have a lot of the same developer experience, it
+never really had the same magic feeling. I had the same problem with my own
+implementation. Sure you can get the app hosting part of Heroku fairly easily
+(and with Docker being as mature as it was at that point yeah it was fairly
+easy). But when it comes to the real experience of addons and the whole
+ecosystem there, you really need either to get very lucky or become an industry
+standard. Realistically though, you aren't going to be either lucky or an
+industry standard and then you need to also reinvent the next 80% of Heroku from
+scratch on hardware that you don't control. It's no wonder that ultimately
+failed (even though one of them was bought out by Microsoft after doing a weird
+Kubernetes pivot).
+
+There was something really magical about the whole thing that I really miss to
+this day. Heroku was at least a decade ahead of its time as far as developer
+experience goes. Things Just Worked in ways that would probably put a lot of us
+out of jobs if they really took off. I miss the process for putting something on
+the internet to just be a `git push` and trust that the machine will just take
+care of it. I wonder if we'll ever really have something like that on top of Nix
+or NixOS.
+
+---
+
+If you're reading this before the 12th, welcome to an experiment! I've been
+wondering about how to make some of my posts Patreon exclusive for a week. This
+post was published for my patrons on the 5th of May. Please don't share this
+link around on social media until the 12th, but privately sharing it is okay.