diff --git a/blog/dhall-kubernetes-2020-01-25.markdown b/blog/dhall-kubernetes-2020-01-25.markdown new file mode 100644 index 0000000..49bab77 --- /dev/null +++ b/blog/dhall-kubernetes-2020-01-25.markdown @@ -0,0 +1,435 @@ +--- +title: Dhall for Kubernetes +date: 2020-01-25 +tags: + - dhall + - kubernetes + - witchcraft +--- + +# Dhall for Kubernetes + +Kubernetes is a surprisingly complicated software package. Arguably, it has to +be that complicated as a result of the problems it solves being complicated; but +managing yaml configuration files for Kubernetes is a complicated task. [YAML][yaml] +doesn't have support for variables or type metadata. This means that the +validity (or sensibility) of a given Kubernetes configuration file (or files) +isn't easy to figure out without using a Kubernetes server. + +[yaml]: https://yaml.org + +In my [last post][cultk8s] about Kubernetes, I mentioned I had developed a tool +named [dyson][dyson] in order to help me manage Terraform as well as create +Kubernetes manifests from [a template][template]. This works for the majority of +my apps, but it is difficult to extend at this point for a few reasons: + +[cultk8s]: https://christine.website/blog/the-cult-of-kubernetes-2019-09-07 +[dyson]: https://github.com/Xe/within-terraform/tree/master/dyson +[template]: https://github.com/Xe/within-terraform/blob/master/dyson/src/dysonPkg/deployment_with_ingress.yaml + +- It assumes that everything passed to it is already valid yaml terms +- It doesn't assert the type of any values passed to it +- It is difficult to add another container to a given deployment +- Environment variables implicitly depend on the presence of a private git repo +- It depends on the template being correct more than the output being correct + +So, this won't scale. People in the community have created other solutions for +this like [Helm][helm], but a lot of them have some of the same basic problems. +Helm also assumes that your template is correct. [Kustomize][kustomize] does +help with a lot of the type-safe variable replacements, but it doesn't have the +ability to ensure your manifest is valid. + +[helm]: https://helm.sh +[kustomize]: https://kustomize.io + +I looked around for alternate solutions for a while and eventually found +[Dhall][dhall] thanks to a friend. Dhall is a _statically typed_ configuration +language. This means that you can ensure that inputs are _always_ the correct +type or the configuration file won't load. There's also a built-in +[dhall-to-yaml][dhallyaml] tool that can be used with the [Kubernetes +package][dhallk8s] in order to declare Kubernetes manifests in a type-safe way. + +[dhall]: https://dhall-lang.org +[dhallyaml]: https://github.com/dhall-lang/dhall-haskell/tree/master/dhall-yaml#dhall-yaml +[dhallk8s]: https://github.com/dhall-lang/dhall-kubernetes + +Here's a small example of Dhall and the yaml it generates: + +```dhall +-- Mastodon usernames +[ { name = "Cadey", mastodon = "@cadey@mst3k.interlinked.me" } +, { name = "Nicole", mastodon = "@sharkgirl@mst3k.interlinked.me" } +] +``` + +Which produces: + +```yaml +- mastodon: "@cadey@mst3k.interlinked.me" + name: Cadey +- mastodon: "@sharkgirl@mst3k.interlinked.me" + name: Nicole +``` + +Which is fine, but we still have the type-safety problem that you would have in +normal yaml. Dhall lets us define [record types][dhallrecord] for this data like +this: + +[dhallrecord]: http://www.haskellforall.com/2020/01/dhall-year-in-review-2019-2020.html + +```dhall +let User = + { Type = { name : Text, mastodon : Optional Text } + , default = { name = "", mastodon = None } + } + +let users = + [ User::{ name = "Cadey", mastodon = Some "@cadey@mst3k.interlinked.me" } + , User::{ + , name = "Nicole" + , mastodon = Some "@sharkgirl@mst3k.interlinked.me" + } + ] + +in users +``` + +Which produces: + +```yaml +- mastodon: "@cadey@mst3k.interlinked.me" + name: Cadey +- mastodon: "@sharkgirl@mst3k.interlinked.me" + name: Nicole +``` + +This is type-safe because you cannot add arbitrary fields to User instances +without the compiler rejecting it. Let's add an invalid "preferred_language" +field to Cadey's instance: + +``` +-- ... +let users = + [ User::{ + , name = "Cadey" + , mastodon = Some "@cadey@mst3k.interlinked.me" + , preferred_language = "en-US" + } + -- ... + ] +``` + +Which gives us: + +``` +$ dhall-to-yaml --file example.dhall +Error: Expression doesn't match annotation + +{ + preferred_language : … +, … +} + +4│ User::{ name = "Cadey", mastodon = Some "@cadey@mst3k.interlinked.me", +5│ preferred_language = "en-US" } + +example.dhall:4:9 +``` + +Or [this more detailed explanation][explanation] if you add the `--explain` flag +to the `dhall-to-yaml` call. + +[explanation]: https://clbin.com/JtVWT + +We tried to do something that violated the contract that the type specified. +This means that it's an invalid configuration and is therefore rejected and no +yaml file is created. + +The Dhall Kubernetes package specifies record types for _every_ object available +by default in Kubernetes. This does mean that the package is incredibly large, +but it also makes sure that _everything_ you could possibly want to do in +Kubernetes matches what it expects. In the [package +documentation][k8sdhalldocs], they give an example where a +[Deployment][k8sdeployment] is created. + +[k8sdhalldocs]: https://github.com/dhall-lang/dhall-kubernetes/tree/master/1.15#quickstart---a-simple-deployment +[k8sdeployment]: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/ + +``` dhall +-- examples/deploymentSimple.dhall + +-- Importing other files is done by specifying the HTTPS URL/disk location of +-- the file. Attaching a sha256 hash (obtained with `dhall freeze`) allows +-- the Dhall compiler to cache these files and speed up configuration loads +-- drastically. +let kubernetes = + https://raw.githubusercontent.com/dhall-lang/dhall-kubernetes/1.15/master/package.dhall + sha256:4bd5939adb0a5fc83d76e0d69aa3c5a30bc1a5af8f9df515f44b6fc59a0a4815 + +let deployment = + kubernetes.Deployment::{ + , metadata = kubernetes.ObjectMeta::{ name = "nginx" } + , spec = + Some + kubernetes.DeploymentSpec::{ + , replicas = Some 2 + , template = + kubernetes.PodTemplateSpec::{ + , metadata = kubernetes.ObjectMeta::{ name = "nginx" } + , spec = + Some + kubernetes.PodSpec::{ + , containers = + [ kubernetes.Container::{ + , name = "nginx" + , image = Some "nginx:1.15.3" + , ports = + [ kubernetes.ContainerPort::{ + , containerPort = 80 + } + ] + } + ] + } + } + } + } + +in deployment +``` + +Which creates the following yaml: + +``` +apiVersion: apps/v1 +kind: Deployment +metadata: + name: nginx +spec: + replicas: 2 + template: + metadata: + name: nginx + spec: + containers: + - image: nginx:1.15.3 + name: nginx + ports: + - containerPort: 80 +``` + +Dhall's lambda functions can help you break this into manageable chunks. For +example, here's a Dhall function that helps create a docker image reference: + +``` +let formatImage + : Text -> Text -> Text + = \(repository : Text) -> \(tag : Text) -> + "${repository}:${tag}" + +in formatImage "xena/christinewebsite" "latest" +``` + +Which outputs `xena/christinewebsite:latest` when passed to `dhall text`. + +All of this adds up into a powerful toolset that lets you express Kubernetes +configuration in a way that does what you want without as many headaches. + +Most of my apps on Kubernetes need only a few generic bits of configuration: + +- Their name +- What port should be exposed +- The domain that this service should be exposed on +- How many replicas of the service are needed +- Which Let's Encrypt Issuer to use (currently only `"prod"` or `"staging"`) +- The [configuration variables of the service][12factorconfig] +- Any other containers that may be needed for the service + +[12factorconfig]: https://12factor.net/config + +From here, I defined all of the [bits and pieces][kubermemeshttp] for the +Kubernetes manifests that Dyson produces and then created a `Config` type that +helps to template them out. Here's my [`Config` type +definition][configdefinition]: + +[kubermemeshttp]: https://tulpa.dev/cadey/kubermemes/src/branch/master/k8s/http +[configdefinition]: https://tulpa.dev/cadey/kubermemes/src/branch/master/k8s/app/config.dhall + +```dhall +let kubernetes = ../kubernetes.dhall + +in { Type = + { name : Text + , appPort : Natural + , image : Text + , domain : Text + , replicas : Natural + , leIssuer : Text + , envVars : List kubernetes.EnvVar.Type + , otherContainers : List kubernetes.Container.Type + } + , default = + { name = "" + , appPort = 5000 + , image = "" + , domain = "" + , replicas = 1 + , leIssuer = "staging" + , envVars = [] : List kubernetes.EnvVar.Type + , otherContainers = [] : List kubernetes.Container.Type + } + } +``` + +Then I defined a `makeApp` function that creates everything I need to deploy my +stuff on Kubernetes: + +```dhall +let Prelude = ../Prelude.dhall + +let kubernetes = ../kubernetes.dhall + +let typesUnion = ../typesUnion.dhall + +let deployment = ../http/deployment.dhall + +let ingress = ../http/ingress.dhall + +let service = ../http/service.dhall + +let Config = ../app/config.dhall + +let K8sList = ../app/list.dhall + +let buildService = + \(config : Config.Type) + -> let myService = service config + + let myDeployment = deployment config + + let myIngress = ingress config + + in K8sList::{ + , items = + [ typesUnion.Service myService + , typesUnion.Deployment myDeployment + , typesUnion.Ingress myIngress + ] + } + +in buildService +``` + +And used it to deploy the [h language website][hlang]: + +[hlang]: https://h.christine.website + +```dhall +let makeApp = ../app/make.dhall + +let Config = ../app/config.dhall + +let cfg = + Config::{ + , name = "hlang" + , appPort = 5000 + , image = "xena/hlang:latest" + , domain = "h.christine.website" + , leIssuer = "prod" + } + +in makeApp cfg +``` + +Which produces the following Kubernetes config: + +```yaml +apiVersion: v1 +items: + - apiVersion: v1 + kind: Service + metadata: + annotations: + external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" + external-dns.alpha.kubernetes.io/hostname: h.christine.website + external-dns.alpha.kubernetes.io/ttl: "120" + labels: + app: hlang + name: hlang + namespace: apps + spec: + ports: + - port: 5000 + targetPort: 5000 + selector: + app: hlang + type: ClusterIP + - apiVersion: apps/v1 + kind: Deployment + metadata: + name: hlang + namespace: apps + spec: + replicas: 1 + selector: + matchLabels: + app: hlang + template: + metadata: + labels: + app: hlang + name: hlang + spec: + containers: + - image: xena/hlang:latest + imagePullPolicy: Always + name: web + ports: + - containerPort: 5000 + imagePullSecrets: + - name: regcred + - apiVersion: networking.k8s.io/v1beta1 + kind: Ingress + metadata: + annotations: + certmanager.k8s.io/cluster-issuer: letsencrypt-prod + kubernetes.io/ingress.class: nginx + labels: + app: hlang + name: hlang + namespace: apps + spec: + rules: + - host: h.christine.website + http: + paths: + - backend: + serviceName: hlang + servicePort: 5000 + tls: + - hosts: + - h.christine.website + secretName: prod-certs-hlang +kind: List +``` + +And when I applied it on my Kubernetes cluster, it worked the first time and had +absolutely no effect on the existing configuration. + +In the future, I hope to expand this to allow for multiple deployments (IE: a +chatbot running in a separate deployment than a web API the chatbot depends on +or non-web projects in general) as well as supporting multiple Kubernetes +namespaces. + +Dhall is probably the most viable replacement to Helm or other Kubernetes +templating tools I have found in recent memory. I hope that it will be used by +more people to help with configuration management, but I can understand that +that may not happen. At least it works for me. + +If you want to learn more about Dhall, I suggest checking out the following +links: + +- [The Dhall Language homepage](https://dhall-lang.org) +- [Learn Dhall in Y Minutes](https://learnxinyminutes.com/docs/dhall/) +- [The Dhall Language GitHub Organization](https://github.com/dhall-lang) + +I hope this was helpful and interesting. Be well.