---
title: "Wasmcloud Progress: Rewritten in Rust"
date: 2020-10-31
series: olin
tags:
- wasm
- wasmcloud
- wasmer
---
# Wasmcloud Progress: Rewritten in Rust
It's been a while since I had the [last update for
Wasmcloud](/blog/wasmcloud-progress-2019-12-08). In that time I have gotten a
lot done. As the title mentions I have completely rewritten Wasmcloud's entire
stack in Rust. Part of the reason was for [increased
speed](/blog/pahi-benchmarks-2020-03-26) and the other part was to get better at
Rust. I also wanted to experiment with running Rust in production and this has
been an excellent way to do that.
Wasmcloud is going to have a few major parts:
- The API (likely to be hosted at `api.wasmcloud.app`)
- The Executor (likely to be hosted at `run.wasmcloud.lgbt`)
- The Panel (likely to be hosted at `panel.wasmcloud.app`)
- The command line tool `wasmcloud`
- The Documentation site (likely to be hosted at `docs.wasmcloud.app`)
These parts will work together to implement a functions as a service platform.
[The executor is on its own domain to prevent problems like this
GitHub Pages vulnerability from 2013. It is on a `.lgbt` domain because LGBT
rights are human rights.](conversation://Mara/hacker)
I have also set up a (slightly sarcastic) landing page at
[wasmcloud.app](https://wasmcloud.app) and a twitter account at
[@usewasmcloud](https://twitter.com/usewasmcloud). Right now these are
placeholders. I wanted to register the domains before they were taken by anyone
else.
## Architecture
My previous attempt at Wasmcloud had more of a four tier webapp setup. The
overall stack looked something like this:
- Nginx in front of everything
- The api server that did about everything
- The executors that waited on message queues to run code and push results to
the requester
- Postgres
- A message queue to communicate with the executors
- IPFS to store WebAssembly modules
In simple testing, this works amazingly. The API server will send execution
requests to the executors and everything will usually work out. However, the
message queue I used was very "fire and forget" and had difficulties with
multiple executors set up to listen on the queue. Additionally, the added
indirection of needing to send the data around twice means that it would have
difficulties scaling globally due to ingress and egress data costs. This model
is solid and _probably would have worked_ with some compression or other
improvements like that, but overall I was not happy with it and decided to scrap
it while I was porting the executor component to Rust. If you want to read the
source code of this iteration of Wasmcloud, take a look
[here](https://tulpa.dev/within/wasmcloud).
The new architecture of Wasmcloud looks something like this:
- Nginx in front of everything
- An API server that handles login with my gitea instance
- The executor server that listens over https
- Postgres
- Backblaze B2 to store WebAssembly modules
The main change here is the fact that the executor listens over HTTPS, avoiding
_a lot_ of the overhead involved in running this on a message queue. It's also
much simpler to implement and allows me to reuse a vast majority of the
boilerplate that I developed for the Wasmcloud API server.
This new version of Wasmcloud is also built on top of
[Wasmer](https://wasmer.io/). Wasmer is a seriously fantastic library for this
and getting up and running was absolutely trivial, even though I knew very
little Rust when I was writing [pa'i](/blog/pahi-hello-world-2020-02-22). I
cannot recommend it enough if you ever want to execute WebAssembly on a server.
## Roadmap
At this point, I can create new functions, upload them to the API server and
then trigger them to be executed. The output of those functions is not returned
to the user at this point. I am working on ways to implement that. There is also
very little accounting for what resources and system calls are used, however it
does keep track of execution time. The executor also needs to have the request
body of the client be wired to the standard in of the underlying module, which
will enable me to parse CGI replies from WebAssembly functions. This will allow
you to host HTTP endpoints on Wasmcloud using the same code that powers
[this](https://olin.within.website) and
[this](http://cetacean.club/cgi-bin/olinfetch.wasm).
I also need to go in and completely refactor the
[olin](https://github.com/Xe/pahi/tree/main/wasm/olin/src) crate and make the
APIs much more ergonomic, not to mention make the HTTP client actually work
again.
Then comes the documentation. Oh god there will be so much documentation. I will
be _drowning_ in documentation by the end of this.
I need to write the panel and command line tool for Wasmcloud. I want to write
the panel in [Elm](https://elm-lang.org/) and the command line tool in Rust.
There is basically zero validation for anything submitted to the Wasmcloud API.
I will need to write validation in order to make it safer.
I may also explore enabling support for [WASI](https://wasi.dev/) in the future,
but as I have stated before I do not believe that WASI works very well for the
futuristic plan-9 inspired model I want to use on Wasmcloud.
Right now the executor shells out to pa'i, but I want to embed pa'i into the
executor binary so there are fewer moving parts involved.
I also need to figure out what I should do with this project in general. It
feels like it is close to being productizable, but I am in a very bad stage of
my life to be able to jump in headfirst and build a company around this. Visa
limitations also don't help here.
## Things I Learned
[Rocket](https://rocket.rs) is an absolutely fantastic web framework and I
cannot recommend it enough. I am able to save _so much time_ with Rocket and its
slightly magic use of proc-macros. For an example, here is the entire source
code of the `/whoami` route in the Wasmcloud API:
```rust
#[get("/whoami")]
#[instrument]
pub fn whoami(user: models::User) -> Json {
Json(user)
}
```
The `FromRequest` instance I have on my database user model allows me to inject
the user associated with an API token purely based on the (validated against the
database) claims associated with the JSON Web Token that the user uses for
authentication. This then allows me to make API routes protected by simply
putting the user model as an input to the handler function. It's magic and I
love it.
Postgres lets you use triggers to automatically update `updated_at` fields for
free. You just need a function that looks like this:
```sql
CREATE OR REPLACE FUNCTION trigger_set_timestamp()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = NOW();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
```
And then you can make triggers for your tables like this:
```sql
CREATE TRIGGER set_timestamp_users
BEFORE UPDATE ON users
FOR EACH ROW
EXECUTE PROCEDURE trigger_set_timestamp();
```
Every table in Wasmcloud uses this in order to make programming against the
database easier.
The symbol/number layer on my Moonlander has been _so good_. It looks something
like this:
And it makes using programming sigils _so much easier_. I don't have to stray
far from the homerow to hit the most common ones. The only one that I still have
to reach for is `_`, but I think I will bind that to the blank key under the `]`
key.
The best programming music is [lofi hip hop radio - beats to study/relax
to](https://www.youtube.com/watch?v=5qap5aO4i9A). Second best is [Animal
Crossing music](https://www.youtube.com/watch?v=2nYNJLfktds). They both have
this upbeat quality that makes the ideas melt into code and flow out of your
hands.
---
Overall I'd say this is pretty good for a week of hacking while learning a new
keyboard layout. I will do more in the future. I have plans. To read through the
(admittedly kinda hacky/awful) code I've written this week, check out [this git
repo](https://tulpa.dev/wasmcloud/wasmcloud). If you have any feedback, please
[contact me](/contact). I will be happy to answer any questions.
As far as signups go, I am not accepting any signups at the moment. This is
pre-alpha software. The abuse story will need to be figured out, but I am fairly
sure it will end up being some kind of "pay or you can only run the precompiled
example code in the documentation" with some kind of application process for the
"free tier" of Wasmcloud. Of course, this is all theoretical and hinges on
Wasmcloud actually being productizable; so who knows?
Be well.