cadey/xesite
cadey
/
xesite
1
0
Fork 1
Code Issues Pull Requests 1 Projects Releases Wiki Activity

add API backend

This commit is contained in:
Cadey Ratio 2016-12-14 06:20:25 -08:00
parent 060a7c913a
commit 90f176019a
19 changed files with 2239 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

97
backend/christine.website/main.go Normal file
View File

@ -0,0 +1,97 @@
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
"os"
"path"
"path/filepath"
"strings"
"github.com/gernest/front"
)
type Post struct {
Title string `json:"title"`
Link string `json:"link"`
Summary string `json:"summary"`
Date string `json:"date"`
}
var posts []*Post
func init() {
err := filepath.Walk("./blog/", func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if info.IsDir() {
return nil
}
fin, err := os.Open(path)
if err != nil {
return err
}
defer fin.Close()
m := front.NewMatter()
m.Handle("---", front.YAMLHandler)
front, _, err := m.Parse(fin)
if err != nil {
return err
}
p := &Post{
Title: front["title"].(string),
Date: front["date"].(string),
Link: strings.Split(path, ".")[0],
}
posts = append(posts, p)
return nil
})
if err != nil {
panic(err)
}
}
func main() {
http.HandleFunc("/api/blog/posts", writeBlogPosts)
http.HandleFunc("/api/blog/post", func(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
name := q.Get("name")
fin, err := os.Open(path.Join("./blog", name))
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
defer fin.Close()
m := front.NewMatter()
m.Handle("---", front.YAMLHandler)
_, body, err := m.Parse(fin)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
fmt.Fprintln(w, body)
})
http.Handle("/dist/", http.FileServer(http.Dir("./frontend/static/")))
http.HandleFunc("/", writeIndexHTML)
log.Fatal(http.ListenAndServe(":9090", nil))
}
func writeBlogPosts(w http.ResponseWriter, r *http.Request) {
json.NewEncoder(w).Encode(posts)
}
func writeIndexHTML(w http.ResponseWriter, r *http.Request) {
http.ServeFile(w, r, "./frontend/static/dist/index.html")
}

122
blog/cinemaquestria-orchestration-2015-03-13.markdown Normal file
View File

@ -0,0 +1,122 @@
---
title: CinemaQuestria Orchestration
date: 2015-03-13
---
CinemaQuestria Orchestration
============================
### Or: Continuous Defenstration in a Container-based Ecosystem
I've been a core member of the staff for [CinemaQuestria](http://cinemaquestria.com)
for many months. In that time we have gone from shared hosting (updated by hand
with FTP) to a git-based deployment system that has won over the other
staffers.
In this blogpost I'm going to take a look at what it was, what it is, and what
it will be as well as some challenges that have been faced or will be faced as
things advance into the future.
The Past
--------
The site for CinemaQuestria is mostly static HTML. This was chosen mainly
because it made the most sense for the previous shared hosting environment as
it was the least surprising to set up and test.
The live site content is about 50 MB of data including PDF transcripts of
previous podcast episodes and for a long time was a Good Enough solution that
we saw no need to replace it.
However, being on shared hosting it meant that there was only one set of
authentication credentials and they had to be shared amongst ourselves. This
made sense as we were small but as we started to grow it didn't make much
sense. Combined with the fact that the copy of the site on the live server *was
pretty much the only copy of the site* we also lost disaster recovery points.
Needless to say, I started researching into better solutions for this.
The first solution I took a look at was AWS S3. It would let us host the CQ
site for about 0 dollars per month. On paper this looked amazing, until we
tried it and everyone was getting huge permissions issues. The only way to have
fixed this would have been to have everyone use the same username/password or
to have only one person do the deploys. In terms of reducing the [Bus
factor](https://en.wikipedia.org/wiki/Bus_factor) of the site's staff, this was
also unacceptable.
I had done a lot of work with [Dokku-alt](https://github.com/dokku-alt/dokku-alt)
for hosting my personal things (this site is one of many hosted on this
server), so I decided to give it a try with us.
The Present
-----------
Presently the CQ website is hosted on a Dokku-alt server inside a container.
For a while while I was working on getting the warts out only I had access to
deploy code to the server, but quickly on I set up a private repo on my git
server for us to be able to track changes.
Once the other staffers realized the enormous amount of flexibility being on
git gave us they loved it. From the comments I received the things they liked
the most were:
- Accountability for who made what change
- The ability to rollback changes if need be
- Everyone being able to have an entire copy of the site and its history
After the warts were worked out I gave the relevant people access to the dokku
server in the right way and the productivity has skyrocketed. Not only have
people loved how simple it is to push out new changes but they love how
consistent it is and the brutal simplicity of it.
Mind you these are not all super-technically gifted people, but the command
line git client was good enough that not only were they able to commit and make
changes to the site, but they also took initiative and *corrected things they
messed up* and made sure things were consistent and correct.
When I saw those commits in the news feed, I almost started crying tears of
happy.
Nowadays our site is hosted inside a simple [nginx
container](https://registry.hub.docker.com/_/nginx/). In fact, I'll even paste
the entire Dockerfile for the site below:
```Dockerfile
FROM nginx
COPY . /usr/share/nginx/html
```
That's it. When someone pushes a new change to the server it figures out
everything from just those two lines of code.
Of course, this isn't to say this system is completely free of warts. I'd love
to someday be able to notify the backrooms on skype every time a push to the
live server is made, but that might be for another day.
The Future
----------
In terms of future expansion I am split mentally. On one hand the existing
static HTML is *hysterically fast* and efficient on the server, meaning that
anything such as a Go binary, Lua/Lapis environment or other web application
framework would have a very tough reputation to beat.
I have looked into using Lapis for [this beta test site](http://cqsite-beta.apps.xeserv.us/),
but the fact that HTML is so dead easy to modify made that idea lose out.
Maybe this is in the realm of something like [jekyll](http://jekyllrb.com/),
[Hugo](http://gohugo.io/) or [sw](https://github.com/jroimartin/sw) to take
care of. I'd need to do more research into this when I have the time.
If you look at the website code currently a lot of it is heavily duplicated
code because the shared hosting version used to use Apache server-side
includes. I think a good place to apply these would be in the build in the
future. Maybe with a nice husking operation on build.
---
Anyways, I hope this was interesting and a look into a side of CinemaQuestria
that most of you haven't seen before. The Season 5 premiere is coming up soon
and this poor server is going to get hammered like nothing else, so that will
be a nice functional test of Dokku-alt in a production setting.

72
blog/coming-out-2015-12-01.markdown Normal file
View File

@ -0,0 +1,72 @@
---
title: Coming Out
date: 2015-12-01
---
Coming Out
==========
I'd like to bring up something that has been hanging over my head for a
long time. This is something I did try (and fail) to properly express way
back in middle school, but now I'd like to get it all of my chest and let
you know the truth of the matter.
I don't feel comfortable with myself as I am right now. I haven't really
felt comfortable with myself for at least 10 years, maybe more; I'm not
entirely sure.
At this point in my life I am really faced with a clear fork in the road.
I can either choose to continue living how I currently do, lying to myself
and others and saying everything is normal, or I can cooperate with the
reality that my brain is telling me that I don't feel comfortable with
myself as I have been for the last almost 22 years. I feel like I don't fit
inside my own skin. I think it is overall better for me to face the facts
and cooperate with reality. I have been repressing this off and on out of
fear of being shot down or not accepted the way I want to be seen to you
all. This has been a really hard thing for me to think through and even
harder for me to work up the courage to start taking action towards. This
is not a choice for me. I need to pursue this.
In fact, I have been pursing this. My current business cards reflect who
I really am. My co-workers accept my abnormal status (when compared to the
majority of society), and even will help stand up for me if something goes
south with regards to it.
I fully understand how much information this is to take in at once. I know
it will be difficult for you to hear that your firstborn son is actually a
daughter in a son's body, but I am still the same person. Most of the
changes that I want to pursue are purely cosmetic, but they are a bit more
noticeable than changing hair color. I feel that transitioning to living
as a woman like this will help me feel like I fit in with the world better
and help to make me more comfortable with who I am and how I want other
people to see me. Below I have collected some resources for you to look
through. They will help for you to understand my views better explained
in language you would be familiar with.
I have been trialing a lot of possible first names to use, Zoe (the name
you were going to give me if I was born a girl) did come to mind, but after
meditating on it for a while I have decided that it doesn't fit me at all.
The name I am going with for now and eventually will change my official
documents to use is Christine Cadence Dodrill.
Additionally I have been in a long-distance relationship with someone
since mid-June 2014. His name is Victor and he lives in Ottawa, Ontario.
He has been helping me a lot as I sort through all this; it has been a
godsend. He is a student in college for Computer Science. He knows and is
aware about my transition and has been a huge part of my emotional line
of support as I have been accepting these facts about who I am.
---
Above is (a snipped version of) the letter I sent to my parents in the
last 48 hours. With this I have officially come out to all of my friends
and family as transgender. I am currently on hormone replacement therapy
and have been living full time as a woman. My workplace is very accepting
of this and has been a huge help over the last 7-8 months as I have
battled some of my inner demons and decided to make things official.
I am now deprecating my old [facebook account](https://facebook.com/shadowh511)
and will be encouraging people to send friend requests and the like to my
[new account under the correct name](https://www.facebook.com/chrissycade1337).
Thank you all for understanding and be well.

281
blog/ffi-ing-golang-from-nim-for-fun-and-profit-2015-12-20.markdown Normal file
View File

@ -0,0 +1,281 @@
---
title: FFI-ing Golang from Nim for Fun and Profit
date: 2015-12-20
---
FFI-ing Golang from Nim for Fun and Profit
==========================================
As a side effect of Go 1.5, the compiler and runtime recently gained the
ability to compile code and run it as FFI code running in a C namespace. This
means that you can take any Go function that expresses its types and the like
as something compatible with C and use it from C, Haskell, Nim, Luajit, Python,
anywhere. There are some unique benefits and disadvantages to this however.
A Simple Example
----------------
Consider the following Go file `add.go`:
```go
package main
import "C"
//export add
func add(a, b int) int {
return a + b
}
func main() {}
```
This just exposes a function `add` that takes some pair of C integers and then
returns their sum.
We can build it with:
```
$ go build -buildmode=c-shared -o libsum.so add.go
```
And then test it like this:
```
$ python
>>> from ctypes import cdll
>>> a = cdll.LoadLibrary("./libsum.so")
>>> print a.add(4,5)
9
```
And there we go, a Go function exposed and usable in Python. However now we
need to consider the overhead when switching contexts from your app to your Go
code. To minimize context switches, I am going to write the rest of the code in
this post in [Nim](http://nim-lang.org) because it natively compiles down to
C and has some of the best C FFI I have used.
We can now define `libsum.nim` as:
```
proc add*(a, b: cint): cint {.importc, dynlib: "./libsum.so", noSideEffect.}
when isMainModule:
echo add(4,5)
```
Which when ran:
```
$ nim c -r libsum
Hint: system [Processing]
Hint: libsum [Processing]
CC: libsum
CC: system
Hint: [Link]
Hint: operation successful (9859 lines compiled; 1.650 sec total; 14.148MB; Debug Build) [SuccessX]
9
```
Good, we can consistently add `4` and `5` and get `9` back.
Now we can benchmark this by using the `times.cpuTime()` proc:
```
# test.nim
import
times,
libsum
let beginning = cpuTime()
echo "Starting Go FFI at " & $beginning
for i in countup(1, 100_000):
let myi = i.cint
discard libsum.add(myi, myi)
let endTime = cpuTime()
echo "Ended at " & $endTime
echo "Total: " & $(endTime - beginning)
```
```
$ nim c -r test
Hint: system [Processing]
Hint: test [Processing]
Hint: times [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: libsum [Processing]
CC: test
CC: system
CC: times
CC: strutils
CC: parseutils
CC: libsum
Hint: [Link]
Hint: operation successful (13455 lines compiled; 1.384 sec total; 21.220MB; Debug Build) [SuccessX]
Starting Go FFI at 0.000845
Ended at 0.131602
Total: 0.130757
```
Yikes. This takes 0.13 seconds to do the actual computation of every number
i in the range of `0` through `100,000`. I ran this for a few hundred times and
found out that it was actually consistently scoring between `0.12` and `0.2`
seconds. Obviously this cannot be a universal hammer and the FFI is very
expensive.
For comparison, consider the following C library code:
```
// libcsum.c
#include "libcsum.h"
int add(int a, int b) {
return a+b;
}
```
```
// libcsum.h
extern int add(int a, int b);
```
```
# libcsum.nim
proc add*(a, b: cint): cint {.importc, dynlib: "./libcsum.so", noSideEffect.}
when isMainModule:
echo add(4, 5)
```
and then have `test.nim` use the C library for comparison:
```
# test.nim
import
times,
libcsum,
libsum
let beginning = cpuTime()
echo "Starting Go FFI at " & $beginning
for i in countup(1, 100_000):
let myi = i.cint
discard libsum.add(myi, myi)
let endTime = cpuTime()
echo "Ended at " & $endTime
echo "Total: " & $(endTime - beginning)
let cpre = cpuTime()
echo "starting C FFI at " & $cpre
for i in countup(1, 100_000):
let myi = i.cint
discard libcsum.add(myi, myi)
let cpost = cpuTime()
echo "Ended at " & $cpost
echo "Total: " & $(cpost - cpre)
```
Then run it:
```
➜ nim c -r test
Hint: system [Processing]
Hint: test [Processing]
Hint: times [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: libcsum [Processing]
Hint: libsum [Processing]
CC: test
CC: system
CC: times
CC: strutils
CC: parseutils
CC: libcsum
CC: libsum
Hint: [Link]
Hint: operation successful (13455 lines compiled; 0.972 sec total; 21.220MB; Debug Build) [SuccessX]
Starting Go FFI at 0.00094
Ended at 0.119729
Total: 0.118789
starting C FFI at 0.119866
Ended at 0.12206
Total: 0.002194000000000002
```
Interesting. The Go library must be doing more per instance than just adding
the two numbers and continuing about. Since we have two near identical test
programs for each version of the library, let's `strace` it and see if there is
anything that can be optimized. [The Go one](https://gist.github.com/Xe/e0cd06d1d93e3299102e)
and [the C one](https://gist.github.com/Xe/7641cdba5657a4e8435a) are both very simple
and it looks like the Go runtime is adding the overhead.
Let's see what happens if we do that big loop in Go:
```
// add.go
//export addmanytimes
func addmanytimes() {
for i := 0; i < 100000; i++ {
add(i, i)
}
}
```
Then amend `libsum.nim` for this function:
```
proc addmanytimes*() {.importc, dynlib: "./libsum.so".}
```
And finally test it:
```
# test.nim
echo "Doing the entire loop in Go. Starting at " & $beforeGo
libsum.addmanytimes()
let afterGo = cpuTime()
echo "Ended at " & $afterGo
echo "Total: " & $(afterGo - beforeGo) & " seconds"
```
Which yields:
```
Doing the entire loop in Go. Starting at 0.119757
Ended at 0.119846
Total: 8.899999999999186e-05 seconds
```
Porting the C library to have a similar function would likely yield similar
results, as would putting the entire loop inside Nim. Even though this trick
was only demonstrated with Nim and Python, it will work with nearly any
language that can convert to/from C types for FFI. Given the large number of
languages that do have such an interface though, it seems unlikely that there
will be any language in common use that you *cannot* write to bind to Go code.
Just be careful and offload as much of it as you can to Go. The FFI barrier
**really hurts**.
---
This post's code is available [here](https://github.com/Xe/code/tree/master/experiments/go-nim).

148
blog/getting-started-with-go-2015-01-28.markdown Normal file
View File

@ -0,0 +1,148 @@
---
title: Getting Started with Go
date: 2015-01-28
---
Getting Started with Go
=======================
Go is an exciting language made by Google for systems programming. This article
will help you get up and running with the Go compiler tools.
System Setup
------------
First you need to install the compilers.
```console
$ sudo apt-get install golang golang-go.tools
```
`golang-go.tools` contains some useful tools that aren't part of the standard
Go distribution.
Shell Setup
-----------
Create a folder in your home directory for your Go code to live in. I use
`~/go`.
```console
$ mkdir -p ~/go/{bin,pkg,src}
```
`bin` contains go binaries that are created from `go get` or `go install`.
`pkg` contains static (`.a`) compiled versions of go packages that are not go
programs. `src` contains go source code.
After you create this, add
[this](https://github.com/Xe/dotfiles/blob/master/.zsh/go-completion.zsh) and
the following to your zsh config:
```sh
export GOPATH=$HOME/go
export PATH=$PATH:/usr/lib/go/bin:$GOPATH/bin
```
This will add the go compilers to your `$PATH` as well as programs you install.
Rehash your shell config (I use
a [`resource`](https://github.com/Xe/dotfiles/blob/master/.zsh/resource.zsh#L3)
command for this) and then run:
```console
$ go env
GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/xena/go"
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
TERM="dumb"
CC="gcc"
GOGCCFLAGS="-g -O2 -fPIC -m64 -pthread"
CXX="g++"
CGO_ENABLED="1"
```
This will verify that the go toolchain knows where the go compilers are as well
as where your `$GOPATH` is.
Testing
-------
To test the go compilers with a simple
[todo command](http://github.com/mattn/todo), run this:
```console
$ go get github.com/mattn/todo
$ todo add foo
$ todo list
☐ 001: foo
```
Vim Setup
---------
For Vim integration, I suggest using the
[vim-go](https://github.com/fatih/vim-go) plugin. This plugin used to be part
of the standard Go distribution.
To install:
1. Add `Plugin 'fatih/vim-go'` to the plugins part of your vimrc.
2. Run these commands:
```console
$ vim +PluginInstall +qall
$ vim +GoInstallBinaries +qall
```
This will install the go oracle and the go autocompletion daemon gocode as well
as some other useful tools that will integrate seamlessly into vim. This will
also run `gofmt` on save to style your code to the standard way to write Go
code.
Resources
---------
[Effective Go](https://golang.org/doc/effective_go.html) and the
[language spec](https://golang.org/ref/spec) provide a nice overview of the
syntax.
The Go [blog](http://blog.golang.org) contains a lot of detailed articles
covering advanced and simple Go topics.
[This page](https://golang.org/doc/#articles) has a list of past articles that
you may find useful.
The Go standard library is a fantastic collection of Go code for solving many
problems. In some cases you can even write entire programs using only the
standard library. This includes things like web application support, tarfile
support, sql drivers, support for most kinds of commonly used crypto, command
line flag parsing, html templating, and regular expressions. A full list of
the standard library packages can be found [here](http://godoc.org/-/go).
Variable type declarations will look backwards. It takes a bit to get used to
but makes a lot of sense once you realize it reads better left to right.
For a nice primer on building web apps with Go, codegangsta is writing a book
on the common first steps, starting from the standard library and working up.
You can find his work in progress book
[here](http://codegangsta.gitbooks.io/building-web-apps-with-go/).
Go has support for unit testing baked into the core language tools. You can
find information about writing unit tests [here](http://golang.org/pkg/testing/).
When creating a new go project, please resist the urge to make the folder in your
normal code folder. Drink the `$GOPATH` koolaid. Yes it's annoying, yes it's the
language forcing you to use its standard. Just try it. It's an amazingly useful
thing once you get used to it.
Learn to love godoc. Godoc lets you document code like
[this](https://gist.github.com/Xe/b973e30d81280899955d). This also includes an
example of the builtin unit testing support.

430
blog/metaprogramming-partial-application-2015-08-26.markdown Normal file
View File

@ -0,0 +1,430 @@
---
title: "Metaprogramming: Partial Application..."
date: 2015-08-26
---
Metaprogramming: Partial Application and Currying 101
=====================================================
The title of this post looks intimidating. There's a lot of words there that
look like they are very complicated and will take a long time to master. In
reality, they are really very simple things. Let's start with a mundane example
and work our way up to a real-world bit of code. Let's begin with a small
story:
---
ACMECorp has a world-renowned Python application named Itera that is known for
its superb handling of basic mathematic functions. It's so well known and
industry proven that it is used in every school and on every home computer. You
have just accepted a job there as an intermediate programmer set to do
maintenance on it. Naturally, you are very excited to peek under the hood of
this mysterious and powerful program and offer your input to make it even
better for the next release and its users.
Upon getting there, you settle in and look at your ticket queue for the day.
A user is complaining that whenever they add `3` and `5`, they get `7` instead
of `8`, which is what they expected. Your first step is to go look into the
`add3` function and see what it does:
```Python
def add1(x):
return x + 1
def add2(x):
return x + 2
def add3(x):
return x + 2
def add4(x):
return x + 4
```
You are aghast. Your company's multi-billion dollar calculator is brought to
its knees by a simple copy-paste error. You wonder, "how in Sam Hill are these
people making any money???" (The answer, of course, is that they are a big
enterprise corporation)
You let your boss know about the bad news, you are immediately given any
resource in the company that you need to get this mission-critical problem
solved for *any input*. Yesterday. Without breaking the API that the rest of
the program has hard-coded in.
---
Let's look at what is common about all these functions. The `add*` family of
functions seems to all be doing one thing consistently: adding one number to
another.
Let's define a function called `add` that adds any two numbers:
```Python
def add(x, y):
return x + y
```
This is nice, but it won't work for the task we were given, which is to not
break the API.
Let's go over what a function is in Python. We can define a function as
something that takes some set of Python values and produces some set of Python
values:
```haskell
PythonFunction :: [PythonValue] -> [PythonValue]
```
We can read this as "a Python function takes a set of Python values and
produces a set of Python values". Now we need to define what a Python value
actually is. To keep things simple, we're only going to define the following
types of values:
- `None` -> no value
- `Int` -> any whole number (Python calls this `int`)
- `Text` -> any string value (Python calls this `str`)
- `Function` -> something that takes and produces values
Python [itself has a lot more types that any value can be](https://docs.Python.org/3.4/library/stdtypes.html),
but for the scope of this blog post, this will do just fine.
Now, since a function can return a value and a function is a value, let's see
what happens if you return a function:
```python
def outer():
def inner():
return "Hello!"
return inner
```
And in the repl:
```
>>> type(outer)
<type 'function'>
```
So `outer` is a function as we expect. It takes `None` (in Python, a function
without arguments has `None` for the type of them) and returns a function that
takes `None` and that function returns `Text` containing `"Hello!"`.
Let's make sure of this:
```
>>> outer()()
'Hello!'
>>> type(outer()())
<type 'str'>
```
Yay! When nothing is applied to the result of applying nothing to `outer`, it
returns the `Text` value `"Hello!"`. We can define the type of `outer` as the
following:
```haskell
outer :: None -> None -> Text
```
Now, let's use this for addition:
```python
# add :: Int -> Int -> Int
def add(x):
def inner(y):
return x + y
return inner
```
And in the repl:
```
>>> add(4)(5)
9
```
A cool feature about this is that now we can dip into something called Partial
Application. Partial application lets you apply part of the arguments of
a function and you get another function out of it. Let's trace the type of the
`inner` function inside the `add` function, as well as the final computation
for clarity:
```python
# add :: Int -> Int -> Int
def add(x):
# inner :: Int -> Int
def inner(y):
return x + y # :: Int
return inner
```
Starting from the inside, we can see how the core computation here is `x + y`,
which returns an `Int`. Then we can see that `y` is passed in and in the scope
also as an `Int`. Then we can also see that `x` is passed in the outermost
layer as an int, giving it the type `Int -> Int -> Int`. Since `inner` is
a value, and a Python variable can contain any Python value, let's make
a function called `increment` using the `add` function:
```python
# increment :: Int -> Int
increment = add(1)
```
And in the repl:
```
>>> increment(50)
1
```
`increment` takes the integer given and increases it by 1, it is the same thing
as defining:
```python
def increment50():
return 51
```
Or even `51` directly.
Now, let's see how we can use this for the `add*` family of function mentioned
above:
```python
# add :: Int -> Int -> Int
def add(x):
def inner(y):
return x + y
return inner
# add1 :: Int -> Int
add1 = add(1)
# add2 :: Int -> Int
add2 = add(2)
# add3 :: Int -> Int
add3 = add(3)
# add4 :: Int -> Int
add4 = add(4)
```
And all we need to do from here is a few simple tests to prove it will work:
```python
if __name__ == "__main__":
assert add(1)(1) == 2 # 1 + 1
assert add(1)(2) == add(2)(1) # 1+2 == 2+1
print("all tests passed")
```
```console
$ python addn.py
all tests passed
```
Bam. The `add*` family of functions is now a set of partial applications. It is
just a set of half-filled out forms.
---
You easily mechanically rewrite all of the `add*` family of functions to use
the metaprogramming style you learned on your own. Your patch goes in for
consideration to the code review team. Meanwhile your teammates are frantically
going through every function in the 200,000 line file that defines the `add*`
family of functions. They are estimating months of fixing is needed not to
mention millions of lines of test code. They are also estimating an additional
budget of contractors being brought in to speed all this up. Your code has made
all of this unneeded.
Your single commit was one of the biggest in company history. Billboards that
were red are now beaming a bright green. Your code fixed 5,000 other copy-paste
errors that have existed in the product for years. You immediately get a raise
and live happily ever after, a master in your craft.
---
For fun, let's rewrite the `add` function in Haskell.
```haskell
add :: Int -> Int -> Int
add x y = x + y
```
And then we can create a partial application with only:
```haskell
add1 :: Int -> Int
add1 = (add 1)
```
And use it in the repl:
```
Prelude> add1 3
4
```
Experienced haskellers would probably gawk at this. Because functions are the
base data type in Haskell, and partial application means that you can make
functions out of functions, we can define `add` as literally the addition
operator `(+)`:
```haskell
add :: Int -> Int -> Int
add = (+)
```
And because operators are just functions, we can further simplify the `add1`
function by partially applying the addition operation:
```haskell
add1 :: Int -> Int
add1 = (+1)
```
And that will give us the same thing.
```
Prelude> let add1 = (+1)
Prelude> add1 3
4
```
---
Now, real world example time. I recently wrote a simple JSON api based off of
a lot of data that has been marginally useful to some people. This api has
a series of HTTP endpoints that return data about My Little Pony: Friendship is
Magic episodes. Its code is [here](https://github.com/Xe/ponyapi) and its
endpoint is `http://ponyapi.apps.xeserv.us`.
One of the challenges when implementing it was how to avoid a massive amount of
copy-pasted code when doing so. I had started with a bunch of functions like:
```python
# all_episodes :: IO [Episode]
def all_episodes():
r = requests.get(API_ENDPOINT + "/all")
if r.status_code != 200:
raise Exception("Not found or server error")
return r.json()["episodes"]
```
Which was great and all, but there was so much code duplication involved to
just get one result for all the endpoints. My first step was to write something
that just automated the getting of json from an endpoint in the same way
I automated addition above:
```python
# _base_get :: Text -> None -> IO (Either Episode [Episode])
def _base_get(endpoint):
def doer():
r = requests.get(API_ENDPOINT + endpoint)
if r.status_code != 200:
raise Exception("Not found or server error")
try:
return r.json()["episodes"]
except:
return r.json()["episode"]
# all_episodes :: IO [Episode]
all_episodes = _base_get("/all")
```
Where `_base_get` returned the function that satisfied the request.
This didn't end up working so well with the endpoints
[that take parameters](https://github.com/Xe/ponyapi#seasonnumber), so I had to
account for that in my code:
```python
# _base_get :: Text -> Maybe [Text] -> (Maybe [Text] -> IO (Either Episode [Episode]))
# _base_get takes a text, a splatted list of texts and returns a function such that
# the function takes a splatted list of texts and returns either an Episode or
# a list of Episode as an IO action.
def _base_get(endpoint, *fragments):
def doer(*args):
r = None
assert len(fragments) == len(args)
if len(fragments) == 0:
r = requests.get(API_ENDPOINT + endpoint)
else:
url = API_ENDPOINT + endpoint
for i in range(len(fragments)):
url = url + "/" + fragments[i] + "/" + str(args[i])
r = requests.get(url)
if r.status_code != 200:
raise Exception("Not found or server error")
try:
return r.json()["episodes"]
except:
return r.json()["episode"]
return doer
# all_episodes :: IO [Episode]
all_episodes = _base_get("/all")
# newest :: IO Episode
newest = _base_get("/newest")
# last_aired :: IO Episode
last_aired = _base_get("/last_aired")
# random :: IO Episode
random = _base_get("/random")
# get_season :: Int -> IO [Episode]
get_season = _base_get("", "season")
# get_episode :: Int -> Int -> IO Episode
get_episode = _base_get("", "season", "episode")
```
And that was it, save the `/search` route, which was acceptable to implement by
hand:
```python
# search :: Text -> IO [Episode]
def search(query):
params = {"q": query}
r = requests.get(API_ENDPOINT + "/search", params=params)
if r.status_code != 200:
raise Exception("Not found or server error")
return r.json()["episodes"]
```
---
Months later you have been promoted as high as you can go. You've been teaching
the other engineers at ACMECorp metaprogramming and even convinced management
to let the next big project be in Haskell.
You are set for life. You have won.
---
For comments on this article, please feel free to email me, poke me in `#geek`
on `irc.ponychat.net` (my nick is Xena), or leave thoughts at one of the below
places this article has been posted.
Comments:
- [Hacker News](https://news.ycombinator.com/item?id=10127389)
- [Reddit](https://www.reddit.com/r/programming/comments/3ijyyz/metaprogramming_partial_application_and_currying/)

100
blog/nim-and-tup-2015-06-10.markdown Normal file
View File

@ -0,0 +1,100 @@
---
title: Nim and Tup
date: 2015-06-10
---
Nim and Tup
===========
I have been recently playing with and using a new lanugage for
my personal development, [Nim](http://nim-lang.org). It looks like
Python, runs like C and integrates well into other things. Its
compiler targets C, and as a result of this binding things to C
libraries is a lot more trivial in Nim; even moreso than with go.
For example, here is a program that links to the posix `crypt(3)`
function:
```
# crypt.nim
import posix
{.passL: "-lcrypt".}
echo "What would you like to encrypt? "
var password: string = readLine stdin
echo "What is the salt? "
var salt: string = readLine stdin
echo "result: " & $crypt(password, salt)
```
And an example usage:
```
xena@fluttershy (linux) ~/code/nim/crypt
➜ ./crypt
What would you like to encrypt?
foo
What is the salt?
rs
result: rsHt73tkfd0Rg
```
And that's it. No having to worry about deferring to free the C
string, no extra wrappers (like with Python or Lua), you just
write the code and it just works.
At the idea of another coworker, I've also started to use
[tup](http://gittup.org/tup/) for building things. Nim didn't
initially work very well with tup (temporary cache needed, etc),
but a very simple set of tup rules were able to fix that:
```
NIMFLAGS += --nimcache:".nimcache"
NIMFLAGS += --deadcodeElim:on
NIMFLAGS += -d:release
NIMFLAGS += -d:ssl
NIMFLAGS += -d:threads
NIMFLAGS += --verbosity:0
!nim = |> nim c $(NIMFLAGS) -o:%o %f && rm -rf .nimcache |>
```
This creates a tup !-macro called `!nim` that will Do The Right
Thing implicitly. Usage of this is simple:
```
.gitignore
include_rules
: crypt.nim |> !nim |> ../bin/crypt
```
```
xena@fluttershy (linux) ~/code/nim/crypt
➜ tup
[ tup ] [0.000s] Scanning filesystem...
[ tup ] [0.130s] Reading in new environment variables...
[ tup ] [0.130s] No Tupfiles to parse.
[ tup ] [0.130s] No files to delete.
[ tup ] [0.130s] Executing Commands...
1) [0.581s] nim c --nimcache:".nimcache" --deadcodeElim:on --verbosity:0 crypt.nim && rm -rf .nimcache
[ ] 100%
[ tup ] [0.848s] Updated.
```
Not only will this build the program if needed, it will also
generate a gitignore for all generated files. This is an amazing
thing. tup has a lot more features (including lua support for
scripting complicated build logic), but there is one powerful
feature of tup that makes it very difficult for me to work into
my deployment pipelines.
tup requires fuse to ensure that no extra things are being
depended on for builds. Docker doesn't let you use fuse mounts
in the build process.
I have a few ideas on how to work around this, and am thinking
about tackling them when I get nim programs built inside Rocket
images.

150
blog/plt-1-the-beginning-2015-02-14.markdown Normal file
View File

@ -0,0 +1,150 @@
---
title: The Saga of plt, Part 1
date: 2015-02-14
---
The Saga of plt, Part 1
=======================
The following is adapted from a real story. Parts of it are changed to keep it
entertaining to read but the core of the story is maintained. I apologize that
this issue in the epic will be shorter than the others, but it gets better.
The Beginning of The Interesting Pain
-------------------------------------
It all started when I got this seemingly innocuous PM on Freenode:
```
2015-01-23 [18:32:48] <plt> Hello. I am writting a new ircd and can I have the channel ##ircd please?
```
This is a fairly common event on larger IRC networks, especially given the
length of the channel name and the fact that it references IRC daemons
specifically. At this point I had *forgotten* I owned that channel. So
naturally I decided to give it a join and see if the person who requested the
channel was worthy of it or had brought enough activity to it such that it was
morally correct to hand it off.
This was not the case.
```
[18:33:54] *** Joins: Xe (xe@unaffiliated/xe)
[18:34:02] <plt> Hello xe.
[18:35:17] <plt> Xe the project name pbircd.
[18:37:09] <plt> Xe the project site is http://sourceforge.net/p/pbircd
```
In case the site is removed from SourceForge, it is the default sourceforge
page.
After taking a look at this and then getting off the call with my family I was
on at the point, I decided to reply.
```
[20:30:49] <Xe> plt: I've decided against giving you my channel
[20:31:03] <Xe> you have no code in your repo.
[20:31:31] <plt> I am currently working on the project. Can I help you in the channel?
[20:32:04] <Xe> if you are working on it
[20:32:11] <Xe> I'd expect to see at least something
[20:32:25] <Xe> for example: https://github.com/Xe/scylla
[20:32:35] <Xe> that's mostly autogenerated code and makefiles, but it's something
[20:33:31] <plt> Take a look at this http://pastebin.com/F8MH3fSs
[20:34:04] <plt> You know it takes a while to write ircd code.
[20:34:16] <Xe> I don't see any commits
[20:34:20] <Xe> not even framework code
[20:34:24] <Xe> or design
[20:34:26] <Xe> or an outline
[20:34:30] <Xe> all I see is that pastebin
[20:34:39] <Xe> which is in no way connected to that git repo
[20:35:07] <plt> I am still adding more features so its not going to be posted on the main web site yet.
```
The contents of the pastebin looked like a changelog, but that pastebin has
since expired or was explicitly deleted. He was all talk and no game. I admit
at this point I was pretty tired and frustrated, so I told him off:
```
[20:35:19] <Xe> fucking commit it then
[20:35:52] <plt> I was going to wait until the code was completed.
[20:36:43] <Xe> yeah good lick then
[20:36:45] <Xe> luck*
[20:37:14] <plt> Itgoing to get done and I am the only one working on the project so what do you expect?
[20:37:29] <Xe> to be able to look at the in-progress code?
[20:39:24] <plt> The code will do you no good because you will not be able to compile it.
[20:39:51] <Xe> then you have nothing
[20:40:06] <plt> I am not required to approve it.
[20:41:08] <plt> I can post the run program on the web site.
[20:42:33] <Xe> then do that
[20:43:28] <plt> Done.
```
The "run program" was nothing but a wrapper around the nonexistent binary for
pbircd and seemed to be compiled in a language that doesn't respect assembly
functions and all of the forms of RE that I know how to do were useless. If you
know how to better do RE on arbitrary binaries please let me know.
```
[20:44:12] <Xe> there are binaries
[20:44:15] <Xe> not source code
[20:44:25] <Xe> this is what you use git for
[20:44:35] <plt> The source code will do you no good since you can not compile it.
[20:52:02] <plt> In order for you to compile it you need the encryption program and I am not going to release the source code.
[20:54:43] <Xe> lol
[20:55:34] <plt> The program is freeware and I have no obligation to release the code under the License agreement.
[21:00:56] <Xe> you also will get no users
[21:03:13] <plt> The company that wrote Conferenceroom has a lot of customers.
```
ConfrenceRoom was a company that made a commercial IRC daemon. They have lost
to Slack and other forms of chat like HipChat. Note here that he says "you can
not compile it". This is true in more ways than you would think. He also claims
it is Freeware and not full fledged open source software. As someone who is
slightly proactive and paranoid after the Snowden bullshit, I find this highly
suspect. However, this "encryption program" was something I was oddly most
interested in.
```
2015-01-24
[12:11:14] <plt> Xe why do you always demand to see the source code?
```
Curiosity? To learn from other people's ideas? To challenge myself in
understanding another way of thinking about things? To be able to improve it
for others to learn from? Those seem like good reasons to me.
```
[22:46:33] <plt> PBIRCD is a irc daemon.
[22:46:36] <plt> Hello xe
```
The `PB` in that name will become apparent later.
```
[23:09:31] <plt> Would you like to see what I have in the updates?
[23:09:40] <Xe> sure
[23:09:47] <plt> http://pastebin.com/2udHPSyP
[23:13:10] <plt> Tell me what you think about it?
[23:16:32] <plt> I need to take a short break.
```
Again, the paste is dead (I should really be saving these gems) but it was
another set of what appeared to be patch notes.
```
[23:22:37] <plt> Do you like what I have in the notes?
[23:23:49] <Xe> I still think it's ridiculous that you don't have the balls to release your code
[23:24:36] <plt> I understand what you telling me.
[23:25:48] <plt> There is no way to working around protecting the encrypted information.
[23:34:19] <plt> Why are you do want to see the code?
[23:43:36] <plt> Xe The encryption is used to encrypt the Operators, Link and the other passwords.
```
This sounds suspect. Any sane system of encrypting passwords like this would be
a mathematical one-way function. By not showing the code like this, is this
a two-way function?
```
2015-01-25
[00:05:55] <plt> Xe Question if the authors that wrote free pgp do not release their source code then why should I have do
```

205
blog/plt-2-entering-the-cave-2015-02-14.markdown Normal file
View File

@ -0,0 +1,205 @@
---
title: The Saga of plt, Part 2
date: 2015-02-14
---
The Saga of plt, Part 2
=======================
So I ended with a strong line of wisdom from `plt` last time. What if the
authors that wrote free PGP did not release their source code? A nice rehash of
the [Clipper Chip](https://en.wikipedia.org/wiki/Clipper_chip) anyone?
```
2015-01-25
[00:06:15] <Xe> but they did release their code
[00:06:40] <plt> I saw a few that did not release their source code.
[00:07:09] <plt> Its up to the author if they want to release it under the U.S Copyright Laws.
[00:08:50] <plt> http://copyright.gov/title17/circ92.pdf
```
Note that this is one of the few external links `plt` will give that actually
works. A lot of this belief in copyright and the like seems to further some
kind of delusional system involving everyone being out to steal his code and
profit off of it.
Please don't pay this person.
```
[00:57:18] <plt> The ircd follows the Internet Relay Protocols
[00:57:35] <Xe> which RFC's?
[00:57:43] <plt> Yep
[00:58:01] <plt> Accept for the IRCD Link works a little bit different.
[00:58:57] <plt> Version 2.0 or 3.0 will include it's own IRC Services that will work with PBIRCD.
[01:01:53] <plt> Later version will include open proxy daemon
[01:02:34] <plt> Version 1.00 will allow the ircd owner to define the irc command security levels which is a lot different from the other ircds.
[01:04:27] <plt> Xe that is the file /Conf/cmdlevs.conf.& the /Conf/userlevs.conf
[01:05:24] <plt> Adding a option for spam filtering may be included in the future version of PBIRCD.
[01:07:03] <plt> Xe PBIRCD will have not functions added to allow the operators to spy on the users.
```
Oh lord. Something you might notice quickly is that `plt` has no internal
filter nor ability to keep to one topic for very long. And that also `plt` has
some strange belief that folder names Should Start With Capital Letters, and
that apparently all configuration should be:
- split into multiple files
- put into the root of the drive
Also note that last line. Note it in bold.
Some time passed with no activity in the channel.
```
[18:50:49] <plt> Hey Xe
[18:51:06] <Xe> hi
[18:58:54] <plt> How did you like the information that I showed you yesterday?
[19:02:56] <Xe> it's useless to me
[19:03:03] <Xe> I don't run on a standard linux setup
[19:03:15] <Xe> I need source code to evaluate things
[19:03:17] <Xe> :P
```
When I am running unknown code, I use a virtual machine running [Alpine
Linux](http://alpinelinux.org). I literally do need the source code to be able
to run binaries as Alpine doesn't use glibc.
```
[19:04:24] <plt> It's the standard irc commands and I am still working on
adding some more features.
[19:04:38] <Xe> what language is it in?
[19:04:48] <Xe> how does it handle an accept() flood?
[19:09:17] <plt> Are you refering to accept() flood while connecting to the ircd or a channel?
[19:20:42] <plt> You can not compare some of the computer languages with C since some of they run at the same speed as C. Maybe some of them where a lot slower but in some cases that is not the same today!
```
These are some very simple questions I ask when evaluating a language or tool
for use in a project like an IRC server. How does it handle when people are
punishing it? So the obvious answer is to answer that some languages are
comparable to C in terms of execution speed!
How did I not see that before?
```
[19:26:05] <Xe> what language is it?
[19:27:23] <plt> Purebasic [...]
```
I took a look at the site for [PureBasic](http://www.purebasic.com). It looks
like Visual Basic's proprietary cousin as written by someone who hates
programmers. Looking at its feature set:
- Huge set of internal commands (1400+) to quickly and easily build any
application or game
- All BASIC keywords are supported
- Very fast compiler which creates highly optimized executables
- No external DLLs, runtime interpreter or anything else required when creating
executables
- Procedure support for structured programming with local and global variables
- Access to full OS API for advanced programmers
- Advanced features such as pointers, structures, procedures, dynamically
linked lists and much more
If you try to do everything, you will end up doing none of it. So it looks like
PureBasic is supposed to be a compiler for people who can't learn Go, Ruby,
Python, C, or Java. This looks promising.
I'm just going to paste the code for the 99 bottles of beer example. It
requires OOP. I got this from [Rosetta Code](http://rosettacode.org/wiki/99_Bottles_of_Beer/Basic#PureBasic).
```
Prototype Wall_Action(*Self, Number.i)
Structure WallClass
Inventory.i
AddBottle.Wall_Action
DrinkAndSing.Wall_Action
EndStructure
Procedure.s _B(n, Short=#False)
Select n
Case 0 : result$="No more bottles "
Case 1 : result$=Str(n)+" bottle of beer"
Default: result$=Str(n)+" bottles of beer"
EndSelect
If Not Short: result$+" on the wall": EndIf
ProcedureReturn result$+#CRLF$
EndProcedure
Procedure PrintBottles(*Self.WallClass, n)
Bottles$=" bottles of beer "
Bottle$ =" bottle of beer "
txt$ = _B(*Self\Inventory)
txt$ + _B(*Self\Inventory, #True)
txt$ + "Take one down, pass it around"+#CRLF$
*Self\AddBottle(*Self, -1)
txt$ + _B(*self\Inventory)
PrintN(txt$)
ProcedureReturn *Self\Inventory
EndProcedure
Procedure AddBottle(*Self.WallClass, n)
i=*Self\Inventory+n
If i>=0
*Self\Inventory=i
EndIf
EndProcedure
Procedure InitClass()
*class.WallClass=AllocateMemory(SizeOf(WallClass))
If *class
InitializeStructure(*class, WallClass)
With *class
\AddBottle =@AddBottle()
\DrinkAndSing =@PrintBottles()
EndWith
EndIf
ProcedureReturn *class
EndProcedure
If OpenConsole()
*MyWall.WallClass=InitClass()
If *MyWall
*MyWall\AddBottle(*MyWall, 99)
While *MyWall\DrinkAndSing(*MyWall, #True): Wend
;
PrintN(#CRLF$+#CRLF$+"Press ENTER to exit"):Input()
CloseConsole()
EndIf
EndIf
```
We are dealing with a professional language here folks. Their evaluation
version of the compiler didn't let me compile binaries and I'm not going to pay
$120 for a copy of it.
```
[19:27:23] <plt> Purebasic it does not make one bit of difference since it runs at the same speed as c
[19:27:44] <plt> The compiler was writting in asm.
[19:28:02] <Xe> pfffft
[19:28:04] <Xe> lol
[19:28:20] <Xe> I thought you would at least have used VB6
[19:28:37] <plt> VB6 is so old dude.
```
At least there is some sense there.
```
[19:28:44] <Xe> so is purebasic
[19:28:54] <plt> You can not compare purebasic with the other basic compilers.
[19:29:51] <Xe> yes I can
[19:29:56] <Xe> seeing as you post no code
[19:29:59] <Xe> I can and I will
[19:30:16] <plt> Makes no logic what you said.
[19:30:24] <Xe> I'm saying prove it
[19:31:18] <plt> I am not going to give out the source code because of the encryption and no one has any reason to use it to decrypt the other irc networks passwords or traffic.
[19:31:40] <Xe> so you've intentionally backdoored it to allow you to have access?
[19:32:00] <plt> I dn not trust anyone any more.
[19:32:29] <plt> Not after the nsa crap going on.
[19:32:50] <Xe> so, in order to prove you don't trust anyone
[19:33:06] <Xe> you've intentionally backdoored the communications server you've created and intend to sell to people?
[19:33:37] <Xe> also
[19:33:45] <Xe> purebasic is semantically similar to vb
[19:34:06] <plt> There is no backdoors included in the source code. A course if a user gets a virus or hacked that is not going to be my fault.
```

116
blog/pursuit-of-dsl-2014-08-16.markdown Normal file
View File

@ -0,0 +1,116 @@
---
title: Pursuit of a DSL
date: 2014-08-16
---
Pursuit of a DSL
================
A project we have been working on is [Tetra](http://github.com/Xe/Tetra). It is
an extended services package in Go with Lua and Moonscript extensions. While
writing Tetra, I have found out how to create a Domain Specific Language, and
I would like to recommend Moonscript as a toolkit for creating DSL's.
[Moonscript](http://moonscript.org) is a high level wrapper around Lua designed
to make programming easier. We have used Moonscript heavily in Tetra because of
how easy it is to make very idiomatic code in it.
Here is some example code from the Tetra codebase for making a command:
```moonscript
require "lib/elfs"
Command "NAMEGEN", ->
"> #{elfs.GenName!\upper!}"
```
That's it. That creates a command named `NAMEGEN` that uses `lib/elfs` to
generate goofy heroku-like application names based on names from [Pokemon Vietnamese Crystal](http://tvtropes.org/pmwiki/pmwiki.php/JustForFun/PokemonVietnameseCrystal).
In fact, because this is so simple and elegant, you can document code like this
inline.
## Command Tutorial
In this file we describe an example command `TEST`. `TEST` will return some
information about the place the command is used as well as explain the
arguments involved.
Because Tetra is a polyglot of Lua, Moonscript and Go, the relevant Go objects
will have their type definitions linked to on [godoc](http://godoc.org)
Declaring commands is done with the `Command` macro. It takes in two arguments.
1. The command verb
2. The command function
It also can take in 3 arguments if the command needs to be restricted to IRCops
only.
1. The command verb
2. `true`
3. The command function
The command function can have up to 3 arguments set when it is called. These
are:
1. The [Client](https://godoc.org/github.com/Xe/Tetra/bot#Client) that
originated the command call.
2. The [Destination](https://godoc.org/github.com/Xe/Tetra/bot#Targeter) or
where the command was sent to. This will be a Client if the target is an
internal client or
a [Channel](https://godoc.org/github.com/Xe/Tetra/bot#Channel) if the target
is a channel.
3. The command arguments as a string array.
```moonscript
Command "TEST", (source, destination, args) ->
```
All scripts have `client` pointing to the pseudoclient that the script is
spawned in. If the script name is `chatbot/8ball`, the value of `client` will
point to the `chatbot` pseudoclient.
```moonscript
client.Notice source, "Hello there!"
```
This will send a `NOTICE` to the source of the command saying "Hello there!".
```moonscript
client.Notice source, "You are #{source.Nick} sending this to #{destination.Target!} with #{#args} arguments"
```
All command must return a string with a message to the user. This is a good
place to do things like summarize the output of the command or if it worked or
not. If the command is oper-only, this will be the message logged to the
services snoop channel.
```moonscript
"End of TEST output"
```
See? That easy.
```moonscript
Command "TEST", ->
"Hello!"
```
This is much better than Cod's
```python
#All modules have a name and description
NAME="Test module"
DESC="Small example to help you get started"
def initModule(cod):
cod.addBotCommand("TEST", testbotCommand)
def destroyModule(cod):
cod.delBotCommand("TEST")
def testbotCommand(cod, line, splitline, source, destination):
"A simple test command"
return "Hello!"
```

94
blog/the-origin-of-h-2015-12-14.markdown Normal file
View File

@ -0,0 +1,94 @@
---
title: The Origin of h
date: 2015-12-14
---
The Origin of h
===============
For a while I have been pepetuating a small joke between my friends, co-workers and community members of various communities (whether or not this has been beneficial or harmful is out of the scope of this post). The whole "joke" is that someone says "h", another person says "h" back.
That's it.
This has turned into a large scale game for people, and is teachable to people with minimal explanation. Most of the time I have taught it to people by literally saying "h" to them until they say "h" back. An example:
```
<Person> Oh hi there
<Xena> h
<Person> ???
<Xena> Person: h
<Person> i
<Xena> Person:
<Xena> h
<Person> h
<Xena> :D
```
Origins
-------
This all started on a particularly boring day when we found a video by [motdef](https://www.youtube.com/user/motdef) with gameplay from [Moonbase Alpha](https://www.nasa.gov/offices/education/programs/national/ltp/games/moonbasealpha/index.html), an otherwise boring game made to help educate people on what would go on when a moonbase has a disaster. This game was played by many people because of its [text-to-speech engine](http://knowyourmeme.com/memes/moonbase-alpha-text-to-speech), which lead to many things like flooding "JOHN MADDEN" or other inane things like that.
Specifically there was a video called ["Moonbase 4lpha: *****y Space Skeletons"](https://www.youtube.com/watch?v=SnTludRdZDw) that at one point had recorded the phrase "H H H RETURN OF GANON". Me and a few friends were flooding that in an IRC room for a while and it eventually devolved into just flooding "h" to eachother. The flooding of "h" lasted over 8 hours (we were really bored) and has evolved into the modern "h" experience we all know and love today.
The IRC Bot
-----------
Of course, humans are unreliable. Asking them to do things predictably is probably a misguided idea so it is best to automate things with machines whenever it is pragmatic to do so. As such, I have created and maintained the following python code that automates this process. An embarassing amount of engineering and the like has gone into making sure this function provides the most correct and canonical h experience money can buy.
```
@hook.regex(r"^([hH])([?!]*)$")
def h(inp, channel=None, conn=None):
suff = ""
if inp.group(2).startswith("?"):
suff = inp.group(2).replace("?", "!")
elif inp.group(2).startswith("!"):
suff = inp.group(2).replace("!", "?")
return inp.group(1) + suff
```
The [code was pulled from here](https://git.xeserv.us/xena/h/src/f33fad269cc2c900079bae1e5bfc0b1f5536b223/plugins/shitposting.py#L7-L14).
Here is an example of it being used:
```
(Xena) h
(h) > h
(Xena) h???
(h) > h!!!
(Xena) h!!!!
(h) > h????
-- [h] (h@h): h
-- [h] is using a secure connection
-- [h] is a bot
-- [h] is logged in as h
```
I also ended up porting h to matrix under the name [`h2`](https://git.xeserv.us/xena/h2). It currently sits in `#ponydevs:matrix.org` and has a bad habit of getting broken because [Comcast is a bad company](http://comcast.apps.xeserv.us/) and doesn't believe in uptime.
Spread of h
-----------
Like any internet meme, it is truly difficult to see how far it has spread with 100% certainty. However I have been keeping track of where and how it has spread, and I can estimate there are at least 50 guardians of the h.
However, its easily teachable nature and very minimal implementation means that new guardians of the h can be created near instantly. It is a lightweight meme but has persisted for at least 2 years. This means it is part of internet culture now, right?
There has been one person in the [Derpibooru](https://derpibooru.org) IRC channel that is really violently anti-h and has a very humorous way of portraying this. Stop in and idle and you'll surely see it in action.
Conclusion
----------
I hope this helps clear things up on this very interesting and carefully researched internet meme. I hope to post further updates as things become clear on this topic.
---
Below verbatim is the forum post (it was deleted, then converted to a [blog post](http://parclytaxel.tumblr.com/post/135227842874/derpibooru-xena-h) on his blog) that inspired the writing of this article.
> > [Parcly Taxel](http://parclytaxel.tumblr.com/)
>
> Lately, if you’ve been going up to our [Derpibooru](https://derpibooru.org) [IRC channel](https://derpibooru.org/irc), you may notice that a significant portion of sayings and rebuttals are countered with the single letter h (lowercase). So where does this come from?
>
> This is a joke started by Xena, one of the administrators of the [Ponychat](https://ponychat.net) IRC system which the site uses. It came from a [video showing gameplay, glitches and general tomfoolery in the simulation game Moonbase Alpha](https://www.youtube.com/watch?v=SnTludRdZDw). Starting from 1:32 there is shown a dialogue between two players, one of which makes grandiose comments about how they will "eradicate" everyone else, to which the other simply replies "h" or multiples of it.
>
> Hence when h is spoken in IRC, do know that it’s a shorthand for "yes and I laugh at you". I do not recommend using it though as it could be confused with hydrogen or UTC+8 (the time zone in which I live).

150
blog/the-universal-design-2015-10-17.markdown Normal file
View File

@ -0,0 +1,150 @@
---
title: The Universal Design
date: 2015-10-17
---
The Universal Design
====================
As I have been digging through existing code, systems and the like I have been wondering what the next big direction I should go in is. How to design things such that the mistakes of the past are avoided, but you can benefit from them and learn better how to avoid them. I have come to a very simple conclusion, monoliths are too fragile.
## Deconstructing Monoliths
One monolith I have been maintaining is [Elemental-IRCd](http://elemental-ircd.com). Taking the head of a project I care about has taught me more about software engineering, community/project management and the like than I would have gotten otherwise. One of these things is that there need to be five basic primitives in your application:
1. State - What is true now? What was true? What happened in the past? What is the persistent view of the world?
2. Events - What is being changed? How will it be routed?
3. Policy - Can a given event be promoted into a series of actions?
4. Actions - What is the outcome of the policy?
5. Mechanism - How should an event be taken in and an action put out?
Let's go over some basic examples of this theory in action:
### Spinning up a Virtual Machine
- the event is that someone asked to spin up a virtual machine
- the policy is do they have permission to spin that machine up?
- the mechanism is an IRC command for some reason
- the action is that a virtual machine is created
- the state is changed to reflect that VM creation
### Webserver
- the event is an HTTP request
- the policy is to do some database work and return the action of showing the HTML to the user
- the mechanism is nginx sending data to a worker and relaying it back
- the state is updated for whatever changed
And that's it. All you need is a command queue feeding into a thread pool which feeds out into a transaction queue which modifies state. And with that you can explain everything from VMWare to Google.
As a fun addition, we can also define nearly all of this as being functionally pure code. The only thing that really needs to be impure are mechanisms and applying actions to the state. Policy handlers should be mostly if not entirely pure, but also may need to access state not implicitly passed to it. The only difference between an event and an action is what they are called.
## Policy
Now, how would a policy handler work? I am going to be explaining this in the context of an IRC daemon as that is what I intend to develop next. Let's sketch out the low level:
The relevant state is the state of the IRC network. An event is a command from a user or server. A policy is a handler for either a user command or another kind of emitted action from another policy handler.
One of the basic commands in RFC 1459 is the `NICK` command. A user using it passes the new nickname they want. Nicknames must also be unique.
```lua
-- nick-pass-1.lua
local mephiles = require "mephiles"
mephiles.declareEvent("user:NICK", function(state, source, args)
if #args ~= 1 then
return {
{mepliles.failure, {mephiles.pushNumeric, source, mephiles.errCommandBadArgc(1)}}
}
end
local newNick = args[1]
if state.nicks.get(newNick) then
return {
{mephiles.failure, {mephiles.pushNumeric, source, mephiles.errNickInUse(newNick)}}
}
end
if not mephiles.legalNick(newNick) then
return {
{mephiles.failure, {mephiles.pushNumeric, source, mephiles.errIllegalNick(newNick)}}
}
end
return {
{mephiles.success, {"NICKCHANGE", source, newNick}}
}
end)
```
This won't scale as-is, but most of this is pretty straightforward. The policy function returns a series of actions that fall into two buckets: success and failure. Most of the time the success of state changes (nickname change, etc) will be confirmed to the client. However a large amount of the common use (`PRIVMSG`, etc) will be unreported to the client (yay RFC 1459); but every single time a line from a client fails to process, the client must be notified of that failure.
Something you can do from here is define a big pile of constants and helpers to make this easier:
```lua
local actions = require "actions"
local c = require "c"
local m = require "mephiles"
local utils = require "utils"
m.UserCommand("NICK", c.normalFloodLimit, function(state, source, args)
if #args ~= 1 then
return actions.failCommand(source, "NICK", c.errCommandBadArgc(1))
end
local newNick = args[1]
if state.findTarget(newNick) then
return actions.failCommand(source, "NICK", c.errNickInUse(newNick))
end
if not utils.legalNick(newNick) then
return actions.failCommand(source, "NICK", c.errIllegalNick(newNick))
end
return {actions.changeNick(source, newNick)}
end)
```
## Thread Safety
This as-is is very much not thread-safe. For one the Lua library can only have one thread interacting with it at a time, so you will need a queue of events to it. The other big problem is that this is prone to race conditions. There are two basic solutions to this:
1. The core takes a lock on all of the state at once
2. The policy handlers take a lock on resources as they try to use them and the core automatically releases locks at the end of it running.
The simpler implementation will do for an initial release, but the latter will scale a lot better as more and more users hit the server at the same time. It allows unrelated things to be changed at the same time, which is the majority case for IRC.
In the future, federation of servers can be trivialized by passing the actions from one server to another if it is needed, and by implicitly trusting the actions of a remote server.
---
This design will also scale to running across multiple servers, and in general to any kind of computer, business or industry problem.
What if this was applied to the CPU and a computer in general at a low level? How would things be different?
## Urbit
Over the past few weeks I have been off and on dipping my toes into [Urbit](http://urbit.org). They call Urbit an "operating function" and define it [as such](http://urbit.org/preview/~2015.9.25/materials/whitepaper#-definition):
V(I) => T
where `T` is the state, `V` is the fixed function, and `I` is the list of input events from first to last.
Urbit at a low level takes inputs, applies them to a function and returns the state of the computer. Sound familar?
`~hidduc-posmeg` has been putting together a set of tutorials^\* to learn Hoon, its higher-level lisp-like language. At the end of the first one, they say something that I think is also very relevant to this systems programming ideal:
> All Hoon computation takes [the] same general form. A subject with a fomula that transforms that subject in some way to produce a product which is then used as the subject for some other formula. In our next tutorial we'll look at some of the things we can do to our subject.
Subjects applied to formulae become results that are later applied to formulae as subjects. Events applied to policy emit actions which later become events for other policies to emit actions.
Because of this design, you can easily do live code reloading, because there is literally no reason you can't. Wait for a formula to finish and replace it with the new version, provided it compiles. Why not apply this to the above ideas too?
---
\* Link here: http://hidduc-posmeg.urbit.org/home/pub/hoon-intro/ as of publishing this revision of the article hidduc's urbit is offline, so they cannot be accessed at the moment. If that link fails, the source code for it is apparently [here](https://bitbucket.org/zaphar/hoon-intro/src/6a69c2048036f3e6d4ea1ef065577adf0b21be5f/0/hymn.hook?at=default&fileviewer=file-view-default). Thanks `mst` on Freenode!
For comments on this article, please feel free to email me, poke me in `#geek` on `irc.ponychat.net` (my nick is Xena, on freenode it is Xe), or leave thoughts at one of the places this article has been posted.

85
blog/this-site-text-stack-2015-02-14.markdown Normal file
View File

@ -0,0 +1,85 @@
---
title: This Site's Tech Stack
date: 2015-02-14
---
This Site's Tech Stack
======================
> Note: this is out of date as this site now uses [PureScript](http://www.purescript.org/) and [Go](https://golang.org).
As some of my close friends can vouch, I am known for sometimes setting up and
using seemingly bizarre tech stacks for my personal sites. As such I thought it
would be interesting to go in and explain the stack I made for this one.
The Major Players
-----------------
### Markdown
This is a markdown file that gets rendered to HTML and sent to you via the lua
discount library. As I couldn't get the vanilla version from LuaRocks to work,
I use Debian's version.
I like Markdown for thigns like this as it is not only simple, but easy for
people to read, even if they don't know markdown or haven't worked with any
other document system than Office or other wisywig document processors.
### Lapis
Lapis is the middleware between Lua and Nginx that allows me to write pages
simply. Here is some of the code that powers this page:
```
-- controllers/blog.moon
class Blog extends lapis.Application
["blog.post": "/blog/:name"]: =>
@name = util.slugify @params.name
@doc = oleg.cache "blogposts", @name, ->
local data
with io.open "blog/#{@name}.markdown", "r"
data = \read "*a"
discount data, "toc", "nopants", "autolink"
with io.open "blog/#{@name}.markdown", "r"
@title = \read "*l"
render: true
```
And the view behind this page:
```
-- views/blog/post.moon
import Widget from require "lapis.html"
class Post extends Widget
content: =>
raw @doc
```
That's it. That even includes the extra overhead of caching the markdown as
HTML in a key->value store called OlegDB (I will get into more detail about it
below). With Lapis I can code faster and be much more expressive with a lot
less code. I get the syntactic beauty that is Moonscript with the speed and raw
power of luajit on top of nginx.
### OlegDB
OlegDB is a joke about mayonnaise that has gone too far. It has turned into
a full fledged key->value store and I think it is lovely.
### Container Abuse
I have OlegDB running as an in-container service. This means that OlegDB does
hold some state, but only for things that are worth maintaining the stats of
(in my eyes). Having a cache server right there that you can use to speed
things up with is a brilliant abuse of the fact that I run a container that
allows me to do that. I have Oleg hold the very HTML you are reading right now!
When it renders a markdown file for the first time it caches it into Oleg, and
then reuses that cached version when anyone after the first person reads the
page. I do the same thing in a lot of places in the codebase for this site.
---
I hope this look into my blog's tech stack was interesting!

98
blog/thoughts-on-community-2014-07-31.markdown Normal file
View File

@ -0,0 +1,98 @@
---
title: Thoughts on Community Management
date: 2014-07-31
---
Thoughts on Community Management
================================
Many open source community projects lack proper management. They can put too
much of their resources in too few places. When that one person falls out of
contact or goes rogue on everyone, it can have huge effects on everyone
involved in the project. Users, Contributors and Admins.
Here, I propose an alternative management structure based on what works.
## Organization
Contributors and Project Administrators are there to take input/feedback from
Users, rectify the situation or explain why doing so is counterproductive.
Doing so will be done kindly and will be ran through at least another person
before it is posted publicly. This includes (but is not limited to) email, IRC,
forums, anything. A person involved in the project is a representative of it.
They are the face of it. If they are rude it taints the image of everyone
involved.
## Access
Project Administrators will have full, unfiltered access to anything the
project has. This includes root access, billing access, everything. There will
be no reason to hide things. Operational conversations will be shared. All
group decisions will be voted on with a simple Yes/No/Abstain process. As such
this team should be kept small.
## Contributions
Contributors will have to make pull requests, as will Administrators. There
will be review on all changes made. No commits will be pushed to master by
themselves unless there is approval. This will allow for the proper review and
testing procedures to be done to all code contributed.
Additionally, for ease of scripts scraping the commits when something is
released, a commit style should be enforced.
### Commit Style
The following section is borrowed from [Deis' commit
guidelines](https://github.com/deis/deis/blob/master/CONTRIBUTING.md).
---
We follow a rough convention for commit messages borrowed from CoreOS, who borrowed theirs
from AngularJS. This is an example of a commit:
feat(scripts/test-cluster): add a cluster test command
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
To make it more formal, it looks something like this:
{type}({scope}): {subject}
<BLANK LINE>
{body}
<BLANK LINE>
{footer}
The {scope} can be anything specifying place of the commit change.
The {subject} needs to use imperative, present tense: “change”, not “changed”
nor “changes”. The first letter should not be capitalized, and there is no dot
(.) at the end.
Just like the {subject}, the message {body} needs to be in the present tense,
and includes the motivation for the change, as well as a contrast with the
previous behavior. The first letter in a paragraph must be capitalized.
All breaking changes need to be mentioned in the {footer} with the description
of the change, the justification behind the change and any migration notes
required.
Any line of the commit message cannot be longer than 72 characters, with the
subject line limited to 50 characters. This allows the message to be easier to
read on github as well as in various git tools.
The allowed {types} are as follows:
feat -> feature
fix -> bug fix
docs -> documentation
style -> formatting
ref -> refactoring code
test -> adding missing tests
chore -> maintenance
---
I believe that these guidelines would lead towards a harmonious community.

56
blog/trying-vagga-2015-03-21.markdown Normal file
View File

@ -0,0 +1,56 @@
---
title: Trying Vagga on For Size
date: 2015-03-21
---
Trying Vagga on For Size
========================
[Vagga](https://github.com/tailhook/vagga) is a containerization tool like
Docker, Rocket, etc but with one major goal that is highly ambitious and really
worth mentioning. Its goal is to be a single userspace binary without a suid
bit or a daemon running as root.
However, the way it does this seems to be highly opinionated and there are some
things which annoy me. Let's go over the basics:
All Vagga Images Are Local To The Project
-----------------------------------------
There is no "global vagga cache". Every time I want to make a new project
folder with an ubuntu image I have to wait the ~15 minutes it takes for Ubuntu
to download on my connection (Comcast). As such I've been forced to use Alpine.
No Easy Way To Establish Inheritance From Common Code
-----------------------------------------------------
With Docker I can create an image `xena/lapis` and have it contain all of the
stuff needed for [lapis](http://leafo.net/lapis) applications to run. With
Vagga I currently have to constantly reinvent the setup for this or risk
copying and pasting code everywhere
Multiple Containers Can Be Defined In The Same File
---------------------------------------------------
This is a huge plus. The way this all is defined is much more sane than Fig or
Docker compose. It's effortless where the Docker workflow was kinda painful.
However this is a bittersweet advantage as:
Vagga Containers Use The Same Network Stack As The Host
-------------------------------------------------------
Arguably this is because you need root permissions to do things like that with
the IP stack in a new namespace, but really? It's just inconvenient to have to
wrap Vagga containers in Docker or the like just to be able to run things
without the containers using TCP ports on the host up.
http://vagga.readthedocs.org/en/latest/network.html is interesting.
Overall, Vagga looks very interesting and I'd like to see how it turns out.
---
Interesting Links
-----------------
- https://www.joyent.com/blog/dockers-killer-feature

1
frontend/.psc-ide-port
View File

@ -1 +0,0 @@
15098

12
frontend/src/BlogIndex.purs
View File

@ -8,8 +8,9 @@ import Network.HTTP.Affjax (AJAX, get)
import Prelude (($), bind, map, const, show, (<>), pure, (<<<)) import Prelude (($), bind, map, const, show, (<>), pure, (<<<))
import Pux (EffModel, noEffects) import Pux (EffModel, noEffects)
import Pux.Html (Html, br, div, h1, ol, li, button, text, span, p) import Pux.Html (Html, br, div, h1, ol, li, button, text, span, p)
import Pux.Html.Attributes (key, className) import Pux.Html.Attributes (key, className, id_)
import Pux.Html.Events (onClick) import Pux.Html.Events (onClick)
import Pux.Router (link)
data Action = RequestPosts data Action = RequestPosts
| ReceivePosts (Either String Posts) | ReceivePosts (Either String Posts)
@ -58,9 +59,9 @@ update RequestPosts state =
post :: Post -> Html Action post :: Post -> Html Action
post (Post state) = post (Post state) =
div div
[ className "col s4" ] [ className "col s6" ]
[ div [ div
[ className "card pink lighten-1" ] [ className "card pink lighten-4" ]
[ div [ div
[ className "card-content black-text" ] [ className "card-content black-text" ]
[ span [ className "card-title" ] [ text state.title ] [ span [ className "card-title" ] [ text state.title ]
@ -68,6 +69,9 @@ post (Post state) =
, p [] [ text ("Posted on: " <> state.date) ] , p [] [ text ("Posted on: " <> state.date) ]
, span [] [ text state.summary ] , span [] [ text state.summary ]
] ]
, div
[ className "card-action pink" ]
[ link state.link [] [ text "Read More" ] ]
] ]
] ]
@ -76,5 +80,5 @@ view state =
div div
[] []
[ h1 [] [ text state.status ] [ h1 [] [ text state.status ]
, button [ onClick (const RequestPosts) ] [ text "Fetch posts" ] , button [ onClick (const RequestPosts), id_ "requestbutton", className "hidden" ] [ text "Fetch posts" ]
, div [ className "row" ] $ map post state.posts ] , div [ className "row" ] $ map post state.posts ]

2
frontend/src/Layout.purs
View File

@ -47,7 +47,7 @@ view state =
navbar :: State -> Html Action navbar :: State -> Html Action
navbar state = navbar state =
nav nav
[ className "light-blue lighten-1", role "navigation" ] [ className "pink lighten-1", role "navigation" ]
[ div [ div
[ className "nav-wrapper container" ] [ className "nav-wrapper container" ]
[ link "/" [ className "brand-logo", id_ "logo-container" ] [ text "Christine Dodrill" ] ] [ link "/" [ className "brand-logo", id_ "logo-container" ] [ text "Christine Dodrill" ] ]

27
frontend/support/index.html
View File

@ -4,7 +4,7 @@
<meta charset="UTF-8"/> <meta charset="UTF-8"/>
<meta http-equiv="Content-type" content="text/html; charset=utf-8"/> <meta http-equiv="Content-type" content="text/html; charset=utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Pux Starter App</title> <title>Christine Dodrill</title>
<link href="http://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet"> <link href="http://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/materialize/0.97.8/css/materialize.min.css"> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/materialize/0.97.8/css/materialize.min.css">
@ -14,5 +14,30 @@
</head> </head>
<body> <body>
<div id="app"></div> <div id="app"></div>
<script>
// Forgive me
function eventFire(el, etype){
if (el.fireEvent) {
el.fireEvent('on' + etype);
} else {
var evObj = document.createEvent('Events');
evObj.initEvent(etype, true, false);
el.dispatchEvent(evObj);
}
}
interval = setInterval(function() {
e = document.getElementById("requestbutton");
if(e != null) {
eventFire(e, "click");
} else {
console.log("Tried so hard been so far")
}
clearInterval(interval);
e.style.display = "none";
}, 250);
</script>
</body> </body>
</html> </html>