From dcd1e6d8f616fd763ff6069fef7bdb430062bfb0 Mon Sep 17 00:00:00 2001
From: niv <n@e-ix.net>
Date: Fri, 20 Jan 2017 21:22:22 +0100
Subject: [PATCH] Created Fuzzing your nim code to rabbit out all the hard bugs
 (markdown)

---
 ...im-code-to-rabbit-out-all-the-hard-bugs.md | 116 ++++++++++++++++++
 1 file changed, 116 insertions(+)
 create mode 100644 Fuzzing-your-nim-code-to-rabbit-out-all-the-hard-bugs.md

diff --git a/Fuzzing-your-nim-code-to-rabbit-out-all-the-hard-bugs.md b/Fuzzing-your-nim-code-to-rabbit-out-all-the-hard-bugs.md
new file mode 100644
index 0000000..daa1faf
--- /dev/null
+++ b/Fuzzing-your-nim-code-to-rabbit-out-all-the-hard-bugs.md
@@ -0,0 +1,116 @@
+[afl-fuzz](http://lcamtuf.coredump.cx/afl/) is quite rabid at ferreting out issues that are hard-to-impossible to find with manual testing.
+
+While classically, afl-fuzz attempts to detect exactly the sort of issues that nim is attempting to prevent with its runtime checks in debug mode, it is by no means of no use to the nim developer. afl is extremely good at discovering programmer errors that cannot be caught with simple user testing, even when all the safety nets are turned on.
+
+Please note that afl will catch errors by checking for the segfault signal; nim is designed to never do that and exception instead, so we have to simulate this behaviour for all error types we consider to be a unintended "crash" issue.  This is why the example transforms any exceptions into a SIGSEGV exit code.
+
+## Work by example
+
+The general idea behind afl is that you shrink down your code path enough that you can just feed it some input and either have it work/do nothing, or crash.  There are other ways to inject afl hooking into a more complex program (google: afl-fuzz persistent mode), but for this example, we'll stick with that.  afl can feed data either over stdin (recommended! speed!) or over a temporary file.
+
+Let's start with a simple example program everyone can get behind:
+
+```nim
+import os, streams, strutils, posix
+
+try:
+  let s = newFileStream(stdin)
+
+  var texts: seq[string] = newSeq[string]()
+  var picks: seq[int] = newSeq[int]()
+
+  let textCount = s.readInt8()
+  for i in 0..<textCount:
+    let elemSz = s.readInt8()
+    let text = s.readStr(elemSz)
+    texts.add(text)
+
+  echo "Read ", texts.len, " texts"
+
+  let pickCount = s.readInt8()
+  for i in 0..<pickCount:
+    picks.add(s.readInt8())
+
+  echo "Read ", picks.len, " picks"
+
+  for p in picks:
+    echo p, "=", texts[p]
+
+except:
+  echo repr(getCurrentException())
+  discard kill(getpid(), SIGSEGV)
+
+```
+
+This is a pretty contrived thing, but just imagine this being somewhere in a file reader or network parser, or anything like that.  Now, nim in debug mode will catch some of the issues you can probably spot immediately, but let's assume the code is complex enough to confound you into missing them all.
+
+## Make it work with afl
+
+Save that example in a file called "afl.nim".  Because afl needs to instrument the generated binary code, you need to tell nim to use the compiler frontend afl comes with, instead of whatever your distro defaults to.  Create a file called "afl.nim.cfg" in the same directory and put this in:
+
+```
+cc = clang
+clang.exe = "afl-clang"
+clang.linkerexe = "afl-clang"
+clang.path = "/usr/local/Cellar/afl-fuzz/2.36b/bin"
+```
+
+This is a working example for OSX, where afl was installed via homebrew. To make it work on any Linux, simply adjust clang.path.  This will tell nim to use afl to compile afl.nim, instead of whatever your distro defaults to.
+
+Very important: `-d:noSignalHandler`, because AFL installs it's own signal handler inside instrumented code.
+
+```
+$ nim c -d:noSignalHandler afl
+```
+
+You should see some output from afl confirming that it instrumented your binary properly, like:
+
+```
+Hint:  [Link]
+afl-cc 2.36b by <lcamtuf@google.com>
+```
+
+## Testcases
+
+afl best works when given a bunch of valid testcases. It *can* make up random data, but if you have some idea of what working input looks like, it'll help speed things up:
+
+```
+$ mkdir afl-in/
+$ echo "\x03\02hi\05there\x03nim\x03\x02\x01\x00" >! afl-in/testdata
+$ ./afl < afl-in/testdata
+Read 3 texts
+Read 3 picks
+2=nim
+1=there
+0=hi
+  
+$ # yay, works!
+```
+
+
+## Running afl
+
+Finally, we can run afl. As afl feeds it's mutating input on stdin by default, we're already all set.
+
+```
+$ mkdir afl-out/
+$ afl-fuzz -i afl-in/ -o afl-out/ -- ./afl
+```
+
+Let it run for a while, but it should start finding issues pretty much immediately.  While crash triage is way out of scope of this document, it should give you something to start - as soon as it starts finding crashes or hangs, look into afl-out/crashes|hangs for the input data that made your code throw an exception or hang:
+```
+$ ./afl < afl-out/crashes/id:000000,sig:11,src:000000,op:flip1,pos:0
+Read 0 texts
+Read 2 picks
+ref 0x10f911050 --> [parent = nil,
+name = 0x10f918260"IndexError",
+msg = 0x10f911088"index out of bounds",
+trace = 0x10f913050"Traceback (most recent call last)\10"
+"afl.nim(25)              afl\10"
+"system.nim(2581)         sysFatal\10"
+""]
+```
+
+You can limit the exceptions that are considered crashes by adjusting the except: clause in the example.
+
+Then I'd strongly suggest to start reading up on afl if you want to employ it with confidence.
\ No newline at end of file