cadey
/
nim-wiki
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Updated Fuzzing your nim code to rabbit out all the hard bugs (markdown)

This commit is contained in:
Federico Ceratto 2017-01-20 20:47:11 +00:00
parent d9ea735453
commit fbf8b7b9c5
1 changed files with 14 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
Fuzzing-your-nim-code-to-rabbit-out-all-the-hard-bugs.md
View File

@ -1,12 +1,17 @@
[afl-fuzz](http://lcamtuf.coredump.cx/afl/) is quite rabid at ferreting out issues that are hard-to-impossible to find with manual testing.
While classically, afl-fuzz attempts to detect exactly the sort of issues that nim is attempting to prevent with its runtime checks in debug mode, it is by no means of no use to the nim developer. afl is extremely good at discovering programmer errors that cannot be caught with simple user testing, even when all the safety nets are turned on.
While classically, afl-fuzz attempts to detect exactly the sort of issues that nim is attempting to prevent with its runtime checks in debug mode, it is by no means of no use to the nim developer. AFL is extremely good at discovering programmer errors that cannot be caught with simple user testing, even when all the safety nets are turned on.
Please note that afl will catch errors by checking for the segfault signal; nim is designed to never do that and exception instead, so we have to simulate this behaviour for all error types we consider to be a unintended "crash" issue. This is why the example transforms any exceptions into a SIGSEGV exit code.
Please note that AFL will catch errors by checking for the segfault signal; nim is designed to never do that and exception instead, so we have to simulate this behaviour for all error types we consider to be a unintended "crash" issue. This is why the example transforms any exceptions into a SIGSEGV exit code.
## Work by example
The general idea behind afl is that you shrink down your code path enough that you can just feed it some input and either have it work/do nothing, or crash. There are other ways to inject afl hooking into a more complex program (google: afl-fuzz persistent mode), but for this example, we'll stick with that. afl can feed data either over stdin (recommended! speed!) or over a temporary file.
The general idea behind AFL is that you shrink down your code path enough that you can just feed it some input and either have it work/do nothing, or crash. There are other ways to inject AFL hooking into a more complex program (google: afl-fuzz persistent mode), but for this example, we'll stick with that. AFL can feed data either over stdin (recommended! speed!) or over a temporary file.
Let's start with a simple example program everyone can get behind:
@ -46,9 +51,9 @@ This is a pretty contrived thing, but just imagine this being somewhere in a fil
The general idea is to boil down your testcase to just the code path you want to test; like part file handling library. The smaller, the faster; and the faster, the less time you have to wait for it to find all those eye-opening bugs you never thought of.
## Make it work with afl
## Make it work with AFL
Save that example in a file called **afltestcase.nim**. Because afl needs to instrument the generated binary code, you need to tell nim to use the compiler frontend afl comes with, instead of whatever your distro defaults to. Create a file called **afltestcase.nim.cfg** in the same directory and put this in:
Save that example in a file called **afltestcase.nim**. Because AFL needs to instrument the generated binary code, you need to tell nim to use the compiler frontend AFL comes with, instead of whatever your distro defaults to. Create a file called **afltestcase.nim.cfg** in the same directory and put this in:
```ini
# nim.cfg for afl-clang
@ -66,7 +71,7 @@ gcc.exe = "afl-gcc"
gcc.linkerexe = "afl-gcc"
```
This will tell nim to use afl to compile afltestcase.nim (and only that file, not any others!), instead of whatever your distro defaults to. Now, compile it:
This will tell nim to use AFL to compile afltestcase.nim (and only that file, not any others!), instead of whatever your distro defaults to. Now, compile it:
```
$ nim c afltestcase
@ -81,7 +86,7 @@ afl-cc 2.36b by <lcamtuf@google.com>
## Testcases
afl best works when given a bunch of valid test data. It *can* make up random data, but if you have some idea of what working input looks like, it'll help speed things up:
AFL best works when given a bunch of valid test data. It *can* make up random data, but if you have some idea of what working input looks like, it'll help speed things up:
```
$ mkdir afl-in/
@ -97,9 +102,9 @@ $ # yay, works!
```
## Running afl
## Running AFL
Finally, we can run afl. As afl feeds it's mutating input on stdin by default, we're already all set.
Finally, we can run AFL. As AFL feeds it's mutating input on stdin by default, we're already all set.
```
$ mkdir afl-out/
@ -142,8 +147,3 @@ template AFLAssert*(cond: bool) =
```
Also add `-d:afl` to your nim.cfg to enable this.