Charlie and the Fuzzing Factory

July 30, 2009 by kowsik

It’s cool that Charlie Miller fuzzed the iPhone and broke it, but the catch phrase for me was (paraphrased) “When I start the fuzzer, I want to get some sleep and when I wake up find tons of 0-days“. I remember watching the movie with my kids and it wasn’t that the factory made awesome chocolates, but the whole thing was automated with elves and such.

If you’ve looked at my CanSecWest slides from last year, you might infer that fuzzing is essentially an algorithmic alternate serialization (please don’t abbreviate, ‘cos it doesn’t sound good). In other words you don’t (and never) build a fuzzer. You build a data structure that’s capable of serializing differently in order to produce alternate byte streams. And when you do, you no longer have to build protocol-specific fuzzers. You focus instead on modeling the individual message structures and state machines.

Anyways, back to automation.

The thing is, it’s easy to generate random inputs. Takes more effort to generate calculated inputs in the right way. Takes a whole lot more work to ensure that the input actually caused some problem. At Mu, we call these monitors and these form the basis of automating the fuzzing process. The monitors are fundamental to correlating the execution of the test cases to what actually happens to the target.

Here are the monitors we have in our product:

command_monitor.png Execute commands on a host and use the output to identify possible failures or faults occurring on the Target.
console_monitor.png Passively monitor output generated (like a terminal server or a serial console)
dio_monitor.png Monitor the expected frequency associated with an input line on an I/O device
remote_log_monitor.png Monitor a log file output (like tail -F) to identify faults
snmp_monitor.png Execute SNMP requests on a host to identify possible failures or faults occurring on the Target
syslog_monitor.png Monitor the content of syslog messages received from the Target

Systems fail in a very many ways (lessons learned over 4 years) and you want the fuzzer to adapt the tester’s needs. We’ve seen processes restart themselves so quickly that it’s not visible externally. With monitors, fuzzing doesn’t just have to be black-box testing.

The process of correlating crashes to the monitor-readout is called as fault isolation and the Mu Platform does this automatically. As and when we execute the test cases, we look at the output of the various monitors and correlate the monitors against the test cases. When the monitors determine it’s a problem (as you define it), we can automatically restart the SUT (system under test) and figure out which test case caused the problem. Now how cool is that? The fault isolation process takes a few passes similar to what you would do if you were in front of a fuzzer.

  • You run the fuzzer
  • Something bad happens (like a seg fault or a kernel panic)
  • Now you spend the time figuring out which test case caused the problem

Guess what? I’m a lazy programmer, to the core. Laziness, Impatience and Hubris resonate with me. I hate anything that’s manual and I’m proud to be working with engineers that think the same way. Wouldn’t it be cool if the test harness could back-track and figure out automatically which test-case caused the core dump? Or better yet, in the process of figuring it out also captured the packets for you and told you which test-case (exactly) caused the crash?

Welcome to the Fuzz Factory!

Categories: Fuzzing, Mutations, Research
2,647 Comments
Bookmark and Share