Iron Chef Interviews Part 1: Charlie Miller

I’m cleaning up my machine, and I came across the interviews we did for the Iron Chef competition at Black Hat in August 2008. They’re too much fun for me to keep to myself. Our lead Chefs were Charlie Miller from ISE and Sean Fay from Fortify. Charlie fuzzed his way to victory, but Sean gave him a run for his money with static analysis. Here’s what Charlie had to say before the game. I’ll post Sean’s interview in a few days.

Charlie Miller is a principle analyst at Independent Security Evaluators, but he’s better known as the first guy to hack the iPhone, and he’s made a habit of walking away with a Mac in the annual CanSecWest Pwn2Own contest. We just think he’s an Apple hater. Charlie, why you gotta hate?

I’m not a hater. I just like pointing out how the “cool” company isn’t perfect. That and I like to irritate all the fanboys. Oh wait, I guess I am a hater.

How did you prepare for the competition today?

Actually, I did A LOT of preparation. More than I thought I’d need to do when I signed up. I was told the target program would speak Bonjour, HTTP, and DAAP. So I made sure I had some fuzzers that would work on those protocols.

For some fuzzers this meant getting packet captures or client programs (for mutation) or learning the protocol and writing a simple specification for generation-based fuzzers. Unfortunately for me, there aren’t a lot of DAAP or Bonjour fuzzers sitting around.

It’s easy to get started with fuzzing, but should people build their own fuzzing tools? What are the advantages/disadvantages of doing it yourself? Any advice for people just getting started?

The advantage of doing it yourself is the ability to customize it. However, just like most development, it’s usually best to use something that’s been around for a while and been tested rather than rolling your own version.

I’m using 3 fuzzers against 3 different protocols and didn’t write any of the fuzzers (although I made some small modifications to some of them.) It turns out that debugging a fuzzer is super difficult!

There are a lot of open source fuzzing tools out there. Should people consider commercial tools too? If you pay money, what do you get that doesn’t come for free with open source?

Yes, commercial fuzzers are an option. If you want the most thorough fuzzing, you need to build in a lot of protocol knowledge into the fuzzer. This takes a considerable amount of time and effort.

When you buy a fuzzer, you are buying the fact someone did all the research for the protocol and built the test cases for you. If you are fuzzing a proprietary format, you probably won’t be able to find a commercial fuzzer for it. It all depends on whether you have more time than money. For me, I have more time.

What’s wrong with static analysis?

Static analysis tools are often plagued by false positives. Instead of looking for bugs, users of static analysis tools end up spending all their time trying to find the one bug out of the 10,000 alerts generated by a tool. I also think static analysis fails to find extremely complex bugs.

Fuzzing is negative testing, so you can never be sure you’ve found all the bugs. How should people balance fuzzing with other techniques?

Fuzzing should be used as only one part of a complete and balanced diet of bug finding techniques. It can find bugs and point static/manual analysis to untested portions of code. However, it isn’t the end-all-be-all. For example, I probably wouldn’t waste my time fuzzing a program written in Java.

You’ve got a PhD in math. If fuzzing is so cool, why didn’t you study it in school?

Yea, I totally wasted those 5 years. When I got my PhD, I could barely program, much less understand fuzzing. However, I knew everything about nonlinear partial differential equations! (Sadly, that last statement is a gross exaggeration) Also, keep in mind; I’m an old dude, so fuzzing wasn’t exactly hip in 2000 when I graduated (much less 1995 when I started grad school). The OULU guys hadn’t blown the world away yet.

Barton Miller coined the term “fuzzing” in 1990. Why didn’t it become a popular technique sooner?

That’s a great question! I’d say that it was a combination of a couple of things:

First, it wasn’t exactly hard to find bugs in something like Microsoft RPC 10 years ago. You didn’t NEED fuzzers.

Another is that Miller was an academic so the guys who actually look for bugs weren’t aware of the work. It wasn’t until OULU broke the Internet with their ASN.1 fuzzer that everyone took notice.

After you fuzz a program, do you still respect it in the morning?

It matters how easy it gave it up to me. Apache is a total prude! I currently have a restraining order against QuickTime.

What do you expect the static analysis results will be like?

Lots of alerts about coding style. Lots of “lame” bugs that don’t get you a shell. Lots of “bugtraq”-esque bugs that say “if they configure it this way and run it as root and write their configuration file in UTF-8, there is a bug”. Maybe there will be a good bug thrown in by accident.

Why do you think you’re going to win?

Because my buddy from work picked the target program. No, wait, that’s why Sean is going to win. I’m going to win because I’ve got mad skillz…and I hand picked a couple of the judges.

What’s the biggest mistake people make when they fuzz?

Probably that they don’t check what they are doing. The biggest problem with fuzzing is if you do it wrong, you don’t find any bugs, which means either there weren’t any bugs or there were bugs and you missed them. Often, it’s hard to tell which.

The other common mistake is not fuzzing long enough. For me at least, I’m very impatient and want to turn off the fuzzer after a couple of hours when I should let it run for a week.

What's your favorite vulnerability or security incident from the last year?

I kinda liked the cross-site scripting attack that redirected Obama’s site to Clinton’s, even though in general I think XSS is lame.

What other disciplines do you take inspiration from?

They don’t have anything to do with computer hacking, but I like to race my bike (like Lance Armstrong except way slower) and I like watching hockey. I’m also possibly the oldest man to play dance dance revolution and I kicked ass at it.

Where did you grow up?

I’m a product of St. Louis County public schools.

If this were the original Iron Chef, what would you cook?

I cook a mean Hamburger Helper. Probably whatever the special ingredient was, I’d just throw it in with a box of the helper. Depending on what it was, I might choose Lasagne, or Three Cheese flavor, or maybe Cheeseburger Macaroni. I’d never use Beef Noodle which is horrible!