Penetration Testing is Dead, Long Live Penetration Testing

Last week, in a CSO magazine article titled Penetration Testing: Dead in 2009, I said that penetration testing is undergoing a transformation - it's going to die and be reborn as part of a more tightly integrated approach to security. This week Ivan Arce responded in defense of penetration testing in a CSO article titled Twelve Reasons Pen Testing Won't Die. Ivan is a class act, which is why his article isn't titled "12 Reasons Brian Chess is a Ninny." But Ivan doesn't address the core of my original argument, so I'm going to further explain myself here. Let me start by saying this: if you misread the original article to mean "on Jan 1, 2009 all penetration testers are going to go out Jonestown-style", then you need to keep reading.

It's December. For me that usually means my travel schedule winds down, I spend more time doing family stuff, and I start thinking about what I'm going to do next year. This year the travel hasn't let up, and as for the future, all I can think about are the things Gary McGraw, Sammy Migues and I have learned interviewing executives at some the world's top software security programs. We're in the process of building a maturity model out of the data we collected, but some of the stuff we learned is too good to wait. We wrote up a Letterman-style Top 10 Surprises article this month. Check it out.

For me the biggest surprise was about penetration testing. Penetration testing is one of the first things many organizations do when it comes to software security. I believe it to be universally true that if you're not paying attention to security, you have security problems, but it seems most people like to pay a penetration tester prove it to them. Now here's the surprise: as organizations get better at software security, their emphasis on penetration testing diminishes. In most cases it doesn't disappear, but it does get wrapped into a much larger and much more comprehensive approach to improving security. The best initiatives balance the yin and the yang of attack and defense.

We passed an inflection point last year. People are now spending more money on getting code right than they are on proving it's wrong. (More on that here.) This isn't the end of the road for penetration testing, but it does change things. Rather than being a standalone "product", it's going to be more like a product feature. Penetration testing is going to cease being an end unto itself and reemerge as part of a more comprehensive security solution. This kind of thing happens all the time in high-tech. The first PC spell checkers were standalone programs, but the market for stand-alone spell checkers died when they became a standard part of any word processor. These days spell checkers are everywhere (even in my IDE), but there is no market for a standalone spell checker. Proof positive: there aren't even any Web 2.0 or iPhone spell checker startups!

Alright, so why 2009? The time is right because back in 2007, IBM bought a company named WatchFire and HP bought a company named SPI Dynamics. The acquired companies both made web application penetration testing products. IBM and HP spent serious money for these companies'not crazy dotcom prices, but even at HP and IBM you have to have to tell a good story before you get to spend north of seventy million dollars. The good story was that the acquired technology would work together with other products and services to fuel a broad entrée into a rapidly growing software security market. It takes a little while to digest any acquisition, but by now it's been long enough. 2009 will be the year this strategy comes together, and when we look back, it will be the year when most of the world began thinking about penetration testing as part of a larger offering. There will always be boutique security consulting companies with funny names and exotic services, but the industry will grow by integrating security yin and yang. If you'd like a sneak preview of what the future holds, check out the work White Hat Security has done to integrate their vulnerability measurement service with Web application firewalls. This is attack and defense working together in a creative new way.

More than ever before, people understand the software security challenge, and penetration testing deserves credit for helping spread the word. But knowing a security problem exists is not the same as knowing how to fix it. In other words, penetration testing is good for finding the problem and bad for finding the solution. Just like the venerable spell checker, it's going to die and come back in a less distinct but more pervasive form.