Microsoft has an excellent security culture. Case in point: Michael Howard posted an in-depth explanation of the latest Internet Explorer bug including the key piece of code involved. Read it. Most organizations go to extraordinary lengths to bury a failure as quickly as possible, but Microsoft seems to understand that they have to really understand their failures if they hope to avoid repeating them.

But the big news here is the bug itself. It was use-after-free caused by a thread safety problem. C++ failed twice in one fell swoop: first, the programmer was forced to do direct memory management, and surprise surprise, eventually an object gets freed while there's still a pointer to it. Second, the use-after-free allowed an attacker to execute arbitrary code. That's a pretty high price to pay for a mistake in code that wasn't supposed to have anything to do with pulling in outside instructions, but that's the way it goes in C++.

But even with a better programming language, thread safety is a tough problem. If you're writing in Java or C# or Ruby a thread safety problem probably won't cause you to end up with a vulnerability that allows arbitrary code execution, but you might inadvertently allow one user to see another user's data, and that's plenty bad. Back in 2005, Katrina Tsipenyuk, Gary McGraw, and I wrote a paper titled Seven Pernicious Kingdoms: A Taxonomy of Software Security Errors. We lumped this kind of race condition together with errors caused by distributed computation and called them problems related to "time and state". In the presentations I gave on the paper, I always referred to Time and State as the kingdom of the future, because we're only headed for more CPU cores, more threads, and more interconnected systems, and that means more opportunities for getting out of sync.

Last week, in a CSO magazine article titled Penetration Testing: Dead in 2009, I said that penetration testing is undergoing a transformation - it's going to die and be reborn as part of a more tightly integrated approach to security. This week Ivan Arce responded in defense of penetration testing in a CSO article titled Twelve Reasons Pen Testing Won't Die. Ivan is a class act, which is why his article isn't titled "12 Reasons Brian Chess is a Ninny." But Ivan doesn't address the core of my original argument, so I'm going to further explain myself here. Let me start by saying this: if you misread the original article to mean "on Jan 1, 2009 all penetration testers are going to go out Jonestown-style", then you need to keep reading.

It's December. For me that usually means my travel schedule winds down, I spend more time doing family stuff, and I start thinking about what I'm going to do next year. This year the travel hasn't let up, and as for the future, all I can think about are the things Gary McGraw, Sammy Migues and I have learned interviewing executives at some the world's top software security programs. We're in the process of building a maturity model out of the data we collected, but some of the stuff we learned is too good to wait. We wrote up a Letterman-style Top 10 Surprises article this month. Check it out.

For me the biggest surprise was about penetration testing. Penetration testing is one of the first things many organizations do when it comes to software security. I believe it to be universally true that if you're not paying attention to security, you have security problems, but it seems most people like to pay a penetration tester prove it to them. Now here's the surprise: as organizations get better at software security, their emphasis on penetration testing diminishes. In most cases it doesn't disappear, but it does get wrapped into a much larger and much more comprehensive approach to improving security. The best initiatives balance the yin and the yang of attack and defense.

We passed an inflection point last year. People are now spending more money on getting code right than they are on proving it's wrong. (More on that here.) This isn't the end of the road for penetration testing, but it does change things. Rather than being a standalone "product", it's going to be more like a product feature. Penetration testing is going to cease being an end unto itself and reemerge as part of a more comprehensive security solution. This kind of thing happens all the time in high-tech. The first PC spell checkers were standalone programs, but the market for stand-alone spell checkers died when they became a standard part of any word processor. These days spell checkers are everywhere (even in my IDE), but there is no market for a standalone spell checker. Proof positive: there aren't even any Web 2.0 or iPhone spell checker startups!

Alright, so why 2009? The time is right because back in 2007, IBM bought a company named WatchFire and HP bought a company named SPI Dynamics. The acquired companies both made web application penetration testing products. IBM and HP spent serious money for these companies'not crazy dotcom prices, but even at HP and IBM you have to have to tell a good story before you get to spend north of seventy million dollars. The good story was that the acquired technology would work together with other products and services to fuel a broad entrée into a rapidly growing software security market. It takes a little while to digest any acquisition, but by now it's been long enough. 2009 will be the year this strategy comes together, and when we look back, it will be the year when most of the world began thinking about penetration testing as part of a larger offering. There will always be boutique security consulting companies with funny names and exotic services, but the industry will grow by integrating security yin and yang. If you'd like a sneak preview of what the future holds, check out the work White Hat Security has done to integrate their vulnerability measurement service with Web application firewalls. This is attack and defense working together in a creative new way.

More than ever before, people understand the software security challenge, and penetration testing deserves credit for helping spread the word. But knowing a security problem exists is not the same as knowing how to fix it. In other words, penetration testing is good for finding the problem and bad for finding the solution. Just like the venerable spell checker, it's going to die and come back in a less distinct but more pervasive form.