From an Undisclosed Location in Washington State

This week I’m attending a conference for the participants of the BSIMM project, which seeks to objectively measure the software security assurance activities conducted by leading ISVs, financial services firms, and a variety of other development organizations.

At the conference, Gary McGraw, Sammy Migues, and Brian Chess (who created BSIMM) are sharing the results of the third round of measurements spanning 42 different firms with representatives of those firms. Most of the results make sense and fit with a motherhood-and-apple-pie understanding of the industry and what counts. However, as the number of participants increases and the set of measured data points grows, so to does the risk of misconstruing the results to promote pet arguments or other fallacies.

Now things are getting fun. As I write this, the group is preparing to spend the bulk of the afternoon discussing how measurement efforts like BSIMM could be used to assess and compare vendors in terms of software security maturity. Do the activities measured in BSIMM provide a meaningful differentiator? Do strong maturity levels correlate to better security? Do third-party vendors behave the same way as internal efforts?

I will enjoy debating these topics, but more importantly, I’m excited that the debate is occurring. The more energy we spend discussing these topics, the more likely we are to collect and share data that will help us reach better answers.