An organization can say they’re successfully conducting security testing when 1) they can trace test cases back to security requirements that embody the application’s ability to resist viable attack that would cause the business to suffer impact to its mission and 2) they enter security bugs in their bug-tracking software. They must then prioritize and fix security bugs like any other software change request.

Sounds like QA right? Well, it should. Rest assured, there will be techniques, tools, and knowledge required to make the above statement true that even great QA people, process, and tools won’t be able to accomplish without some help from Security. Can QA get to 80% on their own? I don’t think so. Can they make more than 20% progress? I think so. My intuition is that QA folk with good tooling and a small amount of training will be able to specify and implement about half of what I call both a good test suite and good security tests.

In my experience companies don’t do security testing. Some of the more advanced companies with respect to Software Security have it on their ‘08 and ‘09 roadmaps. Product companies are more likely to have integrated security testing practices because of their comparatively tester-centric culture and SDL (vs. IT shops).

What are you seeing out there?

Technorati Tags: software testing

Gunnar Peterson wrote an excellent post lamenting the lack of market forces in the security space.

I don’t know when we’ll see such market forces affecting companies but do agree they would have a positive impact. Certainly, I get why the security space hasn’t been subject to market forces yet though:

• People haven’t historically been ready to pay for it
• Companies haven’t entered the market with something valuable enough to invest in

First: what people are willing to pay for. Simple question: if a development team has a $10M budget, how much does it, say, spend on quality? 10-40%? OK. How much does it spend on software security?

Second: The security space, at least with respect to Software or Application Security, has indeed moved beyond its purely missionary roots: people know they face a problem. People do not, however, know what to do about it.

Lack of direction hasn’t resulted from the vendors’ failed productization of a tool set or services though. No, it’s resulted from the lack of mature solutions in the space. Penetration testing companies were gobbled up last year but my experience has been that, for the most part, customers haven’t been willing to invest in these tools far beyond their initial purchases and the smallest amount of shepherding.

In fact, most organizations I’m working with have seen either a reduction in reliance on or an outright backlash towards penetration testing. Use of static analysis tools and code review, while on the rise, has not reached ubiquity in response (We’ve yet to see what will happen in a broader context, having only completed what I consider to be the early-adopter phase).

What do we do about it? Partnering more closely with customers has been something I’ve felt very strongly about. Listening more closely to them remains crucial. Even involvement at this level can be tricky to fund at all but the most interested customers.

There’s work to do as vendors too. Because, while we’re through the missionary phase, we’re not through the education phase in security. We must spend more time helping clients understand what attainable next steps look like for them. Moreover, I think we have to work with them to solve some of the problems we’ve avoided: we need to clarify salient next steps ;-)

How are we really going to get enough security knowledge in the hands of developers to change their behavior in a sustainable way? How are we going to scale code analysis so that it has some of depth of expertise, manually applied, but all of the consistency and speed of automation?

Technorati Tags: software security