OWASP just posted an interview with me as part of their budding podcast series. It’s nice to have the tables turned after doing all the Silver Bullet (and Reality Check) interviews! It’s also nice to be able to answer some of the questions that OWASP types have about Cigital’s approach to software security.

Download the podcast here.

The OWASP interviewer is Jim Manico, and he did a great job. He was a little worried about some of the questions he asked. In fact, off the record he kept saying he was sorry and telling me that I did not have to address certain questions. Personally, I enjoyed the questions he asked immensely. Though some of his questions were loaded, I do hope that my answers may serve to clarify our position and eliminate OWASP concerns.

Here are a few of the many more questions I address in the podcast:

• Why do you insist on use of the term “software security” as opposed to “application security”?
• What is static analysis good for and what is it no good for?
• What is the exact relationship between Cigital and Fortify?
• Why do you think your “top 19” is any better than the OWASP top 10 or the CWE top 25? (Special note, the 19 Sins work is Mike Howard’s and John Viega’s…I was not involved.)
• Why does Cigital have a proprietary approach to IP?
• What makes the Touchpoints any better than the SDL or CLASP?
• What is your relationship with Allan Paller and SANS?
• Who picked the “porn music” theme for Silver Bullet?

As an extra bonus, the theme music for this episode is a song written and recorded by my band Where’s Aubrey.

Anyway, enjoy the podcast, and let me know what you think about my answers!

More links:

• Show notes
• MP3
• OWASP Podcast RSS Feed
• Direct iTunes link

Myself and others have been getting Webinar invites from IBM’s developerworks regarding Rational’s AppScan Developer Edition.

This is of course part of the re-launch of the re-tooled AppScan product. It now includes a set of analysis types, some static and some dynamic. They’ve got fancy new names for subsets of each, some hair splitting to my ear and I’ve been reading/writing on this topic for 10 years.

The market for static analysis is plumping nicely year over year. Static analysis (SA) vendors, such as Fortify, are one-or-two revisions into producing dynamic analysis tools and suites that leverage hybrid approaches. With the big dynamic analysis tools controlled by IBM and HP a certain amount of this cross-over (and much more I predict) was inevitable.

It was also inevitable that that dynamic shops would take shots at the SA space. The dynamic guys got shot up pretty bad as their tools–sold as application security’s first ’silver bullet’–begun to meet with resistance on how much manual effort was still required and on how many false positives and/or missed vulnerabilities remained. Static analysis swept in promising a better solution. “Look at the source code”, they smiled, “and you can be more complete and accurate than if you just poke the application from the outside.” Well, now it’s SA’s turn in the tank and the dynamic shops are enjoying the reversal.

But, in a hybrid world–where the major tool vendors offer both static and dynamic tooling this is technically absurd. AppScan’s explanation on “String Analysis” is particularly absurd. They’ve chosen to attack the effectiveness of static analysis with their ’solution’, a static analysis technique.

It’s true that both Fortify and Ounce principally find SQL-injections through data flow techniques that propagate ‘taint’ and rely on tagging source, sink, and cleansing logic at the resolution of function calls. But I can say with authority that both tools model the potential values of strings as they propagate their data flow analysis as well. AppScan’s innovation isn’t so innovative.

Yes both SA tools, in my opinion, leave a bit to be desired in the core products’ support for what aspects of string propagation and manipulation they can follow. I won’t get into detail, but I advise security managers that it’s worth it to understand where the tool will fail you in this regard.

Coverity and the now defunct CodeAssure did an exceptionally good job modeling string values throughout code’s ’speculative execution’, as part of their static analysis. When I tested these tools, they surprised me in both what they were able to find and what other tools’ false positives they left out of reports. Alas, these two don’t help today’s security manager much if they’re focusing on Java EE or .NET web applications. CodeAssure is gone and Coverity’s product has (in my opinion) limited language support outside C/C++ (though, again, I can not stress enough how positive my experience with it in C/C++ has been).

The dynamic shops had to level the playing field. But, as near as I can tell, the current situation is this:

• The major vendors believe there’s benefit to static and dynamic analysis
• There’s a lot of room for technical improvement in the market leaders’ SA products, with respect to modeling
• The future’s tool will leverage static and dynamic techniques, because each is suited for find a particular class of problems well.

You, as a security manager, will still need to sift through the marketectures and promises and figure out which tool works best on the kind of code your organization builds/buys. A major component of this will be how customizable the tool is.

Over-reliance on ANY automated tools (static or dynamic) leaves you with un-found vulnerabilities and a false sense of security. Cigital’s assessment services rely on these tools for speed and scale, and so we’ve taken great pains to understand where their modeling bends and where it breaks. In our own practice, we augment static techniques with dynamic tests and have even begun writing some next-generation static techniques to counter these limitations. We’ve spent hundreds of hours helping many many clients wring more out of their preferred suite of tools with the same understanding: opting instead for less invasive customization and tuning.

The bottom line is (and I’ve been saying this since at least ‘03 now), there are strengths to each tool and each type of analysis. NONE will get you to the assessment finish line alone. Anyone who tells you otherwise is sellin’ you a load.

Let the posturing begin.

On January 12th, the CWE/SANS Top 25 Most Dangerous Programming Errors list was released. Sean Barnum (a Principal Consultant) participated in the creation of the list, and I did some off the record review myself (not for attribution).

There are some important good things about top ten lists that are worthy of mention. The notion of knowing your enemy is essential in security (as it is in warfare), and top ten lists can help get software people started thinking about attacks, attackers, and the vulnerabilities they go after. These days almost any attention paid to the problem is good attention, and the fact that the technical media is paying attention to software security at all is a good thing. Top ten lists help in that respect.

But I have some serious concerns about these kinds of lists that I wrote about in my informIT article this month:

Top Eleven Reasons Why Top 10 (or Top 25) Lists Don’t Work

Here are the reasons, stripped of history and commentary which you can find in the article:

1. Executives don’t care about technical bugs.
2. Too much focus on bugs.
3. Vulnerability lists help auditors more than developers.
4. One person’s top bug is another person’s yawner.
5. Using bug parade lists for training leads to awareness but does not educate.
6. Bug lists change with the prevailing technology winds.
7. Top ten lists mix levels.
8. Automated tools can find bugs–let them.
9. Metrics built on top ten lists are misleading.
10. When it comes to testing, security requirements are more important than vulnerability lists.
11. Ten is not enough.

I’m happy to announce the launch of my new podcast, the Reality Check Security Podcast with Gary McGraw:

The Reality Check Podcast with Gary McGraw focuses directly on software security practitioners and practical software security. Reality Check’s sister podcast, the Silver Bullet Security Podcast with Gary McGraw, follows a free form interview style tailored highlight the ideas and experience of security gurus. By contrast, Reality Check is concerned with practical questions centered on running large-scale software security initiatives in the real world.

Reality Check targets experienced leaders working to solve software security problems in large organizations every day. We use a standard script to guide each conversation with questions about history, methodology, best practice, and measurement. We plan to interview leaders of mature software security programs and leaders of programs just getting started.

Your feedback is absolutely welcome. Please subscribe to the series through or RSS feed or through iTunes.

In the last entry, I responded to dashboard initiatives and organizations’ attempt to measure. Examples focused on vulnerability-detection because a lot of software security expenditure has as well.

Some practitioners, as well as software security managers, have realized that a focusing only on uncovering the problem won’t get it fixed. They’ve begun to document secure sample code, construct secure-development frameworks, and even harden existing open-source frameworks in order to make secure application development easier. This trend cuts across those IT shops supporting non-software businesses (such as finance, insurance, gaming, and the like) as well as software vendors. Let’s look at the 2nd surprising finding from SAMM data gathering, as published in Gary’s InformIT article:

8. Secure-by-default frameworks can be very helpful, especially if they are presented as middleware classes (but watch out for an over focus on security “stuff”).

Secure-by-default frameworks for aiding in development are absolutely my favorite part of software security. Not only do they make sense as part of any program, they represent an essential corner-turn from a hopeless attempt to keep developers abreast of what they’re doing wrong to the downward push towards the finish line by helping developers release better software. In 2001, plainly sick of documenting and demonstrating the same vulnerabilities over-and-over again in meetings and reports, I begun to talk about, seek money for, and outline what such a framework might look like for use of Servlets in the Java EE platform. Frankly, sometimes it’s just plain easier to describe how to do something correctly than it is to conceive of pen-tests or static analysis rules that will find all instances of vulnerability, some omission, or an incorrect implementation.

But, I’m not the only one that’s been climbing this hill: A broad range of approaches to constructing such frameworks are visible within my own client base.

Some organizations have taken their cue from what the network and host security guys have done in their own spaces. These software security groups see software security as a mapping between vulnerability and the need for control. This drives two sets of behavior:

With regards to the software they build, these organizations tend to strive towards having some central engineering team (or a particular development group) provide a ’security control’ that they can plug in for each vulnerability. This has had two effects in my experience:

1. The ’security framework’ ends up as an amalgam of functions designed in very point-solution fashion. For example OWASP ESAPI’s addCSRFToken() method. (see code)
2. ‘design review’ becomes a checklist activity in which security folk (playing the role of auditor) walk through a list of features they expected to include, such as those API calls described in #1 or at a higher level the inclusion of a particular security package: IE: “You’ve used ClearTrust for authentication–right?”

With regards to the software, infrastructure, and COTS these organizations purchase, they tend to apply a modified checklist (such as #2) during acquisition, and then–in the case in which they’re acquiring a security component itself–add it to the checklist for future reviews.

There are benefits to this level of framework creation/use (namely, it raises awareness of the problem, standardizes solutions making compliance easier to gauge, and does in fact place some useful functionality in the hands of developers they might have otherwise omitted or bumbled themselves).

However, I think this approach misses the mark for several important reasons.

Others build their security frameworks from scratch. Convinced that their organization is different enough to warrant their own unique approach they set to construction. Some organizations cull framework material from ‘centers of excellence’ within their organization: development teams that have already done a great job at something like demanding authentication and invalidating sessions. Others architect the security framework centrally, construct it, and then deploy it mandating development teams integrate with it as they continue otherwise normal maintenance of their apps.

There’s intrinsic benefit and difficulty to a purely bottom-up or top-down approach. If an organization goes purely bottom-up, their difficulty lies in coordinating and architecturally cohering different teams’ widgets into a common framework. The strength they can rely on is buy-in: developers are hard-pressed to complain about what they and their peers have written. The top-down approach lacks buy-in and (at times) practicality, but often enjoys a coherent unified design.

Early in ‘07, I wrote on a model for mixing the two approaches successfully.

Whether executed top-down, bottom-up, or through some compromise, I like this approach better than the one I described above. My biggest issues with it are:

1. It can become a political monster, especially if top-down
2. Being overly unique (ala: “not-invented here” or “I’m a snowflake” pathologies) can be damning

The more top-down an approach is, the more likely developers are to balk at mandates for the framework’s use. “Pfft, but that won’t work the way we built our app.”, they’ll lament. If you hear that, prepare for a deluge of exception forms. ;-)

So, how are people overcoming limitations of an ineffective feature-centric audit-model and the pains suffered from inventing their own framework from scratch? By, “meeting the developer on ‘their side of the river’”, as it was once said to me.

When a developer sits down to write or maintain a web application, they’re doing so within a very rich platform already. The .NET platform has well-integrated set of tools from the webserver up to application-support functionality provided by a single vendor: Microsoft. The Java EE platform focuses on open standards, and as such has a bevy of open-source packages that support the lower-levels of the stack that Sun provides. Simply put: a developer writing a Java EE web app is already conforming to the Servlet framework, if not the Struts, JSF, or some other MVC package. They know the platform’s packages, they’re already using them, and there are tons of great books, tutorials, and examples out there.
This is their side of the river. Demanding they build on top of those packages, but integrate with another package–for the sake of security alone–is folly.

A better answer to creating one’s own security framework–and I’ve seen this in play in more than one organization–is to secure a framework the developers already know. This:

• Maintains current platform design idiom - If Spring is ‘the right design paradigm’–great, you’re using with without having to have invented it;
• Allows consideration of the framework’s threat model - It’s easier to secure a system you know the design/deployment model of, than it is to write ‘the perfect’ security feature in a vacuum;
• Maintains transparency, as best as possible - Those securing the framework can place implementation ‘under the hood’ allowing developers to call functions as normal;
• Moves security ‘lower in the stack’
• Reduces integration points for developers

To me, these advantages are overwhelming. Showing the benefits of faithfully maintaining the underlying framework’s design could be the subject of its own entry entirely and–I feel–places the nail in the coffin of those arguing that security can be represented simply as features. I’ll defer that for now however.

Fixing problems instead of identifying them? Security guys getting traction by helping developers write code and get it out the door faster? Meeting them within the realm of the frameworks they’re already using: hardening their configuration, implementation, and deployment? This works?

Not surprising ;-)