Cigital has always favored delivery of holistic assessments (those assessments that consider architecture/design as well as code, leveraging static & dynamic tools, and using both tool-assisted and manual techniques). During my tenure as Director of Security, we found the following distribution of findings:

• Static tool: 20%


• Dynamic tool: 5% (*1)


• Manual SCR: 15%


• Architecture Risk Analysis: 60%

Those numbers may surprise and demand some explanation. For instance, though Microsoft at one point reported 50% of their findings were “flaws”, Cigital was finding 60% (on average, with as much as 70%) of its issues as ‘flaws’. If anything, our classification of ‘flaw’ was less inclusive though, and at the time I looked into this, I concluded that our larger percentage could be due to technique, focus on architecture in methodology (LoE), or level of customer familiarity/maturity with the tech-stack they developed on.

Likewise, our process started with a “whitebox” triage then shifted to exploratory testing (aka: dynamic) rather than the other way around (which I find common elsewhere) and that order-of-operations probably moved balance from dynamic to static. So, in practice we “found” vulnerabilities statically and then supported those findings with dynamic tests to validate their exploit-ability and impact (though perhaps dissatisfying, static gets the point in our data set here).

Slicing by technique
Our experience made it pretty evident to us: skip assessment techniques and expect to miss some critical vulnerabilities.

Slicing by vulnerability type
As I’ve mentioned in previous blog entries and mailing list posts, security managers need the capability to assess and report on their ability to find particular classes of security vulnerability as well. WASC’s Web Application Security Statistics provide a starting point for both classes of attacks you might expect to find and their probability. There’s no shortage of vulnerability taxonomy work out there, so we’ll leave this topic at this superficial level.

Optimizing a Security Practice
Whether you know what percentage of your reported findings come from each assessment technique, or you’re just starting to build your assessment practice, optimization (measurement, analysis, iterative improvement) should be a goal. To do this we combine our knowledge of the techniques we’re employing and the enumeration of vulnerabilities we want to find and report.

Confined to the context of static analysis, I described the following loop in a previous blog post to do just that:

• Use a representative sample of your applications to observe a static analysis tool’s performance, not a contrived test suite.
• Consider the tool’s findings relative to pen-testing findings on the same app.
• Did new and interesting findings result?
• Did pen-testing findings found by the static tool provide adequate root-cause analysis and remediation advice to fix problems earlier?
• How long did it take to on-board an app? How will this scale to your portfolio of apps?
• How long did it take to triage the results? How will this scale?

The purpose of this trial experimentation was to provide some basic comparison between assessment techniques (though you could easily come up with a more thorough approach). Of course, you’d want to consider findings from the perspective of all techniques, not just static tools.

Internally, we had our assessment lab managers write up a qualitative comparison of assessment techniques we employ, comparing how well each is “suited” to finding vulnerabilities against common vulnerability taxonomies (*2) and came up with the following:

• ARA: 14%


• Static tool: 12%


• Manual SCR: 21%


• Manual Pen: 21%


• Dynamic Tool: 12%


• Sec Testing: 20%

At this point, the reader may be asking why my experience differs from our lab managers’. Specifically, for instance, “why are dynamic techniques over-estimated here as compared to to historical findings rates?” Two reasons (at least) come to mind:

1. The balance of techniques that constitute our assessments are “moving back” towards dynamic as software-under-test and market forces evolve


2. There’s a difference between how suited a technique is to discover something and how we choose to find it (I guarantee that in some circumstances this gap represent ‘optimization opportunities’, while in others it represents explicit technology or methodological choices).
   

The Caveat You’ve Come to Expect From Me
Whether your considering a vendor’s assessment service, an assessment standard, or your own internal assessment practice, it’s key to be able to detect biases. When others supply you assessment data you’ll likely uncover biases (though it may not be apparent to those reporting initially). Be careful, I doubt you’ll be able to compile, combine, or relate numbers you receive from varying sources for this reason.

If you’re compiling assessment data for the purpose of motivating practice improvement, you may want to either 1) extract universal similarities from the data you get and present those lessons learned to management, or 2) just list the others’ experience separately. This can provide those to whom you’re presenting a compelling view to your perspective on differing assessment approaches.

In fact, many factors will muck up assessment comparison. For instance, when people report a percentage (like I did above), did they mean finding instances, findings, by type, or something else (*3)? What was their methodology, level-of-effort, and how did they conduct assessment. For instance, risk-based assessments produce a very different distribution (with wider variation) than time-boxed techniques, which more frequently drive to an explicit (or worse, implicit) checklist.

One thing universally seems true though: the more assessment techniques a software security practice makes use of, the more types of vulnerabilities it can expect to report.

-jOHN

(*1) – Clarification: our use of dynamic tools is human-assisted (both in terms of crawl and in follow-up with manual effort or leveraging internally developed tools).

(*2) – WASC, OWASP Top 10, CWE Top 25

(*3) – My data is comprised of reported vulnerabilities. Answering questions about what risk-level I reported requires some abstraction as clients differ in their requirements there. Likewise, behavior regarding instances vs. buckets of ‘related vulns’ requires addl. explanation.

In the software development circles I travel, the topic of adding a mobile channel onto existing application/system infrastructure has come up a bunch recently. With regards to threat modeling a ‘move to mobile’ represents an ideal opportunity to revisit threat modeling. Remember, I don’t necessarily prescribe threat modeling for well-known system arch-types (such as classic n-tier) and technology stacks as much as when teams attempt new and lesser known architectures.

The natural question: how do my threats change when I bring a mobile channel into my existing application? Consider, for instance, a generic application, as depicted below:

Existing Application w/ New Mobile Channel

This application supported a User and CSR through a browser interface, and had other connections to RESTful services in its middle tier. Now, we’ve added more controller and presentation tier logic, perhaps opting to reuse almost all the former application’s model. In this case, the new functionality aims to provide a more numerous set of users (2-4X as many as the plain browser interface) access to their accounts and services but with considerably lower bandwidth. The intent is that a sub-set of services would be offered (rate comparison and ACH transfer being omitted) and all available services will be provided in a crisp, simple, and responsive format.

Since we’ve discussed this example (and many like it) in threat modeling classes, we’ll leave these users and threats to the original system alone for now.

New portions of the system, being constructed to support mobile, are depicted in blue. In a future post, we’ll discuss how to more clearly identify the change in attack surface due to these additions. For now, let’s just consider the new Threats themselves. Again:

Threat – A class of individuals or software agent executing on behalf of such an individual

When we model threats we describe a threat’s capabilities, level of access, and skills to start. Let’s apply a few of Threat Modeling techniques to identify new threats to consider.

1. Consider the system’s users
When teams add new functionality to a system, or when they start a new development effort entirely, they commonly create user stories or perhaps even detailed use cases and requirements.

The first (and easiest) way to identify new threats to the system is to mine user stories, use cases, and other usage documentation (even marketectures often work) for their users. Then, consider:

• What evil or insidious behaviors could a user engage in?
• What obnoxious or stupid behaviors could a user cause trouble with?

Two users come to mind as we look at proposed new mobile functionality:

1. Mobile device user (in this case, a smart phone)
2. Neighboring network user

Combining the first items from our two lists above, we come to our first threat: a malicious mobile device user.

1. Malicious Mobile Device User – Device users possess the credentials to the device (including any UI, ‘app store’, or other username/password tuples) and likely possess the carrier account/credentials. Access includes physical access to the device and use of both of its applications and browsers. This threat can install applications, sync, and explore device contents with their computer. Of course, this threat has access to device SDK and simulators as any developer would. See this threat depicted in the figure above with label “1″.

When we discuss attack surfaces, we’ll discuss what kinds of access and capabilities these threats possess in a technology-specific fashion. Some conceptual actions include:

• Transfer device contents to computer for debugging, reversing, etc.


• Install purpose-built (malicious) applications on device


• Manipulate application settings, set up a proxy for web interaction, etc.


• Use both browser and application to access the same services/resources


• Jailbreak device

Consider Evil/Insidious User Action
The last behavior piques particular interest. When the user decides, with malicious intention, to jailbreak their device, additional malicious behaviors are available to them. Regardless of whether or not the jailbreak was motivated by malicious intent, the device’s security controls are compromised. In this case, applications can no longer trust that security features such as application signing apply or that a confidential path for their data exists between the application and network. Documenting threats in a traceability matrix allows one to show this escalation of privilege from one threat to another directly:

In summary, this behavior gives rise to another class of threat:

2. Malicious Mobile Device – This threat has all the capabilities and access of a malicious mobile device user, but can also compromise or augment the device OS and its drivers. The figure above depicts this threat labeling it “4″.

A broad range of capabilities results from this opportunity and if a device is compromised prior to the a victim application being loaded, malicious behavior can observe and affect the application installation process itself, or user signup/registration processes–which may include initial credential/key material exchange. This imagined scenario may quickly turn an application’s entire security proposition into a house of cards.

Consider Obnoxious/Stupid User Action
Once you start, it’s hard not to imagine a multitude of stupid things a user could do with their mobile device. Let’s focus on a common and important one: the user could leave the device somewhere, for it to be stolen. This scenario has two effects: first, this dramatically increases the population of Threat #1: malicious device user. Let’s split this threat into 1.A (previously described) and 1.B: malicious device user (same as 1.A but without the user’s credentials). However, Threat 1.B also increases the population of Threat #2–malicious devices because thieves can jailbreak devices even without user credentials.

An important third scenario impacts certain applications, like the generic one we describe above. If our threat traceability matrix (above) was filled out during a previous secure design exercise, it might include “password reset” use described in the mitigation column. Indeed, during many threat modeling courses jOHN teaches his participants indicate “we can remove the malicious CSR threat entirely by doing password reset out-of-band using a mobile phone.”

In the case that a phone is stolen and possesses the bank’s MobileBankingApp, perhaps which caches the user’s name, how effective is this control?

Many password reset implementations jOHN observes (using his accounts) suffer full compromise under the ‘stolen phone’ scenario. In this case, the identification of a new threat causes us to reconsider or augment the design of our system so that it again demonstrates the security properties we desire.

A Brief Diversion into the Mobile Threat Population
Looking superficially at search results, it appears as though ~120,000 phones are stolen in the UK annually; ~200,000 in Australia. Another site indicated 26M phones are stolen annually and resold. Regardless of how accurate these numbers are, this represents a much larger threat population than expected from security researchers and nefarious parties alone.

Not Stolen, “Something Borrowed”

Consider more generally the notion of the device being “out of sight” interaction briefly, perhaps through the following vectors:

• Because stolen (unknown to the device’s user) and returned after being tampered with


• The device is plugged into another individual’s machine (perhaps for charging or for app download)


• Device lent to individual for momentary use (phone call, game demo, contact/weather lookup)


• Grey-market phone, donated phones, recycled/returned phones

Each of these cases exposes the whole of the device’s attack surface to exploitation without visual cues to the phone’s user.

And Who ‘Owns’ These Devices Anyways

This is perhaps a good time to point out that the phone’s user may not be the phone’s owner. Family and corporate phone plans often provide access to the phone, remotely, unbeknownst to the user. In these cases, the phone’s owner must be considered a (yet unpictured in our diagram) threat from the user’s perspective.

From the phone (or account) owner’s perspective, the user represents a similar threat: the user can potentially add/modify/remove software without visual cues or notification. Our ‘single-user’ device actually serves multiple masters:

1. Me


2. The Account Owner


3. Our Benevolent Dictators: Google/Apple… RIM/Microsoft?


4. Carriers: ATT, VeriZon…


5. Device Manufacturer

We know the benevolent dictators retain the right/capability to remove applications from our devices. What other capabilities must an application publisher be concerned about within their Threat Model?

Here in the diagram, you see the inevitable separation of the application store curator and the current set of benevolent dictators listed–Amazon’s new application store is a prime example.

Reconsider Old Model’s Threats
Having conducted a threat model on the classic n-tier system before, a man-in-the-middle (MiM) threat was undoubtedly considered. Reconsider this threat in terms of the new architecture.

A common mistake modelers make in considering mobile MiM is to forget that devices contain not only applications that make Internet connections but that they also possess browsers:

3. MiM – This threat has access to traffic sent over the Internet (OR carrier network) sent either from the app to server or
vice versa. When a device contains a browser, this threat can see both app to server traffic and browser to server traffic from the same account/device. This threat may be able to see traffic through (active or passive) interposition, or through landing code (script) w/in the device’s browser using classic web attacks (Depicted as Threat #4 in the figure above).

Whether or not one needs to consider both Internet and carrier-based sniffing depends on other factors within the threat model. Cigital has always considered carrier-level compromise part of its mobile threat model, at a low barrier to entry. However, it bears mentioning that as recently as 2004 it was challenge to convince organizations that individuals could “be the mobile network”. It’s somewhat vindicating to see this as a consumer-grade scenario now.

This isn’t the half of it though (and we’re getting ahead of ourselves a little bit in talking about attack surface here, but). Consider the following ‘radio-based’ surfaces on the phone:

1. GSM Stack


2. CDMA Stack


3. 802.11


4. Bluetooth


5. NFC (coming soon to a device near you!!!)

Remember, vulnerability in any of these implementations may cause our user to suffer a “drive-by owning“, leaving him/her with Threat #2: Malicious Device.

Other Old Threat
Taking another cue from former efforts, modelers must consider the possibility that malicious applications will run alongside their applications within a mobile device. This scenario is no different than that of a browser, or any application running on the almost forgotten host computer.

4. Malicious Application – Threat represents either a compromised victim application on the device, a Trojan, or other malware (depicted in the figure above as #3). The causal vector may have been data interpreted and executed by a vulnerable app, a malicious application placed in an app store (or otherwise available for download), or different entirely. Such applications (malicious or corrupted) can attempt to poke and prod at your victim app directly, exploit the underlying device OS, or focus on server resources directly. The malicious app may possess a valid signature (or not, as advantageous) and has access to the device’s services (as per what the user has granted it).

Summary
Hopefully, this thought exercise has shown its reader threats they hadn’t considered. Hopefully it shows the reader the advantages of reconsidering a threat model as the architecture changes through normal means:

1. Start with the users/user-stories


2. Think maliciously


3. Think ‘stupid’


4. Re-consider old threats in the new architecture


5. Understand, you’re not the only device ‘user’, ‘owner’

It continues to surprise me that these techniques pay off even when only threats (the ‘who’) themselves are considered.

Next, we’ll write up how consideration of the attack surface differs as we ‘move to mobile’ and begin to see how this expanded surface can have dramatic implications on the security posture of existing functionality.

you will find our observations regarding static analysis shared by others. It’s encouraging to note that Flash Sheridan observes many of the same difficulties and more formally treats them in his ISSRE ’10 publication. It’s worth a read.

A few commentators shared some decent points I wanted to address in a comment. I’m cross-posting my commentary here:

Comments treating the application of static tools as a distinct (and unfavorable) alternative to “fixing your SDLC” disappoint me, especially from the OWASP community (of which I am a contributing paid member). In fact, the community’s OpenSAMM project actively prescribes use of static analysis in not one but four (4) unique activities. So, both BSIMM and SAMM see static analysis as a part of fixing one’s SDL. **Note that in both models, all activities are not compulsory.

“Fix your code” is another red-herring alternative to static analysis. I believe static analysis inseparable from effective usage of a secure programming toolkit (such as OWASP’s ESAPI or an organization’s own toolkit) because static analysis tools can meet the need for organizations to 1) detect consistent and thorough use of secure APIs (and non-use of dangerous ones) as well as 2) detect incorrect usage of such APIs (whether through broken call-order, un-paired functions, or other bugs). We shouldn’t forget, as Mr. Manico indicates, 3) running static tools on these security toolkits themselves finds problems that careful review may not otherwise have found.

So, secure SDL models indicate usage and we–as a community–see use in these tools in helping remind developers how to correctly build secure code. I think we have to agree static analysis tools find exploitable bugs (even though sometimes in an overly expensive or time-consuming way). So, I think we’re stuck with them as a ‘best practice’ in the short term, however little some of us may like them. To dynamic vendors out there, I’d even go so far as saying, “Early adopters have begun to over-rely on static means to find problems more effectively/efficiently found with dynamic tools. Don’t worry, the ones I’m working with will begin to tilt the balance back using assessment data”.

I would like to put a face-and-name on the key notion that almost all commentators have thus-far touched: the static analysis tool that gets used by a majority of application developers out there, in the future, will likely not look like the ones we have today. And that’s fine too.

Dan Cornell wrote a blog post on static analysis coverage. He observed that while the static tool he used completed its scan, it didn’t find things it should. He implicitly referred to this as coverage.

I’ve often put things as follows:

Your tool can’t find what it can’t see and it can’t see what it doesn’t parse.

This got said often years back because at the time it was quite difficult to integrate tools effectively into a builds so that they would in fact ‘see’ everything and successfully parse it (*1). We referred to this concept of how well we, as operators, had integrated as providing visibility into the code. These days, market leading tools don’t have nearly the problem they used to with finding the code they need to parse… or parsing that code successfully (*2).

These days, the frameworks developers use for persistence, object remoting, application navigation, and view display challenge visibility most. Tools see the code for your controller and parse it successfully but they don’t necessarily understand what it represents. This happens in almost every application I scan, even the painfully simple ones.

We couldn’t have our Consultants going out into the world armed with tools bearing only spotty visibility into applications so I wrote a utility identifyEntryPoints to find entry points on its own. We don’t want Consultants guessing at what their static analysis tool missed or popping the hood open to try to discern what it found and not doing what they’re paid to do: assess the application, so I wrote another utility entryPointCompare to compare what the tool found vs. what I was able to find programmatically. See actual output below (I’ve replaced the static tool’s name with ‘TOOL XXX’):

Sanguine:demo jsteven$ /Users/jsteven/code/demo/working/engine/util/entrypointCompare.py
2011-02-06 09:07:08,158 INFO     bois_1.0        bin.execPythonRobot STARTING: /Users/jsteven/code/demo/working/engine/util/entrypointCompare.py
Factory 31, [TOOL XXX] 0
 TOOL XXX missed set(['Index', 'csrchat', 'accountsList', 'Transfer', 'editProfileFormBean', 'newAccountFormBean', 'fileList', 'Auth', 'newClientFormBean', 'qaFormBean', 'UserProfile', 'SetupProfileAction', 'uploadFile', 'VerifyAnswerAction', 'BeanEditProfileAction', 'existingClientFormBean', 'GetDocuments', 'ChatPoll', 'announcements', 'modifyDocument', 'BeanCreateAccountAction', 'ChangePasswordAction', 'VerifyClientAction', 'ChatSend', 'Help', 'fileUploader', 'BackOfficeAdmin', 'passwordFormBean', 'BeanCreateClientAction', 'action', 'Accounting'])

Here, I ran the test on Cigital’s own internal “Bank of Insecurities”, which is a simple Struts1 application. The utility’s output represents all the actions and forms that account for web-based input in their own respective ways. Sure looks like some pretty important stuff was missed in there huh?

The next step was to write a third program registerEntryPoints which would write rules for the missing entry/exit points as appropriate so that the beefy commercial static analysis tool could do what it does best. These utilities represent just some of the internal workings of Cigital’s ESP platform.

Yes, but my tool finds stuff all the time
Would a tool not find any findings without detecting entry points? Well, it would likely find potential vulnerability but only using more local syntactic analysis. Remember, “you can’t find what you can’t see…”

I’m sensing a theme in your posts jOHN
OK, so, that’s two blog posts (Mr. Cornell and my own) complaining about what I refer to visibility. Now what?

Spend time cataloging your chosen tool’s performance
Cigital experience indicates that as few as 10-13 reasonably chosen applications can serve as a representative sample for as many as 300 applications on the Java platform. When you pilot those 10-13 applications, conduct the following steps to avoid the visibility failures seen above in the portfolio as a whole:

1. On-board the application using whatever tool interface gives the most feedback about missing artifacts (*3)
2. Explore scan logs for identified entry points
3. Manually explore the application’s deployment descriptors and critical configuration files
4. Document controller logic as framework default or developer extended
5. Identify key entities within the DAO/persistence framework

When you’ve done that for 10-13 apps, our representative sample, stop and take stock of what you’ve collected. You’ll have created a very good list of items for which you’ll want to create custom rules. Different tools call the kind of rules you’ll create different things, but suffice it to say you’ll be writing rules to tell the tool that it’s:

• Entry: Taking input from untrusted web sources
• Entry: Taking input from untrusted partner applications
• Exit: Placing data in a untrusted view (browser, service repsonse, etc.)
• Exit: Conducting CRUD operations on entity data

The above list is by no means exhaustive. Indeed, you’ll want to look at data originating from the persistence tier and headed to the web (not listed above) but we often consider this as a second phase of customization. You’ll also want to begin considering data entry/exit from 2nd and 3rd party components within the applications being tests. Again, another good subsequent customization phase.

Gosh, this sounds expensive
Yes; ‘well. Compared to simply running a static analysis tool using its IDE-based GUI, triaging results, and calling it quits this is darned expensive. However, it dramatically raises your visibility into an application’s vulnerability (you should also expect false-positive reduction).

Compare this to the potential difference in quality and depth of a manual, tool-assisted penetration test and an automated vulnerability scan. An engineer running AppScan costs much less than an expert penetration test and organizations have come to not only expect different results, but see these as two very different services. I predict that we’ll see a similar distinction between two services (automated and more expert-driven) in the static space in a few years.

Leveraging Manual Efforts Across Assurance Activities
If your organization doesn’t conduct manual penetration testing and is unwilling to pay your engineers to properly on-board applications into its static analysis tool, then neither static nor dynamic analysis will benefit from high visibility into applications under test. The cost will look “high” and “extra” as well.

Conducting the kind of analysis described by this entry (targeting entry and exit points) can inform both static and dynamic analysis alike. Conduct this exercise for applications built with representative technology stacks. On the static front, write custom rules based on your work. On the dynamic side, furnish this work to your dynamic testers (even if they are external vendors) so that their exploration benefits from it.

(*1) – Coverity’s Prevent was always a notable exception and able to integrate with even challenging build environments without much pain.
(*2) – Limitations still exist with complex (distributed) build systems, code-generation schemes, and even the more mundane JSP compilation process.
(*3) – Ask if you need help here, last time I published tool details, vendors got cranky.

Today, Microsoft announced the launching of its SDL Pro Network. Cigital is proud to be part of this pilot offering, and pleased to continue to take the message (and the delivery) of software security to the market. As a network of independent software security professionals, the SDL Pro Network will collectively take our best of breed experiences and work collaboratively to develop unified service offerings around Microsoft’s SDL methodology.

At Cigital we are proud of our extensive experience running more than six large-scale enterprise software security initiatives spanning customers in financial services, independent software vendors, and embedded systems. We have trained several thousand developers, architects and executives on the fundamentals of software security. We have rolled out tools and best practices for many of our best customers. We have helped to grow the software security market from its infancy. Cigital is the largest and most experienced software security services provider in the world, and we look forward to continuing our market leadership through our partnership with Microsoft.

The number of firms delivering software security services is small and forms a tightly knit community, including companies of varying sizes, experience and areas of expertise. As a group, we have all read and embraced the three top software security methodologies, including CLASP from OWASP, the Touchpoints from our own CTO Dr. Gary McGraw, and of course, Microsoft’s SDL. Regardless of what flavor of methodology our customers subscribe to, we all share the common goal of educating and delivering services that protect our clients’ assets and good name through better software security. Collaborative efforts that bring together the best minds in the business can only help improve what we do with our own customers and broaden our thoughts on the subject.

Kudos to Microsoft for pulling the SDL Pro Network together. Our clients will all benefit from the experience…stay tuned to this space for more.

I am very optimistic about the growth of the software security field over the last few years. Things are certainly moving in the right direction (toward white box analysis instead of outside->in black box, out of the myopic focus on Web apps, and toward full-lifecycle programs based on the touchpoints). The numbers show this growth and these trends objectively.

Some pointers:

From Slashdot
Posted by: kdawson, on 2008-01-09 01:20:00

Stony Stevenson alerts us to a US Department of Homeland Security program in which subcontractors have been examining FOSS source code for security vulnerabilities. InformationWeek.com takes a glass-half-empty approach to reporting the story, saying that for FOSS code [1]on average 1 line in 1000 contains a security bug. From the article: ‘A total of 7,826 open source project defects have been fixed through the Homeland Security review, or one every two hours since it was launched in 2006…’ ZDNet Australia prefers to emphasize those FOSS projects that [2]fixed every reported bug, thus achieving a clean bill of health according to DHS. These include PHP, Perl, Python, Postfix, and Samba.

Firstly, I am a big fan of code scanning and believe that use of static analysis tools should always be one of the basic security steps integrated into every SDLC. However, there are huge problems with declaring security after passing a code scan with an arbitrary tool and a random set of rules. The most obvious issue is that security defects come in two flavors—bugs and flaws—each accounting for roughly 50% of defects in practice. Code scanning tools can only find bugs. Here are two stupid examples for effect: can a code scanning tool determine that no user authentication was performed? How about whether or not a playback attack will work?

The second most obvious problem is that the list of rules enforced by a static analysis engine can never be complete. Discussion about this is left as an exercise for the reader.

Architectural risk analysis (crazily called “threat modeling” by Microsofties) is, like code scanning, an essential software security best practice. Formal methods are one way to go about attacking the flaw problem. In the US we rely on flakier heuristic-based approaches such as the one we use at Cigital. But no matter the approach, we can’t ignore the architecture.

References

1. http://www.informationweek.com/story/showArticle.jhtml?articleID=205600229&cid=RSSfeed_IWK_All
2. http://www.zdnet.com.au/news/security/soa/11-open-source-projects-pass-security-health-check/0,130061744,339284949,00.htm

We’re pleased to have a guest blogger for this Justice League entry. Michael Cohen is a Senior Security Consultant at Cigital where he is responsible for leading, assessing, architecting and implementing secure software for Fortune 500 companies. Michael also works with Cigital teams on enterprise-wide security solutions intended to improve an organization’s security posture and help them meet audit and regulatory requirements. Michael just gave a talk to the Washington, DC IASA chapter that was well received, and is now the subject of this entry:

When I hear software architects talk about the architecture they’ve crafted to depict the various structures and behaviors of a system, they point out interesting techniques they’ve applied that best convey how the system should work and be put together. An architect’s enthusiasm for the little details reveals a great sense of accomplishment and creativity, but most of all good architecture conveys sorely needed information in order to help those who develop, test and maintain the system. Sharing the tips and tricks gathered from the field is what helps our community move forward to building better software. A classic example of this is the Design Patterns book written by the Gang of Four.

Not too long ago I was also sharing my ideas on architecture and security at a local IASA chapter here in Washington, DC. The group was a crowd of like-minded architects who work for large Fortune 500 companies, government agencies, and promising local startups. My topic was pragmatic secure architecture. The basic idea was to look at some real ways to incorporate security into architecture using Cigital’s risk management and threat modeling practices. You can find the slides for my presentation here.

For the uninitiated, threat modeling is a way of depicting where threats (think malicious people, attackers, and so on) can touch a software architecture and how they may be able to exploit critical assets using various attack patterns. Threat modeling is valuable for determining an architecture’s security posture. In addition, identifying risks in user requirements and business goals, and tying those risks to a threat model results in a map of how design flaws impact the system, its users and the overall business. Threat models coupled with identified high-level risks are a great way to get other stakeholders involved with security decisions. And mind you, these are stakeholders who would otherwise be unable to contribute and supply critical feedback.

The attendees at the chapter meeting were glad they attended the presentation and heard something worthwhile that they could use in their daily architectural activities. Many people brought up interesting points about how to best protect critical assets, what a real risk to a system is, and what is considered a good enough mitigation. What I found particularly interesting was how threat modeling provided a way for everyone to contribute interesting ideas about where security vulnerabilities may exist and how they could be mitigated (which, if you think about it, is the entire point of threat models).

I encourage any architect to share their ideas on building a better architecture with their peers, whether it be at a local organization or at a conference. As a senior security consultant at Cigital, I like to take shared ideas and mold them into my own unique way of thinking about the world. The best results come when I can apply new ideas to my daily activities as I help our customers assess and create more secure software.

… On an entirely separate note, McGraw still owes me money. I’m watching you, Gary.
[tags]software architecture,software quality,software security,touchpoints[/tags]

On the first day of class, mechanical engineers learn a critical lesson—pay attention and learn this stuff or the bridge you build could fall down. This lesson is most powerfully illustrated by a video of the Tacoma Narrows Bridge shaking itself to death. Figure 0-1 shows a 600-foot section of the bridge falling into the water in 1940. By contrast, on the first day of software engineering class, budding developers are taught that they can build anything that they can dream of. They usually start with “hello world.???

Figure 0-1: A 600-foot section of the Tacoma Narrows bridge crashes into Puget Sound as the bridge twists and torques itself to death. Mechanical engineers are warned early on that this can happen if they don’t practice good engineering.

An overly-optimistic approach to software development has certainly led to the creation of some mind-boggling stuff, but it has likewise allowed us to paint ourselves into the corner from a security perspective. Simply put, we neglected to think about what would happen to our software if it were intentionally and maliciously attacked.

Much of today’s software is so fragile that it barely functions properly when its environment is pristine and predictable. If the environment that our fragile software runs in turns out to be pugnacious and pernicious (as much of the Internet environment turns out to be) software fails spectacularly, splashing into the metaphorical Puget Sound.

The biggest problem in computer security today is that most systems aren’t constructed with security in mind. Reactive network technologies such as firewalls can help alleviate obvious script kiddie attacks on servers, but they do nothing to address the real security problem—bad software. If we want to solve the computer security problem, we need to do more to build secure software.

Software security is the practice of building software to be secure and function properly under malicious attack. This book is about one of software security’s most important practices—code review with a static analysis tool.

As practitioners become aware of software security’s importance, they are increasingly adopting and evolving a set of best practices to address the problem. Microsoft has carried out a noteworthy effort under its Trustworthy Computing Initiative. Many Cigital customers are in the midst of enterprise scale software security initiatives. Most approaches in practice today encompass training for developers, testers, and architects; analysis and auditing of software artifacts; and security engineering. There’s no substitute for working software security as deeply into the development process as possible and taking advantage of the engineering lessons software practitioners have learned over the years.

In my book Software Security, I introduce a set of seven best practices called touchpoints. Putting software security into practice requires making some changes to the way most organizations build software. The good news is that these changes don’t need to be fundamental, earth shattering, or cost prohibitive. In fact, adopting a straightforward set of engineering best practices, designed in such a way that security can be interleaved into existing development processes, is often all it takes.

Figure 0-2 specifies the software security touchpoints and shows how software practitioners can apply them to the various software artifacts produced during software development. This means understanding how to work security engineering into requirements, architecture, design, coding, testing, validation, measurement, and maintenance.

Figure 0-2: The software security touchpoints as introduced and fleshed out in Software Security: Building Security In.

Some touchpoints are by their very nature more powerful than others. Adopting the most powerful ones first is only prudent. The top two touchpoints are Code review with a static analysis tool, and Architectural risk analysis. This book is all about the first.

All software projects produce at least one artifact—code. This fact moves code review to the number one slot on our list. At the code level, the focus is on implementation bugs, especially those that static analysis tools that scan source code for common vulnerabilities can discover. Several tools vendors now address this space, including Fortify Software the company that Brian and Jacob work for.

Implementation bugs are both numerous and common (just like real bugs in the Virginia countryside) and include nasty creatures like the notorious buffer overflow, which owes its existence to the use (or misuse) of vulnerable APIs (e.g., gets(), strcpy(), and so on in C). Code review processes, both manual and (even more important) automated with a static analysis tool, attempt to identify security bugs prior to the software’s release.

Of course, no single technique is a silver bullet. Code review is a necessary but not sufficient practice for achieving secure software. Security bugs (especially in C and C++) are a real problem, but architectural flaws are just as big a problem. Doing code review alone is an extremely useful activity, but given that this kind of review can only identify bugs, the best a code review can uncover is around 50% of the security problems. Architectural problems are very difficult (and mostly impossible) to find by staring at code. This is especially true for modern systems made of hundreds of thousands of lines of code. A comprehensive approach to software security involves holistically combining both code review and architectural analysis.

By its very nature, code review requires knowledge of code. An infosec practitioner with little experience writing and compiling software is going to be of little use during a code review. The code review step is best left in the hands of the members of the development organization, especially if they are armed with a modern source code analysis tool. With the exception of information security people who are highly experienced in programming languages and code-level vulnerability resolution, there is no natural fit for network security expertise during the code review phase. This may come as a great surprise to those organizations currently attempting to impose software security on their enterprises through the infosec division. Even though the idea of security enforcement is solid, making enforcement at the code level successful when it comes to code review requires real hands-on experience with code.

The problem is that most developers have little idea what bugs to look for, or what to do about bugs if they do happen find them. That’s where this book, Secure Programming with Static Analysis, comes in. The book that you have in your hands is the most advanced work on static analysis and code review for security ever released. It teaches you not only what the bugs are (what I sometimes call the “bug parade??? approach to software security), but how to find them with modern static analysis tools, and more importantly what to do to correct them. By putting the lessons in this book into practice, you can go a long way toward helping to solve the software security problem.

You can purchase a copy of Secure Programming with Static Analysis from Amazon.

[tags]touchpoints, software security, static anlaysis, Brian Chess, Jacob West[/tags]

The higher up you are in the policy framework, the more general and time-enduring the content should be. The farther you progress down the framework to a more detailed level, the more perishable the content will be, out of necessity.

Later he continues:

…is because implementers need it. They’re not security experts (usually) and do not necessarily grok security the same way a seasoned (salty?) security person might.because “implementers need it”.

This tiering was implicit to my first post. In fact your most senior security resources can probably use nothing but Security Principals (as described by McGraw’s BSS Book and the famous Saltzer paper) and find both insidious vulnerabilities as well as brand-new “Game over” architectural flaws with new development technologies they aren’t familiar with. But, the more junior (inexperienced) or development-oriented (constructive) the person being targeted, the more specific the guidance must be in order to be valuable without requiring inordinate effort.

Because we’re trying to change the behavior of the majority of our Developers–who range in skill from OK to Hero and whom may have never had even a security awareness class–I find “technology-specific” guidance moves the ball the furthest.

In my previous two posts I talk about forms various levels of standards take, and the way in which one might create it. It occurs to me that I all but showed the bigger picture and might as well follow up to do so. Below, you’ll find a map of how I show security guidance flowing throw and effecting a software development team (click-through for full detail):

As information moves from top to bottom and from left to right it becomes more specific and actionable, but also more perishable (as has been said). To build security in, one must think about security’s implications throughout the lifecycle, so I see no reason why security knowledge (regardless of how specific) shouldn’t mirror artifacts used to construct the application itself: software requirements, design, and the code itself.

Though not central to this discussion, the diagram has been annotated to indicate who should produce and consume this information. Here, I’ll point out that your centralized Application Security Resources can probably most effectively and efficiently create the generic security guidance, but will need help of Security Architects to create the more technology-specific guidance and garner broad buy-in.

My last post presented a brief model of how one might organize and fund this in practice.
-jOHN

[tags]knowledge architecture,software security[/tags]