Some people start “Security Testing” by buying and using a pen-test tool on project. Such tools uncover security vulnerabilities (though they seldom help with root cause analysis or even obtaining double-digit code coverage).

These tools are degenerate, at best, in facilitating a security testing strategy. Why? Because, these tools are “black box” tools. What are black box tools?

The term “black box” stems from old testing literature and means “without internal knowledge”. An external perspective has always excluded “code” but sometimes goes as far as (in my opinion appropriately) software design. Obviously, you need to know something about the application’s architecture and design to test it though and the slope gets slippery.

In the realm of penetration testing, the term “black box” often has meaning beyond the tester’s knowledge of the product. OSSTM defines “black box” to mean that neither attacker nor defender are given knowledge of each other. They mean for this test procedure to accurately represent the kinds of opportunities, need for stealth, accessibility, and exploit that a real attacker would have, and also to evaluate the defender’s abilities to identify and prevent or recover from attack.

Because black box tools to a large extent run canned tests they will not satisfy my security testing goal (see previous entry) of having run tests that one traces back to requirements. ‘Requirements that one created as a result of doing risk analysis that determines exactly what behaviors (and their impacts) should be avoided were the software attacked.

Arguably, security folk have “cached” this risk analysis and these implicit requirements in the pen testing tool. Fine, this is that small benefit that I mentioned pen tests do provide. And, they DO find bugs. Again, this is at best a degenerate case of security testing in the same way running a fuzz testing tool is a degenerate way of conducting functional testing.

For QA folk wary of accepting the previous statement, it will suffice to say that you wouldn’t defend your job based on achieving less than fifty percent coverage would you?

Vendors have begun using hybrid approaches (this will only become more common). Do these approaches solve our coverage problem and allay our concerns?

I demo’d Compuware’s DevPartner, which has a poorly advertised (and perhaps now nacent) security scanning capability, a few years back and was pretty impressed with the start they had made in hybrid .NET analysis. I’m not sure where it’s gone since then. Fortify’s PTA also combines static and dynamic analyses to help prioritize static findings and provide root cause analysis for dynamic ones.

These hybrid tools don’t get our security testers off the hook either though, as they’re still not addressing the project-specific risk analysis nor are they anchoring tests in requirements.

An organization can say they’re successfully conducting security testing when 1) they can trace test cases back to security requirements that embody the application’s ability to resist viable attack that would cause the business to suffer impact to its mission and 2) they enter security bugs in their bug-tracking software. They must then prioritize and fix security bugs like any other software change request.

Sounds like QA right? Well, it should. Rest assured, there will be techniques, tools, and knowledge required to make the above statement true that even great QA people, process, and tools won’t be able to accomplish without some help from Security. Can QA get to 80% on their own? I don’t think so. Can they make more than 20% progress? I think so. My intuition is that QA folk with good tooling and a small amount of training will be able to specify and implement about half of what I call both a good test suite and good security tests.

In my experience companies don’t do security testing. Some of the more advanced companies with respect to Software Security have it on their ‘08 and ‘09 roadmaps. Product companies are more likely to have integrated security testing practices because of their comparatively tester-centric culture and SDL (vs. IT shops).

What are you seeing out there?

Technorati Tags: software testing

Gunnar Peterson wrote an excellent post lamenting the lack of market forces in the security space.

I don’t know when we’ll see such market forces affecting companies but do agree they would have a positive impact. Certainly, I get why the security space hasn’t been subject to market forces yet though:

• People haven’t historically been ready to pay for it
• Companies haven’t entered the market with something valuable enough to invest in

First: what people are willing to pay for. Simple question: if a development team has a $10M budget, how much does it, say, spend on quality? 10-40%? OK. How much does it spend on software security?

Second: The security space, at least with respect to Software or Application Security, has indeed moved beyond its purely missionary roots: people know they face a problem. People do not, however, know what to do about it.

Lack of direction hasn’t resulted from the vendors’ failed productization of a tool set or services though. No, it’s resulted from the lack of mature solutions in the space. Penetration testing companies were gobbled up last year but my experience has been that, for the most part, customers haven’t been willing to invest in these tools far beyond their initial purchases and the smallest amount of shepherding.

In fact, most organizations I’m working with have seen either a reduction in reliance on or an outright backlash towards penetration testing. Use of static analysis tools and code review, while on the rise, has not reached ubiquity in response (We’ve yet to see what will happen in a broader context, having only completed what I consider to be the early-adopter phase).

What do we do about it? Partnering more closely with customers has been something I’ve felt very strongly about. Listening more closely to them remains crucial. Even involvement at this level can be tricky to fund at all but the most interested customers.

There’s work to do as vendors too. Because, while we’re through the missionary phase, we’re not through the education phase in security. We must spend more time helping clients understand what attainable next steps look like for them. Moreover, I think we have to work with them to solve some of the problems we’ve avoided: we need to clarify salient next steps ;-)

How are we really going to get enough security knowledge in the hands of developers to change their behavior in a sustainable way? How are we going to scale code analysis so that it has some of depth of expertise, manually applied, but all of the consistency and speed of automation?

Technorati Tags: software security

Last week, I gave a talk at QCON’s inaugural US conference, as part of Gunnar Peterson’s security track. There were some pretty serious speakers giving talks and I was thrilled to be amongst a set of developers with such deep quals.

I spoke about threat modeling because I think its a reasonable first-step towards implementing “Architectural Analysis”, one of McGraw’s most important touch points. Threat models aim to illuminate the intersection of technical risks (from technical vulnerabilities) and the business assets they target. An active SDLC work product, Security Architects (conducting secure design), Security Analysts (conducting code analysis), and test managers (planning destructive tests) should consume these models. They’re not shelfware.

Slides from my talk are available from the first link above, but I wanted to distill the key steps from threat modeling I covered therein:

1. Anchor the threat model on software architecture, that’s where the majority attacks against your application will occur
2. Identify assets (critical or sensitive data, and functionality) key to the application’s business function
3. Translate use case actors into threats, use into misuse/abuse. Start with normal use cases’ error conditions and alternative flows
4. Enumerate attack vectors each attacker may use against the application by pilfering the “low-hanging fruit” from community resources on vulnerability (such as the OWASP Top-10)

Tricks emerge along the way:

1. Show privilege escalation or increased system exposure (for instance from unauthorized Internet threats to authorized system use) through simple, common attacks. Show escalation to ‘insider’ in turn through similar attacks
2. Layer or combine simple attacks to penetrate systems more deeply
3. Show selective targeting of assets (victim clients, data stores, or application functionality) through particular attack selection

Finally, annotate threat diagrams by showing mitigating or compensating controls. While this wasn’t covered by my 50 minute session, its a critical part of a larger ‘loop’ of behavior that begins to create “design for security” out of “model threats”.

Enjoy.

Technorati Tags: threat modeling

I was talking with one of my consultants the other day about a common confusion Developers sometimes have regarding a pretty mundane piece of security guidance. Specifically, “What does it mean I have to turn off logging/debug in production?”

In my mind, these two separate issues exist here, intertwined.

Almost every logging framework has an in-place configuration option for increasing/decreasing log-level; most allow you to reconfigure on a running system without restart. The point here being that an application developer, with use of such a package, get to 1) keep a single code base, 2) turn a high level of logging off by default in a production environment, and 3) ratchet up log-level in circumstances requiring it (incident response or in-place debugging).

Often, people mix the above guidance with “don’t roll debug code out to production.” Code Analysis tools, for instance, give this advice. Two things principally concern me here. First, you don’t want a bunch of “main” or other “debug-only” entry points to the code. Attackers capable of compromising host systems could shoe-horn their way into an application this way. The situation becomes worse when said debug interfaces are web-based. Second, especially for thick clients or code running in less trusted environments, debug code tends to ‘express’ things about the design (or leak critical data) in an undisciplined manner. Removing such code from deployments to untrusted zones CAN be advisable, but one must balance complete removal with the need to ‘turn on’ nascent debug code in times of need as described by the previous paragraph. Only protection of the most sensitive information, and most invasive code qualifies for the extreme guidance “remove before deploying” that this paragraph describes.

The best thing one can do is give the developer guidance as to what types of test/debug code might qualify for wholesale removal. I’ll start you a list:

1. Client ‘harness’ code allowed to connect to server interfaces w/o
   authentication
2. Client code allowed to edit/manipulate local data repositories/stores
   directly
3. Command ‘harnesses’, allowed to (re)set or manipulate server state
4. Client code that dumps secret keys, tokens, or stored-but-otherwise-protected credentials
5. Backdoors
6. Code allowing for ‘insecure failure’ or back off, say to an older protocol
7. ‘Re-implementation’ outside a sandbox (native equivalents of a managed binary)

Just a small tidbit, away from the glib towards concrete guidance.

-jOHN

Technorati Tags: logging, debugging

In a recent article iPhone shell code hits the web HD Moore, creator of the Metasploit Framework describes how combining members of a set of implementation bugs in applications on Apple’s iPhone with a design flaw results in a ripe opportunity for landing shell code or otherwise controlling the phone’s various hardware goodies (camera, mic, speaker, etc.) remotely.

As you read the above article, note the classic vulnerability-types mentioned. The iPhone suffers from a design flaw: all apps running as root. Moore also mentions some of the simplest kinds of buffer problems (such as the stack-based buffer overflow).

These bugs, this flaw represent a low-level of organizational maturity in security. It almost seems a throw-back to any 90’s era code assessment I’d done. We’ve understood the stack-based buffer overflow well for years-upon-years and even rudimentary static analysis tools catch it. Threat modeling or simple architectural analysis should have dissuaded architects from moving forward design intact, bearing so fundamentally “risk amplifying” a flaw as “run as root”.

Make no mistake: Apple represents a company leveraging security by obscurity. I cringed when Apple used its then largely untarnished record as a marketing ploy, (from what I understand) without listening to what’s been said about the actual resistance of his products to attack.

While aspects of their desktop OS were truly “designed with security in mind” and represent solidly secure OS technology, it’s clear to me that Apple does not address software security programmatically to what I consider an acceptable level.

When Apple created an asset of reasonable enough value and appeal, threats easily broke a shoddy system by retraining their attention on the protective mechanisms shielding that asset.

While a lot of our customers have adopted software security early, and have committed to addressing it in all phases of their software’s lifecycle, in all appropriate organizations touching software, others have not. If, reading this, you look at the well-covered transition from Microsoft to Apple as the vulnerability target and think by analog, “this stuff is happening to my competitor, but can’t happen to me”, or perhaps feel invulnerable because, “No one would figure these vulns. out.” or, “We’re not on people’s radar.” BEWARE. One ostrich is having to pull its head out of the sand and peck at crow… and this is sad because, with the exception of my Crackberry, I largely believe in and purchase from Apple.

A final thought: Regardless of the actual security in OS X, what chance does Apple have of touting Leopard’s security over Vista in the market place after these events?

Benjamin Tomhave wrote about “tiering” security guidance when I cross-posted a comment to my last blog entry on the SC-L mailing list. Quoting him:

The higher up you are in the policy framework, the more general and time-enduring the content should be. The farther you progress down the framework to a more detailed level, the more perishable the content will be, out of necessity.

Later he continues:

…is because implementers need it. They’re not security experts (usually) and do not necessarily grok security the same way a seasoned (salty?) security person might.because “implementers need it”.

This tiering was implicit to my first post. In fact your most senior security resources can probably use nothing but Security Principals (as described by McGraw’s BSS Book and the famous Saltzer paper) and find both insidious vulnerabilities as well as brand-new “Game over” architectural flaws with new development technologies they aren’t familiar with. But, the more junior (inexperienced) or development-oriented (constructive) the person being targeted, the more specific the guidance must be in order to be valuable without requiring inordinate effort.

Because we’re trying to change the behavior of the majority of our Developers–who range in skill from OK to Hero and whom may have never had even a security awareness class–I find “technology-specific” guidance moves the ball the furthest.

In my previous two posts I talk about forms various levels of standards take, and the way in which one might create it. It occurs to me that I all but showed the bigger picture and might as well follow up to do so. Below, you’ll find a map of how I show security guidance flowing throw and effecting a software development team (click-through for full detail):

As information moves from top to bottom and from left to right it becomes more specific and actionable, but also more perishable (as has been said). To build security in, one must think about security’s implications throughout the lifecycle, so I see no reason why security knowledge (regardless of how specific) shouldn’t mirror artifacts used to construct the application itself: software requirements, design, and the code itself.

Though not central to this discussion, the diagram has been annotated to indicate who should produce and consume this information. Here, I’ll point out that your centralized Application Security Resources can probably most effectively and efficiently create the generic security guidance, but will need help of Security Architects to create the more technology-specific guidance and garner broad buy-in.

My last post presented a brief model of how one might organize and fund this in practice.
-jOHN

Technorati Tags: knowledge architecture, software security

The process of writing security guidance is just as important to the quality of the resulting standards as is the target: technology-specific, code-centric constructive statements. How do you succeed? By using the same muscles you exercise when you conduct secure design.

When I write Security guidance, such as the technology-specific standards I blogged about last week, I start with a threat model for the system (or design paradigm) I’m trying to protect. This could be as specific as a single system or as generic as all my client’s customer-facing 3-tier J EE Apps. I aim to remove attack vectors with sets of standards. I don’t write standards that don’t directly prevent a particular attack I’ve already documented in the same way Agile Developers ignore abstraction not mandated by the current user stories. Eventually though, removing a subsequent attack vector will require a smidge of refactoring or (more likely) augmentation.

Thinking about each attack vector individually gives necessary focus to the process and will help get past hapless statements of context-free specificity. Years ago I saw, “All data must be protected with 128-bit encryption”, within a customer’s standards. Admirable though the specificity was, the limitations of this as a standard become immediately obvious. Would 128-bit encryption suffice for archival-grade needs—such as log files? What does the plain-text or cipher-text data get used for? If we’re looking at attacks on session management within our imaginary web-app, simply encrypting a session token doesn’t effectively protect against theft and replay does it? Remember last week’s entry. There, in my example, I sought to protect against the circumstance in which (due to omission or later maintenance) a Developer forgot to explicitly demand authentication prior to accessing a piece of functionality.

Think of security standards as requirements driving a design that resists the attacks you outlined in your model. This focuses authoring and often alleviates the writer’s block a blank page can impose. Improving and completing guidance becomes tractable. In my crypto example above, the client was attempting to secure exchange of data between two systems (thick Java clients) over the Internet in a circumstance in which they could not rely on SSL being present. Data from each transition persisted beyond a single session, but had a short span of value—say a day, or week (depending on the volatility present in pricing). You’re probably already conjuring ideas about what you’d expect to see in a secure implementation. With as little context as this, my explanation of the design begins to illuminate attack vectors. From these, you can begin to answer the questions I posed above about ciphers, structure of data, and from there secure design becomes an exercise in trading off difficulties of key maintenance, performance/overhead of the encryption, and others with resisting the attacks you’d enumerate.

Not shockingly, HOW you build this guidance is as important as ending up in courier font. And, (again) it shouldn’t surprise you that I’ve chosen to get end up with code (courier) through design (via threat modeling). This is how software is built—and security should be no different. This does mean you’ll need security architects (who can code—not the corporate librarian types) to write good, detailed security standards though.

Technorati Tags: software security

While speaking at a conference out west an interested attendee challenged me: “You said I should make my security standards as specific as possible, but the other speaker said, ‘Keep them general’, what gives?” This type of exchange happens all too often in the software security space these days. I could do a piece on that alone, but instead, I’ll address the challenge.

The confusion stems from two competing goals driving standards creation: 1) providing useful security know-how that benefits developers and 2) obtaining ‘coverage’ of all the security concepts, technology stacks, and development/deployment platforms your organization uses. To be useful to developers–to truly change the way they behave when “Their butt hits the seat in front of their compiler”—one has to speak their language. Developers speak and write code. Documents like security policy, tend to be written by Corporate Security, or worse: lawyers. These groups speak and write legalese. There’s a big difference and it’s easily detected: one usually comes in 12pt. Courier.

Your objective: answering questions about how to do things right for developers by showing them the right way… while leaving enough flexibility and room in the guidance for them to remain creative and solve the business problems their application was intended to.

Writing technology-specific guidance engages Security Architects in helping directly solve Developer problems. Rather than specifying “Do not allow direct access to Servlets by name” (a decent agnostic standard, when used in concert with others) show them how:
——
Using Struts, map an impossible-to-assign role, such as noaccess to every Servlet but one–a single front controller–that mediates access to your other Action Servlets like this:

 <web-resource-collection>
   <web-resource-name>Application</web-resource-name>
              <url-pattern>/functionality</url-pattern>
       </web-resource-collection>
  <auth-constraint>
   <role-name>noaccess</role-name>
  </auth-constraint>
 </security-constraint>
 <login-config>
  <auth-method>DIGEST</auth-method>
 </login-config>
 <security-role>
  <role-name>noaccess</role-name>
 </security-role>

Place all Action Servlets in a single directory, for ease of maintenance (/functionality in the example above). Demand authentication prior to access to the single front controller and delegate actions from that Servlet.
——

Alternatives may be necessary. For instance, while the standard prescribes lumping functionality in one directory–that may not be possible. For those cases, the standard should describe how extension based url-patterns can aid in casting the broadest net possible.

Standards, at this level, should always state a preference however. The worst offense of failing to do so is nearly every J2EE book’s discussion of both declarative and programmatic means of authorization without indicating which should be used when.

Next week I’ll move on and discuss detailed, technology-specific security guidance in more detail, but first I would like to recognize the value less specific guidance provides. Detailed, technology-specific guidance requires significant time and effort to produce. Such guidance is perishable and becomes useless as you upgrade or update your technology stack. Technology agnostic guidance, or guidance kept at the level of security concepts insulates you a bit more. Organizations should certainly start with this level of guidance, getting coverage over the broad array of security topics needed to educate their developers before diving down a rabbit hole and writing technology-specific guidance.

In other words, one level of guidance does not replace the other. Instead, less specific guidance serves as safety net underneath the more specific, catching inquiring minds when the specific guidance hasn’t been written yet or when it doesn’t apply (as often happens when a team faces constraints like deploying an old version of Tomcat).

I hope, however, that in the meantime I’ve shown an example of how being technology-specific, code-centric, and detailed about standards can engage security folk in development, engage Developers in their own language, and actually push projects forward more quickly by making hard security decisions for them. This is just one of the activities your Security Architects can undertake when they parachute into development teams… a concept I introduced in my blog entry on research in the 50’s.

Technorati Tags: software security

Sammy aimed two recent entries at those attempting to govern security and expenditure in an organization. I’m using his posts as license to wax more philosophically. Specifically, I’m going to use Digital Rights Management (DRM) as a lightning rod for conversation about protecting data end-to-end in one’s system (the topic of my next–far more focused–post). I’ve been thinking about this ever since McGraw’s Dark Reading column: on Vista, and it’s driven me mad.

In posting this message, I’ll skirt more topics than I cover; I apologize. In return, I’ve included a lot of links worth reading. I don’t presume to answer the question, “Is DRM ever a good idea?” I believe most computer security folk simply answer “no,” bemused because in the end they know consumers MUST use protected data and algorithms in their full quality, to be satisfied. Some gifted albeit misguided security folk attempt to trade data quality for what they perceive to be security. Peter Gutmann’s working paper covers Microsoft’s attempts through Vista.

When I replied to Gary’s Dark Reading article, defending Apple’s DRM, I had purchased about $350 worth of music from iTunes. The standard caveats applied:

1. Apple could change their DRM scheme out from under consumers at any time
2. DRM still provides no value from the consumer’s perspective
3. DRM imposes limitations, potentially limiting competition between vendors

but I still found myself unable to come to any other conclusion but the mighty unpopular: Apple’s DRM seems a reasonable compromise between affording protection and remaining flexible to consumers. Remember, I said ‘unpopular’, so feel free to argue with me.

Now, $450 into the fray, something very interesting has happened: Apple and EMI have decided to sell some music electronically but DRM-free (see:
http://news.bbc.co.uk/2/hi/technology/6516189.stm ). Unprotected music will be encoded at 256KBps (twice the bit rate of Apple’s protected files) and will cost $1.29, rather than $0.99.

Now, consumers have an interesting choice to make: “Do I want to pay an extra $0.30 for the ability to copy the music I purchased freely?” Some may be willing to pay extra for the quality–but that’s not what we’re interested in right now.

In asking local “kids,” I’ve gotten consistent reply: “I’d only pay more if I liked the band.” Questioning them further immediately reveals they perceive the music to be free (and rightly so–it’s just too easy to pirate). They make their purchase/steal decision based on their feeling of loyalty towards the band. Can your business rely on such good will? I used to pay for single pieces of bulk candy at the grocery store–but I wouldn’t bet my business on others doing so. At the same age (prior to driving) I’d hack games’ copy protection, give copies to my friends and say, “Ok, but call me before you play it… to make sure only one of us is using it at a time.” Why am I only now explicitly aware they never called? Guess they never played it ;)

More fundamentally, will consumers perceive your product or service as being free? Think back to Windows ‘95… did anyone you know have a problem copying it?

An interesting question to ask one’s self is this: At what increase of price can I afford to protect my data less for the sake of other business drivers? Presumably, EMI has come to the conclusion that the answer, in their case, is $0.30. Interesting. If protected music accounts for only 2% of sales (dubious estimation by extrapolation of iPod size + sales compared to iTunes sales), how much does EMI expect that number to jump if the price has moved to $1.29? If physical sales still occur at $1.99 per single, how much less piracy does EMI expect from electronic sources as compared to physical ones? What does this even mean–given that pirated physical wares will almost immediately take the electronic form for ease of distribution?

Finally, what numbers could Apple put on the cost of developing and maintaining their DRM scheme? What affect does that price have on their profits (if any) in the case of EMI’s unprotected content versus that which is protected?

Part of me wonders if any of this will have even the slightest effect; given the ubiquity of the protected content, unlocked and freely available from peer-to-peer sites. If all’s for naught, is EMI making a good decision, or a mistake? God, I hope there’s no analog for that in your data.

I have no idea how successful Apple’s iTunes model will be, or whether or not consumers will accept Vista’s DRM, but with EMI’s decision to distribute unprotected music along side protected songs, those of us in security-land potentially gained data to look at. What will it tell us? And, while it’s very unlikely that you’re distributing electronic music, dear reader, or even passing content directly to end-consumers, EMI’s move should raise questions about how you’ve calculated the value of data, the protective mechanisms you’ve placed around it, and the impact on usability those protections imposed.

Technorati Tags: drm, apple, data security