I was part of a panel at a university recently speaking to prospective computer science students. The panel members were from industry — a few of the biggest companies and few smaller ones. We each had ten minutes to speak before QA, so I stripped my talk down to two simple points. The first point was that IT is fun. We truly build amazing things. I’ve been in the business 35 years, and I have never been bored. Each day offers a new challenge. The other point was that there will be plenty of jobs for decades to come just trying to get things right. Software sucks. No product on the market has more defects than software and no projects have a failure rate close to that of IT development work.

The next generation can live off the argument that they can certainly do better than the fogies doing the work now. The challenge for them, as it was for us we made the same claim, is living up to the bold promises. I have been through five distinct new approaches to building software, each designed to deliver better products and better delivery management. Each has moved the ball forward a bit, but we still have a LONG way to go.

Just how bad are we? There are many studies of failure in IT, but the results are all over the place. The most optimistic estimate I can find says that about 30% of projects fail. When I mention that number, a fair number of people tell me it is too low, but let’s grant ourselves the benefit of the doubt. We fail 30% of the time.

How does that compare to other industries? Before we take on the masters of quality lets start with an easier opponent – how about used car salesmen? Every year in United States, about 45 million used cars are sold. The Feds estimate that the odometers of about 450,000 of these have been tampered with, and that in another 500,000 cases the seller fails to disclose a serious known defect. That works out to a failure rate around 2%.

We in the IT business are 15 times worse than used-car salesmen. Yet, the demand for our products and services is tremendous. IT related professions dominate the top-20 positions on the US Department of Labor’s list of high-growth jobs. Why is this so, if our products are so bad?

The answer is that software, bugs and all, delivers real value. We would love our programs to work without error, to never crash, to never need patching, but even with these sorts of problems, software does amazing things. Once in awhile we all encounter a program that makes us go wow. Remember the first time you used online mapping? That was a wow, but we quickly get bored. The amazing becomes commonplace — part of the wallpaper. That’s sad. I wish it was possible to flip a switch, and show ourselves and the world juts how far we’ve come. It would be a cruder, poorer, and much more boring world without us.

That said, software is still crap – amazing crap so customers will continue buying, but crap nonetheless and we owe them better. The challenge of making software better is a difficult one. All of the Justice League blog entries are, at root, about this one subject. Sometimes we write about broad issues other times about the very specific, but our objective is always to make software better. Better functionality, more reliable, and more secure.

This blog, rather than being about solutions, is about the intriguing, challenging, and sometimes frustrating dichotomy of software, it is both amazing, with a capacity to dazzle and inspire while it is also mundane – just a bit of bug ridden code. The Roman God Janus has two faces – one facing forward, one backward.

Building software is much the same. We constantly envision the future and it drives us to imagine and to realize that imagination we build ever more complicated systems. There is the rub. The other face of Janus looks back to the past. In IT that is the hard job of not making the software, but making it good. Janus was the god of doorways, a wonderful image for IT. The doorway to the future in IT rests on equal measure on vision and quality, looking forward to what can be done and backwards at what we have. We must aspire to be Janus.

A final observation that will set the stage for a future blog entry. Our troubles lie in the imbalance between the forward looking face and the backward. There is so much potential in software that we are constantly reaching farther; farther in some cases than we can deliver. It is the reaching that gets us unto trouble, but it also the reaching that keeps us energized and fresh, and once in while makes our customers say “Wow.”

While reading last week’s Economist, I stumbled on an article on Innovation (available without a subscription online). The article discussed how commercial entities have changed the way they fund R&D. They’ve fundamentally changed the structure of research and development groups–as well as their interaction. I began my Cigital career in the company’s research division and later moved into a purely operational role as “Consultant.” “Principal,” my current role, demands artful compromise between the two, so the article immediately caught my attention.

When a client asked about “Getting in front of AJAX from a security perspective…” this week, I connected dots between organizations’ need to innovate and the realities development require.

The big change: Big research labs (such as PARC, Bell Labs, IBM’s labs) have become extinct and hoping for “tech transfer” after stand-alone research investment represents a fatal chasm between “Research” and “Development” unlikely to be crossed… or improperly exploited even when crossed.

I see an analog for this in today’s software shops, especially pursuant of security. Right now, organizations don’t understand software security well enough to treat it as development. For them, it’s still research. While not science, in its strict sense, people don’t know much about software security. They’ll have to read, think, experiment, and be very creative before moving into a engineering/development mode.

That’s why my client said, “We want to get in front of AJAX…”; he, his organization’s software security guardian, sees a problem development can’t solve, and he doesn’t yet have the answers either. He needs a leap or two forward before he can communicate solutions to hard problems well enough to empower development to “build security in.” These problems include personal data protection, authentication, and even the very process by which his organization creates software. That’s probably why he called Cigital.

My client faces a common situation. Organizations drive their businesses, for the most part, reactively. Product companies react to customer needs (features, deadlines) and IT shops react to opportunities and risk the businesses they support face. Unless driven by compliance, publicized incident, or a similarly dramatic driver, these businesses’ development teams don’t react to software security’. As a result, development doesn’t possess engineering skill and experience in software security.

What to do about it: Organizations depending on innovation must 1) foster practical engineering (development) behaviors in those individuals with the horsepower for research and 2) embed them in development teams towards a practical end.

First, get qualified individuals. But, I’m not referring to wild-haired mathematicians. You probably already employ some of what you need in your org. I’m talking about your gifted individuals that attend conferences, avidly consume literature, and possess an advanced degree in computer science (or similar). These individuals frame the problem, hypothesize a solution, observe, then iterate.

This person probably has an uncanny knack for coming up with lateral approaches to your design problems. They may be a complainer–ineffectively shackled by their current position. A former mentor to me used to comment, “You seem to always have ‘just read something’ useful in every meeting I bring you to.” That’s what you’re looking for.

I like to focus on “Security Architecture,” so I look for these types in the Enterprise Architecture group (though, the do-nothing librarian type need not apply). Specifically, you don’t want someone that ONLY reads, or with no (or outdated) hands-on technology experience. We’ll call these people your innovators.

Second, embed these people IN development teams. Preferably, innovators should be ‘leased’ to the team solving the difficult problem: securing credit card data without increasing its (representational) size unduly, for instance.

As part of a development team, innovators should be held to a schedule and budget, like everyone else. I like the idea of managing them to a final delivery a release or two out, but with important constructive deadlines within the current release, showing progress. Managing these interim deadlines between “Hail Mary checkpoint” and “incremental main-line progress” depends on how ambitious the overall goal is.

I argue that organizations lose when they fund research with development project money directly or comp. (bonus) innovators with project metrics directly. Having these individuals report to someone else (shared services?) will allow them to focus on their technology goals. If their manager sits as a peer to those managing development, tough conflicts regarding project schedule and security program priority can be resolved reasonably.

However, that’s not to say that the development team shouldn’t participate in cost though a “tax.” After all, they do benefit from the extra horsepower applied to their project. Likewise, the researcher should be evaluated, in part, based on development project impact.

This approach naturally lends itself to a consulting model. You can hire someone or use your own internal team (which one of our clients does particularly effectively). First, make sure they’re qualified: exploiting code can be pretty easy and some consultants give bad advice generalizing quickly from something they’ve seen once. The “use POST not GET” advice OWASP gives serves me consistently as an example. I’ll cover this example and poor advice in a later post.

Beware the “’sploits-and-splash” failure mode. Embedded security resources must be constructive. Simply pointing out how not to do things doesn’t produce secure code faster and disenfranchises developers. Advice about how to pay and measure these individuals applies here. Instead, focus innovators on the tough design problem it will likely take two-to-three releases to iron out, and that might be useful on a whole class of products within a line of business. Make them responsible for producing a solution; not just pointing out problems in every product team’s smaller attempt at one.

Cigital believes in research, in the true sense of the word. We talk about the value a lab provides in each of our strategy meetings. We believe in practically applying more theoretical solutions in a consulting model as well. I believe injecting a research mentality, as a ‘thread’, into development serves as counter-weight to reactive mentality, helps solve software security problems, and gives the necessary horsepower boost to “build security in.”

I think you’ll find that architectural flaws your organization’s software exhibit require solutions “bigger than a single release” and set the stage for this topic nicely. The alternative? Fall prey to a purely engineering mentality making small incremental improvements that remove a single attack vector but leave the endemic flaw at large. This causes continual development churn and leaves the software exposed, ultimately.

Technorati Tags: ajax, software security, enterprise software security

Sure, you can’t throw out all your existing code and start over, but you also can’t hide it forever. It wouldn’t surprise me even a little bit to find that your super-slick, Ajax-improved front-end that’s written in the latest Java is actually using a JNI call to kick off some C code that no one knew how to rewrite because it’s calling some COBOL that no one has looked at since before Y2K that may actually rely on some Fortran 77 for a particularly complex calculation. And, of course, your authorization and entitlement database is on some Unix-clone variant on, say, an old IBM mainframe somewhere that talks some weird protocol that only one guy understands (and he’s about to get a gold watch in his stocking). In addition, did I mention that imminent end-of-life notice about both the hardware and the software you’re running on that box in the corner that no one touches (the one that spits out quarterly financials).

If you’re waiting for the one and only security answer to appear before you start making improvements in this installed base, you should probably polish up the old resume. You’re going to be selling yourself before you’re able to sell that wait-and-see strategy. To put an even finer point on it, even if the perfect security solution came along today, it would probably still take you years to get it in place because you’re not set up for success (because you’ve been waiting for the perfect drop-in software security solution).

Software security isn’t going to happen that way. No one knows the one-size-fits-all answer yet, or even if there can be one. But we sure have a pretty good handle on how “…chance favors only the prepared mind” (with apologies to Mr. Pasteur). Even if you can do only a few things, get started.

Institute some governance. If your executive group has application and software security on their radar, then it’s much more likely to get in the budget.

Know your business. You cannot make informed and wise security decisions if you don’t know what you’re trying to accomplish.

Document your software development lifecycle. If you can describe and demonstrate your process, then your process can be improved when the time comes.

Give skill-appropriate awareness training. If your product managers, architects, business analysts, developers, and testers develop a sensitivity to the security issues associated with what they’re working on right now, it will get better. And the next thing will be better still.

Go start now.

Technorati Tags: software security

Never one to mince words, I coined the term badness-ometer to describe “application security testing tools” like the ones made by SPI Dynamics and Watchfire. For whatever reason, people read more into the term than I intended. I guess they see the term as having only negative connotations. I stick by my nomenclature–black box application security testing tools are in fact badness-ometers–but badness-ometers are a good thing and everyone should use them!

Here’s part of what I wrote about badness-ometers in my book Software Security:

That said, application security testing tools can tell you something about security—
namely, that you’re in very deep trouble. That is, if your software fails any of the canned tests, you have some serious security work to do. The tools can help uncover known issues. But if you pass all the tests with flying colors, you know nothing more than that you passed a handful of tests with flying colors.

Put in more basic terms, application security testing tools are “badness-ometers,” as shown in [the Figure below]. They provide a reading in a range from “deep trouble” to “who knows,” but they do not provide a reading into the “security” range at all. Most vulnerabilities that exist in the architecture and the code are beyond the reach of simple canned tests, so passing all the tests is not that reassuring. (Of course, knowing you’re in deep trouble can be helpful!)

I also wrote an article for Dr. Dobbs called “Beyond the Badness-ometer.” That article stresses the fact that solely relying on a badness-ometer as your only software security activity is a really bad idea.

So what’s good about badness-ometers, and why do I think you should buy one right away? Well, many organizations that build software are woefully in the dark about their software security risk. A badness-ometer can do wonders to turn the lights on with respect to software security, especially when it comes to Web applications.

The sad fact is that many purveyors of Web applications believe that their apps are “bulletproof” if they use simple security features like authentication and SSL. Of course we all know by now that software security goes well beyond security features and deep into enemy territory, concerning itself with things like software defects (bugs and flaws) that lead to software security failure and unacceptable business consequence. Badness-ometers can help expose this “myth of security features” for what it is–a myth–by automatically attacking and taking down Web applications through obvious everyday security tests. Automated testing is cheap and sometimes powerful.

The great irony of badness-ometers is that you can be sure that your enemy will use them. In fact, throughout the decade that I have been practicing software security, bad guys have been more adept at adopting advanced tools and techniques than the good guys generally have.

When it comes to deciding whether your organization needs a badness-ometer, you should ask yourself whether you already know your software is at risk or whether you need some more convincing. If you, or anyone else in your organization, need more convincing, grab a badness-ometer and find out whether your code is in “deep trouble.” I bet it is.

If you are using a badness-ometer today, and it’s finding issues in your code, don’t simply fix the issues and call it a day. Never forget that the badness-omemter can’t tell you that you’re secure. It can only tell you that you’re not. To get beyond simple badness-ometer tests or to fix the problems that your badness-ometer is finding, seek professional help from Cigital.

Technorati Tags: software security, badness-ometer

Because Cigital spends time helping clients document “technology-specific” security standards, to aid developers and architects, I get asked, “What do you think about [new technology XXXX]” alot. Questions regarding AJAX have crossed the threshold, so I’ll post what I think here.

Quick disclaimer: I make no comment about the technologies or security in a Web 2.0 world (I’ll leave that to Captain Technology Curmudgeon, Scott).

The world still wrestles with how to best use AJAX, so a lot remains in flux. But, I believe if you’ve got teams prototyping its use, you’ll need to consider the following:

1) Picking an AJAX toolkit - There’s a quite a few out there and each covers different ground. What’s more, people are plugging in their own approaches to things (like password management) into toolkits and riding the wave.

People always used to ask, “Websphere or Weblogic?”. I’m no product’s salesman. Usually each possesses strong advantages and disadvantages. I wanted organizations to understand tradeoffs and make an informed decision themselves. One group heard me out, took a look at price, and picked Weblogic. Their rushed decision forced them to need _me_ to implement clustering robust enough to handle the scale they desired; Plain shortsighted.

Just like your choice of containers in the Enterprise Java world, your choice of AJAX toolkits provides much of the backdrop for your software development efforts. Because the toolkit handles a lot of client-server plumbing, the security posture of your application will depend on the toolkit’s implementation. Do not make this choice lightly. Consider elements of security along-side supportability, usability, and the other decision factors.

You need to technically vet the toolkit. As I said in the case of containers, this toolkit will act as a security “table cloth” that could be pulled out from under ALL the applications you implement on top of it. Have your security group leverage any existing threat models it’s constructed for n-tier applications so that they’re not starting their analysis from scratch.

My next entry will indicate what “development scenarios” one should put each toolkit through the paces on in order to help their development teams be productive.

Moral: Spend enough time prototyping with candidate toolkits to make an informed decision about “best for you” and the realities of integration.

2) Exploring Toolkit Architecture - A natural ‘next step’ after #1, perhaps the later stage of a thorough technical assessment into a particular toolkit. Any time you move to a new technology, you need to “get smart”. Most people skip this step unfortunately. An architecture team can save security guys a lot of time by steeping themselves in a technology, writing a primer, and distributing it to developers.

When financial companies started writing their back-ends in Java, they all fell down on String processing. They just weren’t prepared for how differently (and expensively) this ‘new’ technology manipulated strings. I spent a lot of time with development teams showing them, on a whiteboard, how XML-processing affected Java’ memory management–and how to ‘game’ it. You want to “fight these battles” once, centrally, and then share the battle scars before they occur again.

In AJAX, we’re finding toolkits handle different segments of requests client-side and impose differing amounts of requests on client-server interaction. Find out how your candidate toolkits handle requests and which side of the client-server divide weight for those factors affecting security fall. WRITE THIS DOWN. Get an adoption and user’s guide out to your earlier adopter teams. Have them write version 1.1 of the document.

Moral: Share institutional knowledge gained by vetting toolkits.

3) Don’t leave real design time with short-shift to ‘buzzword’ architecture - Selection of a platform, framework, or toolkit (perhaps all three) does not constitute an architecture. Patterns, messaging protocols, threading/event models, state machines, and similar high-brow concepts constitute the language of Design. Design should address both the structure and behavior of a system. Toolkits may help address design concerns and certainly impact design–but they do not themselves constitute the design.

During an architectural analysis of a 1.0 –> 2.0 product transition, the chief architect said, “Well, we’re just going to use struts on top of tomcat, the end. Oh, for authentication we’re using Acegi.” After extended silence I asked, “In terms of how many factors does the authentication decision get made and how will you represent those factors in Acegi and elsewhere?” The answer to these questions plagued their development team for over a month. As it turns out, they couldn’t just ‘outsource’ authorization design to the fine members of the Acegi framework.

“Buzzword” architecture produces no better a result when AJAX makes the list. You still have decide what’s asynchronous and what you’ll make more persistent connections to resolve. The matter of dividing model, view, and controller (especially the last: controller) between components and tiers becomes more difficult. With a distributed application controller, application state transition becomes sticky.

Moral: Don’t let enterprise or application architects off the hook for design because they say [Technology: AJAX] will handle it. Make them specify real design; it’s their job.

More news @ 11…

-jOHN

Technorati Tags: software security, ajax

Recently, someone at Cigital asked me to characterize the difference between our approach to software security and Microsoft’s. Before I get to comparing things I want to note that we’re big fans of Microsoft when it comes to software security. Under the leadership of Michael Howard and Steve Lipner, Microsoft has made great progress in software security over the last few years. There are many lessons that we can all learn from what they’re doing.

That said, Microsoft’s approach to software security does differ from ours in a few fundamental ways that are worth discussing.

The biggest difference to highlight is that their approach has a static threat model that is great for operating system vendors. People in other sectors such as finance or hospitality will not have the same kind of threats that Microsoft does. In this case, what I mean by threat is an actor or agent who causes risk. That is, your possible attacker(s). Building a threat model that is tailored for a customer is a key aspect of our approach at Cigital. Microsoft has already done this for their business, but chances are you are not an operating system vendor.

At the heart of Cigital’s approach to software security are a number of best practices that we call the touchpoints. You can read about the touchpoints in my book Software Security. The top two touchpoints are “Architectural Risk Analysis??? which helps uncover and mitigate flaws at the design and architecture level, and “Code Review with a Static Analysis Tool??? which helps ferret out bugs in code at the implementation level.

Our approach to Architectural Risk Analysis is much better suited to finding new risks not yet ever seen before than Microsoft’s list-based approach. Undiscovered new risks are the kinds of things that lead to 0days and help you to wind up on the front page of the Wall Street Journal. Our three-phase analysis goes way past Microsoft’s STRIDE model with regard to dependencies and ambiguities. Our approach to risk is much more sophisticated and grounded in business as well. The reasons for this probably have to do with having to demonstrate to our customers the importance of the things we’re finding. By contrast, Microsoft doesn’t need to be convinced to fix security problems, Bill Gates already said that they have to.

When it comes to code review, Microsoft and Cigital approaches are similar, but Microsoft leverages proprietary unreleased tools that are not available outside of Microsoft. (Many of you will have heard of Prefix…Prefast, now available with some Microsoft compilers, is not the same thing.) The Cigital approach leverages customizable commercial tools and a best of breed approach. We often use Fortify’s supremely good static analysis tool, but we have been known to wield Coverity’s tool and Klokwork’s tool as well, depending on the situation. In both cases, however, the use of a static analysis tool to find bugs is absolutely critical.

Our approach to security testing is much more white box than Microsoft’s is, and involves using the risks uncovered during Architectural Risk Analysis to drive inside-out test planning. Microsoft’s approach is grammar driven and focused on APIs. The fact that James Whittaker now works for Microsoft is likely to help to evolve Microsoft’s approach to security testing, but James has a tendency to start with API fault injection as well (see his great book How to Break Software Security).

As far as I know, Microsoft does next to nothing at the requirements level. At Cigital, we have a decent approach to security requirements that covers both functional security mechanisms and abuse cases.

There are many other differences between our approaches, but those are the highlights. The most interesting thing is just how closely aligned the two approaches are.

Technorati Tags: software security, microsoft, stride, sdl

If I were to say penetration testing, what would you expect? After conversations with lots of colleagues and friends it’s become fairly clear to me that the term is massively overloaded. It actually got me thinking about the evolution of pen-testing over the last several years. And I’m not going to preach from the you-need-design-and-code-review-too pulpit. I’m just gonna talk about the innards of the pen-test itself (breaking a running system).

The old-school notion of a pen-test was one where you’d hire a bunch of l33+ h4×0r5 and turn them loose on a system to see what they are able to bust. The general idea here was to simulate a real-world attack scenario (black-box pen-test). Given adequate time, Cheetos^® and Red Bull^® you’ll probably be left with some findings that are interesting. But take a closer look at the breakdown of how those testers spent their time. Those guys likely spent a ton of their time trying to figure out how the app works, trolling for non-obvious interfaces and partially reverse-engineering the logic of the app. When you’re paying by the hour, this is bad news since they didn’t spend as much time as they could have in actually trying to break the app (they spent a bunch in just getting their arms around it.)

Evolving from these obvious limitations in the completely black-box approach the notion of the “pen-test with code” was born. This was a similar scene, but instead of poking at the running app as much, the testers would go and read code. While the testers are still at arm’s length from the dev team, it’s still better since access to the code gives a truer picture of what’s actually happening under the hood without time spent on reverse engineering. Sometimes, there would even be design docs available which really helped move things along. In the end, this is close to the modern-day version of what many expect when they buy a “pen-test.”

The major weakness of this style of pen-test is that there’s often no business context to ground the direction of the testing (on which aspects of the system you concentrate) or to baseline the value of the findings (to the company using the app, what do the discovered exploits really mean?) Some pen-testers are starting to bridge the technical-to-business gap, but they’re definitely in the minority. This step in the advancement of the pen-test is definitely a step in the right direction since it focuses the testers on generating and prioritizing findings that show demonstrable impact to your core business (as opposed to sheets of meaningless XSS vulns that are all prioritized “high” since XSS is bad.) The pen-tester gets to concentrate on breaking business logic in your application and scheming about combinations of technical vulns that lead to an interesting business problem. You’ll still get all the technical vulns, they’ll just be low priority if they can’t directly contribute to a bigger problem. Since this all plays into making risk management decisions, I call this style of white-box pen-testing the “risk-based pen-test.”

If you’re using pen-test today, you really want as close to a risk-based pen-test as you can get. Although I have heard a few reasons why you might not:

• “I don’t have access to code or design docs.” I hear this reason all the time as justification for why a black-box approach is used. In the end, even if you don’t have code, someone has gotta know something about how the application actually works. Even if it’s only from a user or sysadmin perspective, it goes miles to setting the testers off in the right direction. Find those people and link them up with the pen-testers! And business risk mapping is still possible even without code (but you’ll probably need at least ad hoc design info).
• “I use black-box pen-test nowadays as a way to simulate real-world attacks.” This notion is just off-base due, perhaps, to the lack of understanding about real-world attacks. Yes, the people “in the wild” that might attack you will have a similar skill-set to your pen-testers. The kicker is that the time available to an outside attacker is virtually infinite compared to that of a for-hire pen-tester. When it comes down to exploiting software often it’s just a matter of how much time you spend bashing your head against the problem. Outsiders can spend 2 hours a day for 6 months working on getting a single exploit working. Your pen-testers have 8 hours a day for a week to find lots of impressive results. Thus, cheat the problem by giving the pen-testers all the info you have.

In any case, let’s get back to talking about that notion of the risk-based pen-test. When I think about the requisite skills to complete that job I see many similarities to what QA folks do on a routine basis. Foremost, the QA testers will already know the application components and UIs. They should know the business value of the app and in many cases they’ll also know how the business logic of the app is supposed to work. And also, they are routinely asked to assess the specific business impact of problems in the application (they do it every time they open a bug and assign a priority.) That’s a huge advantage over someone from the “outside” coming into a pen-test engagement. Now what about differences? The biggest red-flag is that the QA people aren’t trained to attack applications to find security flaws. That’s a big disadvantage since you need those skills in order to be even remotely effective at a pen-test. But do keep in mind, QA folks are very much trained to break applications in general (it is, in fact, their job.)

So where are we going with this? Simple: let’s give the pen-testing responsibility to the QA team. They’ve already got a leg-up since they know the application. To counter the point about them not being trained in security attacks employ two simple techniques: tools and training. Some of the automated pen-testing tools out there are really great now. They’re basically extremely potent packages of electronic subject-matter expertise. Now, I would never advocate just buying a tool and hitting the “go” button and calling the job done (you don’t get very good results at all this way.) Enter the training. The automated pen-testing tools, in many ways, are very similar to other types of tools in a QA tester’s belt. They run a series of attack test-cases and report vulnerabilities when a test fails (this is a massive oversimplification of what they’re doing under the hood, but in terms of usability, the analogy holds.) So, train the QA testers on effectively using those tools. Teach them about how to feed application-specific details to the tools to make sure your coverage is high and ensure the results are more accurate.

Further, teach them about the classic notion of security testing (starting with requirements and deriving test-cases to ensure that functional requirements are implemented securely) and show them how to automate it with the tooling. In fact, that’s where this is all ending up: a risk-based security test with a wide blast-radius for getting the bulk of the benefit of a pen-test. It’s cheaper in the long-run (front-loaded cost for buying tools and building the internal skill vs. repeated, indefinite fixed-price cost.) In terms of effectiveness vs. an external pen-test, it’s definitely on the positive side of the 80%-20% rule (and all the things you might miss in that 20% will likely be caught by code review or architecture analysis.) It’s even “faster” in the sense that an organization could move through each assessment more quickly, thus enabling more assessments within a given timeframe.

In summary, I see the future of pen-testing as a push toward QA environments. Since the notion of a code review and architecture review are becoming more mainstream, this makes sense. Why continue to spend a lot on a service that ultimately should be used as a sanity-check of the running system?

Red Bull^® and Cheetos^® are trademarks of their respective companies.

Technorati Tags: penetration testing, software security