(This is part one of a three part series. Parts two and three will appear on Wednesday and Friday.)

The beautiful thing about learning is that no one can take it away from you. -B. B. King

We’ve all been to training. More specifically, we’ve all been in training where we were bored to tears, not because we were already experts, but because there was just no connection between our individual learning style, base knowledge, interest level, or job goals and whatever it was the instructor was doing. Invariably, however, there will be a few people in the same class who feel this training was the best they’ve ever attended. Two days later, you can’t even remember what the course was about and two months later others are still quoting the instructor. Why does this seem to happen so often? In my experience, that it was effective even for those few people was probably just luck.

When you’re thinking of making or commissioning some technical training, stop. Completely. Not one of those rolling stops you do at a stop sign in the middle of nowhere. Sit down and take a deep breath. Now, write three complete sentences:

• “This training is appropriate for people who…???
• “In this training, you will learn how to…???
• “After this training, you will be able to…???

If you cannot specifically and succinctly define your audience, tell them what they will learn, and describe how their new ability will benefit the organization, you’re definitely not ready to start building training. This would be just like writing code without requirements and you’re likely to achieve the same spectacular lack of success.

If you need to do a formal performance needs assessment to get requirements because you’re teaching someone a life-or-death skill (e.g., how to disarm landmines in the field), then do it. On the other hand, if you’re teaching employees the basics of physical security for the average office building, you can probably just survey the existing literature to get your main points.

I understand that there is some training that must be given to everyone regardless of their interest, skill level, learning style, or anything else. Examples might include employee orientation training, sexual harassment training, security procedures training, and so on. I feel it’s easy to show how poorly the average one-size-fits-all training approach has worked in these areas, especially when reduced to a poorly-scripted eLearning module or abysmally-acted short film.

Even after you get a good course description and, presumably, the right audience, tell people right up front what they’re going to learn in your course. Don’t make it something cheesy like “Today we’re going to learn about software security.??? That’s just silly, even if it’s for a general software security course. Tell them something useful like “Today, I’m going to show you how to generate crypto seed values correctly in various languages and technology stacks. By doing this correctly, you will help the company meet ongoing regulatory compliance requirements.??? If it is the general course, tell them something like “Today, I’m going to give 10 reasons to care about software security and help save millions of dollars.???

Well, that’s different. Now I know why I’m here and how what I’m going to learn will really help; it’s not just some flavor-of-the-month exercise because it’s “time to shake things up around here.??? Continually reinforcing why this training is important, both during the class and afterwards, means you’ll end up not only with increased competence in the target audience, but, in this case, you’ll also end up with practitioners who are more business-risk aware. That’s a good thing. Why? Because now, they’ll ask themselves questions about “How does this affect the business???? and, when they don’t know the answer, they’ll go ask someone rather than just putting some code together. Give them the chance to do the right thing.

Remember, it is a significant task to get a diverse group of people to leave a room with the basics of a new skill (e.g., developers using certificates for mutual application authentication), with necessary knowledge (rand() is not cryptographically secure), or even with just a common awareness of a given problem (e.g., web application input validation is tricky). This is compounded when we let training creation wait until the last minute and then “throw something together??? or we “just use last year’s slides.???

The biggest evidence of such procrastination and lack of planning is ending up with training materials (e.g., PowerPoint slides) that are simply about the topic, not on how to do the topic. How often have you gone to a class guaranteeing to turn you into a practitioner, only to spend the majority of the time on vocabulary and concepts and fluffy cloud process diagrams! What a waste of time. Training about test process automation, for example, is something that you give to people making budgeting and resourcing decisions. Training detailing a method for doing test process automation is what you deliver to practitioners.

Speaking of planning, in my experience you have to budget about 100 minutes per technical information slide to create useful material that focuses on imparting a single important idea in a memorable fashion. This includes sufficient time for instructor and student notes. The instructor notes should be what the instructor says while the slide is being projected, along with any prompts for what to do, what to hand out, and so on. The student notes should include specific reinforcements, references, and related materials applicable to the information on the slide. This 100-minute block also includes internal review, discussion, and bootstrapping the first instructor.

I know 100 minutes per slide sounds like a lot, but if you’re spending less than three days to plan, draft, and complete one hour (about 15-20 information slides) of technical training (or any training that is telling someone how to do something), you’re probably messing it up. In particular, take the time to look for every group of words, bullets, or slides that you can turn into a real, accurate picture. Decorate the picture appropriately, but don’t make it so busy that it becomes an eye chart. Animate it whenever necessary to show how and when things actually happen. People will remember and refer to these pictures (because they can cut and paste them), but they probably won’t ever reuse some random set of bullet points.

This reuse habit, by the way, is also why it is universally idiotic to include on-the-fly sample code that someone just banged out for the slide; it’s almost certainly wrong, not secure, and against some policy and someone will be embedding it in some critical code module tomorrow. If they do that, it’s just as much your fault as it is theirs.

Another word of advice: Don’t have technical experts create the final training product. No, really. Even if you’re the technical expert and you think you know better. Trust me. The only exception to this might be when a technical expert is creating slides for very similar technical experts. Even then, run the slides by someone a level or two below and a level or two above “technical expert,??? as well as someone more skilled in training presentation, and heed their feedback. It’ll be a valuable experience and you’ll end up with better material that is less bogged down with jargon, with incomplete thoughts that presume too much historical knowledge, with acronyms that are never explained, and with words that apparently have a different definition on the author’s home planet. One hopes you’ll also end up with something that isn’t 100 slides of wall-to-wall words.

The long-term benefits derived from putting the required time and skill into building good training will far exceed the initial investment. Spend a little extra time on it and make your students happy to be there. Everyone will benefit.

Technorati Tags: training

I’d like to give a slightly different perspective on a topic John Steven talked about a few weeks ago (“Keeping up with the Jones’ Security Initiatives???).

Be a goose; don’t spend “10%??? just because it’s a popular number.

I spent the first four years of my career, in the early 1980s, in the Air Force. I worked as a systems programmer in the Pentagon and had direct responsibility for system security (Go Multics!). This was a timesharing mainframe with directly connected VT100 terminals in secure locations, so threat was fairly well understood. It was all about availability then, even though security was paramount. If the system was down, heads rolled. On the other hand, if some MLS control prevented the general from doing something he thought would be cool, well that was just tough. No one ever asked me, “Do we have the right level of security?”; it was always some question about specific vulnerabilities and how to remediate each one on a case-by-case basis. These were ducks.

As a defense contractor employee, I worked with dozens of classified and unclassified systems, some on the burgeoning Internet and some not. I performed virtually every kind of security review, pen test, IV&V, and tiger team you can imagine. No one ever asked me, “Do we have the right level of security?”; it was always some question about specific vulnerabilities and how to remediate each one on a case-by-case basis. These were ducks, too.

After 12 years in the commercial world, I’ve seen or worked with virtually every information security technology. And, although I gave up software development a long time ago and pen testing more recently, I still try to keep current. I’ve worked with hundred of organizations on thousands of security issues. In my experience, only in the last few years have some organizations begun to look past the individual assessment results and ask about their level of security and its overall appropriateness (first in financial services and later in other public companies). At last, a goose or two.

However, the vast majority are asking about it solely in relation to their peers. These organizations are not asking, “Do we have the right level of security????, they’re asking “Do we have about the same amount of security as everyone else, good or bad????

This is wrong thinking and here are two reasons why it bothers me.

The first is the large number of organizations that are insulted at the mere insinuation that I can “know them” even if I have years of experience and I’ve worked with other firms in their vertical, or even with other business units in the same company!

The second is that they’re right. You can’t really know a given organization just because you’ve worked with its competitors. I can understand implicitly the risk associated with their transaction processing systems, with their SOA framework, with their Internet-facing systems, with their overall approach to security, and so on. On the other hand, I really have to work with them to understand what drives them, what is the tone at the top, what decision will they make when push comes to shove, their risk appetite, where they will cut IT dollars first, whether they really are trying to act strategically as opposed to simply having a 3-year plan of tactical initiatives, and so on.

So, why would these organizations think that I can’t know them by working with their competitors, think they can know something about themselves by comparing furlongs per fortnight of security spending with their competitors?

Here’s are two admittedly loosely related stories:

I did my taxes a few weeks ago and was told by the application the percentage of tax-paying Americans who were “like me” in income and tax burden, with no real additional information. Were these families or single filers? Did we have similar kinds of deductions? Did we have the same cost of living? What did these comparisons mean? Duck or goose?

I went to my doctor recently and was told the percentage of Americans whose weight, cholesterol, and related items were similar to mine. Here, however, I was also told how each of these items factored into overall health. In gruesome detail, I was told about various mortality rates, stroke rates, heart attack rates, cancer rates, and so on until I simply wanted to nibble lettuce for lunch and stay out of the sun forever. But, still, did these other people have my heritage, my work and exercise habits, my eating habits, or anything else that made them like me? Again, duck or goose?

In the information security space, we’ve had (mostly by the analysts and the press) huge discussions about whether 10% of the total IT budget was the right amount to be spending on security. According to Forrester, that number has hovered in the 7.5%-9% range for the past few years. That’s good to know because it gives us a general guideline (which is all we can have in the absence of any real actuarial security failure data, but that’s a rant for another time). However, in multi-billion dollar corporations where a 1% difference in IT security spending could equal the annual revenue of many of small security firms, what does this percentage really mean? If one organization consistently spends significantly more than it’s competitors on hardware, data centers, and related IT items, should it necessarily also be spending more on IT security?

I realize these percentages are just guidelines, but they’re the kind of guideline a sharp litigator will latch on to. Remember that no one wants to be the odd man out. No organization wants to have to explain to some regulatory or law enforcement organization the possibly coincidental facts that it suffered a security breach and was also spending somewhat less on IT security than the average for their industry, or country, or whatever.

So, much like I am, I’m sure you’re wondering whether I have a point or whether I’m simply writing this at 4am because my allergies are kicking Claritin’s butt. My long-winded point is simple: We’re all the goose. Every single organization has its idiosyncrasies that make the 10% rule of thumb somewhat less than useful for anything other than selling research reports.

Organizations should spend as required to adjust risk to acceptable levels and realize that not all of that spending will be in IT security dollars. By and large, IT security doesn’t pay for governance, it doesn’t pay for attitude, it doesn’t pay for commitment for excellence. With these things being paid for elsewhere, the IT security budget may be lower and likely result in lower risk (i.e., improved “security???).

We shouldn’t dwell on the size of this ratio; we should worry about the environment in which it exists. A spend of 10% in an immature, ad hoc, no-vision company, probably means they’ve spent the entire 10% on point security solutions ranging from desktop AV to firewalls to IDS and so on. Which means they spent little or nothing on policy, training, proper tools for developers and testers, and so on. Which means they are an accident waiting to happen – 10% not withstanding.

On the other hand, a lower percentage spent within a mature organization that also spends to foster and reward good thinking will almost certainly produce lower risk. Sure, mistakes will still make it into production and there will be problems, but there will be much fewer of them, they will likely be of reduced consequence because the organization knew to look for the big problems and also had effective response capabilities, and the organization will learn and not make those mistakes again. They will make new mistakes, but everyone does.

Be a proud goose. Organizations must not be afraid to use good governance, good training, and good process to their corporate and competitive advantage. If you do good strategic things, you will achieve better security with a smaller capital outlay that doesn’t all come from IT security. Organizations must be comfortable with their risk management story, and their efficient spending, and be able to tell it to the market, to customers, to regulators, and, if necessary, to juries.

Technorati Tags: governance

(with apologies for complete lack of artistic merit)

feng shui governance
plan, influence, and conduct
policy for all

from boardroom to bits
everyone get on board
a single train forward

a balanced approach
harmonious existence
with stakeholders all

set tone at the top
the key of transparency
all must understand

solving all problems
a terrible goal to bear
just cut barriers

how to change things now
like escape from klein bottle
reverse of trip in

business objectives
publicly painted for all
now all can align

our key resources
named, owned, prioritized, staffed
requirements sketched

cooperation
embrace management’s vision
collaboration

internal control
believable proof for all
this is a good thing

need innovation
old way causes much sadness
delightful change now

who must get what done
true responsibility
good authority

what must get done when
relate business to people
and goal them quite well

when must it get done
everything can’t be first
true order defined

where does it happen
are all things prepared for it
measure twice, cut once

how to accomplish
training, coaching, mentoring
lead by example

why is it crucial
all must recite the drivers
to you and to me

it’s about people
enable them to succeed
show you care about them

expect and inspect
balanced scorecard works for most
dashboards are fun, too

you are not there yet
a continuous journey
goals ever-changing

quite learned you are
required knowledge deep inside
express yourself now

P.S. Although I though I was the first to use “feng shui governance” as a term, I noticed that there was a single hit in Google (a three-word GoogleWhack!) used by a Mr. Foldvary back in 1999 in a somewhat different context.

Technorati Tags: governance, doggerel

“We have long had death and taxes as the two standards of inevitability. But there are those who believe that death is the preferable of the two. ‘At least,’ as one man said, ‘there’s one advantage about death; it doesn’t get worse every time Congress meets.’”

~Erwin N. Griswold

Just look at them grow… they’re like weeds. Unfortunately, in this case it’s not a compliment for your sibling’s kids. This particular growth is in the data breach lists at Privacy Rights Clearinghouse, Attrition, PogoWasRight, and others. (And please accept my thanks for the time you all put into this public service; I apologize for not including you all by name.)

I have no doubt these disclosures represent the tip of the proverbial iceberg of data that have actually left the control of those to which it has been entrusted. I also have no doubt that we’ve been given this glimpse into the shoddy treatment afforded many of our most intimate personal details solely through some forward-looking state legislation (thanks, California, for helping to start a good thing in the U.S.).

Let’s just jump ahead of all this piecemeal personal information loss for a minute. (Did the WABAC machine have a “forward??? lever?) Let’s declare it not only inevitable, but also unavoidable, that, for each and every adult American, their name, social security number, top one or two credit card numbers, street address, date of birth, brief genealogy, and a few other interesting data items are known, collated, and cross-referenced.

But, not by legitimate companies like Experian, TRW, and Equifax. Let’s assume that individuals and groups who do not have our best interests at heart have all of these bits of information correctly assembled into mini-virtual-self-referentially-consistent identities, even if they have no “real life” information on the actual person (e.g., photo, current job, salary, type of vehicle, where your kids go to school, etc.). And, let’s assume they have the ability to keep this information current enough to meet their goals of acceptable cash flow for acceptable risk.

[Aside: Is this feasible? How many databases would I have to subvert in order to get most of this information? One at the IRS, one at a credit agency, one each at a few major banks, and maybe a few others? Remember, I don’t actually have to hack in, I can use social engineering, pay someone, coerce someone, steal the back-up tapes, and so on, and on, and on.]

What then? Is it the collapse of American civilization? I doubt it, but I think it would certainly accelerate some good things and some bad things.

The good things might include something like two-factor credit card use at all times (e.g., you have to show a [new government issue?] driver’s license for all card present credit/debit card transactions). But, what would have to change to allow card-not-present credit card transactions to continue to happen (e.g., most Internet orders, catalog phone orders, Chinese food orders, etc.)? What would it take for Americans to embrace single-use credit card numbers, for example? What parts of the infrastructure would have to change? How would we get and carry around these numbers? And so on.

The bad things might include a new state or national ID that would be required by the credit card companies in order to get a “secure” (whatever that would mean) credit card and a lot of high-end establishments that would start accepting only “VMCAMEXD Secure” for transactions over certain dollar limits or that meet certain risk profiles (e.g., buying for delivery to another country). A treaty with foreign countries could require that “secure” credit cards are the only ones accepted in some situations (e.g., buying a one-way plane ticket to the U.S.).

Beyond that, what would have to change at the IRS now that all SSNs are “public”? What about elsewhere in the Federal government? What would this mean to the Privacy Act? And so on. The mind reels, but it still feels like a good thought exercise.

For me, the question is not “What happens if someone else knows all the data associated with me?” The question is “What financially and socially acceptable capability will make it such that I can unequivocally prove that I’m the only me that goes along with my (now public) mini-identity data (e.g., credit card numbers, social security number, address, genealogy, etc.)?” And, in terms of requirements creep, I’d also like to be able to prove it without giving away my actual identity.

Thoughts? I hate to take you this far and not have something to offer, but I feel qualified only in asking the question, not in postulating an answer. And, of course, my follow-up question is “How are we going to make that software better than the software we have now????

Technorati Tags: data security, privacy

Sure, you can’t throw out all your existing code and start over, but you also can’t hide it forever. It wouldn’t surprise me even a little bit to find that your super-slick, Ajax-improved front-end that’s written in the latest Java is actually using a JNI call to kick off some C code that no one knew how to rewrite because it’s calling some COBOL that no one has looked at since before Y2K that may actually rely on some Fortran 77 for a particularly complex calculation. And, of course, your authorization and entitlement database is on some Unix-clone variant on, say, an old IBM mainframe somewhere that talks some weird protocol that only one guy understands (and he’s about to get a gold watch in his stocking). In addition, did I mention that imminent end-of-life notice about both the hardware and the software you’re running on that box in the corner that no one touches (the one that spits out quarterly financials).

If you’re waiting for the one and only security answer to appear before you start making improvements in this installed base, you should probably polish up the old resume. You’re going to be selling yourself before you’re able to sell that wait-and-see strategy. To put an even finer point on it, even if the perfect security solution came along today, it would probably still take you years to get it in place because you’re not set up for success (because you’ve been waiting for the perfect drop-in software security solution).

Software security isn’t going to happen that way. No one knows the one-size-fits-all answer yet, or even if there can be one. But we sure have a pretty good handle on how “…chance favors only the prepared mind” (with apologies to Mr. Pasteur). Even if you can do only a few things, get started.

Institute some governance. If your executive group has application and software security on their radar, then it’s much more likely to get in the budget.

Know your business. You cannot make informed and wise security decisions if you don’t know what you’re trying to accomplish.

Document your software development lifecycle. If you can describe and demonstrate your process, then your process can be improved when the time comes.

Give skill-appropriate awareness training. If your product managers, architects, business analysts, developers, and testers develop a sensitivity to the security issues associated with what they’re working on right now, it will get better. And the next thing will be better still.

Go start now.

Technorati Tags: software security

I was a young Air Force lieutenant when Ken Thompson released his 1984 piece, Reflections on Trusting Trust. Assigned to a data center in the Pentagon, I was working on the B2 evaluation of Honeywell Multics with the fine folks at the National Computer Security Center and contributing some words to the growing Rainbow Series books. We were in ongoing debates over the meaning of phrases such as “top-level specification” and “on behalf of” in the Orange Book (a mail thread that went on for several years) and trying to perpetuate the wave of talking about “trusted computers” instead of “secure computers.”

We talked a lot about “how much trust do I get for how much analysis and testing” and “how much trust do I need for certain scenarios, like allowing a computer to automatically downgrade data from Top Secret to Secret.” These kinds of concepts ended up in capability maturity models, testing models, and risk models.

I took Ken’s words to heart and quickly made up my mind that we could really only move the trust bar up slightly (like from 10% to 20%), even with enormous expense. It was immediately obvious that getting up to, say, 75% would require so much calendar time that no one would wait for the result (and I think later experience with Orange Book evaluations, Common Criteria evaluations, and related programs have borne this out). Between hardware, software, firmware—and the completely unpredictable human factors—we really had no idea how even the most reviewed code would operate in relatively controlled environments (e.g., in a government facility where everyone was cleared), much less how it would operate in a hostile environment (hostile mobile code was not really a problem yet) where people might actively be up to nefarious deeds.

Why should you care about my reminiscing? Well, because I think a flavor of trust is still a major problem today and it’s costing everyone a bunch of money that could be put into real long-term solutions.

As I talk with my operations friends these days, I’m seeing a subtle shift in their thinking. They’re thinking more and more about appliances (web application firewalls, IDS and stuff like that), but not for direct security value. They seem to be thinking that since the software they install, whether purchased or built internally, will certainly have security problems, they have to install more bells and whistles so that operations can protect itself. This is beyond healthy paranoia; it’s an unhealthy distrust of people who should be active partners, even if it’s been earned by years of spectacular failures.

Then I started wondering why so many development organizations throw out requirements documents from product managers and just start over. Is it only because the requirements are so bad, or is it also because the developers just don’t trust the managers to know what they’re talking about?

And why do so many managers try to tell development organization how to create applications, instead of simply what the creation must accomplish? Do they simply not trust the developers to be aware of business objectives, or do they just not understand the creative processes involved?

And so on.

What an enormous amount of wasted cycles that could be used to greater organizational good.

We may never get to the point where we can implicitly trust software. But, can’t we at get to the point where we can trust each other?

Technorati Tags: software security, software quality

The PCAOB is introducing new guidance to help lower the overall cost and, presumably, increase the effectiveness of SOX 404 audits. It needs to use this opportunity to help fix some root causes, not just tell us how to find more symptoms.

This past December, the PCAOB announced that it would propose for public comment a “new standard on auditing internal control…designed to focus the auditor on the most important matters, increasing the likelihood that material weaknesses will be found…” The proposal itself can be found at http://www.pcaobus.org/Rules/Docket_021/2006-12-19_Release_No._2006-007.pdf.

Starting on page 93 of the document, there is a section on “Benchmarking of Automated Controls.” It includes guidance like “Entirely automated application controls are generally not subject to breakdowns due to human failure,” which is clearly not true since bad input makes functional application security controls fail all the time. Telling auditors that automated application controls automatically get a gold star is not a step in the right direction.

It does go on to suggest that the benchmarking strategy take into account the importance of the effect of related files, tables, data, and parameters on the consistent and effective functioning of the automated application. That’s a good thing.

The document then suggests that “If general controls over program changes, access to programs, and computer operations are effective and continue to be tested, and if the auditor verifies that the automated application control has not changed since the auditor established a baseline (i.e., last tested the application control), the auditor may conclude that the automated application control continues to be effective without repeating the prior year’s specific tests of the operation of the automated application control.”

So, never mind that new attacks are discovered with unnerving frequency and that perfectly good code can suddenly become not so great (think crypto algorithms), the apparent recommendation is that if you didn’t or couldn’t poke hard enough to break it over some time period, then it’s okay to skip it later. Approaches like this where we’re considering only functional changes and not testing skill and depth can’t be effective.

How many times has someone walked up and spotted a problem you failed to notice time and time again. As organizations periodically change auditing firms, expect huge increases in reported problems.

The proposed guidance gives the following factors to use when deciding to use benchmarking:

• The extent to which the application control can be matched to a defined program within an application;
• The extent to which the application is stable (i.e., there are few changes from period to period); and
• The availability and reliability of a report of the compilation dates of the programs placed in production. (This information may be used as evidence that controls within the program have not changed.)

This wording is still neglecting changes on the threat side of the equation. Just as many castles were considered impregnable until about five seconds before the first cannonball hit, many lines of code were considered secure right up to the point where the breach story appeared in the newspaper.

The guidance gives the following factors to use when deciding whether to reestablish the benchmarking baseline:

• The effectiveness of the IT control environment, including controls over application and system software acquisition and maintenance, access controls and computer operations;
• The auditor’s understanding of the nature of changes, if any, on the specific programs that contain the controls;
• The nature and timing of other related tests;
• The consequences of errors associated with the application control that was benchmarked; and
• Whether the control is sensitive to other business factors that may have changed. For example, an automated control may have been designed with the assumption that only positive amounts will exist in a file. Such a control would no longer be effective if negative amounts (credits) begin to be posted to the account.

Okay, so if I know that most auditors and organizations use COSO as a governance model, and COBIT 4 to interpret COSO to arrive some IT control objectives, and I consider AI2, Acquire and maintain application software, important to my environment, then I might understand what “controls over application and system software acquisition and maintenance” means above. And, I might even read COBIT and see AI2.4, Application Security and Availability, which states “Address application security and availability requirements in response to identified risks, in line with data classification, the organization’s information security architecture and risk profile. Issues to consider include access rights and privilege management, protection of sensitive information at all stages, authentication and transaction integrity, and automatic recovery.” Oh wait, I only need security software, not software security. Sigh. Another opportunity missed.

There is a need here for any words that introduce something like the following:

The customer may have written their own software that is directly used in the financial reporting process and you, the auditor, should be aware not only of the software’s functional controls (e.g., I&A, encryption, entitlements), but you must also accrue confidence that their internal development practices and testing are sufficient to produce quality software that has at least some capability to protect itself from attack even in the event of catastrophic failures other general and IT security controls being considered as part of this SOX audit. While security software may comprise the majority of IT security controls, software security is the property that gives us confidence in their continued successful operation even in the certainty of ongoing attack. You must use a risk-based approach to accruing confidence, focusing on relevant factors that have a material effect on the software’s ability to meet business objectives even in an overtly hostile environment.

Now we’re thinking more about the problem and less about the symptoms.

Technorati Tags: software security