Archive for June, 2007

Training Material, Training, and Behavior Modification: Part 3 of 3 - Behavior Modification

(This is part three of a three part series. Part one is here and part two is here.)

G.I. Joe and his friends (e.g., http://en.wikipedia.org/wiki/G.I._Joe) gave cartoon addicts in the mid-1980s United States a series of television public service announcements. The cartoon kids being given the lesson invariably responded with “Now we know!” to which the G.I. response was always “And knowing is half the battle.” Unfortunately, we often forget that knowing is, in fact, only half the battle.

So, let’s you say have some awesome training materials (slides, eLearning module, video, or whatever) and training day goes beautifully. Everyone now knows exactly what they’re supposed to do. How are you going to get them to do it? Periodic and direct reinforcement are key to real behavior modification.

Start by ensuring that reinforcement messages are pitched at the right level. If you want people to learn the theme, then repeat the theme, act out the theme, and make sure the theme is in their words. Don’t start with “It was the best of times, it was the worst of times” to explain the quarterly software bug report trends to executives (although that would be pretty cool) and don’t tell testers to ensure that the “trend in bug reports must have a positive impact on EBITDA.” For example, tell people who work with software bugs about bugs, and their bosses about trends in bugs, and their bosses about trends in application quality, and their bosses about trends in the application portfolio, and so on.

Don’t start from nowhere with your students. I believe it will be somewhat uncommon for you to be teaching people something entirely new (imagine the blank stares in the first calculus class sometime around 1704 or the first Fortran class in 1954!). Even after training, continue to work diligently to identify the gap between what the students are doing and what you need them to do. Then, use that as a stepping stone to build upon your audience’s existing knowledge and skill. Continue to remind them of what they already know and explain to them what you want them to do that’s extra or different. If you make it sound like they must start from scratch, or make them cover old ground, or if you omit some key knowledge, you’re unlikely to get any positive behavior change at all.

Make sure your learners continue to be motivated to learn and change. Make it part of their objectives. If you took the time to train someone on how to do something, you should check to see that it’s getting done. If the students know you’re going to check, they’ll be a little more motivated to learn it and do it. There are few activities more frustrating than standing in front of a class all day, teaching important stuff that the students really need to know, and having them stare vacantly ahead (remember, you can make them put away their PDAs, but you can’t make them like it). A little positive motivation goes a long way.

Unfortunately, so does a little negative motivation. Just as organizations get peeved when some new law generates an unfunded mandate that’s all stick and no carrot, employees get a little perturbed when any kind of “do it or else” message is sent their way. While this kind of motivation may work in the short term, it’s just not worth it. In many organizations, the employees are the company. For consultancies, independent software vendors, application houses, and so on, the biggest corporate asset—the employees—go home every night. In a job market that offers some easy mobility, don’t give them an unnecessary reason to go somewhere else.

The best way I’ve found to ensure ongoing performance is to institutionalize the activity as the way things are done. Don’t make people have to go out of their way to do the right thing. If you’re training software architects to do threat models and architecture risk assessments, then account for that time in development projects, give them the tools they need, and make it’s part of the software development lifecycle. Ensure there’s not only prescriptive guidance for doing it, but that the results of these activities are part of someone’s metrics and the software governance dashboard. It’s the same for teaching developers to improve their development patterns, teaching testers to perform non-functional security tests, or teaching executives how to institute software governance.

Let me mention metrics once again. If you took the time to train someone how to do it, then “it” should probably be part of the metrics that someone tracks. If you taught a roomful of testers how to do some inside→out white-box security testing, then you should probably keep track of how many software flaws they catch as part of this process, along with some trends. You might even want to, for example, track which code bases are exhibiting the most problems and track that back to the composition, training, and automation support for the various development teams. If you can discover which teams are routinely making similar errors due to, say, having a broken unit testing process, everyone wins.

And, by all means, find and encourage the stars. The people who really get it should be groomed appropriately. They may be the next crop of team leaders or managers. If nothing else, get one of those individuals to improve the course material or even to teach the next class.

I certainly don’t mean that you should ignore the stragglers. If some folks have attended training and have received some follow-up attention and still don’t get it, they may simply be in the wrong profession. That’s always a tough decision, but I find it’s usually best for everyone. If someone has arrived at a position where they are failing, even with training and attention, kindly help them find a place where they can be effective. And I do mean kindly; that person may be your boss someday.

Someone must follow up with the instructors as well. Do post mortems after each class and give constructive feedback. Ensure each student completes a feedback form. And don’t ask those silly questions on the form like “Did you enjoy this class?”; that’s just useless. Ask things like “Did you understand why you were here and the problem we’re addressing?” and something like “Will the information given to you today actually improve your day-to-day performance?”. Of course, you also want to ask things such as “Did the instructor communicate in a clear manner?” and “Do you feel this instructor truly understands the material?” and stuff like that.

As some parting advice, keep in mind that it’s not just the individual student that must get something out of the class. For every course you give, someone is paying a hefty bill. Whether you’re a consultant charging by the hour or an internal expert delivering some training to your friends, someone’s paying the price for you and for those students to not be working. That economic buyer needs to get something for spending that money (real money, opportunity cost, or whatever) if you expect it to happen again. Ensure someone comes up with a way to measure training success, that they actually measure it, and that they let the stakeholders know how it went.

“Now we know!”

Training Material, Training, and Behavior Modification: Part 2 of 3 - Training

(This is part two of a three part series. Part one is here and part three will be posted on Friday.)

“Keep away from people who try to belittle your ambitions. Small people always do that, but the really great make you feel that you, too, can become great.” - Mark Twain

My guess is that, on any given day, thousands in the world-wide work force attend some form of corporate training. It could be software governance training for the pointy-haired bosses, how to use a juicer for department store workers, or keeping fingers away from the business end of a chainsaw for lumberjacks, but it’s training nonetheless.

The training might be delivered in the form of a short video, some eLearning module, or a class led by an instructor. It may have an integrated quiz, or require you to go do some additional exercises, or be taught one day per week for some period of time, or whatever. The possibilities aren’t endless, but there are a fair number of them.

Beyond the training requirements discussed previously, you also need to figure out what training styles works for your people in your environment. Having said that, I think there are a few rules that are fairly universal:

  • Be as formal in your presentation method as you need to and then just a touch more. I find that too little formality in training is far worse than too much. Too little formality is often perceived by students as “We had to do this training to check the box, so let’s get through it.”
  • Stick to your schedule. If it says “Start at 9am,” then start at 9am. If you’re breaking for 10 minutes, get started in 10 minutes. To some degree, we all want to wait for that last straggler. Don’t do it; you’re just penalizing everyone else who had the professionalism to be punctual. Class management is always hard. You may in fact be the tech dweeb presenting to a room full of executives. On that day and at that time, however, you are in charge of that class; act like it. Professionally. Doing it unprofessionally is definitely a career-limiting move.
  • Use eLearning (or computer-based training or whatever you call it) for awareness and to give basic skills to the masses. It’s very important, however, to use instructor-led training (ILT) for teaching important execution skills to your practitioners. It doesn’t matter how many times you watch the video of the expert performing the trick, even in slow motion with arrows and explanations, most people still won’t learn a new skill that way.
  • Make sure you can actually teach the material in the time allotted. That usually means you must ban the typical distractions. That includes Blackberries, laptops, various PDAs, and so on. Yes, even for the executives. Stop the side conversations, keep everyone on topic, and tell people who have long, rambling, just-don’t-get-it questions that you’ll get back to them after class. When you rush through the last 20% of the material to stay on schedule, students feel cheated. On the other hand, if you scheduled two hours for a thirty minute class, that’s a completely different screw up and your students will likely be just as irritated.
  • Give people sufficient breaks so that they don’t have to violate your other rules. I find that most people, including the instructor, can concentrate only for about 90 minutes at a time. For really dense stuff, that time might be reduced significantly so partition your material appropriately. Also, ensure you provide what people typically want during breaks: snacks, beverages, easy access to a restroom, and a network connection. I imagine millions of person-minutes of training are lost every year just because someone didn’t think of spending $20 to set out some mid-morning and mid-afternoon snacks.

I argue with myself, sometimes successfully and sometimes not, over whether the material or the instructor is the most important part of technical training. Today, I believe that it’s far more likely that a good instructor can save truly awful material than it is likely that a bad instructor will successfully deliver even the best of material. If you’re good at getting a class excited about a topic and you can talk about the topic authoritatively, you’ll rise above the material.

Let me emphasize that more. If you’re the instructor, you must know more than what’s in the training materials. If you can’t go off-script even a little, you’re the wrong person for the job. Excuse yourself graciously rather than wasting everyone’s time. The amount by which you need to go off-script will depend on the skill level of your students, but you certainly should be able to answer questions about rationale, goals, why another way isn’t the right answer, and how it would be done in a scenario somewhat different from the one presented.

When going off-script, it would be great if you can do so in three ways:

  • Visual stuff, for the people who learn from what they see. For these people, you must be able to provide visual aids. Draw a picture on a white board, or make a hand-out, or have an exercise where people draw out the process, or whatever. Always give the visual people something to imprint on.
  • Auditory stuff, for the people who learn from what they hear. For these people, it’s not just what you say, but it’s also how you say it. They’ll pick up on subtle clues in your delivery that can sabotage your purpose. In other words, be able to be excited about digging deeper. If you think Section 4 is boring, you’ll present it that way, and they’ll believe it. Be interested, not just interesting, and give the auditory people something to pick up on.
  • Kinesthetic stuff, for the people who learn from what they feel, physically or emotionally. For these people, you have to get them involved in the problem and the solution. For some, that will be an emotional connection and for others that will be some manner of hands-on demo. Whenever possible, let the kinesthetic folks touch something.

Good speakers always have good notes from which to speak. Don’t be afraid to have a cheat sheet of important points. And don’t be afraid to lecture. Just don’t do it the whole time. Give people the motive, the rationale, the philosophy, or whatever it takes to get them involved in the problem. Then switch to some practical visual, auditory, or kinesthetic stuff to show them how to execute the solution.

Now, I’m not talking about any of those group hug things, because there are also introverts and extroverts in the world. Making an introverted, visually-oriented person do role playing as part of training is just torture. They’ll hate you forever. On the other hand, not only are the extroverted kinesthetic folks excited at the prospect, they’ll show up with streamers and brownies and hold annual reunions. Sigh.

When you’re presenting technical training material, whether as part of ILT or an eLearning quiz, make sure you ask some open-ended questions. This is a great opportunity to let the students use their own words to tell you what they’re thinking. By listening carefully, you’ll be able to tell who really gets it and who struggling to keep up. That will help out a lot in an ILT setting when you’re making teams for group exercises. (Hint: Don’t let all those struggling students end up in one group.)

Training creators and instructors must remember that there will be students with whom you have no shared experiences. They don’t know Babe Ruth, don’t get your Simpson’s references, and are quite alarmed at your claims that sometimes you just need to “kick some ass,” “gore some oxen,” or “kill some sacred cows” to get what you want. This is not my soapbox for political correctness; this is just my experience telling you that teaching via analogy, colloquialism, and local cultural reference often results in a lot of blank stares (like with my upcoming G.I. Joe reference). Life may be like “a box of chocolates” for millions of people, but there may be billions who’ve never even seen a box of chocolates, much less a Tom Hanks movie.

Take the time to tell students what you want them to do when they leave the class. Don’t just talk around it. Really lay out the activities they should undertake and the changes that should occur when they do. Think of it like writing a software requirement. You can write “The system shall…” all day long, but until you actually show the person who has to do it a use case or a scenario or a drawing or something, there’s virtually no chance of getting the behavior (or code or process or whatever) you intended.

Remember, our brain is likely to simply reject any message that it can’t process. Why might that processing not happen? In my experience, the top causes are:

  • For new skills, you’ve provided no frame of reference for the student. There no place to latch onto and say “Oh, this is like that other thing I know how to do, only different.”
  • For extensions to existing skills, there’s too big a gap between the student’s skill and what you’re teaching, and you’re not providing enough framework for the student to make it over. Either you jumped right into the guts of the material without adequate preamble or someone did a poor job of choosing the students.
  • One or more of the material, the presentation method, or the instructor is simply lousy. If the materials seem good and the method is well thought-out and constructed, then it’s probably the instructor. It’s impossible to describe all the ways training material can be lousy, but one of the most prevalent is that it’s just stale. Teaching the same old stuff year after year without ever updating it is just silly. As for the instructor, not everyone is cut out for teaching. I’ve seen some real world-class technical experts do indescribably horrible jobs in front of a class, just as I’ve seen non-practitioners (never actually did this work for money in their lives, but truly understand the topic) teach wonderful classes.

By the way, don’t send the wrong subliminal message with your training. If you’re having a session on the importance of corporate intellectual property, ethics, and how file sharing is wrong, you probably shouldn’t sprinkle some “borrowed” Dilbert, Calvin and Hobbes, and Far Side cartoons in your presentation, while playing a downloaded Van Halen soundtrack in the background. Duh.

Trust me, if you really care about the training you’re giving, as well as the behavior you’re trying to encourage, it will show in the material and in your delivery. This energy is contagious. Chances are that you want these students to stretch themselves in ways that are new and possibly even undesired. Give them every opportunity to succeed. If the training appeals to their senses, they’ll be more engaged and they’ll retain more. If the training appeals to their sensibilities, they’ll be more engaged and they’ll actually follow through.

Training Material, Training, and Behavior Modification: Part 1 of 3 – Training Material

(This is part one of a three part series. Parts two and three will appear on Wednesday and Friday.)

The beautiful thing about learning is that no one can take it away from you. -B. B. King

We’ve all been to training. More specifically, we’ve all been in training where we were bored to tears, not because we were already experts, but because there was just no connection between our individual learning style, base knowledge, interest level, or job goals and whatever it was the instructor was doing. Invariably, however, there will be a few people in the same class who feel this training was the best they’ve ever attended. Two days later, you can’t even remember what the course was about and two months later others are still quoting the instructor. Why does this seem to happen so often? In my experience, that it was effective even for those few people was probably just luck.

When you’re thinking of making or commissioning some technical training, stop. Completely. Not one of those rolling stops you do at a stop sign in the middle of nowhere. Sit down and take a deep breath. Now, write three complete sentences:

  • “This training is appropriate for people who…”
  • “In this training, you will learn how to…”
  • “After this training, you will be able to…”

If you cannot specifically and succinctly define your audience, tell them what they will learn, and describe how their new ability will benefit the organization, you’re definitely not ready to start building training. This would be just like writing code without requirements and you’re likely to achieve the same spectacular lack of success.

If you need to do a formal performance needs assessment to get requirements because you’re teaching someone a life-or-death skill (e.g., how to disarm landmines in the field), then do it. On the other hand, if you’re teaching employees the basics of physical security for the average office building, you can probably just survey the existing literature to get your main points.

I understand that there is some training that must be given to everyone regardless of their interest, skill level, learning style, or anything else. Examples might include employee orientation training, sexual harassment training, security procedures training, and so on. I feel it’s easy to show how poorly the average one-size-fits-all training approach has worked in these areas, especially when reduced to a poorly-scripted eLearning module or abysmally-acted short film.

Even after you get a good course description and, presumably, the right audience, tell people right up front what they’re going to learn in your course. Don’t make it something cheesy like “Today we’re going to learn about software security.” That’s just silly, even if it’s for a general software security course. Tell them something useful like “Today, I’m going to show you how to generate crypto seed values correctly in various languages and technology stacks. By doing this correctly, you will help the company meet ongoing regulatory compliance requirements.” If it is the general course, tell them something like “Today, I’m going to give 10 reasons to care about software security and help save millions of dollars.”

Well, that’s different. Now I know why I’m here and how what I’m going to learn will really help; it’s not just some flavor-of-the-month exercise because it’s “time to shake things up around here.” Continually reinforcing why this training is important, both during the class and afterwards, means you’ll end up not only with increased competence in the target audience, but, in this case, you’ll also end up with practitioners who are more business-risk aware. That’s a good thing. Why? Because now, they’ll ask themselves questions about “How does this affect the business?” and, when they don’t know the answer, they’ll go ask someone rather than just putting some code together. Give them the chance to do the right thing.

Remember, it is a significant task to get a diverse group of people to leave a room with the basics of a new skill (e.g., developers using certificates for mutual application authentication), with necessary knowledge (rand() is not cryptographically secure), or even with just a common awareness of a given problem (e.g., web application input validation is tricky). This is compounded when we let training creation wait until the last minute and then “throw something together” or we “just use last year’s slides.”

The biggest evidence of such procrastination and lack of planning is ending up with training materials (e.g., PowerPoint slides) that are simply about the topic, not on how to do the topic. How often have you gone to a class guaranteeing to turn you into a practitioner, only to spend the majority of the time on vocabulary and concepts and fluffy cloud process diagrams! What a waste of time. Training about test process automation, for example, is something that you give to people making budgeting and resourcing decisions. Training detailing a method for doing test process automation is what you deliver to practitioners.

Speaking of planning, in my experience you have to budget about 100 minutes per technical information slide to create useful material that focuses on imparting a single important idea in a memorable fashion. This includes sufficient time for instructor and student notes. The instructor notes should be what the instructor says while the slide is being projected, along with any prompts for what to do, what to hand out, and so on. The student notes should include specific reinforcements, references, and related materials applicable to the information on the slide. This 100-minute block also includes internal review, discussion, and bootstrapping the first instructor.

I know 100 minutes per slide sounds like a lot, but if you’re spending less than three days to plan, draft, and complete one hour (about 15-20 information slides) of technical training (or any training that is telling someone how to do something), you’re probably messing it up. In particular, take the time to look for every group of words, bullets, or slides that you can turn into a real, accurate picture. Decorate the picture appropriately, but don’t make it so busy that it becomes an eye chart. Animate it whenever necessary to show how and when things actually happen. People will remember and refer to these pictures (because they can cut and paste them), but they probably won’t ever reuse some random set of bullet points.

This reuse habit, by the way, is also why it is universally idiotic to include on-the-fly sample code that someone just banged out for the slide; it’s almost certainly wrong, not secure, and against some policy and someone will be embedding it in some critical code module tomorrow. If they do that, it’s just as much your fault as it is theirs.

Another word of advice: Don’t have technical experts create the final training product. No, really. Even if you’re the technical expert and you think you know better. Trust me. The only exception to this might be when a technical expert is creating slides for very similar technical experts. Even then, run the slides by someone a level or two below and a level or two above “technical expert,” as well as someone more skilled in training presentation, and heed their feedback. It’ll be a valuable experience and you’ll end up with better material that is less bogged down with jargon, with incomplete thoughts that presume too much historical knowledge, with acronyms that are never explained, and with words that apparently have a different definition on the author’s home planet. One hopes you’ll also end up with something that isn’t 100 slides of wall-to-wall words.

The long-term benefits derived from putting the required time and skill into building good training will far exceed the initial investment. Spend a little extra time on it and make your students happy to be there. Everyone will benefit.

Technorati Tags:



Resources
> Overview
> Your Account
> Podcast
> Blog
> Case Studies
> White Papers
> Publications
> Books
> Security Articles
> Presentations


RSS

You are currently browsing the Justice League weblog archives for June, 2007.

About the Bloggers
  • Pravir Chandra
  • Scott Matsumoto
  • Gary McGraw
  • Sammy Migues
  • Craig Miller
  • John Steven
  • Categories
  • Admin (3)
  • Assurance (6)
  • Data Security (3)
  • Defects, Bugs, and Flaws (3)
  • Enterprise Software Security (11)
  • General Interest (3)
  • Governance and Regulation (5)
  • Risk Management (4)
  • Security Features (2)
  • SOA and Web 2.0 (2)
  • Software Quality (4)
  • Software Security (32)
  • Software Security Touchpoints (7)
  • Software Testing (2)
  • Training (3)
  • Archives
  • April 2008
  • March 2008
  • February 2008
  • January 2008
  • December 2007
  • November 2007
  • October 2007
  • September 2007
  • August 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007
  • March 2007
  • February 2007
  • By Blogger
  • Craig
  • Gary
  • John
  • Pravir
  • Sammy
  • Scott
  • Guest bloggers
  • Recent Comments
  • gem on Three New Books: Thanks Adam (and sorry not to make your role explicit Andrew). I’m...
  • Adam on Three New Books: Thanks Gary! your copy is on its way. Just a little nit, I’m the...
  • Andre Gironda on Is Penetration Testing Security Testing?: From a book I recently read: Functional...
  • Tom Van Vleck on Security And Market Forces: I can’t come up with a number for how much money I...
  • -jOHN on Security And Market Forces: Tim, I’ll let the next 12-24 months of...
  • Recent Entries
  • Unsafe at any bitrate?
  • Three New Books
  • Is Penetration Testing Security Testing?
  • Externalizing Access Control Quandary
  • Making a move
  • Links
  • Cigital
  • Silver Bullet Podcast
  • Blogroll
  • 1 Raindrop
  • Fortify Software's Blog
  • Freedom to Tinker
  • In the Wild
  • Jon Udell
  • Michael Howard's Blog
  • Microsoft Security Vulnerability Research and Defense
  • News.com Security Blog
  • Schneier on Security
  • Security Fix
  • SilverStr's Blog
  • Tao Security