Ifwe want to improve our corporate training, we need to constantly assess ourprograms in order to determine what is working well and what is not. And so Iwas a bit distressed when, during my session at the recent Learning Solutions Conference,several audience members said, “I am too busy training. I do not have time todo research.”
Thisremark got me thinking and I want to share some ideas to help you reconsiderthe importance of building assessment into your organization’s training.
Whydo we assess?
Let’sstart with an analogy. My wife Jane complains about my driving. It’s not that Iam careless or aggressive. The problem is that I am usually in a hurry and Idon’t take the time to check a map or GPS before I leave the house. Like manymen, I trust my intuition and assume I know the right way. Unfortunately, mybest guess is often wrong and we end up hopelessly lost.
Trainingis like that. We are so busy training, so busy creating one lesson after thenext, that we do not take the time to assess where we are and whether we aremaking progress. And when our employees fail to thrive, we just train them somemore.
Doesn’tit make more sense to slow down, take stock of our position, and make strategicdecisions that will help us find the best route to our destination?
Intuition is not enough
Thereis a long history of people assuming that they were doing good work when theywere not. For example, a few years ago, a group of well-intended folksdelivered seminars warning college women of the risks of eating disorders. The trainers delivered workshops on campus after campus,always too busy with their good work to assess their effectiveness. When theyfinally did do systematic experiments they were shocked to discover that on thecampuses where they made presentations there was actually an increase in theprevalence of these disorders. Tragically, it turned out that their workshopsdestigmatized eating disorders and made women more likely to partake.
Isit possible that your training is having no effect on learners, or worse, thatit is actually detrimental to the good of the company? The answer is “Yes” itis possible, and until you conduct systematic assessment you will have no ideawhether you are really helping the organization reach its goals.
When should weassess?
If youdecide that you want to deliver assessments, the next question is “when” theseassessments ought to take place. I polled my audience in Orlando about when, ifever, they delivered their assessments. Here are the results:
Never: | 65% |
Immediatelyafter training: | 30% |
In the daysand weeks after training: | 5% |
As you cansee, only about one-third of the audience actually conduct assessments, and ofthose that do, almost all of them conduct the assessment immediately after thetraining takes place. One trainer explained, “We have the learner availableright after training so it makes sense to do the assessment at the same time.”
But does it make sense? Is an immediateassessment going to give you the meaningful data about what people are likelyto retain? To best answer this question, I need to give you a bit of backgroundon how memory works.
The three stages of memory
Mostneuroscientists agree that memory consists of three distinct stages: sensorymemory, short-term memory, and long-term memory.
Sensory memory
Your sensorymemory is a buffer that stores all of the sensory information you receive.Sensory memory seems to store everything you perceive (sight, sound, touch, forexample), but this storage lasts at most only a few seconds. The fact is thatmost of this sensory information is unimportant to you, and if you pay noattention to it, it quickly fades away, never to be remembered. On the otherhand, if you do focus on a particular aspect of your sensory memory, the fullrange of the information is available to you.
Toillustrate, assume you are watching your favorite TV show while at the sametime, your spouse is going on and on talking to you about his or her busy day. Suddenly,you hear them say loudly, “Are you even listening to me?”
You snapaway from the TV and say, “Of course I am, dear.” And in a surge of cognitiveeffort you search your sensory memory and you recall the last sentence or two. “Uh…You were just saying that ‘George is not finishing his homework and you aregetting frustrated with him.’”
The fact isthat you were NOT really paying attention to what your spouse was saying. Butthe information was stored within your sensory memory and by shifting yourattention to it, you were able to retrieve it before it faded. Whew!
Sensorymemory is the shortest-term element of memory and it decays very quickly. Forexample, iconic memory (the memory of vision) usually decays within a half-second,whereas echoic memory (the memory of sounds) can last up to three or fourseconds.
As you cansee in Figure 1, if you ignore information in your sensory memory, it willfade. If you “pay attention” to it, it is transferred into your short term memory.
Figure 1:Memory components
Short-term memory
Short-termmemory (STM), also known as primary or active memory, is the information we arecurrently thinking about. Our short-term memory has the ability to contain small amounts of information in areadily available state for shortperiods of time. In a famous article titled The Magical Number 7, plus or minus two, George Miller argued thatpeople can usually hold only about seven items, for some it is a bit more, forothers a bit less. (Recent research has adjusted the number down a bit).
We canusually maintain information within STM for as long as we continue to activelyprocess or rehearse it. However once we stop actively processing theinformation, perhaps because we are distracted by another thought, the originalmemory trace can fade in a few seconds.
Toillustrate STM, assume you are being introduced to a series of new juniorexecutives in your organization. Miller’s theory predicts you will be able toactively store and rehearse about seven names at a time. When you areintroduced to the eighth person, however, you then exceed the capacity of STMand something has to be pushed out of STM. If the ejected item has been insufficientlyrehearsed when it is pushed out of STM that item will be lost. On the otherhand, if that item has been sufficiently rehearsed, the memory will be encodedand transferred to long term memory.
Long-term memory
Long-termmemory (LTM) is where encoded information is stored permanently. Unlike sensoryand short-term memories, long-term memory can store (we think) unlimitedamounts of information and can do so permanently. As trainers, our goal is toget our ideas transferred into the learner’s LTM and to ensure that the learneris able to retrieve that information when they need it.
So when should we assess learning?
The value ofassessments depends on when you deliver it. For example, if your goal is todetermine whether the learner understood complex material and you want to knowif they understood the distinction between “consensual leadership” and “directiveleadership,” it is entirely reasonable to ask them a quiz question immediatelyafter training that requires them to demonstrate the distinction.
On the otherhand, however, if you want to determine whether a learner is likely to rememberand utilize information, an immediate assessment is near to useless. Theproblem is that an immediate assessment simply measures whether the informationis contained within the learner’s STM and this has little correlation withwhether it will be transferred to LTM. As a result, an immediate assessmentleads to the false impression that people have learned a lot. If you want topredict long term retention, you need to delay the assessment, for an hour oreven a day, so that you can know that you are really testing a person’s longterm memory.
Next month,we will look at other ways that one can use assessment and how best to deliverit.
Digging deeper
If you wouldlike to have your memory of this article reinforced, send an email to[email protected]. You will automatically receive a series ofboosters on this article. The boosters take only seconds to complete, and theywill profoundly increase your ability to recallthe content of these articles.
If you want to learn more about the mishap with the eating disorder training, here are some good references:
- T. Mann et al. (1997). Health Psychology, 16, 215-225
- E. Stice & H. Shaw (2004). Psychological Bulletin, 130, 206-227.
- C.B. Taylor et al. (2006). Archives of General Psychiatry, 63, 881-888.
A short cut for deliveringassessment
If youreally need to deliver training and assessment during a single session, thetrick is to find a way to fully occupy the learner’s mind so that you can besure that the STM is purged. Memory researchers use the Brown and Peterson procedure whereinstudents count backward from 1,000 by threes for about a minute. Within atraining environment, however, a more realistic strategy might be to give thelearner a second training experience before assessing them on the first.
Is the three-stage model of memorytrue?
The three-stage model iscompelling and easy to understand but technically, it is not “true.” No oneknows how the brain works and the three-stage theory is just a representationof our best guess. Our knowledge of the brain will continue to grow, and youcan be sure that this model will be replaced by one that does an even betterjob explaining the data. All models are wrong—but some models are useful for awhile.
