The best eLearning content should track learners. I do not mean as a monetization strategy using Google/Facebook surveillance techniques, although that is already happening to be sure.

Rather, the trackable nature of online content today is opening up many other opportunities that expand significantly the influence of enterprise learning. While we as an industry already use completion rates, scoring, and bookmarking, here are four other uses of tracking that may become just as necessary. I have drawn from my experience here with short sims but the best practices should generally apply to didactic content as well.

Gaining insights through TADA

The most strategic of the four, teaching and data acquisition—a phrase coined by scholar Peter Shea—is an emerging approach whereby educational content is used in an enterprise not just to roll out best practices but to gain deeper insight into the environment. Learning is two-way.

For example, consider a question in a refresher short sim for experienced wholesale paint sellers.

You are selling paint to a potential customer who fixes up rental condominiums. Do you emphasize

a) Price per gallon

b) Price per coverage

c) Durability, or

d) Trendy colors?

While the rest of the short sim covers new products and processes, the point of this interaction was to help corporate understand what the field thought was important as input to marketing material.

These TADA questions can fit organically into a scenario, or they can be asked explicitly. A coach may ask the player such questions as:

? You just closed a sale. Great! In your real job, what is the average length of time from getting a lead to closing a sale? [Less than 1 week, 1 to 4 weeks, More than 4 weeks.] (Figure 1)

Coach question (TADA)

Figure 1: Coach question (Teaching and Data Acquisition)

? Or “Well done on completing that situation. In your real job, in a week, how many times do you see your fellow health care workers not wash their hands when they should?”

Players of short sim could be asked such questions as:

? What is a more valuable bonus: time off, a more interesting assignment, or a one-time lump of cash?

? Do they speak Chinese?

? What is the most important buying reason for cloud services over installed software?

In an environment of regular short sims deployment, most questions can be answered in 24 hours.

Conviction measuring

Consider the questions:

What is 3!
a) 6
b) 9
c) 27


A person applying for a passport is using a photograph that is nine months old, and her hair is a different color and cut. Do you, as the agent:

a) Accept the photograph

b) Tell the applicant that the photograph will not be accepted

While getting the right answer is the most important variable, the speed at which a player answers a multiple choice question or makes a decision can often serve as a surrogate for their level of conviction. When trying to understand what a player knows, "fast and wrong" should be interpreted differently than "slow and wrong," just as "fast and right" tells a different story than "slow and right."

People with conviction know the answer without thinking, which is increasingly the target level for certain types of mission-critical education. And the new pedagogy around short sims can now rigorously target conviction building.

Note: ‘Speed to answer’ is not something that is natively measured directly in many standards, such as xAPI, but can be easily gleaned by subtracting the time a player is presented with a question from the time that they answer it.

Debugging and remixing

When I launch a short sim, I look to see what users are doing in it. And, when content is cloud-based, I can make changes in real time. Some of my questions and responses:

? Are some people not finishing a 10-minute sim? Track their paths and look for patterns where they got stuck.

? Is everyone making the same decision during some interesting challenge? Perhaps a sim can be reworded or restructured.

? Are less than 50 percent of the players opting to do some optional level? We might make the description more enticing or, worse case, make it mandatory. (I find over 85 percent of players will elect to take a final, optional level if the sim is well-designed.)

? Is no one answering an assessment question incorrectly? We can either make the alternative answers more interesting, or get rid of the question all together.

Retries as a surrogate metric for fun

Finally, while not strategic, I do want to mention one more trackable metric because it is the one I use more than the others. For more interesting sims, I like to measure engagement/fun. Specifically, I like to measure how often a player replays a short sim after they get a right answer (Figure 2). My ideal player-use pattern for a conviction-building sim is:

? A player fails in the first two play-throughs, then

? Discovers the right answer on the third, and then

? Plays through the sim two more times just to experiment in the world.

Retries are a trackable metric useful in conviction-building sims

Figure 2: Retries are a trackable metric useful in conviction-building sims

Tracking as enablers

There is so much more opportunity with tracking, from the clinical A/B testing to the more tricky gaining insight into a player’s leadership style or ethics.

We are just beginning to peer into the broad power of tracking as an indispensable tool for instructional design. As we use more cloud-based services, and have access to better visual presentation tools, this will give all of us the next level of power and speed, whatever the form of the content.

Want more? (From the editor)

If you are new to short sims, see the July interview with Clark Aldrich in Learning Solutions.

On September 19, Clark Aldrich will present “Short Sims: A Game Changer” as Session 501 of The eLearning Guild’s Microlearning Design Online Conference. Learn more about the short sims approach, where it fits in your organization, and project management tips that will help you create your first short sim. Complete program information and registration is available here.