Shifting focus from objectives to outcomes can aid in creating authentic assessments that actually test what we’re trying to measure, according to Jean Marrapodi of Applestar Productions.
In a recent presentation at The eLearning Guild’s eLearning Foundations Spotlight, Marrapodi explained that an authentic assessment examines the learner’s understanding and application of information and ability to apply it in a context close to the real world. That’s different from evaluating learners’ reactions to training or the ROI a company realizes from sending employees to training.
Measuring whether learners can do tasks following training is really the crux of eLearning evaluation. Yet lots of so-called assessment questions measure vocabulary or information recall, she said. “It doesn’t really matter if they know terminology if they can’t do what we need them to do with the learning they’ve gained.” That’s why assessments should focus on outcomes.
Learning objectives or outcomes?
Instructional designers write learning objectives as a roadmap to spell out where the eLearning is going, tell learners what to expect, and describe knowledge or tasks that learners will encounter in the course. They should be aligned to the overall course goal. Marrapodi suggests that designers rethink using objectives and switch to writing outcomes that begin with the end in mind.
Objectives identify the knowledge and abilities learners are intended to glean from the learning. Objectives, which Marrapodi said “focus on the teacher or designer,” are task-based and point to how learners will get to the goal or accomplish an outcome. They are more like a lesson plan, defining what will be presented. Here’s an example:
In this lesson, you will learn how to use the Format Painter to copy font styles.
It tells the learner what is coming.
Outcomes are learner-focused and point to the results of the lesson, providing the WIIFM (What's In It For Me?). In the end, the learner will know and be able to do something specific. The designer states the desired end results, not the process of attaining the results, and they should be components of the overall goal. Here is the outcome statement for that same concept:
At the end of this lesson, you should be able to use the style features of Word to create professional-looking documents.
Marrapodi encourages instructional designers to write the outcomes before writing the learning objectives, and recommends using the objectives to define how the outcome will be achieved. Multiple objectives generally map to a single outcome. In the above example, the objectives might also include learning about styles and templates, being able to update a style, learning how to use tabs and paragraph spacing, and much more. Each of those learning objectives helps the learners reach the outcome of creating professional-looking documents.
Measure what matters
Instructional designers should be sure to assess each objective and outcome, as well as the overarching goal. The assessments should reliably measure whether the learner has met the goal of the eLearning course. Can she demonstrate the skills learned? Does she recall the information studied? Sometimes the activity within a course serves as assessment—allowing the learner to show what she knows. In the end, a summative assessment allows the learner to demonstrate that she has achieved the goal of the course.
High-quality assessment questions require the learner to apply knowledge in order to get the correct response. This often means moving beyond learner surveys and multiple choice questions into more authentic assessments.
Questions should test their understanding and application of key terms and phrases rather than just being a recall of definitions. And scenarios provided in the assessment should ask learners to apply their new knowledge in a relevant context that is different from the examples used in the eLearning. Choosing correct responses in a new context—one that resembles a situation they are likely to encounter on the job—allows learners to show that they’ve understood how to use the training in a realistic scenario and reinforces the transference that is a key goal of training: application on the job.
Be careful not to use questions that can be answered correctly without true understanding of how to do the task—and questions that learners are unlikely to answer correctly even if they do understand the material. The goal is for learners to show what they know.
Remain focused on the goal: “It’s not ‘knowing about.’ It’s doing!” Marrapodi emphasized.
Delve into Evaluating Learning
Evaluating learning effectively takes more effort than simply pulling together a few survey questions. A recent Guild research report, Evaluating Learning: Insights from Learning Professionals, examines how L&D professionals are evaluating eLearning, how they’d like to be doing it, and what they and author Will Thalheimer suggest as best practices for more authentic assessments. Download the report today to explore how your peers approach assessments.