Asconferences, blogs, and forums are abuzz with talk of extracting better reportsfrom learning, big data, analytics, and ROI, a common barrier in thinkingemerges time and time again. How do we get from where we are today to where thebusiness wants us to be?
Sometimes inorder to see the best way forward, we first need to understand how we arrived wherewe are today. If we cast our minds back to the 1980s, we saw the first significantmovement outside of the education sector toward the commercial adoption oftechnology being used to enhance training. The focus was on the science behind improvinglearning efficiency, retention, and ultimately the return on investment. Thisspawned a new industry with incredible potential.
The teams formedto tackle this new evolution in training comprised skilled adult educators,behavioral psychologists, and computer science graduates. The early tools usedto build training content were driven by the science behind learning coupledwith a motivation for tangible results.
As theindustry grew, we settled on the term “eLearning,” and key infrastructurearound learning management became central to an organization’s ability toharness this process. eLearning became noted for its potential to trackcompliance, and reporting standards soon followed. The LMS quickly became thesingle largest capital expense within corporate training departments, and thestandards they dictated drove the requirements of the training they delivered.
Asorganizations pushed to take advantage of their investments, the need forquicker development and delivery methods followed. LMS reporting focused on passrates, completions, scores, time, and attempts. Given these targets,justification for compromised learning design in favor of development speednaturally followed, which paved the way for rapid development tools. Whenlearners started to complain of unengaging experiences, we looked toward thegaming and movie industries for clues to successfully engage our audiences. Toolvendors began to respond to the request for enhanced media and interactivity, andwe saw a further broadening in the target users of these tools. The role of theinstructional designer blurred to embraceweb and media developers, graphic designers, and PowerPoint enthusiasts, and wesaw a move away from the early emphasis on the science behind learning design.
As otherindustries now reap the benefits of innovations in the cloud, big data,analytics, machine learning, and artificial intelligence, it can sometimes feellike our industry is well behind—and we are, but we have some difficult historyto overcome.
Joining the dots
If we teleportourselves to imagine a future point where training is truly effective andmeasurable, and where it represents a solid voice in business strategy, itbecomes a little easier to see why our industry is struggling to join the dots.Almost four decades after realizing the potential of pairing technology withlearning, the industry has come to rest on its ability to efficiently scaledelivery. The key piece of learning management infrastructure used by mostorganizations of size has shown little sign of innovation. The standards andthinking that dominate our industry still focus on tracking completions when,in order to advance, we really need tomeasure understanding, behavioral change, our ability to execute knowledge, andreturn on investment. The tools we’re using to build content lie in thehands of programmers, graphic designers, and scriptwriters while skillededucators are often relegated to a back seat, if they haven’t already been droppedoff at a bus stop along the way.
Theinability to demonstrate our worth has led corporate training divisions tobecome the only major division within an organization unable to clearlydemonstrate tangible financial impact. Due to the lack of science involved inthe creation of learning content today, we now commonly see learners having todigest linear information filled with meaningless assessments lacking statisticalcredibility.
More thanever, the ability to break the cycle seems unachievable. Today, we see anindustry, once filled with incredible potential, completely broken by a lack offoresight and understanding of the true function of training in a corporateenvironment. A training department exists to increase profit. When we deliversales training, we do it so our teams can sell more products. We train ourleaders and managers to increase team morale, productivity, efficiencies, andreduce employee churn—all of which aim to have a positive financial effect. Eventopics like compliance, health and safety, and code of conduct aim to reducerisk that eats into company profits. When we consider this in relation to thetype of tracking our LMS delivers, we gain further clues into the problems weneed to overcome.
So, while breakingthis cycle may be difficult, it’s also inevitable. As technological advancessuch as the cloud, artificial intelligence, machine learning, big data, andanalytics drive unparalleled innovation across industries, training will not gountouched. So the question is: How do we prepare?
Finding the catalyst
Reinventingour industry requires us to revisit our purpose, fully comprehend our problems,and reconsider the way training should be perceived. We need to find directionthat will embody not only solutions to today’s innovation drivers, but solutionsthat provide us the ability to catch up. This means identifying the catalystcapable of facilitating rapid and effective change.
As much aswe may not like it, the reality of our current position is that we’re unable totruly measure the impact of training on business. Armed with a check list ofstaff completions, post-training survey forms, and smile sheets, we’re still along way off from any credible insights that can garner the attention of ourexecutive stakeholders.
It’sundeniable that training needs to become accountable. Any business divisionunable to demonstrate its worth resigns itself to a pole position for cutbacksand faces an uphill battle to attract budget. Without budget, it’s difficult toinfluence change. This brings us to a realization that the catalyst we’relooking for relates to measurement.
Critical thinking needed for change
It makessense that if we can demonstrate business impact or even a willingness tobecome accountable, we possess a business case executive-level managers can’tignore.
“Great,” yousay, “but how does knowing this help us get from here to there?” It doesn’tdirectly, but it highlights the goal we need to work toward. When I speak withCLOs actively seeking solutions to this problem, a pattern emerges. Our abilityto join the dots is clouded by our investment, reliance, and view of the world throughthe eyes of our existing tools, infrastructure, and skill sets. It’s a littlelike being invested heavily in a fleet of cars, skilled drivers, and mechanics armedwith the specialized tools needed to maintain them, then realizing you want to goplaces faster—a lot faster. You find yourself trying to figure out how to make yourcars travel at the speed of a Boeing 777, rather than thinking about a strategyto transition your method of transport.
Measuringlearning is scientifically and technically complex – extremely complex. Butthat doesn’t mean it’s unachievable, out of reach, or even difficult toimplement.
Just for amoment, let’s reflect on the mapping and GPS solutions commonly found on the smartphoneswe have in our pocket. It’s easy to forget the complexity that’s occurringunder the seemingly simple interface. When we ask for walking directions frompoint A to point B, algorithms quickly identify the route that best achievesour priorities. Over time, an interesting thing starts to occur. The estimatedtime to walk from point A to B changes—in actual fact, it becomes moreaccurate, not just for us but for everyone. This is due to data. As we follow itsdirections, the GPS continually tracks our position. This data is captured, andover time the software begins to learn not just our walking speed, butobstacles, traffic, and even variables (such as weather) that impact ourefficiency. It’s easy to overlook this complexity. The systems that capture,store, and process this data are well hidden under what we see as a simple “estimatedwalking time” value. This complexity under the hood is a key element we need torecognize and think critically about.
Measurement—where do we start?
The LMS is theobvious place to look for better measurement. This infrastructure promotes reportingas a key feature, so it makes sense. In actual fact, this is where most of usare looking for answers. Surveys have identified nearly 50 percent of participatingorganizations are looking for a new LMS to obtain better reporting.
This begsthe question—with such a high number of organizations looking to move, why haveLMSs not provided better reporting to date? This simple question reveals ananswer that LMS vendors may prefer to avoid discussing: They can’t. Better insightsrequire better data. An LMS’s reporting functionality is based on the dataprovided to it. This data comes from the content via the standards the LMS hasadopted. This leads us to ask: Can we retrieve better data from our existingcontent? At first, it would seem that the answer could be yes. We’re now seeingseveral vendors that are providing solutions with this line of thinking—wrapthe existing content in another layer able to retrieve further insights.
Survey form analytics—taking advantage of the LMS conundrum
Wrappingexisting content with a post survey form is technically easy—it will stillallow the LMS to manage the content, and it will even work alongside ourexisting tracking standards. On the surface this looks like the easy solution thatwe’ve been looking for. It should be about now that our common-sense filterstarts to twitch. Surely deep and meaningful business insights can’t be thateasy. If so, why didn’t anyone come up with this a decade ago?
Return oninvestment, behavioral change, competency revelations, ability to execute,understanding—these are extremely complex to measure. For the scientificallyminded, logic tells us it would be absurd to think we could somehow get thislevel of insight by asking the trainees 10 or 15 general questions at the endof a lesson. Unfortunately, vendors making these claims are either ignorant oftheir flawed approach or are knowingly taking advantage of the growing numberof companies looking for an easy solution. Solutions that further suggest theirmethodology could prove ROI from old training content consisting of some slidesand a handful of multiple-choice questions should be turning that twitch intoan event in need of medical intervention. Achieving these types of insightsfrom existing content is comparable to painting a Model T Ford red and thenexpecting it to perform like a Ferrari. While we can all live in hope, the well-knownsaying “If it seems too good to be true, it is” certainly applies here. Inactual fact, basing any strategic decisions on data captured in this way is notonly counterproductive but also dangerous. However, these solutions areappearing due to the lack of understanding and answers around the big nut we’reall trying to crack—how do we measure learning?
The path to data
Bettermeasurement starts with the tools. First we need sophisticated tools designedfor creating a new generation of measurable training content. Next we need torecognize the elephant in the room—if we continue creating the same style ofcontent we’re used to building on these new tools, what will the data show? Unfortunately,it won’t be the magical rainbow of joy we’re hoping for. The efforts by thelikes of the Serious eLearning Manifesto have been working to educate us around this issue. While thereality may be hard to swallow, investing in significant education andpotential restructuring to regain the science-driven learning design skills thatwe need at the center of our training development teams is going to beessential if we want to see positive results when these new tools reveal themeasurement we’ve been dreaming of. As for the LMS, we need to consider onequestion: With content able to deliver unprecedented levels of data, how will today’sLMS analyze such complex data without full context of the learning design? Significantand rapid change happens through disruption. My money is on the emergence of anew generation of companies getting ready to overturn the status quo.
Want more?
Glenn Bullwill be presenting two sessions on this topic at The eLearning Guild’s Learning Solutions 2017 Conference & Expo, March 22– 24 in Orlando, Florida:
- “The Essentials of Getting Your Organization Ready for Advanced Analytics” (1:00 PM on Wednesday, March 22)
- “Adaptive Learning: Using Measurement and Analytics to Customize Training” (4:00 PM on Thursday, March 23)
