Research over the past few years has shown that while learning analytics is seen as a critical focus area for learning and development, learning professionals are not confident in their data skills and see big data challenges ahead as they respond to calls from the business to be more data-driven. Where are the potential pitfalls for L&D, and how should they avoid them?
1. Taking too narrow a view of the subject
People often box themselves in by taking a very limited view of what can be done with learning data and thinking of it solely in terms of historical evaluation of past training programs.
Take a 360 approach to learning data. You wouldn’t drive using only your rear-view mirror, so don’t restrict your focus to learning evaluation (important as that is). A more holistic approach will also give you side mirrors, dash-cam, and a clearer view of the road ahead. It’s not all about evaluation or analytics.
So here’s a better box for your thinking: a handy data schema given by Donald Clark in his book Artificial Intelligence for Learning that will help you appreciate the full scope and opportunity offered by taking on big data challenges. The schema is based on goals, because as Clark writes, “it is almost pointless to gather data if it remains unloved and unused. There must be a purpose to the endeavor.”
The four goals of the schema
2. Being too reactive
If you wait to be asked for better data by the business, there is a theory called the ‘conspiracy of convenience’ that suggests it might never happen. It maintains that learning departments and their internal clients collude in a largely unspoken way to avoid the subject of learning impact. The client doesn’t ask for data to determine whether a learning intervention had any impact on performance, and the learning department doesn’t volunteer to supply it.
However, if you spot this happening in your organization and you go along with it, you will run the risk of irrelevance and ultimately obsolescence. Be proactive. Start today in making data central to your practice, and educate your internal and external customers on the benefits of taking a data-led approach.
3. Getting in a rut with data sources
Many learning people fixate on particular data sources that are close to hand and relatively easy to gather—e.g., course completions and test scores from your learning management system—and fail to look further afield. There is an ever-increasing amount of data available now from all sorts of sources. The danger of sticking to limited sources is that it’s rare to make an easy, causal link between one data source on learning and an improvement in performance. In other words, it’s hard to find a ‘smoking gun’ when it comes to evaluating impact.
Solution? Use multiple data sources to evaluate and take a ‘portfolio of evidence approach’. Chances are you will only be able to prove that your intervention might be the reason something changed, but you can stack that deck in your favor with more data points.
4. Siloed thinking
It can be all too tempting to ‘stay within your lane’ and ignore what is going on in the rest of the organization. However, learning doesn’t happen in a vacuum. Learners are primarily employees with day-to-day work that takes up a lot more of their time than their learning activities, so it follows that the vast majority of data about their performance, skills gaps, and knowledge needs are not in the learning management system but in other parts of the organization.
Today’s organizations are awash with data. Starting from where you are, use the data resources that are on hand in other parts of the business, though it might mean forging new alliances and learning new skills to get hold of it.
5. Bunker mentality
Given all the demands and stresses of dealing with the business, it is all too easy for learning professionals to feel undervalued and misunderstood, and to start to adopt a bunker mentality. The defensive mindset this promotes is the very worst thing that can happen in a situation where the learning function is needing to open up and learn new skills to deal with big data challenges. Showing a gap in knowledge and skills of any kind can run the risk of being perceived as a weakness. But you can’t be a learning professional and not have the courage to learn yourself!
Start small, fail fast, learn by doing. Learning Pool has valuable tools available to help you. Find out more.
This article is sponsored by: