The eLearning Guild’s newest research report, Beyond ROI: The Value of Learning, explains that training ROI studies are flawed because they measure learning “events.” But because we are continually learning at work we need to look for more holistic and contextual training impacts, such as through Robert O. Brinkerhoff’s models (which are presented and explained in the report, and in Brinkerhoff’s book—see “Reference” at the end of this article).
Nic Laycock, the author of the report, asked C-level execs what they wanted to have measured and, just as we found in the Guild’s Getting Started in e-Learning: Measuring Success research report, they are not nearly as interested in ROI as we think they are. Instead, they are more interested in knowing that people have the skills needed to do their job and contribute to the needed performance of the organization.
But … do people in our organizations have the skills they need?
The Organisation for Economic Co-operation and Development (OECD) has begun performing a survey of adult skills to assess key information-processing skills that adults aged 16 – 65 need in order to participate in education, training, work, and social and community life.
They interviewed people in their homes, asked people to answer questions, and collected a broad range of data about understanding and using information from written texts; using, applying, interpreting, and communicating mathematical information and ideas; and using technology to solve problems and accomplish tasks.
Participants in Round 1 (2008 – 2012) came from the following countries: Australia, Austria, Belgium (Flanders), Canada, Czech Republic, Denmark, Estonia, Finland, France, Germany, Ireland, Italy, Japan, Korea, Netherlands, Norway, Poland, Russian Federation, Slovak Republic, Spain, Sweden, England, Northern Ireland, and the United States.
They recently published what they found from Round 1. The results highlight a number of concerns. Figure 1 shows some of the results for literacy (understanding and using information from written texts).
Figure 1: OECD Survey of adult skills literacy levels by country
(Source:
http://skills.oecd.org/documents/OECD_Skills_Outlook_2013.pdf, pg. 29)
The skills in Figure 1 are learnable skills. The report makes a clear case that each country shapes the level of skill in their population through learning opportunities (in home, education, and workplace).
We can see that some countries are struggling with basic skills and in others, a large percentage of the population have advanced skills. The study points out, for example, that in Italy and Spain only one in 20 adults is proficient at the highest level of literacy. And the US is below average for this group of countries. A closer look shows that in all but one of these countries, at least one tenth of adults are proficient only at or below Level 1 in literacy or numeracy.
In most countries, a significant portion of adults, “…have trouble using digital technology, communication tools, and networks to acquire and evaluate information, communicate with others, and perform practical tasks … From 7% to 27% of adults lack the most elementary computer skills, such as the ability to use a mouse.”
The demand for these types of skills continues to shift towards higher and higher levels, even with lower-level jobs, because using and communicating with technology is pervading every aspect of our lives. At this point, lack of basic computer skills is beginning to limit adults’ access to services, jobs, and even participation in training and education. My daughter told me she recently went into a bank branch where customers communicated with a virtual teller via computer screen. There weren’t any people working in the branch! I recently visited a dental specialty office where they computerized everything. There was literally no paper in the office. I filled out “papers” via an iPad and they emailed my receipt and other paperwork to me. How would those 7% to 27% fare in these instances?
There are certainly huge implications here for those of us in education and training. What is our role with those who do not have adequate literacy, numeracy, and technical skills? Will those people even make it to us? Are you dealing with this in your workplace? What are your thoughts?
Reference
Brinkerhoff, Robert O. Telling Training’s Story: Evaluation Made Simple, Credible, and Effective. 2006.