There are many application and software types that can be used to perform data analytics as part of your instructional development project. In this article, I will describe a process for selecting software or applications for data analytics during front-end analysis (also called needs assessment).
The first steps: before tool selection
The first step is to decide what you want to know. Rather than just begin to copy data from your LMS or through xAPI, at the very beginning, the instructional development team must know what it is looking for. Usually this means you have a question about performance or accomplishment that you want to answer. Or you may want to know whether a learning experience had an impact, made a difference, or the information was used some time later, and from that study you learned something else and you then need to go back and learn some more. Or maybe you used a new approach to the design or delivery of a learning experience and you want to know if it worked. Data analytics can help answer all of these questions, but there's a catch.
You should only be looking to get data to answer one specific question at a time. Data analytics can only deal with one question at a time. If you have two questions, you will need to do two separate analyses. Start with what you want to know. You may need to select team members on an interdisciplinary basis. Some potential members may be expert at using some tools, and others may have different skill sets. If you are a team manager, this matching of skill sets and tools (or managing the mix) will be essential to success.
Tool selection comes next, but I am going to jump ahead for a moment.
The next steps after tool selection
Use the application(s) you select to build a baseline. This gives you a starting place against which you can assess your progress. As you build the baseline, you may be able to craft your learning objective, or modify it. This may be the most difficult step in terms of the need to create or reiterate tests of what you have designed as an implementation. Remember to avoid letting the perfect become the enemy of the good, or, as they say, know when the results you are getting are good enough. You can always come back and revise things later.
Some development managers find it helpful to journal the quantitative and qualitative changes achieved at each step.
Software reviews and tool selection
This is the step between identifying the question you want to answer and choosing the data to analyze in order to answer it. It is also the step where you must learn some things about the largest number of potential tools. In Learning Solutions and in The Learning Guild newsletters and research reports, as well as in your professional reading, you will find many ads and mentions of various tools.
Each of these tools will have a purpose, benefits, cost, features, considerations for purchase, and operating system. The ads themselves will contain a mixture of descriptive jargon that describes the various ways in which the given tools are intended to fit into the process of data analysis. It is in sorting through this jargon—the tool selection vocabulary of the practitioners—that you will begin to understand data analysis itself.
Fortunately, there are good guides to develop your understanding. These are the various software reviews that you can find compiled online, and at the end of this article I have linked the three review sites that I believe are most reliable. You will find some overlap in their listings and some unique reviews that address commonly used software, but in different ways.
Basics (not necessarily reviewed individually)
There are two basic categories of software that development teams must be able to deal with. Not every person on a development team is required to be able to handle every one of them, but collectively, here is the skillset that a team must cover in whole or in part, depending on the project.
If there is a foundational tool that many if not most team members must be able to work with, it is Microsoft Excel. This shows up on every list of data analytics software.
Programming languages and multi-language applications
Everyone on an eLearning development team need not be a programmer, and in fact on some projects there will be team members who do not need to use these languages at all. But some team members will need to be able to program. In some cases "no programming" software will help the non-programmers carry out their functions.
- Python (popular programming language that can be used on a server to create web applications)
- R (programming language for statistical computing and graphics used among data miners and statisticians for data analysis and developing statistical software.)
- SAS (statistical software suite developed by SAS Institute for data management, advanced analytics, multivariate analysis, business intelligence, criminal investigation, and predictive analytics.
- SQL (standard language for storing, manipulating, and retrieving data in relational databases)
- Jupyter Notebooks (web application for creating and sharing computational documents)
- Apache Spark (a multi-language engine for executing data engineering, data science, and machine learning)
Software review sites
Capterra: A Buyer’s Guide in the right-hand column of each category addresses such points as purpose of the category entries, benefits, cost range, features, considerations for purchase (business requirements, scalability, integration with other business tools, mobile accessibility), trends in category, and breakdown by operating system.
- Data Analysis
- Qualitative Data Analysis
- Embedded Analytics Software
- Business Intelligence
- Data Visualization
G2: Note that G2 listings for data analytics contain many products that are industry-specific or sales/marketing oriented—you may have to look down the lists to find products that are more applicable to instructional development; start with the “Popular Data Analytics Software”.
- Popular Data Analytics Software
- Data Visualization Software
- Embedded Business Intelligence Software
- Self-service Business Intelligence Software