A client recently asked us to help them keep track of their learningoutcomes throughout the development and delivery of their online course, asthey needed to satisfy strict compliance requirements. The customer neededin-depth objective tracking; this meant that they needed to be able to maplearning outcomes to individual parts of their course and evaluate theseoutcomes via rubric-based assessment. They also needed analytics that couldprovide detailed reporting on standard metrics and learning outcomes across alllevels of their organization.
Based on these requirements, we conducted extensive research ontools that could improve the tracking and reporting of learning objectives. Asactivities such as tracking and reporting take place within an organization’s learningmanagement system(s) (LMSs), we started our research by evaluating thecapabilities of several popular learning management systems, and we found thatsome are far more capable of managing learning objectives than others.
Keyfactors
LMSs that allow for strong learning-objective management can bedifferentiated based on several key factors:
1. Rubric creation and customization
Learning management systems should allow users to easily createand customize rubrics. Rubrics are assessment tools that define the criteriabeing used to evaluate an assignment, and they are used to communicateexpectations to students as well as for effectively grading assignments. SomeLMSs allow rubrics to be stored in a repository that staff members andadministrators can manage, edit, and customize.
One of the best popular LMSs we found for easy rubric generationwas Canvas. Within Canvas, users have the option of creating fully customizable learning outcome rubricsquickly and easily (Figure 1); users can also searcha database of existing outcomes.
Figure 1: A rubric created in Canvas LMS (source: Canvas Help Center)
Moodle had great potential for flexible rubriccreation due to its open-source nature. The fact that Moodle is open-sourcemeans that you can almost fully customize it in every aspect—including rubricgeneration. However, while customizing Moodle was possible, we found that inpractice it required an expert level of training.
2. Mapping learning outcomes
Ideally, an LMS should allow users to create and map learning outcomesacross multiple areas—from interactive content to summative assessments andassignments. This provides facilitators with visibility over learningobjectives to see exactly what components of individual eLearning modules aresatisfying their respective learning outcomes, and by whom.
Some learning management systems offer rubric-style learningoutcome mapping, though these rubrics are differentiated from theassessment-based rubrics mentioned above, as they’re used to ensure that acourse adheres to training-delivery criteria rather than ensuring that astudent’s work adheres to assessment criteria.
One of the most comprehensive offerings we found in regard tomapping learning outcomes was eCoach LMS. With thisplatform, you can map learning outcomes at a granular level, from specificlesson screens to individual quiz questions (Figure 2).
Figure2: eCoach LMS allows you to map learning outcomes at a granular level (source:eCoach)
Furthermore, eCoach LMS can also be bundled with eCoach Author, a cloud-based authoring tool that syncs to the LMS and allowsfor real-time mapping of learning objectives as you create content (Figure 3).
Figure3: eCoach Author allows for real-time mapping oflearning objectives as you create content (source: eCoach)
Uniquely, this approach means that learning objectives can bereported on, not only in assessment items, but through all course content. Thiswas important to our client, who needed to view specific learning outcomes inorder to ensure compliance.
3. Assessment and grading
A well-considered LMS should ideally allow users to assess andgrade students’ work in line with a rubric or set of learning outcomes. Inother words, rubrics and learning outcomes should be viewable during grading,with options provided for tallying marks using the rubric itself. The best LMSsalso offer integrated feedback options; Canvas was again a standout LMS in thisregard, with users able to comment on and grade student work against anexisting rubric from a single window. eCoach LMS doesn’t currently have thisfeature, but it is apparently planned in a future release.
4. Reporting and analytics
Our customer also required granular reporting and visualizationoptions in their course analytics to provide “at a glance” reporting that they candrill down into for each course component. eCoach LMS ticked all of theseboxes, providing individual participant reports (to see which students hadcompleted which learning objectives), while also tracking overall cohortperformance on specific objectives (by department or across the entire organization).
Overalllearning management system assessment
Based on the aforementioned factors, the best LMSs we found formanaging learning objectives were Canvas and eCoach LMS. Moodle’s customizationoptions meant that rubric generation was a strong point, though actual customizationhad to be outsourced. Moodle also lacked grading and feedback options and faredpoorly in terms of analytics and reporting, which must be completed manually inexternal platforms such as Excel. Another LMS we evaluated was Blackboard, which lacked thecustomization options of Moodle, had dense and unwieldy menu systems forcreating rubrics, and did not provide the same range of feedback options as systemslike Canvas. As such, Blackboard proved one of the more limited LMS products interms of managing learning objectives.
SpecializedLMS platforms
A further option we examined (and one that may be suitable fororganizations whose eLearning courses are almost entirely made up of compliancetraining) was the installation of a CLMS, or compliance learning managementsystem. CLMSs are LMSs that focus specifically on compliance training andoutcome mapping as their key priority. GRC Solutions, for example, offersa CLMS that allows for comprehensive and specialized compliance training.
However, CLMSs don’t offer the same range of learning managementoptions as more orthodox learning management platforms: Training is deliveredthrough a course library, meaning that users cannot be split into individualdepartments or courses, while user management is limited to assigning eachuser’s job title. CLMSs are recommended only for organizations focused solelyon compliance training.
Add-onsoftware for existing LMSs
Clearly, our client’s existing LMS did not allow for stronglearning objective management, but switching platforms was not a preferred optiondue to the time and effort involved. For this reason, we also evaluated arange of add-on software (or plugins) that can integrate with existing learningmanagement systems and that focus entirely on compliance training and mappinglearning outcomes.
One of the most powerful add-on software products we found was Chalk & Wire. Chalk & Wire offers both LTI and ATP integration, meaningthat it can integrate with almost any LMS on the market. From within their LMS,users can use Chalk & Wire to create, customize, and store rubrics, provideassessment feedback in-line, and access detailed granular analytics acrossmultiple institutional levels. Chalk & Wire integrated with our client’sLMS with a minimum of fuss and was easy to begin using. The downfall here wasthat Chalk & Wire is suited to assignment-style content only, and notinteractive eLearning.
Another external software suited to educators is Atlas Curriculum Mapping, which is fullycustomizable and has very flexible mapping options, but cannot integrate withLMSs, meaning that you have to enter data into the platform manually. AtlasCurriculum Mapping looked like a good tool for creating curriculums incollaboration with other educators, but did not suit our particular client’sneeds.
Conclusion
Learning management systems
The two standouts here were Canvas and eCoach LMS, both of whichoffered rubric generation and learning-outcome mapping, assessment options, anddetailed analytics. We would recommend Canvas for more straightforward,assessment-style learning, as it currently offers better feedback and gradingoptions. eCoach LMS is recommended for more interactive eLearning, as you caneasily map and track authored content in the eCoach platform. eCoach LMS alsoprovided the best granular reporting and analytics, with the potential to breaklearning outcomes down across the entire organization, department, group,individual, or specific resource.
Add-on software
Using an LMS that contains features for managing learningobjectives should be the preferred option for organizations that want to keeptrack of their learning outcomes, but it’s not always possible.
If your organization’s LMS does not offer adequate tracking features,add-on software such as Chalk & Wire can be helpful, though it is limitedto assignment-style learning rather than interactive eLearning. Chalk &Wire successfully integrated with all the major learning management platformswe tried it on, including Moodle, Blackboard, and Canvas.
