(Editor’s Note: This is the fourth of five articles by Steve Foreman on learning management systems.)

Is your existing LMS becoming an L-M-MESS?

After you first installed your LMS, you expected that it might grow and change—and you were right. Since then, more content has been added, more users have become engaged, administrators have come and gone, and your organization’s needs have evolved.

Steve Foreman’s five articles on LMSs:

Over time, your LMS may have become a beast. It may contain outdated courses, a seemingly haphazard organizing structure, duplicate user accounts, catalog dead ends, lack of accountability or course ownership, inconsistent naming and numbering conventions, reporting anomalies, lack of distinction in course titles and descriptions, unused features, and spam-like email notifications. Does any of this sound familiar? If you think you may be heading this way or you’re already there, read on.

If your organization is planning to migrate to a new LMS or undertake a major upgrade of your current product, it could be a great opportunity to clean up and optimize your LMS. Then, while the organization is still mobilized around the LMS project, take steps to ensure your new system doesn’t end up in the same state as your old one.

If your organization is implementing a brand-new LMS for the first time, it is a good idea to take a number of steps to avoid these problems.

So, how do you avoid an L-M-MESS?

How to optimize the operation and governance of your LMS

You must address five major areas: standards, taxonomy, configuration management, housekeeping, and governance (Figure 1).

Figure 1: These are the five areas you must address in order to tame your LMS

 

These are not sequential steps. You can start anywhere, as long as you attend to all five areas. If you do this, your organization will be able to gain the most benefit from your LMS.

Standards

LMS standards include policies, procedures, guidelines, conventions, and criteria related to course properties and structures. Standards ensure that all administrators and stakeholders are using the LMS in a consistent and uniform way, which in turn improves its usability and manageability.

Following are several standards issues to consider. Some examples are included, but I don’t intend for you to cut and paste them into your organization’s implementation documentation. You will need to define standards that are unique and appropriate to your implementation.

Policies

Policies provide a set of rules that all LMS administrators and stakeholders must follow. Your organization may define any number of policies. As a starting point, there are four common LMS policies that you should consider.

  • Content inclusion policy

    A content-inclusion policy (Sidebar 1) defines what content should reside in the LMS and what content should not. Some organizations start out using their LMS for training—its intended purpose. But, when other departments discover the system’s tracking and reporting capabilities, they start using it for other purposes, such as delivering corporate communications, organizational meetings, and other events. While the LMS may be useful in tracking and reporting on this type of content, it can become harder for learners to find the training content in the system. Your content inclusion policy addresses this problem by clarifying the rules for what content is appropriate for the LMS.

    Sidebar 1

    Example of a content inclusion policy

    To be included in the LMS, the learning program must:

    • Support the organization’s goals and strategic objectives.
    • Be sponsored and funded by a department, function, or project team.
    • Be offered in the appropriate languages, based on the needs of the target population.
    • Be available in the appropriate delivery methods, based on the needs of the target population.

    Items to be excluded from the LMS include:

    • Messages, announcements, policy statements, and other communications that are not learning programs.
    • College preparatory courses such as Graduate Management Admission Test (GMAT), Graduate Record Examination (GRE), Miller Analogies Test (MAT), and Law School Admission Test (LSAT).

  • Content ownership policy

    A content ownership policy (Sidebar 2) defines the requirements for establishing and tracking ownership of each learning activity in the LMS. It defines the responsibilities that content owners have, how ownership passes from person to person, and what happens to learning activities that have no owner. One of the chief complaints from organizations with outdated content in their LMS is that, after they publish the content, no one is responsible for its continued relevance, accuracy, and timeliness. Your content ownership policy addresses this problem by establishing the ongoing responsibilities of LMS content owners.

    Sidebar 2

    Example of a content ownership policy

    You must assign every learning program to an individual within the organization, who serves as the content owner. The content owner is responsible for (a) ensuring the course is relevant, up-to-date, and accurate, and (b) responding to learner inquiries related to the course and its content. All course sponsors with active learning programs in the LMS are required to participate in scheduled content reviews and shelf-life-management inventories.


  • Content lifecycle policy

    A content lifecycle policy (Sidebar 3) defines how often you should review content and what criteria to use for removing content from circulation. Without a content lifecycle policy it is unlikely that content will be reviewed, updated, and archived when appropriate.

    Sidebar 3

    Example of a content lifecycle policy

    You will manage the lifecycle of learning programs by monitoring usage, working with content owners to keep the programs up-to-date, and retiring courses that are no longer relevant. Content lifecycle review shall occur annually, at which time you will run course registration reports. You will retire any online courses with less than (n) registrations in the previous (n) months, and any instructor-led courses with no scheduled offerings in the previous (n) months.


  • Training information retention policy

    A training information retention policy (Sidebar 4) defines how long you must retain student transcript data. If the LMS contains training history data that is older than other non-training data in your enterprise, then you may not have established, implemented, or acted on your policy. Most organizations have a broad-based enterprise information retention policy, which can provide some guidance for how long you must retain training data. It may be shorter than you think. To find out more about your organization’s information retention policy, check with your legal and IT departments.

    Sidebar 4

    Example of a training information retention policy

    You will maintain learner-training histories in the LMS for (n) years; they are accessible online and on demand through LMS transcripts and reports. You will archive older training histories that are accessible by request within a turnaround time of (n) weeks.

    In some cases, you may maintain learner records for some courses for a longer period due to either of the following reasons: (a) you still recognize the course as a prerequisite for active courses, or (b) the course is associated with regulatory compliance or professional certification records that you must maintain for a longer timeframe.

Procedures

Procedures outline the steps for administrators and content sponsors to follow when interacting with the LMS. You may support procedures by online request forms, documented workflows, roles and responsibilities, step-by-step instructions, and information for setting stakeholder expectations such as turnaround times, confirmations, and other communications.

Your organization may define any number of procedures. Some common procedures include requesting a new learning program, updating or deactivating an existing learning program, adding or inactivating a user, requesting a custom report, and assigning administrator permissions to a user.

Guidelines

Guidelines (Sidebar 5) provide a benchmark for administrators to use when entering information into the LMS. For example, you may provide guidelines related to creating titles and descriptions for learning programs.

Sidebar 5

Example of guidelines for course titles

  1. Use a shorter title, just long enough to accurately describe the learning program and distinguish it from other programs with similar titles.
  2. Search the LMS for other courses with similar titles and make sure your title is distinct.
  3. Avoid abbreviations and acronyms unless they are commonly known.
  4. Spell out words like “at” and “and” instead of using symbols like “@” and “&” unless the symbol is part of a product or brand name.
  5. When the course is part of a series, ensure consistent titling of all courses in the series.
  6. Use initial caps for all major words as you might see in the title of a book or article.
  7. Be sure to check your spelling.

Conventions

You use conventions (Sidebar 6) to ensure the consistency of items such as course numbers. A course number can be useful in sorting report and search results in some systems. It is sometimes helpful to embed keys to the nature of the course in the course number.

Sidebar 6

Example of convention for course numbers



Standards for course properties

Every LMS contains a set of configurable course properties. Some course properties, such as title and description, are consistent for all types of courses. Other course properties vary based on the type of course. For example, an online course has a launch method and URL, while an instructor-led course has an instructor, location, start date and time, and end date and time.

From a system point of view, some course properties are required fields. Your LMS will not allow you to publish a course without setting these properties. Others are optional.

When considering your organization’s needs, you may designate some course properties as required by your organization, even if the system does not require them. It is important to document these standards and communicate them to all LMS administrators. Adherence to these standards will ensure the consistency of course configuration in the system, which will improve your reports and make the system easier to use.

Standards for course structures

Course structures are the frameworks in which you assemble course activities. A course structure may contain a variety of activities such as classes, self-paced modules, tests, or surveys. You may arrange activities in an enforced sequence or at the user’s option. A higher-level curriculum or learning path structure may contain multiple courses, each of which contains its own learning activities.

Since many LMS products offer a variety of ways to accomplish the same result, it is important to define, document, and communicate standards for how you should structure your organization’s learning programs. When courses of a similar nature are structured consistently, students taking those courses become more familiar with how to access and complete them.

Once you have established your standards, it is important to communicate them through administrator training, job aids, and administration guides.

Taxonomy

You may organize content in your LMS in a number of ways. The LMS may contain a configurable catalog structure and a number of metadata tags that you can associate with courses. Together, these organizing components comprise the taxonomy of your content in the LMS.

Metadata is simply data that describes other data. In an LMS, you can use metadata to describe and classify courses by parameters such as topic, delivery format, or language. Metadata typically consists of a property, such as language, and a number of values, such as English, Spanish, Japanese, etc. Typically, each course is assigned a single value for each metadata property.

A catalog structure may consist of a flat or hierarchical set of categories. Each category represents a node in the structure. Courses are associated with nodes in the catalog, allowing students to browse the catalog to find courses relevant to each category or node.

When designing your taxonomy, it is important to design through the eyes of those who will use the catalog and metadata to find courses and evaluate whether the courses they have found meet their needs. Try to maintain a consistent level of detail in your taxonomy design so that items in a menu are categorically similar. Avoid ambiguity by ensuring items are distinct so that administrators will know how to tag content. Balance hierarchical structures so that menus are kept to a reasonable number of items and can be taken in at a glance.

It is a good idea to test the usability of your taxonomy with a sample of learners and administrators. To do this, establish five to seven goals for the test subject to attempt by using the taxonomy. An example of a goal is, “Find a course that can help you manage your time.” Invite three to five typical end users to participate as test subjects in the usability test. Ask each test subject, individually, to attempt to accomplish the goals you have identified. Do not interfere. If the subject hesitates or gets stuck ask him or her to verbalize his or her thoughts. Observe, take notes, or record the test using an audio and/or video recorder. Analyze the results from all test subjects and use the findings to refine your taxonomy. Repeat the usability test if needed. An iterative approach to taxonomy design and testing can yield great results.

Once you have finalized your taxonomy design, you will need to implement it in your production LMS. For a new implementation this is just a matter of configuring the system. If you are changing the taxonomy of a live system, you will need to proceed carefully. First, finalize the taxonomy in a staging-server environment; that is, an installation of your LMS that is not live, but which mirrors your production system. Then, devise an implementation plan that minimizes disruption to your end users. Engage administrators and content owners to assist in retagging content. Work with your IT organization and/or LMS vendor to explore whether you can automate and perform part or all of the transition to the new taxonomy during off-peak hours.

Configuration management

Many business organizations that use an LMS will tell you that their HR system sends a nightly feed of user data to the LMS. However, in many cases no one knows how the feed works or whether it is still working properly. The people who designed and developed the feed may be long gone and may not have left any documentation. When the feed is changed or expanded, previously working parts may become broken, which may even go unnoticed, contributing to the L-M-MESS. The same problems can occur with other changes to the LMS configuration. For this reason, it is important to document your configuration decisions and keep the documentation up-to-date whenever changes are made.

LMS configuration settings you should document include access and authentication, HR data feeds, user account and profile settings, security roles and permissions, audience rules, catalog and metadata taxonomies, transcripts and certificates, active notifications, and look-and-feel settings. Maintaining LMS configuration documentation enables you to plan and make changes to the configuration more easily, understand the impact of the changes on other settings, and provide clear direction to your vendor or IT department.

You may want to consult your vendor or IT department for a template and/or guidance on how to best document your configuration.

Housekeeping

The best time to clean up your data is when you migrate to a new LMS product or perform a major upgrade to your existing LMS. You can identify clean-up requirements and tasks and build them into your project plan, leveraging the resources already assigned to the implementation.

If you cannot take advantage of a system upgrade or migration, you can still clean up your data, but you may need to create a formal project in order to assign the necessary resources, time, and effort to your clean-up effort.

In either case, your main goal is to bring your LMS data up to the standards your organization has defined.

The first step is to create an inventory of the data that you need to purge or clean up. Identify courses that you should archive. Identify user accounts to deactivate and duplicate accounts to merge.

Next, make a comprehensive list of any configuration changes you intend to make and determine how data will be affected.

Once you have identified the changes to make, carefully choreograph the clean-up process. Define all steps, sequences, and dependencies. Assign who is responsible for each step. Identify the steps that you can do on the current system before the migration or upgrade and those that must happen afterwards. Check and double-check your plan to ensure nothing has been missed.

If you need to take the production system offline to perform some of the steps in order to minimize system down time, carefully plan what must be done, when it will be done, who will do it, and how you will know it has been done accurately. Automate as much of the clean-up process as possible by working with your IT department or LMS vendor to create database scripts to run to clean up the data based on rules and parameters you have defined.

Before you perform the actual clean-up tasks on the production system, it is a good idea to practice the end-to-end process. You can do this by asking your IT department or LMS vendor to copy the production system to a staging-server environment, which is inaccessible to users. Rehearse the process on the staging server. Document any changes or additions to the steps in the plan based on the results of your rehearsal. If necessary, ask IT or your vendor to reset the staging environment and repeat the rehearsal until you are satisfied it can be conducted effectively and expeditiously in the production environment.

When you are ready to perform the clean-up process in your production environment, first ask IT or your LMS vendor to back up the production database. Perform the clean-up tasks that you can do while the server is online. Then notify end users of the system outage, bring the system down, and perform the offline cleanup tasks. Finally, bring the system back up and perform any additional cleanup activities in the live system. Thoroughly test the system configuration to validate that your cleanup has been completed effectively and that no new, unforeseen problems have emerged.

Governance

System governance ensures that the LMS implementation is in alignment with the goals and needs of the organization. Governance establishes appropriate representation from all stakeholder groups and provides a structure for decision-making. Without adequate governance, it is more difficult to establish and enforce standards.

While each organization may establish its own unique governance structure, there is a basic governance model that can be helpful. The groups within this structure interact to ensure that management of LMS issues are at the appropriate level and you escalate any unresolved or systemic issues (Figure 2).

Figure 2: Governance structure

Governing board

A governing board may consist of key stakeholders at the executive level. The board represents the organization’s strategic goals. The governing board does not focus on technology. Its charter is to provide direction to ensure linkage between business strategy and learning strategy. The governing board may convene one or two times per year.

LMS steering team

The LMS steering team is comprised of key stakeholders at the senior management level. Each team member represents one or more user groups that rely on the LMS (e.g., learners, managers, and faculty). The LMS steering team’s charter is to establish learning management practices and policies. The team may convene quarterly.

LMS working groups

LMS working groups consist of key stakeholders at the manager level. Each group represents a special area of focus (e.g., taxonomy, standards). The charter of the working groups is to plan and execute activities related to LMS usability and operations. Groups may convene six times per year and sometimes more often during special initiatives.

LMS operations

The LMS operations structure typically consists of four groups: One group each to manage LMS operations, content owners, administrators, and technical support.

  • LMS operations management

    The LMS operations management team is responsible for ensuring that the LMS operates reliably, is managed in conformance with standards, and meets the needs of the organization. The operations management team works closely with the working groups and steering team, helping to surface issues that need attention.

  • Content owners

    LMS content owners are responsible for the quality of the learning programs they own (i.e., accuracy, relevance, and timeliness). Content owners must provide appropriate information about the learning program to enable LMS administrators to configure the learning program based on the LMS standards. Content owners must be vigilant in monitoring utilization by the learning program’s target audience and ensuring the program is kept up-to-date.

  • LMS administrators

    LMS administrators are responsible for the accuracy and thoroughness of content configuration in the LMS. They must provide timely response to requests from content owners. LMS administrators must consistently implement LMS standards, conventions, policies, and processes.

  • Technical support

    LMS technical support may consist of some combination of training, IT, and vendor staff. Technical support groups may include your helpdesk, eLearning content developers, developers of custom reports, server support, database and application managers, IT security, and network support. Together, these technical support groups are responsible for keeping the application up and running, resolving end-user issues, ensuring any eLearning programs are working properly, developing custom reports, managing any changes to the system configuration, and installing patches and updates.

Concluding thoughts

Your organization may be about to spend, or may have already invested a lot of time, effort, resources, and of course, money on acquiring and implementing (or updating) an LMS. It is important to protect this investment by establishing the necessary operations and governance measures to ensure that your system continues to be easy to use and provides highly relevant content to your end users. By putting these critical pieces into place, you are much more likely to get optimum value from your LMS.

Although it would be nice, it is extremely unlikely that you can simply buy it, install it, and just let it do its thing. Remember, it’s not just about acquiring the technology; it’s also about how well the technology is used. LMS implementation can be challenging, but the payoff is huge. Taking steps to establish and maintain standards, taxonomy, configuration management, housekeeping, and governance will tame the beast, avoid an L-M-MESS, and help to ensure that your LMS remains easy to use and operate for years to come.