LMS Operation and Governance: Taming the Beast

(Editor’s Note: This is the fourth of five articles by Steve Foreman on learning management systems.)

Is your existing LMS becoming anL-M-MESS?

After you first installed your LMS,you expected that it might grow and change—and you were right. Since then, morecontent has been added, more users have become engaged, administrators have comeand gone, and your organization’s needs have evolved.

Over time, your LMS may have becomea beast. It may contain outdated courses, a seemingly haphazard organizingstructure, duplicate user accounts, catalog dead ends, lack of accountabilityor course ownership, inconsistent naming and numbering conventions, reportinganomalies, lack of distinction in course titles and descriptions, unusedfeatures, and spam-like email notifications. Does any of this sound familiar? Ifyou think you may be heading this way or you’re already there, read on.

If your organization is planning tomigrate to a new LMS or undertake a major upgrade of your current product, itcould be a great opportunity to clean up and optimize your LMS. Then, while theorganization is still mobilized around the LMS project, take steps to ensureyour new system doesn’t end up in the same state as your old one.

If your organization isimplementing a brand-new LMS for the first time, it is a good idea to take anumber of steps to avoid these problems.

So, how do you avoid an L-M-MESS?

How to optimize the operation and governance of your LMS

You must address five major areas:standards, taxonomy, configuration management, housekeeping, and governance(Figure 1).

Figure 1: These are the fiveareas you must address in order to tame your LMS

These are not sequential steps. Youcan start anywhere, as long as you attend to all five areas. If you do this,your organization will be able to gain the most benefit from your LMS.

Standards

LMS standards include policies,procedures, guidelines, conventions, and criteria related to course propertiesand structures. Standards ensure that all administrators and stakeholders areusing the LMS in a consistent and uniform way, which in turn improves itsusability and manageability.

Following are several standardsissues to consider. Some examples are included, but I don’t intend for you tocut and paste them into your organization’s implementation documentation. Youwill need to define standards that are unique and appropriate to yourimplementation.

Policies

Policies provide a set of rulesthat all LMS administrators and stakeholders must follow. Your organization maydefine any number of policies. As a starting point, there are four common LMSpolicies that you should consider.

• Content inclusion policy

  A content-inclusionpolicy (Sidebar 1) defines what content should reside in the LMS and whatcontent should not. Some organizations start out using their LMS for training—itsintended purpose. But, when other departments discover the system’s trackingand reporting capabilities, they start using it for other purposes, such asdelivering corporate communications, organizational meetings, and other events.While the LMS may be useful in tracking and reporting on this type of content,it can become harder for learners to find the training content in the system. Yourcontent inclusion policy addresses this problem by clarifying the rules forwhat content is appropriate for the LMS.

  Sidebar 1

  Example of a content inclusion policy To be included in the LMS, the learning program must: Support the organization’s goals and strategic objectives. Be sponsored and funded by a department, function, or project team. Be offered in the appropriate languages, based on the needs of the target population. Be available in the appropriate delivery methods, based on the needs of the target population. Items to be excluded from the LMS include: Messages, announcements, policy statements, and other communications that are not learning programs. College preparatory courses such as Graduate Management Admission Test (GMAT), Graduate Record Examination (GRE), Miller Analogies Test (MAT), and Law School Admission Test (LSAT).




• Content ownership policy

  A contentownership policy (Sidebar 2) defines the requirements for establishing andtracking ownership of each learning activity in the LMS. It defines theresponsibilities that content owners have, how ownership passes from person toperson, and what happens to learning activities that have no owner. One of thechief complaints from organizations with outdated content in their LMS is that,after they publish the content, no one is responsible for its continuedrelevance, accuracy, and timeliness. Your content ownership policy addressesthis problem by establishing the ongoing responsibilities of LMS contentowners.

  Sidebar 2

  Example of a content ownership policy You must assign every learning program to an individual within the organization, who serves as the content owner. The content owner is responsible for (a) ensuring the course is relevant, up-to-date, and accurate, and (b) responding to learner inquiries related to the course and its content. All course sponsors with active learning programs in the LMS are required to participate in scheduled content reviews and shelf-life-management inventories.




• Content lifecycle policy

  A content lifecyclepolicy (Sidebar 3) defines how often you should review content and whatcriteria to use for removing content from circulation. Without a content lifecyclepolicy it is unlikely that content will be reviewed, updated, and archived whenappropriate.

  Sidebar 3

  Example of a content lifecycle policy You will manage the lifecycle of learning programs by monitoring usage, working with content owners to keep the programs up-to-date, and retiring courses that are no longer relevant. Content lifecycle review shall occur annually, at which time you will run course registration reports. You will retire any online courses with less than (n) registrations in the previous (n) months, and any instructor-led courses with no scheduled offerings in the previous (n) months.




• Training information retention policy

  A training informationretention policy (Sidebar 4) defines how long you must retain studenttranscript data. If the LMS contains training history data that is older thanother non-training data in your enterprise, then you may not have established,implemented, or acted on your policy. Most organizations have a broad-basedenterprise information retention policy, which can provide some guidance forhow long you must retain training data. It may be shorter than you think. Tofind out more about your organization’s information retention policy, checkwith your legal and IT departments.

  Sidebar 4

  Example of a training information retention policy You will maintain learner-training histories in the LMS for (n) years; they are accessible online and on demand through LMS transcripts and reports. You will archive older training histories that are accessible by request within a turnaround time of (n) weeks. In some cases, you may maintain learner records for some courses for a longer period due to either of the following reasons: (a) you still recognize the course as a prerequisite for active courses, or (b) the course is associated with regulatory compliance or professional certification records that you must maintain for a longer timeframe.

Procedures

Procedures outline the steps foradministrators and content sponsors to follow when interacting with the LMS. Youmay support procedures by online request forms, documented workflows, roles andresponsibilities, step-by-step instructions, and information for settingstakeholder expectations such as turnaround times, confirmations, and othercommunications.

Your organization may define anynumber of procedures. Some common procedures include requesting a new learningprogram, updating or deactivating an existing learning program, adding orinactivating a user, requesting a custom report, and assigning administratorpermissions to a user.

Guidelines

Guidelines (Sidebar 5) provide abenchmark for administrators to use when entering information into the LMS. Forexample, you may provide guidelines related to creating titles and descriptionsfor learning programs.

Conventions

You use conventions (Sidebar 6) toensure the consistency of items such as course numbers. A course number can beuseful in sorting report and search results in some systems. It is sometimeshelpful to embed keys to the nature of the course in the course number.

Standards for course properties

Every LMS contains a set ofconfigurable course properties. Some course properties, such as title anddescription, are consistent for all types of courses. Other course propertiesvary based on the type of course. For example, an online course has a launchmethod and URL, while an instructor-led course has an instructor, location,start date and time, and end date and time.

From a system point of view, somecourse properties are required fields. Your LMS will not allow you to publish acourse without setting these properties. Others are optional.

When considering your organization’sneeds, you may designate some course properties as required by yourorganization, even if the system does not require them. It is important todocument these standards and communicate them to all LMS administrators. Adherenceto these standards will ensure the consistency of course configuration in thesystem, which will improve your reports and make the system easier to use.

Standards for course structures

Course structures are theframeworks in which you assemble course activities. A course structure maycontain a variety of activities such as classes, self-paced modules, tests, orsurveys. You may arrange activities in an enforced sequence or at the user’soption. A higher-level curriculum or learning path structure may containmultiple courses, each of which contains its own learning activities.

Since many LMS products offer avariety of ways to accomplish the same result, it is important to define,document, and communicate standards for how you should structure yourorganization’s learning programs. When courses of a similar nature arestructured consistently, students taking those courses become more familiarwith how to access and complete them.

Once you have established yourstandards, it is important to communicate them through administrator training,job aids, and administration guides.

Taxonomy

You may organize content in yourLMS in a number of ways. The LMS may contain a configurable catalog structureand a number of metadata tags that you can associate with courses. Together,these organizing components comprise the taxonomy of your content in the LMS.

Metadata is simply data thatdescribes other data. In an LMS, you can use metadata to describe and classifycourses by parameters such as topic, delivery format, or language. Metadatatypically consists of a property, such as language, and a number of values,such as English, Spanish, Japanese, etc. Typically, each course is assigned asingle value for each metadata property.

A catalog structure may consist ofa flat or hierarchical set of categories. Each category represents a node inthe structure. Courses are associated with nodes in the catalog, allowingstudents to browse the catalog to find courses relevant to each category ornode.

When designing your taxonomy, it isimportant to design through the eyes of those who will use the catalog andmetadata to find courses and evaluate whether the courses they have found meettheir needs. Try to maintain a consistent level of detail in your taxonomydesign so that items in a menu are categorically similar. Avoid ambiguity byensuring items are distinct so that administrators will know how to tagcontent. Balance hierarchical structures so that menus are kept to a reasonablenumber of items and can be taken in at a glance.

It is a good idea to test theusability of your taxonomy with a sample of learners and administrators. To dothis, establish five to seven goals for the test subject to attempt by usingthe taxonomy. An example of a goal is, “Find a course that can help you manageyour time.” Invite three to five typical end users to participate as testsubjects in the usability test. Ask each test subject, individually, to attemptto accomplish the goals you have identified. Do not interfere. If the subjecthesitates or gets stuck ask him or her to verbalize his or her thoughts. Observe,take notes, or record the test using an audio and/or video recorder. Analyzethe results from all test subjects and use the findings to refine yourtaxonomy. Repeat the usability test if needed. An iterative approach totaxonomy design and testing can yield great results.

Once you have finalized yourtaxonomy design, you will need to implement it in your production LMS. For anew implementation this is just a matter of configuring the system. If you arechanging the taxonomy of a live system, you will need to proceed carefully. First,finalize the taxonomy in a staging-server environment; that is, an installationof your LMS that is not live, but which mirrors your production system. Then,devise an implementation plan that minimizes disruption to your end users. Engageadministrators and content owners to assist in retagging content. Work withyour IT organization and/or LMS vendor to explore whether you can automate andperform part or all of the transition to the new taxonomy during off-peakhours.

Configuration management

Many businessorganizations that use an LMS will tell you that their HR system sends anightly feed of user data to the LMS. However, in many cases no one knows howthe feed works or whether it is still working properly. The people who designedand developed the feed may be long gone and may not have left anydocumentation. When the feed is changed or expanded, previously working partsmay become broken, which may even go unnoticed, contributing to the L-M-MESS. The same problems can occur withother changes to the LMS configuration. For this reason, it is important todocument your configuration decisions and keep the documentation up-to-datewhenever changes are made.

LMS configuration settingsyou should document include access and authentication, HR data feeds, useraccount and profile settings, security roles and permissions, audience rules,catalog and metadata taxonomies, transcripts and certificates, active notifications,and look-and-feel settings. Maintaining LMS configuration documentation enablesyou to plan and make changes to the configuration more easily, understand theimpact of the changes on other settings, and provide clear direction to yourvendor or IT department.

You may want toconsult your vendor or IT department for a template and/or guidance on how tobest document your configuration.

Housekeeping

The best time to clean up your datais when you migrate to a new LMS product or perform a major upgrade to yourexisting LMS. You can identify clean-up requirements and tasks and build theminto your project plan, leveraging the resources already assigned to theimplementation.

If you cannot take advantage of asystem upgrade or migration, you can still clean up your data, but you may needto create a formal project in order to assign the necessary resources, time,and effort to your clean-up effort.

In either case, your main goal isto bring your LMS data up to the standards your organization has defined.

The first step is to create aninventory of the data that you need to purge or clean up. Identify courses thatyou should archive. Identify user accounts to deactivate and duplicate accountsto merge.

Next, make a comprehensive list ofany configuration changes you intend to make and determine how data will beaffected.

Once you have identified thechanges to make, carefully choreograph the clean-up process. Define all steps,sequences, and dependencies. Assign who is responsible for each step. Identifythe steps that you can do on the current system before the migration or upgradeand those that must happen afterwards. Check and double-check your plan toensure nothing has been missed.

If you need to take the productionsystem offline to perform some of the steps in order to minimize system downtime, carefully plan what must be done, when it will be done, who will do it,and how you will know it has been done accurately. Automate as much of theclean-up process as possible by working with your IT department or LMS vendorto create database scripts to run to clean up the data based on rules andparameters you have defined.

Before you perform the actualclean-up tasks on the production system, it is a good idea to practice theend-to-end process. You can do this by asking your IT department or LMS vendorto copy the production system to a staging-server environment, which is inaccessibleto users. Rehearse the process on the staging server. Document any changes oradditions to the steps in the plan based on the results of your rehearsal. Ifnecessary, ask IT or your vendor to reset the staging environment and repeatthe rehearsal until you are satisfied it can be conducted effectively andexpeditiously in the production environment.

When you are ready to perform theclean-up process in your production environment, first ask IT or your LMSvendor to back up the production database. Perform the clean-up tasks that you cando while the server is online. Then notify end users of the system outage,bring the system down, and perform the offline cleanup tasks. Finally, bringthe system back up and perform any additional cleanup activities in the livesystem. Thoroughly test the system configuration to validate that your cleanup hasbeen completed effectively and that no new, unforeseen problems have emerged.

Governance

System governance ensures that theLMS implementation is in alignment with the goals and needs of theorganization. Governance establishes appropriate representation from allstakeholder groups and provides a structure for decision-making. Withoutadequate governance, it is more difficult to establish and enforce standards.

While each organization mayestablish its own unique governance structure, there is a basic governancemodel that can be helpful. The groups within this structure interact to ensure thatmanagement of LMS issues are at the appropriate level and you escalate anyunresolved or systemic issues (Figure 2).

Figure 2: Governancestructure

Governing board

A governing board may consist ofkey stakeholders at the executive level. The board represents the organization’sstrategic goals. The governing board does not focus on technology. Its charteris to provide direction to ensure linkage between business strategy andlearning strategy. The governing board may convene one or two times per year.

LMS steering team

The LMS steering team is comprisedof key stakeholders at the senior management level. Each team member representsone or more user groups that rely on the LMS (e.g., learners, managers, andfaculty). The LMS steering team’s charter is to establish learning managementpractices and policies. The team may convene quarterly.

LMS working groups

LMS working groups consist of keystakeholders at the manager level. Each group represents a special area offocus (e.g., taxonomy, standards). The charter of the working groups is to planand execute activities related to LMS usability and operations. Groups mayconvene six times per year and sometimes more often during special initiatives.

LMS operations

The LMS operations structuretypically consists of four groups: One group each to manage LMS operations, contentowners, administrators, and technical support.

• LMSoperations management

  The LMS operationsmanagement team is responsible for ensuring that the LMS operates reliably, ismanaged in conformance with standards, and meets the needs of the organization.The operations management team works closely with the working groups andsteering team, helping to surface issues that need attention.

• Contentowners

  LMS content ownersare responsible for the quality of the learning programs they own (i.e.,accuracy, relevance, and timeliness). Content owners must provide appropriateinformation about the learning program to enable LMS administrators toconfigure the learning program based on the LMS standards. Content owners mustbe vigilant in monitoring utilization by the learning program’s target audienceand ensuring the program is kept up-to-date.

• LMSadministrators

  LMS administratorsare responsible for the accuracy and thoroughness of content configuration inthe LMS. They must provide timely response to requests from content owners. LMSadministrators must consistently implement LMS standards, conventions, policies,and processes.

• Technicalsupport

  LMS technicalsupport may consist of some combination of training, IT, and vendor staff. Technicalsupport groups may include your helpdesk, eLearning content developers,developers of custom reports, server support, database and applicationmanagers, IT security, and network support. Together, these technical supportgroups are responsible for keeping the application up and running, resolvingend-user issues, ensuring any eLearning programs are working properly,developing custom reports, managing any changes to the system configuration,and installing patches and updates.

Concluding thoughts

Your organization may be about tospend, or may have already invested a lot of time, effort, resources, and ofcourse, money on acquiring and implementing (or updating) an LMS. It isimportant to protect this investment by establishing the necessary operationsand governance measures to ensure that your system continues to be easy to useand provides highly relevant content to your end users. By putting thesecritical pieces into place, you are much more likely to get optimum value fromyour LMS.

Althoughit would be nice, it is extremely unlikely that you can simply buy it, installit, and just let it do its thing. Remember, it’s not just about acquiring the technology;it’s also about how well the technology is used.LMS implementation can be challenging, but the payoff is huge. Taking steps toestablish and maintain standards, taxonomy, configuration management,housekeeping, and governance will tame the beast, avoid an L-M-MESS, and help to ensure that your LMSremains easy to use and operate for years to come.