MOOCs in Higher Education: Options, Affordances, and Pitfalls (Part 2)

In part1, I began expanding on the poster session I presented at NERCOMP2013on massive open online course (MOOC) pedagogy, technologies, and the roleof the instructor, along with some basic history, the two types of MOOCs thathave evolved to date, and some observations about technology and pedagogy. Inpart two, I aim to expand a bit more on my poster: the role of assessment,credentialing, copyright, and some of my MOOC principles.

Assessment and credentialing

When thinkingof courses we invariably think of how we will assess mastery, or at least gain someunderstanding of those who complete it. This is also true for traditionalcourses that are offered for credit. Alongside assessment there is usually acredentialing issue, which in traditional education is taken care of byawarding degrees or certifications. Even if there is no summative assessment ofmastery by an SME, there ought to be some sort of formative assessment to letthe learner know how well he or she is doing in the course and what areas stillneed to be worked on.

cMOOC practice

Currently,cMOOCs generally don’t provide formal instructor assessment. (See Part 1 ofthis series if you need the definition of “cMOOC.”) There have, however, beenmechanisms for peer review, feedback, and “remixing” of knowledge andinformation. These seem to be at the core of the cMOOC identity. These peerreviews are by identifiable peers. The technological decisions made in cMOOCsthat allow distributed knowledge to be funneled into the course’s town square makeit possible for participants in a cMOOC to demonstrate their work and getfeedback, or to collaboratively work with one another.

cMOOCstypically don’t credential their learners; however, there have been some MOOCsthat have begun offering badges as part of their of the learning process. Thesebadges reward learners both for “staying the course,” (demonstrating certainbehaviors, outcomes, or deliverables), and for deviating from prescribed paths(forging your own learning). Typically, Mozilla’s OBI is used, but Purdue’s Open Passport is also an option. Only one MOOC,mobimooc (2011), has awarded certificates of participation to learners that metcertain criteria.

xMOOC practice

Assessmentin xMOOCs utilizes automated testing and anonymous peer reviews. Formativeassessment can be undertaken while students view course materials, but also aspart of module quizzes. Quite a few courses seem to work on mastery grading,which allows learners a lot of attempts to pass a quiz in a satisfactorymanner. There are also quite a few courses that give learners a limited amountof time and a limited amount of retries at these auto-graded assessments.

Peerassessments tend to be assignments of shorter length (250 to 700 words) around writingprompts. These assignments are graded anonymously by peers with a publishedrubric. In order for a learner to see their own grade on the assignment theyneed to review at least three peers. Anonymous peer reviews are matchedautomatically by the LMS and the feedback received can vary greatly. Rubricsfor assignments are a way for learners to assess each other; however, iflearners aren’t trained on how to provide good feedback, or don’t understandthe rubric, they may not grade good assignments accurately, nor will theyprovide good feedback. They don’t always count participation in the discussionforums of the course as part of the assessment criteria for the course; but ifit is counted, it’s only counted quantitatively, not qualitatively.  

xMOOCs commonlyaward a certificate of completion if a learner achieves over a certain score onassessments. The threshold for this score can vary, but it can start at 50percent. Since the criteria vary on how to achieve a certificate of completion it’shard to really compare what the value is, if any, of a certificate ofcompletion.

Comparing practices

Thechallenge thus lies with the purpose of the course and the value of assessment.cMOOC learners should be able to demonstrate what they learned in manners thatare meaningful to them and applicable in their environments. xMOOCs seem morefocused on replicating existing structures, thus graded and time-constrained summativeassessment seems to be a staple of the course. Assessment by an SME isdifficult in massive environments. What it means to be assessed, how assessmentis undertaken, and for what reasons needs a fundamental rethinking in the MOOC context.Replicating existing structures is not an appropriate course of action, andherein lays the major pitfall for MOOCs. If you replicate those structures tooclosely, free learners may demand free accreditation.

Copyright, Creative Commons, OER

Giventhat MOOCs base their genesis and existence on freely available materials,through Creative Commons (CC) licensing, Open Educational Resources (OER) repositories, or Open Access publishing, I firmly believethat you can’t call something a MOOC if there is a barrier to entry; it’s not“open” if you charge for it. Courses that charge to be part of the course don’tfit the MOOC model. They may be massive online courses, but they are definitelynot “MOOCs.”

There issome disagreement among MOOC participants as to how free a course has to be inorder to qualify to be a MOOC. For instance, some believe that if a paidtextbook is required it disqualifies the course as a MOOC. I don’t. One can getthe textbook from a library and still participate in the MOOC. Some mightbelieve that Coursera’s signature track doesn’t qualify as a MOOC because itcosts. I don’t; the signature track is above and beyond the free version. Thiscould be thought of as freemium MOOC: the certification might cost, but thecost to get the knowledge from the course, sans certification, is stillbasically free. My final MOOC principle is that if you are using openresources, then any new knowledge generated should also be open.

With thatsaid, cMOOCs seem to strive to be as open as possible. cMOOC creators try touse open content, as well as release their MOOC content under an open license,or leave the MOOC available after its end for anyone who wants to use theseresources. This is encapsulated in the four types of activities of cMOOC: aggregate,remix, repurpose, and feed forward (Siemens et al, 2012). These four activitiesalso encapsulate the ethos of thecMOOC. After all, aggregation, repurposing, and remixing is not easy when encumberedby traditional copyright. Without allowing open content to be fed forward, otherscannot aggregate, remix, or repurpose.

xMOOCs tendto retain copyright of all material. A brief survey of the three big platforms(Coursera, Edx, Udacity) earlier this year shows that their terms of serviceare strictly in opposition of the four original MOOC tenets. xMOOC aren’tactively creating open materials. However, this may change. In March 2013, edXannounced a change in its terms of service that make the default for itsmaterials public domain. This may have to do with the fact that edX is anonprofit venture, but it would be interesting to watch this space. Theopportunity here is one of notoriety. If your institution is associated withquality open work, that could have a potential positive impact to your campus. However,good work of that magnitude could cost a lot depending on the discipline andthe course.

Initiative: Personal or institutional?

In thinkingabout who initiates the MOOC, thereare two usual suspects: the faculty and the institutional administration. Thisisn’t really a cMOOC/xMOOC question; rather, the idea here is to think a littleabout the implications underlying who initiates the conversation and any eventualimplementations of a MOOC.

MajorMOOC efforts need to be faculty led, not institutionally forced. This meansthat course design, technology selection, and implementation needs to come fromfaculty and instructional designers, and not based primarily on which providerthe university has a contract with. However, MOOC efforts do need institutionalsupport if they are going to succeed. A group of faculty can design, implement,and facilitate a great MOOC, but the university needs to create conduciveenvironments for MOOCs to flourish, especially with interdisciplinary topics.Traditional courses are siloed into specific departments; however, MOOCs havean opportunity to break down those traditional barriers to create courses thattouch upon many interconnecting disciplines.

MOOCefforts need time for design and internal reflection. What works for one MOOCin one discipline may not work for other courses in other disciplines. Thismeans that you can’t force anyone to facilitate or develop a MOOC, and you can’tjust take an existing course and put it in MOOC format. Also remember thatMOOCs are experimental, and we should be sharing our findings with thecommunity so that we can improve upon them. The challenge here, forinstitutions, is to enable MOOCs as a two-way learning tool and not worry aboutbeing left out of the party, as some institutions might feel.

Rules of thumb

Regardlessof the MOOC you plan on offering, I propose four rules of thumb: Put learnersfirst; do instructional design first; pick your faculty carefully; and don’tworry about the dropout rate!

Under nocircumstances is it acceptable to just close the doors of the course becausethings aren’t working out. This is one important way in which you can think ofyour learners first. We saw a bad example of this early in 2013 when Fundamentalsof Online Education closed its doors after just one week online (Kolowich,2013a). The course did have issues, but instead of working through them andlearning from the experience, the doors were closed with little notice tolearners. Just like in traditional courses, if something isn’t working, modifyit on the fly. In MOOC cases, your support team should be there to help theinstructor(s) of the MOOC to resolve issues. Closing your course’s doors is badfor the learner, and subsequently bad for your own reputation.  

Second,instructional design should come first. While it’s healthy for educationaltechnologists to know of the affordances of each platform, and how thoseaffordances fit with pedagogical goals, there is no reason to go with oneplatform over another exclusively. We shouldn’t adopt the same stance with theMOOC-LMS as we did with our traditional course LMS. We ought to be open toeducational experimentation for the benefit of teaching and learning. Sticking toone technology or provider is potentially detrimental to our learning process.

Third,when deciding who will facilitate the MOOC, if it’s institutionally supported,there needs to be some expectation setting. Recently a professor quit his MOOCbecause of philosophical differences over how the course should run (Kolowich,2013b). While they didn’t shut down this course like the other course was, it’sstill not great PR for the institution. There are institutions that do vettheir online courses before they go live; I am sure that this is probably thecase with xMOOCs as well in some institutions. Why not vet the instructor aswell? Teaching online is different from teaching face-to-face, and facilitatinga MOOC is different from both of them. The medium is experimental andinstructors do need to adapt their teaching. This is how we will all learn moreabout teaching and learning in MOOCs.

Finally,have a better understanding of what the “dropout” numbers mean. I think thatdropout is an inaccurate term because it lumps learners together who don’tbelong together. For instance, those who were just window-shopping in the MOOC;those who know some of the materials and just want a refresher so they onlyparticipate sparingly; and those who are honestly interested in learning, butthe course is failing them in some way or extracurricular issues areinterfering with their participation. I am sure there are other categories aswell.

Thegoals of each category of learner are different, and only the last category’sgoals come close to approximating what I consider a traditional learner’sgoals. Thus, we have to put aside the “funnels” (Sonwalkar, 2012) where we seemany learners starting, but have few “completing,” and put aside snarky remarkscomparing dropout rates between traditional online learning dropout rates toMOOC dropouts (Young et al, 2012) because there is no one-to-one correlation.

Withthis, go forth and MOOC, and remember, “If you’re willing to fail interestingly, you tend to succeed interestingly” (Edward Albee).

References and resources

Kolowich, S. (2013a). GeorgiaTech and Coursera Try to Recover From MOOC Stumble. Chronicle of Higher Education. Retrieved from: https://chronicle.com/blogs/wiredcampus/georgia-tech-and-coursera-try-to-recover-from-mooc-stumble/42167

Kolowich, S. (2013b). Professor Leavesa MOOC in Mid-course in Dispute Over Teaching. Chronicle of Higher Education. Retrieved from: https://chronicle.com/blogs/wiredcampus/professor-leaves-a-mooc-in-mid-course-in-dispute-over-teaching/42381

Lane, L. M. (2009) Insidious Pedagogy:How Course Management Systems Affect Teaching. First Monday, Volume 14, Number 10 – 5 October 2009 Retrieved from:https://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/viewArticle/2530/2303

Siemens, G., Downes, S., andCormier, D. (2012). How This CourseWorks. https://change.mooc.ca/how.htm

Sonwalkar, N. (2012). Sustainabilityof Open Online Courses (MOOCs) in Higher Education. Keynote addresspresented at The Sustainability of MOOCs Forum. University of MassachusettsBoston. December 2012. Retrieved from: https://vpc1.umb.edu/CIEE_SustainabilityForum/

Young, J., Cruz, J., Drimmer, A.,Wilson, J., (2012). Plenary Panel: Evolution orRevolution? What’s Happening with Traditional Online Learning? 18^th Annual SloanConsortium International Conference on Online Learning. Orlando, Fl. November20 – 22, 2012.