As machine learning and other algorithm-driven technologies playa larger role in personalized eLearning and performance support tools, thepotential for algorithmic bias in eLearning increases. In “Biasin AI Algorithms and Platforms Can Affect eLearning,” Learning Solutionsdescribed how algorithms, or their results, might become biased; this articlesuggests methods for evaluating algorithms or platforms and attempting toprevent or limit bias in eLearning and performance support tools based on thesealgorithms.
Black-box algorithms
Machine learning algorithms are capable of amazingfeats—from understanding “natural language” and conversing with learners toguiding self-driving cars—due to a radical leap forward in how the algorithms work.Computer algorithms used to follow clear rules and instructions. These instructionscould be quite complex, but a human had, at some point, programmed theinstructions into the machine.
Machinelearning, in contrast, describes a process by which an algorithm learnsfrom data and results, constantly refining the decisions it makes or actions itselects. This approach makes it difficult or impossible for engineers—merehumans—to explain why an algorithm has produced a particular result.
In “TheDark Secret at the Heart of AI,” Will Knight writes, “The system is socomplicated that even the engineers who designed it may struggle to isolate thereason for any single action.” Though he’s talking about a self-driving car,the same is true of other machine-learning systems. And it’s not a problem thatis easily solved. “There is no obvious way to design such a system so that itcould always explain why it did what it did.”
Complex algorithms are black boxes, impossible to probe,examine, and understand. Some algorithms are simply too complex to explain;some AI algorithms are created by other AI algorithms, resulting in amultilayered system, according to “Ghosts in the Machine”by Christina Couch. In addition, their functioning is often a closely guardedtrade secret. Companies like Google or Amazon don’t want people to be able to“game the system” to achieve higher rankings in a search, for example.
Researchers, ethicists, and engineers have several suggestionsfor detecting and mitigating bias while working within these constraints. Whenevaluating or selecting AI platforms for their eLearning and performancesupport tools, L&D professionals can choose those platforms that havebeen audited or examined for bias. They can also be aware of ways to check theresults of automated tools for bias— for example, by ensuring that the curatedcontent they provide learners is representative and balanced.
Approximation models
Sarah Tan andcolleagues treated black-box algorithms as “teachers” and modeled a“student” algorithm to produce similar results. They then created a transparentmodel whose inputs, processes, and outputs could be studied. By mimicking theproprietary algorithm and then tweaking the inputs, the researchers couldcompare the outcomes or predictions of the black-box model with those of a moretransparent algorithm—and determine which data elements had the most impact ondecisions. They could also compare real-world data with the algorithm’s predictions.In their analysis of algorithms for risk assessment of potential parolees andof loan applicants, the team was able to detect bias and propose less biasedapproaches.
Teaching machines to explain themselves
An alternative to mimicking the algorithm is teaching it toexplain itself, in essence, asking the algorithm to “show its work.” Knightdescribes how a team at Massachusetts General Hospital added code to analgorithm that analyzed pathology reports and predicted which patients hadcharacteristics that merited further study. They taught the system to extractand highlight the pieces of information in patient charts that it hadrecognized as part of a notable pattern, thus providing the humans with insightinto how the algorithm detected a pattern.
An audit study, tweaked for algorithms
It’s not enough to audit or even understand code, researcherChristian Sandvig and coauthors point out in a paperon auditing algorithms. Since a key use of algorithms, particularly ineLearning, is personalization, many algorithms produce different results asdifferent individuals interact with them.
Taking a leaf from in-person or document-based auditstudies, where researchers present test subjects with the same data except fora single controlled variable—for example, sending resumes in response to job adsthat are identical except that one has a conventionally white male name whilethe other uses a female or conventionally African-American name. The study thenmeasures responses and detects bias (or lack of bias) depending on howessentially-identical individuals are treated.
The paper considers several approaches to auditing theresults of algorithms, highlighting legal and ethical issues. It concludes thata crowdsourced or collective audit could use large numbers of volunteers (orpaid workers on an “Amazon Turk”-type model of using low-paid workers for smallor repetitive tasks) to perform prescribed web searches. A large global sampleof users entering the same search would then capture the results and send themto the researchers, providing a large, varied data set with which to evaluatethe algorithm for bias.
Researcher TolgaBolukbasi similarly used Amazon Turk crowdsourcing—and human labor—to “debias”word embeddings in a data set widely used for natural language applications,pointing to a need for human involvement in refining automated systems andplatforms.
Degrees of automation
While AI and machine-learning-based eLearning offer the opportunityto automate and personalize much about how content and assessments aredelivered to learners, the potential for algorithmic bias in eLearning shouldact as a brake. Human—L&D team—involvement is essential to mitigate algorithmicbias in eLearning. Some algorithms act only on instruction from learners ortheir managers; others are subject to human oversight and review. Either ofthese options is preferable to a fully automated system to ensure that contentprovided is both complete and balanced and that learners of comparable abilityare evaluated according to the same criteria, whether for a digital credential,a promotion opportunity, or a prize in a sales competition.
Learn more about relationships between data and content andhow to use data in eLearning at The eLearning Guild’s Dataand Analytics Summit August 22 & 23, 2018.
