Misleading Data Visualizations Can Confuse, Deceive Learners

Data visualizations can be essentialtools for exploring and communicating complicated information—orthey can obfuscate, distort, or misrepresent data. Misleading datavisualizations might be intentional, if the creator has an agenda to promote.Or they might be the result of errors, the creator not understanding the dataor the data visualization process, or allowing engaging or even beautifulvisual design to get in the way of clear communication. Whatever thereason, misleading data visualizations have no place in eLearning; using theseguidelines can help designers avoid confusing or misinforming their learners.

The primary ways that a visualization can mislead learnersare:

• Hiding relevant data
• Presenting too much data
• Distorting the presentation of data
• Describing the data inaccurately in annotations,title, or within the visualization itself

Let’s examine each of these.

Hiding relevant data

Hiding relevant data or highlighting a particularlybeneficial or positive data point can lead learners to focus on a smallfraction of the data story—at the expense of accurate understanding of the biggerpicture. Any individual parameter or statistic can reveal interesting oruseful information. But taken out of context, it can also be misleading.

For example, look at Figure 1, a data visualization based onPew Research Center’s 2018social media use survey. An infographic could zoom in on the linefor Facebook, making much of the fact that 68 percent of American adults useFacebook.

Figure 1: Pew Research data visualization shows use of different social mediaplatforms between 2012 and 2018

An eLearning or marketing strategy might be built aroundFacebook, with the architects believing that Facebook exposure is the goldenticket to reaching more members of their audience.

But a deeper look at the data shows that Facebook use hasbeen flat since Pew’s previous study in 2016. It also shows a whopping 25percent increase in Instagram use during the same period—from 28 percent ofAmerican adults to 35 percent. The increase is primarily among younger users. Acompany seeking to appeal to these learners might want to consider amultiple-platform strategy or focus its efforts on the up-and-coming platforms.Thus, a data visualization like Figure 1, which presents more complete datamight lead eLearning designers to a different approach.

Presenting too much data

Sometimes, showing the big picture can make it hard toidentify salient data or stories.

In Figure 2, the sheer number of lines makes it hard to focus on any one datapoint or trend. If the designer wanted to obscure some bad news, burying it ina massive amount of information could accomplish that—but it also makes thedata visualization essentially worthless.

Figure 2: The number of lines in thisdata visualization makes it hard to isolateany one fact or trend

In other cases, the trends that appear when an entire dataset is visualized are the opposite of trends that appear when subsets of thatdata are studied separately. This phenomenon, known as Simpson’s Paradox, isexplained in Cathy O’Neil’s Weapons of Math Destruction using a nationalreport on school performance as an example. The report, A Nation at Risk,which was the basis of wide-ranging public policy, stated that nationwide, highschoolers’ SAT scores had declined.

Examination of the data revealed that, while this was truein a big-picture sense, the period of data covered an era with tremendousgrowth in the number and range of students taking the exam; universitieswere admitting more minority and lower-income students, vastly increasing thenumbers of these students taking the exam. When each cohort ofstudents—analyzed according to income groups—was examined, the data actuallyshowed increases in the average scores of each group.

When learners will need both a big-picture and a detailedvisualization of data, the designer should consider creating a seriesof data visualizations. News media often do this with large datastories, showing a national map, for example with broad representations of databy region or state, then a series of more narrowly focused visualizations thathighlight important trends, outliers, or other information.

Distorting data

Showing too little or too much data or emphasizing selecteddata could simply be an error that results from choosing the wrongformat for the data visualization or from not fully understandingthe data. These errors can be unintentional, though some presentations of datadistort the data in ways that appear to be intentional or agenda-driven.Examples include using different scales when graphing different variables orstarting the Y axis at a non-zero point, which can de-emphasize differences invalues.

In “Graphics, Lies,Misleading Visuals,” data journalist Alberto Cairo, who holds theKnight Chair in Visual Journalism at the School of Communication of theUniversity of Miami, uses several examples from political campaign ads or mediacoverage. This type of distortion can also be found in consumer advertising,marketing and PR materials, and elsewhere. Figure 3 illustrates how somethingas simple as truncating the Y axis makes a significant difference in howreaders will understand a data visualization.

Figure 3: Both charts show 48 No votes and 52 Yesvotes, but the top figure, whose Y axis starts at 45, appears to show a muchlarger difference between the vote totals

An equally misleading presentation of data could be a“strategic” selection of where to begin and end or use of uneven intervals inthe X or Y axis. Examples Cairo cited include presenting only six months ofunemployment data in an economy where seasonal highs and lows are a knownfactor and switching—mid-chart—from a yearly interval to a monthly intervalwhen presenting information on rate increases. The latter example could hidethe size of an increase by presenting it in smaller chunks, graphed next tolarger annual increases. The unemployment example could appear to show a largedrop (or increase) in unemployment while actually reflecting an expected annualcycle.

Describing data inaccurately

A particularly unethical way to mislead using datavisualizations is to mislabel data or use accompanying text that “explains” itinaccurately.

Figure 4: This map, beloved by President Trump,accurately depicts the county-by-county results of the 2016 presidentialelection. It does not, as many have claimed, reflect the total number of votesin each red or blue square

The county-by-county map in Figure 4 is an accurate data visualization showingresults of every US county for the 2016 presidential election. But when used,as shown in Figure 5, as a representation of voters or “citizens,” it is beingdescribed misleadingly.

Figure 5: The book cover for Citizens for Trump

Using the map to imply a representation of “citizens”mischaracterizes the data map of county election results, conflating them withnumbers of votes. Each county in the heartland states represents far fewervoters (though vastly more physical space) than the densely populated—andprimarily blue—counties clustered along the coasts. While the map itself isaccurate, it has nothing to do with citizen support for either candidate.

A related way to mislead with data visualizations is to presentdata that appear to show correlations—and imply or explicitly state that thereis a causal relationship between them. Data do not show causes, Cairo often remindsstudents. Data sets provide information that can lead to questions. Furtherinvestigation of those questions might turn up a correlation—or it mightnot. Websites and booksdevoted to spurious correlations prove the axiom “correlation doesnot equal causation,” yet well-intentioned (as well as nefarious) designers areprone to the often-fallacious assumption that one trend in the data set somehowcaused another.

Don’t avoid using data visualizations in eLearning

The many ways that data visualizations can go wrong is notan argument for avoiding them. Data visualizations can enhance eLearning andmake complex information clear and instantly accessible to many learners. Choosingan appropriate format for data visualizations in eLearning and applying soundvisual design principles can go a long way toward helping designers avoidmisleading data visualizations. Register now for The eLearning Guild’s Data& Analytics Summit, August 22 & 23, 2018, and learn moreabout using data to enhance eLearning!