Lost Among the Realities: A Guide to Virtual, Augmented, and Mixed Reality

Media enthuse about thelatest gear or games employing virtual reality or augmented reality; scholarlytreatises pontificate about how virtual reality and grounded reality interact.Then there’s something called mixed reality…

It’s easy to get lost amongthe realities. Here’s a primer to sort out the terms and understand how eachmight apply to eLearning.

The real, physical world iscalled “grounded reality.” This article explains the “other” realities inreference and relation to grounded reality.

Immersive experiences and VR

The lure of virtualreality, or VR, is immersing learners in an environment. That environmentcan be created using 360-degree video, be entirely computer-generated, orcombine video with computer-generated elements.

Most learners have probablyencountered 360-degree video online; it shows up on real estate listings,virtual college-campus tours, and much more. While it’s possible to view 360-degreevideo on a computer screen, it is best viewed using a VR headset. Either way,learners can examine a complete scene—up, down, and in front of and behind thelearner’s position—and get a far more complete picture than they get from lookingat still photos or ordinary video. Journalists who’ve used 360-degree video in their storytelling say that news stories presentedin this format have greater emotional impact than text, ordinary video, or eveninteractive packages because the audience has a greater sense of being on thescene. But the “scene” is still something that exists in grounded reality, andthe audience can only look at it, not interact with it.

That’s where VR comes in. VR immersesparticipants in a different world—often a world that doesn’t actually exist.The environment is a digital creation, hence “virtual” reality, though it canbe a digital re-creation of an actual place. Whether based on a real place orentirely fictional, it feels real to learners who are immersed in the digitalworld; while there, learners can interact with other avatars in the virtualworld. According to Tobin Asher, the lab manager at Stanford University’s Virtual Human Interaction Lab (VHIL), people treatavatars—whether other “people” or fictional creatures encountered in a VRenvironment—as real, even if the graphics quality is not top-notch and theavatars don’t look real. Learners also behave as they would in a real space:They are unwilling to walk through a virtual wall, for example, or approachanother avatar too closely.

That’s because of key elementsof VR that make it feel like an actual environment: tracking and rendering.Tracking is the ability of the software to “know” where the learner is in thereal environment by tracing the person’s movements. Rendering moves the avatarwithin the virtual world to reflect the person’s movements in grounded reality.Accurate tracking and rendering creates a strong feeling of “presence”—theperson feels as if he is actually present in the virtual world. When the personmoves, his avatar and the virtual world move along with him—the virtual worldresponds as a real environment would. Since the learner, or, rather, hisavatar, can move through this virtual world and interact with the environmentas well as with things and avatars within the environment, the brain is fooledinto thinking it is real.

A common VR demo scenario iswalking across a (virtual) cavern on a narrow plank of (virtual) wood. Eventhough they “know” that they are on a solid floor, people doing the “plank walk” feel terror; their hands sweat and their bodiesactivate all the responses they would in an actual scary experience. This doesnot happen when people watch a scary movie or participate in a role-playinggame.

VR is making its way intoprofessional and educational eLearning: NFL and college football coaches use VR so their players can practice playsover and over, building muscle memory and honing skills with little risk ofinjury and at considerably less cost than live practices. US Customs and Border Protection uses VR simulators to train officers, preparing them for active-shooterevents and routine confrontations with members of the public. Google Expeditions, which usesGoogle Cardboard viewers and smartphones, enables schoolchildren to takevirtual field trips around the world. And researchers at the VHIL are studyingwhether VR experiences can change people’s attitudes and behavior, with an eyeto developing diversity training programs.

Augmented reality

All of that contrasts sharplywith augmented reality, or AR, in which participants remain who—andwhere—they are, in grounded reality. Using an augmented reality tool, it’spossible to see additional information, objects, and even animated charactersseeming to inhabit the grounded reality. The objects aren’t there, at least notin any physical way. The object or information is on a digital layer thatappears on top of the learner’s view of the world.

Augmented reality is a usefulway to provide information and job aids: Google Glass,an unsuccessful attempt to bring this experience to consumers, has provenuseful in industry. For example, Boeing successfully tested Google Glass as an aid to reduce errors and cut the timetechnicians need to create “wire harnesses” for aircraft—an endeavor with zeroerror tolerance. The techs wear the augmented reality “smart glasses” to accessa “floating” display. They can look up diagrams and access instructions withouthaving to stop what they are doing and turn to a computer screen. Using voicecommands allows employees to keep their hands on their work, which often requiressorting and placing dozens of wires; preventing interruptions saves precioustime and reduces the likelihood of errors.

Cydalion, an augmented reality app, provides navigation assistance to people with low or no vision. The augmentation isa layer of audio and haptic feedback that alerts the user to overheadobstacles, tripping hazards, and items nearby that they could walk into.

Augmented reality is alsopart of games like Zombies, Run! and Pokémon Go, which add a layer to—augment—players’grounded reality experience. Each player sees the zombie or the Pokémon, feelsthe challenge of catching or eluding the fictional character—and changes herbehavior accordingly. But people around her do not see the character (unlessthey are also playing), and, of course, there is no physical character.

Mixing it up

An experience that includeselements of augmented and virtual reality is called mixed reality. A keyelement is that the virtual elements behave as if they are real. A variety ofimplementations are possible. For example:

• Rather than overlayvirtual objects on participants’ view of the world, a mixed reality experiencemight integrate the virtual objects with the grounded reality world, allowinglearners to pick up or interact with them. For example, NASA uses theProtoSpace tool to create a virtual Mars rover; a team of engineers can inspect it, walk around it, kneel and peekunder it, and even poke their heads inside the vehicle—even though it doesn’tphysically exist.
• Learners might useavatars to enter a virtual space rather than immersing themselves in that world.An instructor could hold an online course in a virtual space created to resemblea lecture hall on a college campus or an office conference room, while the learnersare physically dispersed around the world! The learners interact with theinstructor and with one another, collaborate on assignments, and even sit intheir favorite seat during every virtual session—even though they are at theoffice or even at home, wearing their pajamas.

A final term that might gettossed into the mix is alternate reality, an experience that takes place entirely within grounded reality. Analternate reality game or simulation offers realistic—and real-life—experiencesolving problems, generally using the Internet and information, items, andlocations in grounded reality. The problem posed might be fictional; theproblem might be real but the solution fictional; or the entire experience couldbe a simulation. Unlike typical eLearning simulations, though, the outcome ofan alternate reality experience is not preordained. The participants caninfluence or even determine the results.

Whichever reality aneLearning tool inhabits, learners can benefit from the enhancements andincreased opportunities for interaction and engagement.

References

Boeing. You can see the difference. Video, 2015.
https://video.boeing.com/services/player/bcpid4029272025001?bckey=AQ~~,AAAAukPAlqE~,oAVq1qtdRjzvprzv7b_de_OOilq0kg4d&bctid=4051927834001

CNN. “This Is So Like ‘TheMatrix.’” Video excerpt from Morgan Spurlock Inside Man, season 2,episode 2, aired 20 April 2014. Stanford University Virtual Human InteractionLab.
https://vhil.stanford.edu/news/2014/this-is-so-like-the-matrix/

Dennis, Danfung, Nonny de laPeña, Gabo Arora, and Chris Milk. “Where Virtual Reality Takes Us.” New York Times. 21 January 2016.
https://www.nytimes.com/2016/01/21/opinion/sundance-new-frontiers-virtual-reality.html

Hogle, Pamela. “A New Take onAugmented Reality: Cydalion Navigation App Aids People with Low Vision.” Learning Solutions Magazine. 8 November2016.
/articles/2129/?utm_campaign=lsmag&utm_medium=link&utm_source=lsmag

Hogle, Pamela. “BlendingFantasy with Reality Drives Successful Alternate Reality Games.” Learning Solutions Magazine. 28September 2016.
/articles/2070/?utm_campaign=lsmag&utm_medium=link&utm_source=lsmag

Senese, Mike. “NASA Shapesthe Future of Space Design and Exploration with its Mixed Reality Program.” Make. 19 July 2016.
https://makezine.com/2016/07/19/rockets-rovers-mixed-reality/