Don’t Break the Spell: Creating Presence in Virtual Reality

You, as a virtual reality (VR) developer, can—no, you must—payspecial attention to the creation of presence, whether you’re developingeducational games, designing user interfaces, or simply making a virtual spacein which people hang out and interact socially. (This is possibly true for 360-degreecinematographers, though that’s a whole other discussion.) Do it wrong and youbreak the illusion, bringing the user back to the real world.

I hovered over thesunken boat, air tank on my back, drifting gently toward the edge. Severalbrightly colored fish darted past me and over the crumbling railing. I heard alow rumbling moan coming from somewhere far to my left. Seconds later, a grayish-bluemass (a blue whale!) appeared seemingly out of nowhere, larger than any animalI’d ever seen, nearly at arm’s length. That thing was so large it nearly filledmy entire field of vision. The dozens of small fish that had flashed aboutmoments ago had all swum into nooks, hiding from this monster. Its humongouseye moved past, hesitating momentarily right in front of me, as if checking meout, the strange new creature staring right back at it. As it swam on, I couldsee the smoothness of its skin, the grace of its movement, the sheer mass of 80feet of flesh gliding by. I’d have been rendered speechless had I not beenunderwater already. Then, the image froze for maybe half a second while thegraphics card caught up on processing, and the moment was over. The whale keptswimming, but I was already back in the studio, draped in cords from the heavy virtual-realityheadset and earphones strapped to my head, wondering where the nearest wallwas.

Presence

Presence is the feeling that you’re really there, whetherwatching a movie, playing a video game, or experiencing VR. Presence happenswhen technology blurs the line between simulation and reality by adding depth,when people lose themselves in the experience and forget that they are in acreated reality. As VR technology advances in the near future, we may no longerbe able to tell the difference between reality and VR.

Presence in VR is especially difficult to create, easy toruin, and nearly impossible to get back once it’s gone. Without presence, a VRexperience can feel literally and figuratively flat. But with good presence, yourmind forgets everything outside the virtual environment, including the headsetand cord tethering you to a computer. It forgets the rush hour traffic and thebills to pay and everything else. You feel like you’re really soaring throughalien skies, exploring an ancient pharaoh’s tomb with your historyclass instructor, or swimming with whales on a shipwreck. 

In a paper titled “Research on Presence in Virtual Reality: A Survey” (well worth reading, as it divesmuch deeper into the topic of presence than I do in this article), the authorsquote Matthew Lombard’s slightly more scientific definition of presence:

Presence (a shortened version ofthe term “telepresence”) is a psychological state of subjective perception inwhich even though part or all of an individual’s current experience isgenerated by and/or filtered through human-made technology, part or all of theindividual’s perception fails to accurately acknowledge the role of thetechnology in the experience. Except in the most extreme cases, the individualcan indicate correctly that s/he is using the technology, but at some level, and to some degree, her/his perceptions overlook that knowledge andobjects, events, entities, and environments are perceived as if the technologywas not involved in the experience.

In other words, feeling like you’re really there.

Whatever your purpose, presence is the factor that willultimately make or break the success of your program. If you pull it off,people will stay, learn, and come back for more. If you don’t, the wholeexperience could be forgotten within minutes.

How to create presence

VR presence can be defined in multiple ways (as described indetail in the aforementioned paper; see References), but at its most basic,presence can be broken down into three broad categories, listed in order ofmost important to least important for their impact on creating presence:

1. Visual presence
2. Auditory presence
3. Sensory or haptic presence

In this article, we’ll discuss each of these categories anddescribe their importance to elevating the quality of a VR experience. We’llalso explore some miscellaneous methods for creating presence that don’t fitnicely into any categorization.

Visual presence

Visual presence is not only the most important category, butalso the most nuanced. Visual presence itself comprises multiple factors, eachcontributing to the overall sense of “really being there.” We’ll dive intothe technical aspects of creating presence in future articles, but for now, thelist below will kick-start your thinking on what it takes to create visualpresence in VR:

1. Locomotion—Howdo you move around in the virtual space?
2. Tracking—Whereare your head and controllers or hands as they move in and through the virtualspace?
3. Latency—Doesthe image move along with your head?
4. Persistence—Doobjects blur as they move?
5. Resolution—Howdetailed are the images?
6. Field ofview (FOV)—How much of the virtual world can you see at a time? How wideand tall is your peripheral vision?
7. Comfortableeyebox and headphones, with minimal or no cords. Is the headset light andcomfortable enough to be forgotten?

Numbers one through five can all be created within the twoprimary VR development software programs, Unityand Unreal Engine.Together, these five factors make the visual environment look realistic andprevent nausea in the viewer. Numbers six (FOV, which is important because witha limited FOV, it feels like you’re looking down a tunnel instead of existingin a full virtual environment) and seven (the headset and cords, which, ifheavy, restrictive, or uncomfortable, can constantly and painfully remind youof your real world body) are primarily dependent upon the type of VR headsetused by the viewer, which I discussed in detail in this earlier article.

Besides the above technical aspects, a virtual experiencemust satisfactorily create the Four Illusions of VR in order to achieve visual presence:

1. Theillusion of being in a stable spatial space—Does the environment around youlook and feel real?
2. Theillusion of self-embodiment—Do you feel like you really are the avatar youare controlling? Is movement intuitive and realistic?
3. Theillusion of physical interaction—Does your interaction with objects in thevirtual environment look and feel real?
4. Theillusion of social communication—Are you able to effectively and seamlesslyinteract and communicate with other people in the virtual environment?

Admittedly, these Four Illusions often require sound andhaptics to fully work their magic on the viewer, but video is the primarydriver of these illusory effects. 

Auditory presence

Audio presence is important, but not nearly as important asvideo. To prove the point, you can easily experience VR with the sound off, buttry closing your eyes and see how much fun you have in VR. That said, the New York Times’ “Notes on Blindness”project shows just how vivid and fun 360-degree audio can be. 

Despite audio’s secondary effect on virtual presence, good 360-degreeaudio can make or break a VR experience. Game companies and other VR experiencedevelopers devote a lot of attention to sound. Without good 360-degree sound, the virtual worldseems flat. With good 360-degree sound, you can literally hear the shapes ofrooms by hearing sounds “reflect” off virtual walls just like they do in thereal world. Sighted people rarely think about this spatial auditory recognitionconsciously, but take it away, and a room seems uncannily quiet or simply “off.”Similarly, we can innately grasp the size, shape, and velocity of objects,people, and animals by how their sounds (e.g., speaking, breathing, footsteps,rustling of clothes or hair, vibration of air) move through space and bounceoff walls back to our ears.

Binaural or stereoscopic sound is perhaps the most importantfactor in creating a 3-D soundscape. For a very simple example of binaural sound,if your friend is standing to your left and speaking to you, not only will yourleft ear hear the voice as louder, the sounds will reach your left ear slightlybefore your right. These subtle cues are how we can know a friend isstanding to our left in the real world. The same is true in VR. If you want totake your VR experience to the next level, you must pay close attention toaudio. If you’re shooting 360-degree video, consider investing in one ormore high-end microphones in order to better capture the full sound environmentof each scene. In fact, some sound engineers and VR game developers havedevoted whole careers to perfecting 360-degree and VR audio.

Sensory or haptic presence

Haptic presence helps VR users to experience physicalobjects, movement, walls, borders, and, interestingly, data. Experiencing datameans seeing text, chart, or graphs in a virtual environment and being able toread it, click on it, move it, hide it, change it, or otherwise interact withthe data. On traditional 2-D screens, this interaction of data has becomestraightforward and intuitive for most of us; but in VR, developers are havingto create whole new ways of interacting with data that simply don’t apply in 2-D.

In their simplest form, VR haptics include small vibrationsof hand controllers, indicating when your virtual hand has touched, passedthrough, or in some other way interacted with a virtual object. For example,when you move your real hand and real game controller, say, forward and to theright, the virtual hand meets a virtual object, perhaps a wall or a ball. Sinceyour real hand keeps moving and feels no wall or ball, the controller canvibrate to let you know you’ve hit something. Obviously, you can see this wallor ball, and you may even be able to hear it too, if the wall/ball is emittinga noise of some kind. However, the vibratory feedback makes the wall or ball seemeven more real, or at least more noticeable, which is a step toward seemingreal.

Admittedly, a small vibration in your hand isn’t much, butat least it’s something. It engages the brain on yet another level. Several VR hardware companies are currently developinghaptic gloves that track individual finger movements, as well as providevarying resistance and pressure along the fingers and skin, and allow the usersto feel actual heat, texture, moisture, and air movement that matches the programmedexperiences within the virtual world. One company, Monobanda, has even created a haptic belt that allows you to control the locomotion ofyour virtual avatar with your breathing. The flagship product from Virtuix (also a maker of haptic gloves) is aVR treadmill that allows players to literally walk or run in any direction. Stillother companies are working on olfactory andgastronomic haptic devices,allowing VR users to smell and taste virtual objects. As you can imagine, porncompanies are pushing the boundaries of haptic technology with something called“teledildonics,” but I’ll let you Google that for yourselves. 

As haptic technology improves, the range and sensitivity ofvirtual sensory experiences will increase, as will the percentage of the bodythat can interact with virtual objects, until we will eventually be able to donfull haptic bodysuits. People are working on this, but it could be many yearsbefore this kind of hardware is commercially available. Until then, we’ll justhave to enjoy fantastic virtual 360-degree visual and audio worlds with theoccasional hand vibration.

Miscellaneous techniques to create presence

The Void, aninteractive VR experience arcade or theme park with locations in Salt Lake Cityand Toronto, is pushing the boundaries of haptics and presence, going wellbeyond what we normally think of as haptics. Among the many innovative techniquesthey’ve developed, when you’re walking through the experience, they blow moistair in your face, make you traverse a narrow balance beam walkway thatvirtually appears to be suspended high in the air, and, most impressively,redirects you along a curved hallway seemingly indefinitely. This curvedhallway is actually a big circle, but the virtual visual cues make you thinkyou’re walking in an apparent straight line for a distance that far exceeds thedimensions of the actual building, without your brain noticing you’re reallywalking along a curved path.

Further, The Void also uses a good story with engagingnarrative techniques (e.g., you get to be the star of a virtual Ghostbusters story) to create afantastic sense of presence. VR storytelling is very different than 2-D orverbal storytelling because both pace and attention are self-directed. Forexample, as a VR designer, developer, or storyteller, you can’t reveal animportant plot point (puzzle clue, educational instruction) while the player isfacing in another direction and expect the story (puzzle, lesson) to make anysense.

Storytelling, plus these various new techniques employed byThe Void, plus many additional old tricks that have been used by haunted housesand immersive theater productions for ages, perhaps each deserves their owncategories of presence. At the very least, they’ll likely evolve into their ownseparate categories over time as the technology continues to improve at anexponential rate.

The whale, take II

You can create presence in VR in many ways. Ultimately, you’llneed to pay attention to each of the above factors in order to create a trulycompelling experience. Pondering all this complexity, I decided to give thevirtual whale experience (theBlu: Encounter, by WeVR) another try.

I tried on a different VR headset attached to a faster, morepowerful computer. I was instantly transported back to the underwater world Iwas unceremoniously jarred out of minutes prior. This time the whale swam bywithout freezing up. I explored undersea caves, played in coral reefs, andsteered clear of creepy bioluminescent creatures mesmerizing their prey. This isthe kind of VR presence I love. I could play in this world all day long.

References

Bible, Thomas. “Binaural Audio for Narrative VR.” Oculus Story Studio Blog. 31 May 2016.

Ching, Teo Choong. “The Concept of Presence in Virtual Reality.” Medium. 27 August 2016.

Hoagland, Edward. “Feeling My Way Into Blindness.” New YorkTimes. 17 November 2016.

Schuemie, Martijn J., Peter Van Der Straaten, Merel Krijn, andCharles A.P.G. Van Der Mast. “Research on Presence in Virtual Reality: A Survey.” CyberPsychology & Behavior, Vol. 4, No. 2. April 2001.

Sparks, Matt. “Metafocus: Overview of Virtual Reality and Mixed Reality in eLearning.” Learning Solutions Magazine. 23 November2016.

Additional resources on creating presence in VR

Bye, Kent. “VR Presence Researcher Finds Full Embodiment to be Key Component in Plausibility”(podcast). Road to VR: Voices of VR, No.555. 10 July 2017.

Callewaert, Carl. Build Architectural and Gaming Environments That Create Presence in VR (video). Vision VR/AR Summit. 23 February 2016.

Celia, Matthew. How To Deliver on the Promise of Presence When Creating 360 VR (video). Mettle.18 December 2016.

Hoffman, Hunter, Todd Richards, Barbara Coda, Anne Richards,and Sam R. Sharar. “The Illusion of Presence in Immersive Virtual Reality during an fMRI Brain Scan.”CyberPsychology & Behavior, Vol.6, No. 2. April 2003.

Jerald, Jason. The VR Book: Human-Centered Design for Virtual Reality. San Rafael, CA: Morgan & Claypool/ACM Books, 2015.

Lang, Ben. “Oculus Shares 5 Key Ingredients for Presence in Virtual Reality.” Road to VR. 24 September 2014.

Larsson, Pontus, Daniel Västfjäll, Pierre Olsson, and MendelKleiner. “When What You Hear is What You See: Presence and Auditory-Visual Integration in Virtual Environments.” Proceedings ofPRESENCE 2007: The 10^th Annual International Workshop on Presence. October2007.

Marinkovic, Sasa. “First Rule Of VR: Don’t Break The Presence.” TechCrunch.7 February 2015.

Torisu, Takashi. “Sense of Presence in Social VR Experience.” Interactive Architecture Lab Blog, UniversityCollege London Bartlett School of Architecture. 17 August 2016.

Voica, Alex. “Presence in VR: How to provide a superior experience.” VR 360. 24 June 2014.