Perceptual Illusions and Distortions in Virtual Reality
Virtual Reality (VR) enables users to act and perceive as they do during everyday interactions with the word. As a consequence, VR may deliberately or inadvertently elicit a range of perceptual illusions and distortions on behalf of immersed users.
From an early age, people become experts at deciphering the continuous stream of stimuli registered by various sensory systems in response to the perceiver’s own actions and events in the environment. However, even though the senses provide reasonably reliable information, perception is imperfect, leaving the perceiver vulnerable to illusions, i.e., instances of erroneous or misinterpreted perceptions of sensory information (Stevenson 2010).
A defining feature of Virtual Reality (VR) systems is that they support natural perception and actions by means of high fidelity tracking and displays. That is, VR systems support a sensorimotor loop similar to that of the real world and enable users to interact and perceive as they would during unmediated experiences. Thus, VR systems may inadvertently or deliberately elicit a range of perceptual illusions and distortions. Based on our previous writings on the topic (Serafin et al. 2014), this entry details a selection of well-documented perceptual illusions and distortions known to occur during exposure to VR and discusses how they can be leveraged in order to provide more compelling experiences of virtual environments (VEs).
Illusions of Place and Plausibility
What separates VR from traditional audiovisual media is arguably that it elicits an unprecedented degree of presence on behalf of the user. A number of different conflicting and complimentary theories of presence have been proposed (e.g., Lombard and Ditton 1997; Riva et al. 2011; Waterworth and Waterworth 2001). However, the view of presence advocated by Slater and colleagues is particularly relevant to the current discussion because it explicitly focuses on presence as a response to VR (e.g., Sanchez-Vives and Slater 2005; Slater 2009; Slater et al. 2010a). Generally, the degree of presence is viewed as the extent to which the user responds to realistically to the VE. If the VE elicits a high degree of presence then users will respond more or less exactly as they would when exposed to an equivalent scenario in the real world. This implies that the presence response occurs on multiple levels ranging from unconscious and automatic physiological and behavioral responses to higher level processes involving deliberation and thoughts. Thus, even though presence is subjectively felt, the presence response cannot be reduced to a subjective experience (Sanchez-Vives and Slater 2005). Nevertheless, Slater et al. (2010a) have argued that the presence response happens as a function of two illusions: the place illusion (PI) and the plausibility illusion (Psi).
PI can be described as to the illusion of “being there” in the VE. This illusion occurs in response to the VR system supporting the normal sensorimotor contingencies of the user (i.e., the normal actions the user can perform in order to perceive and affect the VE). For example, if the user turns her head or kneels down then displayed image and sound should reflect this change in perspective, and if she reaches out and grasps an object then this action should cause that object to move (Slater 2009). Specifically, Slater et al. (2010a) describe that when “a person perceives by carrying out actions that result in changes in (multi-sensory) perception much as in physical reality, then the simplest hypothesis for the brain to adopt is that what is being perceived is actually there” (p. 3). The concept technological immersion is sometimes associated with systems that support normal sensorimotor contingencies. That is, the degree of immersion is defined as the range of normal sensorimotor contingencies supported by the system.
Psi, the second illusion introduced by Slater (2009), refers to the illusion that the events happening virtually are indeed happening. Rovira et al. (2009) argue that Psi is a more difficult illusion to elicit than PI, and it requires that the VE satisfies at least three criteria: (1) The VE should be correlational. The user’s actions produce correlated reactions within the VE (e.g., a virtual character might avoid eye-contact and step aside if the user stares and exhibit aggressive body language). (2) The VE should be self-referential. The VE responds directly to the user’s presence even when the user does not perform an instigating action (e.g., a virtual character might react to the presence of the user without the user initially approaching or addressing said character). (3) The VE should be credible. The VE conform to the user’s knowledge and expectations accrued through a lifetime of nonmediated interactions. Notably, Skarbez et al. (2017) have introduced the concept coherence as a (possibly) objective characteristic of the VE that gives rise to Psi. Importantly, coherence is viewed as a superset of realism, implying that in order to be coherent the VE need not strictly adhere to real-world laws and norms. Instead, coherence depends on the degree to which the user finds the virtual experience to possess internal logic and behavioral consistency. Thus, a virtual fantasy world may be coherent as long as fantastic events and behaviors are consistent.
Finally, Slater (2009) has described the virtual body as “a focal point where PI and Psi are fused” (p. 3554). The virtual body will be the topic of the following section.
Body Ownership Illusions
Individuals make sense of incoming sensory stimuli based on tacit expectations as to how these stimuli relate to their own actions and events in the environment. Thus, when reaching for an object, you expect to see your extended arm, and when directing your gaze downward, you expect to see your own body. However, when immersed in a VE, you cannot take for granted that such actions will bring your body into view. If the user is wearing a head-mount display (HMD), her real body will be occluded and tracking of the various body-parts is necessary in order to map a virtual representation onto the real body. When done successfully this may cause the user to experience an illusion of virtual body ownership, i.e., the experience that an artificial body is in fact one’s own physical body (Maselli and Slater 2013).
The illusion of ownership of artificial body-parts was first demonstrated without the use of VR technology. Specifically, Botvinick and Cohen (1998) documented the so-called rubber hand illusion where subjects experienced a sense of ownership over a rubber hand placed in front of them when tactile stimuli was synchronously applied to their real hand and the rubber hand. Ten years after the rubber hand illusion was first described, Slater et al. (2008) showed that it can be replicated in VR. Work by Maselli and Slater (2013) explored the perceptual underpinnings for full body ownership illusions in VR. Their work among other things revealed that (1) a first person perspective is central to the virtual body-ownership. (2) Visio-proprioceptive cues can elicit body-ownership illusions when users are exposed to a virtual body that spatially overlaps with the real body from a first person perspective. (3) When the spatial overlap or the realism of virtual body is limited, then multisensory or sensorimotor cues are necessary in order to produce the body-ownership illusion.
Interestingly, virtual body-ownership illusions can occur even when the virtual body differs from the users own body. For example, the illusion can occur when the virtual and real bodies differ in regard to size (Normand et al. 2011), gender (Slater et al. 2010b), age (Banakou et al. 2013), and race (Maister et al. 2013). Moreover, ownership illusions may even occur when the subject is exposed to nonhuman virtual bodies (e.g., humanoid robots and cartoon-like avatars Lugrin et al. 2015) and when these bodies have additional limbs (e.g., a virtual tail Steptoe et al. 2013). Work exploring the effects on virtual body-ownership suggests that ownership of a foreign body affects subject’ perceptions, attitudes, and actions. For example, implicit racial bias may be reduced when subjects have been exposed to a virtual body with a different skin color (Peck et al. 2013) and adult subjects embodying a child’s body may be more likely to overestimate the size of virtual objects (Banakou et al. 2013).
Distortions of Virtual Distances
Perceptual distortions of virtual spaces and objects are not only phenomena that occur when users embody a foreign avatar, such as an adult inhabiting the virtual body of a child. One of the most well-documented perceptual distortions of virtual spaces relate to the perception of egocentric distances. Users tend to underestimate virtual distances. That is, individuals tend to find the distance between their own position and virtual objects shorter than it really is. Based on a review of the literature, Renner et al. (2013) broadly divide the factors influencing virtual distance misperception into four categories: technical factors, compositional factors, human factors, and measurement methods.
Technical factors do as the name implies relate to the properties of the software and hardware used to render and display the VE. Specifically, Renner et al. (2013) highlight a number of properties of HMDs that may account for some of the underestimation. For example, some of the underestimation may be attributed to the combination of a restricted field of view (FOV), the mass and moment of inertia of HMDs, along with the feeling of wearing the HMDs. Moreover, the accommodation-convergence conflict may introduce underestimations, but it is unclear whether graphics quality is of influence.
Compositional factors relate to the features of the VE presented to the user. The review by Renner et al. (2013) suggests that the literature is fairly consistent in regard to the effects of pictorial depth cues on distance perception. Particularly, the authors describe that distance estimates may be positively influenced by the addition of complexity to the VE. Moreover, the use of a correct avatar may also improve distance estimates.
The third category, human factors, relate to the users’ psychological characteristics. Renner et al. (2013) describe that distance estimation does not appear to be influenced by gender or variations in age among adults. The ability to estimate virtual distances may be improved by providing individuals with feedback on their performance. However, the resulting adaptation to virtual distances may necessarily reduce transfer of skills from VR to real life.
In regard to the final group of factors, measurement methods, a number of different measures have been proposed. All measures quite consistently suggest that individuals underestimate egocentric distances in VR. However, different measures are a likely to produce difference estimates of the underestimation. Renner et al. (2013) describe that blind throwing and blind walking estimates yield comparable results; blind reaching estimates may offer higher accuracy and consistency than verbal estimates; timed imagined walking and blind walking produce similar estimates; and blind walking estimates are generally more accurate than estimates resulting from triangulated blind walking and indirect blind walking.
Finally, Renner et al. (2013) recommend that in order to facilitate the best possible distance estimation in VEs, developers should “provide binocular disparity, use high quality of graphics, carefully adjust the virtual camera settings, display a rich virtual environment containing a regularly structured ground texture, and enhance the user’s sense of presence” (p. 23).
Self-motion Illusions and Distortions
It is not just the perception of virtual distances that are prone to misperceptions. Virtual walking accomplished using a treadmill may also be accompanied by perceptual distortions. If a user were walking on a linear treadmill while wearing a HMD, then one would expect that the visual speed presented to the user should match the speed of the treadmill’s belt. However, it has been demonstrated that the walker is likely to perceive matched visual speeds as too slow (Banton et al. 2005; Kassler et al. 2010; Nilsson et al. 2016; Powell et al. 2011). Notably, a similar perceptual distortion appears to be present when users are relying on walking-in-place techniques for virtual travel (Nilsson et al. 2016). While the phenomenon and its causes have yet to be fully understood, previous user studies have yielded the following insights: (1) The distortion may be eliminated if walkers direct their gaze downwards or to the side (Banton et al. 2005). (2) Image jitter does not appear to be responsible for the distortion (Banton et al. 2005). (3) No effect of increased HMD weight or varying peripheral occlusion have been found (Nilsson et al. 2015a, b). (4) The degree of underestimation appears to be inversely proportional to the size of the display FOV and the geometric FOV (Nilsson et al. 2014a; 2015a). (5) The degree of identified underestimation may vary depending on study methods (Nilsson et al. 2015a). (6) High step frequencies may lead to a larger degree of underestimation, but the evidence is somewhat equivocal with respect to this effect (Nilsson et al. 2014b; Durgin et al. 2007; Kassler et al. 2010); (7) Finally, the degree of underestimation may vary slightly depending on whether the user is walking on a treadmill or walking in place (Nilsson et al. 2016).
Turning to the topic of illusory self-motion, one does not need to be immersed in a VE in order to experience visually induced self-motion illusions, that is, vection (Lowther and Ware 1996). The following scenario is likely to elicit a vection illusion. A passenger is sitting on a stationary train, looking out the window at another train in the adjacent track. When this second departs, the passenger experiences a transient, yet compelling, that she is on board the trains which is moving. This example illustrates how optokinetic stimuli may be open to at least two perceptual interpretations (Brandt et al. 1973). The passenger either (falsely) perceives the surroundings as being stationary, while she is moving or (correctly) perceives herself as being stationary while the second train is moving. Riecke et al. (2005b) describe that vection is influenced both by the properties of the physical stimuli indicative of self-motion (bottom-op factors) and by the perceivers’ expectations to and interpretation of said stimuli (top-down factors). Notably, visual stimuli is not a prerequisite for vection, as self-motion illusions can occur during exposed to auditory, vibrotactile, and biomechanical stimulation (Nordahl et al. 2012; Riecke et al. 2005a; Riecke and Schulte-Pelkum 2013; Väljamäe 2009). The bottom-up factors affecting visually induced self-motion illusions include the stimuli’s speed of movement, the perceived depth structure of the visual stimuli, and the area of the visual field occupied by the stimuli (Riecke et al. 2005a). Three of the three primary auditory cues for discrimination of motion are sound intensity, binaural cues, and the Doppler effect (Väljamäe 2009). Recent work on haptically induced self-motion illusions used a combination of vibrotactile stimulation to the main supporting areas of the feet and a virtual elevator to elicit sensations of upward and downward movements (Nordahl et al. 2012). Because the walls of the virtual elevator were opaque, no visual self-motion cues were presented (i.e., optic flow). Thus, it seems likely that the participants’ expectation of vertical movement (a top-down factor) contributed to the illusion. Other examples of top-down factors shown to affect vection include the belief that self-motion is possible (Riecke et al. 2005b, 2008; Wright et al. 2006), and explicitly being asked to attend to either the sensation of self-motion or object motion (Palmisano and Chan 2004).
Leveraging Perceptual Illusions in VR
A defining feature of VR is arguably its ability to elicit both the illusion of “being there” in the VE (the place illusion) and the illusion that the unfolding events are indeed happening (the plausibility illusion). As a consequence, the VR system may cause the user to respond realistically to the VE (a presence response). Bowman and McMahan (2007) have argued that it is precisely this ability to produce experiences greatly resembling “the real thing” that is responsible for the success of many past VR applications. For example, virtual exposure therapy functions because the VE can elicit a elicit a genuine fear response, and military and medical training applications are of value exactly because they can produce a realistic scenario without the presence of any real danger to soldiers or patients. It is arguably also this capacity for providing seemingly real experiences of actual, fictional, and fantastic places that gives VR its appeal with respect to entertainment applications. VR’s ability to provide compelling illusions of virtual body-ownership has applications within a range of different domains beyond mere entertainment (e.g., allowing users become a fictional character). The ability to own a body different from one’s own could potentially inspire empathy towards others, and it also seems possible that virtual body-ownership illusions could be leveraged as an intervention or treatment in relation to body image disturbances.
Misperception of virtual distances and virtual walking speeds may pose a considerable problem with respect to some VR applications. For example, veridical spatial perception of distance, scale, and speed may prove to be a fundamental requirement in relation to certain training scenarios or visualization of architecture). Nevertheless, the limitations of human motion perception in VR have also been leveraged in order to compress larger VEs into smaller physical spaces. Most notably, a collection of techniques referred to as redirected walking makes it possible to scale, rotate, and curve the user’s physical path by subtlety exaggerating or shrinking his real movements (Steinicke et al. 2010). Notably, other perceptual illusions and distortions have also been used to enable redirected walking. For example, Suma et al. (2011) proposed a technique that leverages change blindness illusions in order to move doors and hallways behind the user’s and thereby produce overlapping virtual spaces.
Finally, vection is central to the creation of compelling vehicle simulators. Nevertheless, one should be cautious when presenting stationary users with virtual stimuli indicating self-motion. Specifically, this will introduce a visio-vestibular conflict which is believed to cause cybersickness (Davis et al. 2014). Thus, the factors that produce more compelling illusions of self-motion may also cause sickness on behalf of some users.
To conclude, VR systems make it possible to manipulate the senses unlike any traditional audiovisual media. Thus, VR is a near-perfect medium ripe for eliciting perceptual illusions. Moreover, perceptual illusions may serve as a useful tool for solving practical problems (e.g., enabling unconstrained walking in VR), and they may enable developers to provide more compelling experiences (e.g., the illusion of self-motion during vehicular travel). In fact, the presence response which is characteristic of VR is believed to result from the simultaneous occurrence of the place illusion and plausibility illusion.
- Davis, S., Nesbitt, K., Nalivaiko, E.: A systematic review of cybersickness. In: Proceedings of the 2014 Conference on Interactive Entertainment, pp. 1–9. ACM (2014)Google Scholar
- Kassler, L., Feasel, J., Lewek, M.D., Brooks Jr., F.P., Whitton, M.C.: Matching actual treadmill walking speed and visually perceived walking speed in a projection virtual environment. In: Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization, pp. 161–161. ACM (2010)Google Scholar
- Lowther, K., Ware, C.: Vection with large screen 3d imagery. In: Conference Companion on Human Factors in Computing Systems, pp. 233–234. ACM (1996)Google Scholar
- Lugrin, J.-L., Latt, J., Latoschik, M.E.: Avatar anthropomorphism and illusion of body ownership in vr. In: 2015 I.E. Virtual Reality (VR), pp. 229–230. IEEE (2015)Google Scholar
- Nilsson, N., Serafin, S., Nordahl, R.: Establishing the range of perceptually natural visual walking speeds for virtual walking-in-place locomotion. In: Proceedings of the 2014 I.E. Virtual Reality Conference. IEEE (2014a)Google Scholar
- Nilsson, N.C., Serafin, S., Nordahl, R.: The influence of step frequency on the range of perceptually natural visual walking speeds during walking-in-place and treadmill locomotion. In: Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology, pp. 187–190. ACM (2014b)Google Scholar
- Nilsson, N.C., Serafin, S., Nordahl, R.: The effect of visual display properties and gain presentation mode on the perceived naturalness of virtual walking speeds. In: 2015 I.E. Virtual Reality (VR), pp. 81–88. IEEE (2015a)Google Scholar
- Nilsson, N.C., Serafin, S., Nordahl, R.: The effect of head mounted display weight and locomotion method on the perceived naturalness of virtual walking speeds. In: 2015 I.E. Virtual Reality Conference (VR) (2015b)Google Scholar
- Nilsson, N.C., Serafin, S., Nordahl, R.: The perceived naturalness of virtual walking speeds during wip locomotion: summary and meta-analyses. PsychNol. J. 14(1), 7–39 (2016)Google Scholar
- Nordahl, R., Nilsson, N.C., Turchet, L., Serafin, S.: Vertical illusory self-motion through haptic stimulation of the feet. In: Proceedings of the 2012 I.E. VR Workshop on Perceptual Illusions in Virtual Environments (2012)Google Scholar
- Powell, W., Stevens, S., abd Hand, B., Simmonds, M.: Blurring the boundaries: the perception of visual gain in treadmill-mediated virtual environments. In: Proceedings of the 3rd IEEE VR Workshop on Perceptual Illusions in Virtual Environments, pp. 4–8. IEEE (2011)Google Scholar
- Riecke, B.E., Schulte-Pelkum, J.: Perceptual and cognitive factors for self-motion simulation in virtual environments: how can self-motion illusions (“vection”) be utilized? In: Human Walking in Virtual Environments, pp. 27–54. Springer (2013)Google Scholar
- Riecke, B., Schulte-Pelkum, J., Avraamides, M., von der Heyde, M., Bülthoff, H.: Scene consistency and spatial presence increase the sensation of self-motion in virtual reality. In: Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, pp. 111–118. ACM (2005a)Google Scholar
- Riecke, B., Västfjäll, D., Larsson, P., Schulte-Pelkum, J.: Top-down and multi-modal influences on self-motion perception in virtual reality. In: Proceedings of the 11th International Conference on Human-Computer Interaction (2005b)Google Scholar
- Riecke, B., Feuereissen, D., Rieser, J.: Auditory self-motion illusions (circular vection) can be facilitated by vibrations and the potential for actual motion. In: Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization, pp. 147–154. ACM (2008)Google Scholar
- Rovira, A., Swapp, D., Spanlang, B., Slater, M.: The use of virtual reality in the study of people’s responses to violent incidents. Front. Behav. Neurosci. 3, 59 (2009)Google Scholar
- Serafin, S., Nilsson, N.C., Erkut, C., Nordahl, R.: Virtual Reality and the Senses. Danish Sound Innovation Network (2017). https://issuu.com/danishsound/docs/dtu_whitepaper_2017_singlepages. ISBN 978-87-643-1317-8
- Stevenson, A.: Oxford Dictionary of English. Oxford University Press, Oxford (2010)Google Scholar
- Suma, E., Clark, S., Krum, D., Finkelstein, S., Bolas, M., and Warte, Z.: Leveraging change blindness for redirection in virtual environments. In: Proceedings of the 2011 I.E. Virtual Reality Conference, pp. 159–166. IEEE (2011)Google Scholar
- Wright, W., DiZio, P., Lackner, J.: Perceived self-motion in two visual contexts: dissociable mechanisms underlie perception. J. Vestib. Res. 16(1), 23–28 (2006)Google Scholar