Advertisement

Novel Approaches to Immersive Media: From Enlarged Field-of-View to Multi-sensorial Experiences

  • Iris GallosoEmail author
  • Claudio Feijóo
  • Asunción Santamaría
Chapter

Abstract

This chapter presents a review of current evidence on the influence of immersion (defined in terms of the technical features of the system) on the user experience in multimedia applications. Section 2.1 introduces the concepts of media enjoyment, presence, and Quality of Experience (QoE) that frame our analysis from the user perspective. Section 2.2 discusses the bounding effects of multimodal perception on the previously defined metrics. Section 2.3 analyses the influence of relevant technical factors on presence, enjoyment, and QoE, with emphasis on those characterizing the level of immersion delivered by system across four dimensions: inclusiveness, extensiveness, surrounding, and vividness. Section 2.4 presents recent works integrating some of these factors into multi-sensorial media experiences and highlights open issues and research challenges to be tackled in order to deliver cost-effective multi-sensorial media solutions to the mass market.

Keywords

Sensory Effect Head Mount Display Simulator Sickness Spatial Presence Audiovisual Content 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Sherry JL (2004) Flow and media enjoyment. Commun Theory 14:328–347. doi: 10.1111/j.1468-2885.2004.tb00318.x CrossRefGoogle Scholar
  2. 2.
    Ruggiero TE (2000) Uses and gratifications theory in the 21st century. Mass Commun Soc 3:3–37. doi: 10.1207/S15327825MCS0301_02 CrossRefGoogle Scholar
  3. 3.
    Kapsner JC (2009) The encyclopedia of positive psychology. doi:  10.1111/b.9781405161251.2009.x
  4. 4.
    Csikszentmihalyi M (1990) Flow: the psychology of optimal experience. Harper & Row, New YorkGoogle Scholar
  5. 5.
    Nakamura J, Csikzentmihalyi M (2003) The construction of meaning through vital engagement. In: Keyes CLM, Haidt J (eds) Flourishing Posit. Psychol. Life well-lived. American Psychological Association, Washington, pp 83–104CrossRefGoogle Scholar
  6. 6.
    Nabi RL, Krcmar M (2004) Conceptualizing media enjoyment as attitude: implications for mass media effects research. Commun Theory 14:288–310. doi: 10.1111/j.1468-2885.2004.tb00316.x CrossRefGoogle Scholar
  7. 7.
    Slater MD (2003) Alienation, aggression, and sensation seeking as predictors of adolescent use of violent film, computer, and website content. J Commun 53:105–121. doi: 10.1111/j.1460-2466.2003.tb03008.x CrossRefGoogle Scholar
  8. 8.
    Zillmann D, Vorderer P (2000) Media entertainment: the psychology of its appeal. Lawrence Erlbaum Associates, Mahwah, p 282Google Scholar
  9. 9.
    Raney AA, Bryant J (2002) Moral judgment and crime drama: an integrated theory of enjoyment. J Commun 52:402–415. doi: 10.1111/j.1460-2466.2002.tb02552.x CrossRefGoogle Scholar
  10. 10.
    Wechsung I, Schulz M, Engelbrecht K-P et al (2011) All users are (not) equal – the influence of user characteristics on perceived quality, modality choice and performance. In: Kobayashi T, Delgado RL-C (eds) Proceedings of the paralinguistic information and its integration in spoken dialogue systems workshop. Springer, New York, pp 175–186CrossRefGoogle Scholar
  11. 11.
    Tamborini R, Bowman ND, Eden A et al (2010) Defining media enjoyment as the satisfaction of intrinsic needs. J Commun 60:758–777. doi: 10.1111/j.1460-2466.2010.01513.x CrossRefGoogle Scholar
  12. 12.
    Slater M, Wilbur S (1997) A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence-Teleop Virtual Environ 6:603–616Google Scholar
  13. 13.
    Riva G, Mantovani F, Capideville CS et al (2007) Affective interactions using virtual reality: The link between presence and emotions. Cyberpsychology Behav 10:45–56. doi: 10.1089/cpb.2006.9993 CrossRefGoogle Scholar
  14. 14.
    Västfjäll D (2003) The subjective sense of presence, emotion recognition, and experienced emotions in auditory virtual environments. Cyberpsychology Behav 6:181–188. doi: 10.1089/109493103321640374 CrossRefGoogle Scholar
  15. 15.
    Sylaiou S, Mania K, Karoulis A, White M (2010) Exploring the relationship between presence and enjoyment in a virtual museum. Int J Hum Comput Stud 68:243–253CrossRefGoogle Scholar
  16. 16.
    Skalski P, Tamborini R, Shelton A et al (2010) Mapping the road to fun: Natural video game controllers, presence, and game enjoyment. New Media Soc 13:224–242CrossRefGoogle Scholar
  17. 17.
    Lombard M, Ditton T (1997) At the heart of it all: the concept of presence. J Comput Commun 3:0. doi: 10.1111/j.1083-6101.1997.tb00072.xGoogle Scholar
  18. 18.
    IJsselsteijn W (2000) Presence: concept, determinants, and measurement. In: Proc. SPIE. Spie. pp 520–529Google Scholar
  19. 19.
    Slater M, Steed A, McCarthy J, Maringelli F (1998) The influence of body movement on subjective presence in virtual environments. Hum Factors 40:469–477CrossRefGoogle Scholar
  20. 20.
    Schuemie MJ, van der Straaten P, Krijn M, van der Mast CAPG (2001) Research on presence in virtual reality: a survey. Cyberpsychol Behav 4:183–201. doi: 10.1089/109493101300117884 CrossRefGoogle Scholar
  21. 21.
    Darken RP, Bernatovich D, Lawson JP, Peterson B (1999) Quantitative measures of presence in virtual environments: the roles of attention and spatial comprehension. Cyberpsychol Behav 2:337–347CrossRefGoogle Scholar
  22. 22.
    Lee S, Kim GJ (2008) Effects of visual cues and sustained attention on spatial presence in virtual environments based on spatial and object distinction. Interact Comput 20:491–502CrossRefGoogle Scholar
  23. 23.
    Novak T, Hoffman D, Yung Y (2000) Measuring the customer experience in online environments: a structural modeling approach. Mark Sci 19:22–44CrossRefGoogle Scholar
  24. 24.
    Takatalo J, Nyman G, Laaksonen L (2008) Components of human experience in virtual environments. Comput Human Behav 24:1–15. doi: 10.1016/j.chb.2006.11.003 CrossRefGoogle Scholar
  25. 25.
    Weibel D, Wissmath B, Mast FW (2010) Immersion in mediated environments: the role of personality traits. Cyberpsychol Behav Soc Netw 13:251–256CrossRefGoogle Scholar
  26. 26.
    Alsina-Jurnet I, Gutierrez-Maldonado J (2010) Influence of personality and individual abilities on the sense of presence experienced in anxiety triggering virtual environments. Int J Hum Comput Stud 68:788–801CrossRefGoogle Scholar
  27. 27.
    Le Callet P, Möller S, Perkis A (2012) Qualinet white paper on definitions of quality of experience. In: European network on quality of experience in multimedia systems and services (COST Action IC 1003), Lausanne, Switzerland, Version 1.1, June 3, 2012Google Scholar
  28. 28.
    Jumisko-Pyykkö S (2011) User-centered quality of experience and its evaluation methods for mobile television. Tampere University of TechnologyGoogle Scholar
  29. 29.
    Bracken C, Pettey G, Wu M (2011) Telepresence and attention: secondary task reaction time and media form. In: Proc. Int. Soc. PresenceGoogle Scholar
  30. 30.
    Goldstein EB (2010) Sensation and perception. p 496Google Scholar
  31. 31.
    Lazarus RS (1993) From psychological stress to the emotions: a history of changing outlooks. Annu Rev Psychol 44:1–21. doi: 10.1146/annurev.ps.44.020193.000245 CrossRefGoogle Scholar
  32. 32.
    Bey C, McAdams S (2002) Schema-based processing in auditory scene analysis. Percept Psychophys 64:844–854CrossRefGoogle Scholar
  33. 33.
    Jennings JR, Van der Molen MW, Van der Veen FM, Debski KB (2002) Influence of preparatory schema on the speed of responses to spatially compatible and incompatible stimuli. Psychophysiology 39:496–504CrossRefGoogle Scholar
  34. 34.
    Cui LC (2003) Do experts and naive observers judge printing quality differently? In: Miyake Y, Rasmussen DR (eds) Electron. Imaging 2004. International Society for Optics and Photonics, pp 132–145Google Scholar
  35. 35.
    Werner S, Thies B (2000) Is “Change Blindness” attenuated by domain-specific expertise? An expert-novices comparison of change detection in football images. Vis Cogn 7:163–173. doi: 10.1080/135062800394748 CrossRefGoogle Scholar
  36. 36.
    Sowden PT, Davies IR, Roling P (2000) Perceptual learning of the detection of features in X-ray images: a functional role for improvements in adults’ visual sensitivity? J Exp Psychol Hum Percept Perform 26:379–390CrossRefGoogle Scholar
  37. 37.
    Curran T, Gibson L, Horne JH et al (2009) Expert image analysts show enhanced visual processing in change detection. Psychon Bull Rev 16:390–397. doi: 10.3758/PBR.16.2.390 CrossRefGoogle Scholar
  38. 38.
    Sowden PT, Rose D, Davies IRL (2002) Perceptual learning of luminance contrast detection: specific for spatial frequency and retinal location but not orientation. Vision Res 42:1249–1258. doi: 10.1016/S0042-6989(02)00019-6 CrossRefGoogle Scholar
  39. 39.
    Neisser U (1976) Cognition and reality: principles and implications of cognitive psychology. p 230.Google Scholar
  40. 40.
    Dinh HQ, Walker N, Hodges LF, Kobayashi A (1999) Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In: Proc. IEEE virtual real (Cat. No. 99CB36316). IEEE Comput. Soc., pp 222–228Google Scholar
  41. 41.
    Hecht D, Reiner M, Halevy G (2006) Multimodal virtual environments: response times, attention, and presence. Presence-Teleop Virtual Environ 15:515–523CrossRefGoogle Scholar
  42. 42.
    Coen M (2001) Multimodal integration-a biological view. In: Proc. Fifteenth Int. Jt. Conf. Artif. Intell. Seattle, WA, pp 1417–1424Google Scholar
  43. 43.
    Shimojo S, Shams L (2001) Sensory modalities are not separate modalities: plasticity and interactions. Curr Opin Neurobiol 11:505–509CrossRefGoogle Scholar
  44. 44.
    Mcgurk H, Macdonald J (1976) Hearing lips and seeing voices. Nature 264:746–748CrossRefGoogle Scholar
  45. 45.
    Rock I, Victor J (1964) Vision and touch: an experimentally created conflict between the two senses. Science 143:594–596CrossRefGoogle Scholar
  46. 46.
    Scheier C, Nijhawan R, Shimojo S (1999) Sound alters visual temporal resolution. Invest Ophthalmol Vis Sci 40:4169Google Scholar
  47. 47.
    Chandrasekaran C, Ghazanfar AA (2011) When what you see is not what you hear. Nat Neurosci 14:675–676. doi: 10.1038/nn.2843 CrossRefGoogle Scholar
  48. 48.
    Skalski P, Whitbred R (2010) Image versus sound: a comparison of formal feature effects on presence and video game enjoyment. Psychology J 8:67–84, doi: ArticleGoogle Scholar
  49. 49.
    Biocca F, Kim J, Choi Y (2001) Visual touch in virtual environments: an exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions. Presence-Teleop Virtual Environ 10:247–265CrossRefGoogle Scholar
  50. 50.
    Basdogan C, Ho C, Srinivasan MA, Slater MEL (2001) An experimental study on the role of touch in shared virtual environments. ACM Trans Comput Interact 7:443–460CrossRefGoogle Scholar
  51. 51.
    Welch RB, Warren DH (1980) Immediate perceptual response to intersensory discrepancy. Psychol Bull 88:638–667CrossRefGoogle Scholar
  52. 52.
    De Kort YAW, Ijsselsteijn WA, Kooijman J, Schuurmans Y (2003) Virtual laboratories: comparability of real and virtual environments for environmental psychology. Presence-Teleop Virtual Environ 12:360–373CrossRefGoogle Scholar
  53. 53.
    Bowman DA, McMahan RP (2007) Virtual reality: how much immersion is enough? Computer (Long Beach Calif) 40:36–43. doi: 10.1109/MC.2007.257 Google Scholar
  54. 54.
    Slater M (2009) Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos Trans R Soc Lond B Biol Sci 364:3549–3557. doi: 10.1098/rstb.2009.0138 CrossRefGoogle Scholar
  55. 55.
    Slater M (2003) A note on presence terminology. Presence-Connect, pp 1–5Google Scholar
  56. 56.
    Steuer J (1992) Defining virtual reality: dimensions determining telepresence. J Commun 42:73–93. doi: 10.1111/j.1460-2466.1992.tb00812.x CrossRefGoogle Scholar
  57. 57.
    Ijsselsteijn W, de Ridder H, Freeman J et al (2001) Effects of stereoscopic presentation, image motion, and screen size on subjective and objective corroborative measures of presence. Presence-Teleop Virtual Environ 10:298–311CrossRefGoogle Scholar
  58. 58.
    Hands DS (2004) A basic multimedia quality model. IEEE Trans Multimedia 6:806–816. doi: 10.1109/TMM.2004.837233 CrossRefGoogle Scholar
  59. 59.
    Hwang J, Kim GJ (2010) Provision and maintenance of presence and immersion in hand-held virtual reality through motion based interaction. Comput Animat Virtual Worlds 21:547–559CrossRefGoogle Scholar
  60. 60.
    Hendrix C, Barfield W (1995) Presence in virtual environments as a function of visual and auditory cues. In: Proceedings of Virtual Real Annu Int Symp. pp 74–82. doi:  10.1109/VRAIS.1995.512482
  61. 61.
    Lin JJ-W, Duh HBL, Parker DE et al (2002) Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment. In: Proc. IEEE Virtual Real. IEEE Comput. Soc., pp 164–171Google Scholar
  62. 62.
    Freeman J, Avons SE, Pearson DE, IJsselsteijn WA (1999) Effects of sensory information and prior experience on direct subjective ratings of presence. Presence-Teleop Virtual Environ 8:1–13. doi: 10.1162/105474699566017 CrossRefGoogle Scholar
  63. 63.
    Bleumers L, Lievens B, Pierson J (2011) From sensory dream to television format: gathering user feedback on the use and experience of omnidirectional video-based solutions. In: ISPR 2011 Int. Soc. PRESENCE Res. Annu. Conf.Google Scholar
  64. 64.
    Slater M, Khanna P, Mortensen J, Yu I (2009) Visual realism enhances realistic response in an immersive virtual environment. IEEE Comput Graph Appl 29:76–84CrossRefGoogle Scholar
  65. 65.
    Welch RB, Blackmon TT, Liu A et al (1996) The effects of pictorial realism, delay of visual feedback, and observer interactivity an the subjective sense of presence. Presence-Teleop Virtual Environ 5:263–273Google Scholar
  66. 66.
    Barfield W, Hendrix C (1995) The effect of update rate on the sense of presence within virtual environments. Virtual Real 1:3–15. doi: 10.1007/BF02009709 CrossRefGoogle Scholar
  67. 67.
    Meehan M (2001) Physiological reaction as an objective measure of presence in virtual environments. University of North Carolina at Chapel HillGoogle Scholar
  68. 68.
    Luque FP, Galloso I, Feijoo C, Martín CA, Cisneros G (2014) Integration of multisensorial stimuli and multimodal interaction in a hybrid 3DTV system. ACM Trans Multimedia Comput Commun Appl 11(1s):16:1–16:22. doi: 10.1145/2617992
  69. 69.
    Beerends JG, De Caluwe FE (1999) The influence of video quality on perceived audio quality and vice versa. J Audio Eng Soc 47:355–362Google Scholar
  70. 70.
    Slater M, Usoh M, Steed A (1995) Taking steps: the influence of a walking technique on presence in virtual reality. ACM Trans Comput Interact 2:201–219. doi: 10.1145/210079.210084 CrossRefGoogle Scholar
  71. 71.
    De Ruyter B, Aarts E (2004) Ambient intelligence: visualizing the future. In: Proc. Work. Conf. Adv. Vis. interfaces – AVI’04. ACM Press, New York, p 203Google Scholar
  72. 72.
    Waltl M, Timmerer C, Hellwagner H (2010) Increasing the user experience of multimedia presentations with sensory effects. In: 11th Int. Work Image Anal. Multimed. Interact. Serv. (WIAMIS). IEEE, Desenzano del Garda, pp 1–4Google Scholar
  73. 73.
    Ademoye OA, Ghinea G (2009) Synchronization of olfaction-enhanced multimedia. IEEE Trans Multimed 11:561–565. doi: 10.1109/TMM.2009.2012927 CrossRefGoogle Scholar
  74. 74.
    Murray N, Qiao Y, Lee B et al (2013) Subjective evaluation of olfactory and visual media synchronization. In: Proc. 4th ACM Multimed. Syst. Conf. ACM, New York, pp 162–171Google Scholar
  75. 75.
    Ghinea G, Ademoye O (2012) The sweet smell of success: enhancing multimedia applications with olfaction. ACM Trans Multimed Comput Commun Appl 8:1–17. doi: 10.1145/2071396.2071398 Google Scholar
  76. 76.
    Pallavicini F, Cipresso P, Raspelli S et al (2013) Is virtual reality always an effective stressors for exposure treatments? Some insights from a controlled trial. BMC Psychiatry 13:1–10CrossRefGoogle Scholar
  77. 77.
    (1990) ITU-T J.100 Recommendation. Tolerance for transmission time differences between vision and sound components of a television signal. International Telecommunication Union (ITU) – Telecommunication sector.Google Scholar
  78. 78.
    Slutsky DA, Recanzone GH (2001) Temporal and spatial dependency of the ventriloquism effect. Neuroreport 12:7–10CrossRefGoogle Scholar
  79. 79.
    IJsselsteijn WA, de Ridder H, Vliegen J (2000) Effects of stereoscopic filming parameters and display duration on the subjective assessment of eye strain. In: Proc. SPIE 3957, Stereosc. Displays Virtual Real. Syst. VII. pp 12–22Google Scholar
  80. 80.
    Kooi FL, Toet A (2004) Visual comfort of binocular and 3D displays. Displays 25:99–108. doi: 10.1016/j.displa.2004.07.004 CrossRefGoogle Scholar
  81. 81.
    Meesters L, IJsselsteijn W (2003) Survey of perceptual quality issues in threedimensional television systems. Proc. SPIEGoogle Scholar
  82. 82.
    Banos RM, Botella C, Alcaniz M et al (2004) Immersion and emotion: their impact on the sense of presence. Cyberpsychology Behav 7:734–741CrossRefGoogle Scholar
  83. 83.
    Timmerer C, Waltl M, Rainer B, Hellwagner H (2012) Assessing the quality of sensory experience for multimedia presentations. Signal Process Image Commun 27:909–916CrossRefGoogle Scholar
  84. 84.
    ISO/IEC 23005-3 (2013) Information technology – Media context and control – Part 3: Sensory information. p 104Google Scholar
  85. 85.
    Waltl M, Rainer B, Timmerer C, Hellwagner H (2013) An end-to-end tool chain for Sensory Experience based on MPEG-V. Signal Process Image Commun 28:136–150CrossRefGoogle Scholar
  86. 86.
    Yoon K (2013) End-to-end framework for 4-D broadcasting based on MPEG-V standard. Signal Process Image Commun 28:127–135CrossRefGoogle Scholar
  87. 87.
    Kim J, Lee C-G, Kim Y, Ryu J (2013) Construction of a haptic-enabled broadcasting system based on the MPEG-V standard. Signal Process Image Commun 28:151–161CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Iris Galloso
    • 1
    Email author
  • Claudio Feijóo
    • 1
  • Asunción Santamaría
    • 1
  1. 1.CeDInt-UPM.MadridSpain

Personalised recommendations