Abstract
Serious games are gaining in popularity within a wide range of educational and training applications given their ability to engage and motivate learners in the educational process. Recent hardware and computational advancements are providing developers the opportunity to develop applications that employ a high level of fidelity (realism) and novel interaction techniques. However, despite these great advances in hardware and computational power, real-time high fidelity rendering of complex virtual environments (found in many serious games) across all modalities is still not feasible. Perceptual-based rendering exploits various aspects of the multi-modal perceptual system to reduce computational requirements without any resulting perceptual effects on the resulting scene. A series of human-based experiments demonstrated a potentially strong effect of sound on visual fidelity perception, and task performance. However, the resulting effects were subjective whereby the influence of sound was dependent on various individual factors including musical listening preferences. This suggests the importance of customizing (individualizing) a serious game’s virtual environment with respect to audio-visual fidelity, background sounds, etc. In this paper details regarding this series of audio-visual experiments will be provided followed by a description of current work that is examining the customization of a serious game’s virtual environment by each user through the use of a game-based calibration method.
Chapter PDF
Similar content being viewed by others
References
Robison, R.A., Liu, C.Y., Apuzzo, M.L.J.: Man, mind, and machine: The past and future of virtual reality simulation in neurologic surgery. World Nuerosurgery 76(5), 419–430 (2011)
Farmer, E., von Rooij, J., Riemersma, J., Joma, P., Morall, J.: Handbook of simulator based training. Ashgate Publishing Limited, Surrey (1999)
Hulusic, V., Harvey, C., Debattista, K., Tsingos, N., Walker, S., Howard, D., Chalmers, A.: Acoustic rendering and auditory-visual cross-modal perception and interaction. Computer Graphics Forum 31(1), 102–131 (2012)
Norman, G., Dore, K., Grierson, L.: The minimal relationship between simulation fidelity and transfer of learning. Medical Education 46(7), 636–647 (2012)
Blascovich, J., Bailenson, J.: Infinite Reality. Harper Collins, New York (2011)
Mastoropoulou, G., Debattista, K., Chalmers, A., Troscianco, T.: The influence of sound effects on the perceived smoothness of rendered animations. In: Proceedings of the Symposium on Applied Perception in Graphics and Visualization 2005, La Coruna, Spain, August 26-28, pp. 9–15 (2005)
Larsson, P., Vstjll, D., Kleiner, M.: On the quality of experience: A multi-modal approach to perceptual ego-motion and sensed presence in virtual environments. In: Proceedings of the First International Speech Communications Association Tutorial and Research Workshop on Auditory Quality of Systems, Akademie Mont-Cenis, Germany (2003)
Carlile, S.: Virtual auditory space: Generation and application. R. G. Landes, Austin TX (1996)
Mania, K., Wooldridge, D., Coxon, M., Robinson, A.: The effect of visual and interaction fidelity on spatial cognition in immersive virtual environments? IEEE Transactions on Visualization and Computer Graphics 12(3), 396–404 (2006)
Storms, S.L., Zyda, M.J.: Interactions in perceived quality of auditory-visual displays. Presence: Teleoperators and Virtual Environments 9(6), 557–580 (2000)
Hulusic, V., Debattista, K., Aggarwal, V., Chalmers, A.: Maintaining frame rate perception in interactive environments by exploiting audio-visual cross-modal interaction. The Visual Computer 27(1), 57–66 (2011)
Bonneel, N., Suied, C., Viaud-Delmon, I., Drettakis, G.: Bimodal perception of audio-visual material properties for virtual environments. ACM Transactions on Applied Perception 7(1), 1–16 (2010)
Shams, L., Kim, R.: Crossmodal influences on visual perception. Physics of Life Reviews 7(3), 295–298 (2010)
Rojas, D., Kapralos, B., Cristancho, S., Collins, K., Conati, C., Dubrowski, A.: The effect of background sound on visual fidelity perception. In: Proceedings of ACM Audio Mostly 2011 (Extended Abstracts), Coimbra, Portugal, September 7-9, pp. 1–7 (2011)
Rojas, D., Kapralos, B., Cristancho, S., Collins, K., Hogue, A., Conati, C., Dubrowski, A.: Developing effective serious games: The effect of background sound on visual fidelity perception with varying texture resolution. Studies in Health Technology and Informatics 173, 386–392 (2013)
Rojas, D., Kapralos, B., Hogue, A., Collins, K., Nacke, L., Cristancho, S., Conati, C., Dubrowski, A.: The effect of sound on visual fidelity perception in stereoscopic 3-d. IEEE Transactions on System Man and Cybernetics Part B 43(6), 1572–1583 (2013)
Rojas, D., Kapralos, B., Collins, K., Dubrowski, A.: The effect of contextual sound cues on visual fidelity perception. Studies in Health Technology and Informatics (to appear, 2014)
Cowan, B., Rojas, D., Kapralos, B., Collins, K., Dubrowski, A.: Spatial sound and its effect on visual quality perception and task performance within a virtual environment. In: Proceedings of the 21st International Congress on Acoustics, Montreal, Canada, June 2-7, pp. 1–7 (2013)
Kapralos, B., Moussa, F., Dubrowski, A.: An overview of virtual simulations and serious games for surgical education and training. In: Brooks, A.L., Braham, S., Jain, L.C. (eds.) Technologies of Inclusive Well-Being. SCI, vol. 536, pp. 289–306. Springer, Heidelberg (2014)
Kerr, D.S.: The refractor/phoropter an important tool in vision correction (December 7, 2010), http://dougkerr.net/pumpkin/articles/Refractor.pdf
Tawadrous, M., Hogue, A., Kapralos, B., Collins, K.: An interactive in-game approach to user adjustment of stereoscopic 3D settings. In: Proceedings of Stereoscopic Displays and Applications XXIV, San Francisco, CA, USA, February 3-7, pp. 1–7 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Kapralos, B., Shewaga, R., Ng, G. (2014). Serious Games: Customizing the Audio-Visual Interface. In: Shumaker, R., Lackey, S. (eds) Virtual, Augmented and Mixed Reality. Applications of Virtual and Augmented Reality. VAMR 2014. Lecture Notes in Computer Science, vol 8526. Springer, Cham. https://doi.org/10.1007/978-3-319-07464-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-07464-1_18
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07463-4
Online ISBN: 978-3-319-07464-1
eBook Packages: Computer ScienceComputer Science (R0)