Experiencing the Multisensory Past

  • Alan ChalmersEmail author


The world is multisensory. All five major senses, visuals, audio, smell, feel and taste, are important to how we, humans, perceive the world around us. Furthermore, the interaction of the senses—cross-modal effects—can significantly affect our understanding of an environment, with one sensory stimuli even being ignored in the presence of other more dominant stimuli. This is as true now as it was in the past. So if we are to attempt, through computer simulation, to understand how a past environment may have been experienced, it is essential that these virtual reconstructions are multisensory, otherwise we run the very real risk of misrepresenting the past.


Virtual experiences Multisensory Kalabsha Medieval pottery Real Virtuality 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    P. Martinez, Digital Realities and Archaeology: A Difficult Relationship or a Fruitful Marriage? Proceedings of the 1st International Symposium on Virtual Reality, Archaeology, and Cultural Heritage VAST 2001, 9–16, 2001Google Scholar
  2. 2.
    P. Miller, J. Richards, The good, the bad and the downright misleading: archaeological adoption of computer visualization. Comput. Appl. Quant. Methods Archaeol. 19–22 (1994)Google Scholar
  3. 3.
    K. Devlin, A.G. Chalmers, in Realistic Visualisation of the Pompeii Frescoes. AFRIGRAPH 2001 (ACM SIGGRAPH Publications, 2001), pp. 43–48Google Scholar
  4. 4.
    A. Gonçalves, L. Magalhães, J. Moura, A. Chalmers, Accurate Modelling of Roman Lamps in Conimbriga Using High Dynamic Range. VAST’08: Proceedings of the Symposium on Virtual Reality, Archaeology and Cultural Heritage, 2008Google Scholar
  5. 5.
    L. Itti, C. Koch, A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 40(10–12), 1489–1506 (2000)Google Scholar
  6. 6.
    A.L. Yarbus, Eye movements during perception of complex objects, Chapter VII, in Eye Movements and Vision, ed. by L.A. Riggs (Plenum Press, New York, NY, 1967), pp. 171–196CrossRefGoogle Scholar
  7. 7.
    B. Leonard, L. Martin, in Literature Review on Skill Fade. Human Factors Integration Defence Technology Centre/2/WP10.3/2, Version No. 1/25, May 2007Google Scholar
  8. 8.
    S. Marsland, U. Nehmzow, J. Shapiro, Novelty Detection on a Mobile Robot Using Habituation. From Animals to Animats: The 6th International Conference on Simulation of Adaptive Behaviour, 2000Google Scholar
  9. 9.
    A. Mack, I. Rock, in Inattentional Blindness. Proceedings of Symposium on Interactive 3D Graphics, Massachusetts Institute of Technology Press, 1998Google Scholar
  10. 10.
    D. Nunez, E.H. Blake, in Conceptual Priming as a Determinant of Presence in Virtual Environments. Afrigraph 2003, ACM SIGGRAPH, pp. 101–108, 2003Google Scholar
  11. 11.
    A.G. Chalmers, D. Howard, C. Moir, in Real Virtuality: A Step Change from Virtual Reality. SCCG’09 (ACM Press, 2009)Google Scholar
  12. 12.
    G. Calvert, C. Spence, B. Stein, The Multisensory Handbook (MIT Press, Cambridge, MA, 2004)Google Scholar
  13. 13.
    B. Stein, M. Meredith, The Merging of the Senses (MIT Press, Cambridge, MA, 1993)Google Scholar
  14. 14.
    T.G. Giolas, E.C. Butterfield, S.J. Weaver, Some motivational correlates of lipreading. J. Speech Hear. Res. 17, 18–24 (1974)CrossRefGoogle Scholar
  15. 15.
    K. Sekiyama, Y.J. Tohkura, McGurk effect in non-English listeners: few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. Acoust. Soc. Am. 90, 1797–1805 (1991)CrossRefGoogle Scholar
  16. 16.
    J. Irwin, D. Whalen, A sex difference in visual influence on heard speech. Percept. Psychophys. 68(4), 582–592 (2006)CrossRefGoogle Scholar
  17. 17.
    A. Alsius, J. Navarra, R. Campbell, S. Soto-Faraco, Audiovisual integration of speech falters under high attention demands. Curr. Biol. 15(839–843) (2005)Google Scholar
  18. 18.
    J. Townsend, M. Adamo, F. Haist, Changing channels: an fMRI study of aging and cross-modal attention shifts. NeuroImage 31(4), 1682–1692 (2006)CrossRefGoogle Scholar
  19. 19.
    M.O. Ernst, H.H. Bülthoff, Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162–169 (2004)CrossRefGoogle Scholar
  20. 20.
    T.S. Andersen, K. Tiipanna, M. Sams, Factors influencing audiovisual fission and fusion illusions. Cogn. Brain Res. 21, 301–308 (2004)CrossRefGoogle Scholar
  21. 21.
    A.G. Chalmers, A. Ferko, in Levels of Realism: from Virtual Reality to Real Virtuality. In SCCG'08: Spring Conference on Computer Graphics, ACM SIGGRAPH Press, pp. 27–33, 2008Google Scholar
  22. 22.
    J. Ferwerda, in Three Varieties of Realism in Computer Graphics. Proceedings SPIE Human Vision and Electronic Imaging, pp. 290–297, 2003Google Scholar
  23. 23.
    F. Banterle, A. Artusi, K. Debattista, A.G Chalmers, Advanced High Dynamic Range Imaging. (CRC Press, Boca Raton, 2011). ISBN: 978-156881-719-4Google Scholar
  24. 24.
    H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, A. Vorozcovs, High dynamic range display systems. ACM Trans. Graphics 23(3) (2004)Google Scholar
  25. 25.
    SIM2 HDR 47 Engineering display series, Feb 2017
  26. 26.
    V. Hulusic, C. Harvey, K. Debattista, N. Tsingos, S. Walker, D. Howard, A.G. Chalmers, Acoustic rendering and auditory-visual cross-modal perception and interaction. Comput. Graphics Forum 31(1), 102–131 (2012)CrossRefGoogle Scholar
  27. 27.
    N. Tsingos, E. Gallo, G. Drettakis, in Perceptual Audio Rendering of Complex Virtual Environments. SIGGRAPH 2004 (ACM Press, 2004), pp. 249–258Google Scholar
  28. 28.
    V. Hulusíc, C. Harvey, K. Debattista, N. Tsingos, S. Walker, D. Howard, A.G. Chalmers, Acoustic rendering and auditory-visual cross-modal perception and interaction. Comput. Graphics Forum 31(1), 102–131 (2012)Google Scholar
  29. 29.
    A. Saddik, The potential of haptic technologies. IEEE Instrum. Meas. Mag. 10(31), 10–17 (2007)CrossRefGoogle Scholar
  30. 30.
    K.A. Robson Brown, A.G. Chalmers, T. Saigol, C. Green, F. d’Errico, An automated laser scan survey of the Upper Palaeolithic rock shelter of Cap Blanc. J. Archaeol. Sci. 28, 283–289 (2001)CrossRefGoogle Scholar
  31. 31.
    E. Zányi, Y. Chrysanthou, J. Happa, V. Hulusic, A.G. Chalmers, in The High-Fidelity Computer Reconstruction of Byzantine Art. Cyprus IV International Congress of Cypriot Studies, April 2008Google Scholar
  32. 32.
  33. 33.
    J. Aggleton, L. Waskett, The ability of odours to serve as state-dependent cues for real-world memories: can Viking smells aid the recall of Viking experiences? Br. J. Psychol. 90, 1–7 (1999)CrossRefGoogle Scholar
  34. 34.
    H. Abdi, What can cognitive psychology and sensory evaluation learn from each other? Food Qual. Prefer. 13, 445–451 (2002)CrossRefGoogle Scholar
  35. 35.
    C.M. Delahunty, J.R. Piggott, Current methods to evaluate contribution and interactions of components to flavour of solid foods using hard cheese as an example. Int. J. Food Sci. Technol. 30, 555–570 (1995)CrossRefGoogle Scholar
  36. 36.
    E.T. Rolls, Taste, olfactory, and food reward value processing in the brain. Prog. Neurobiol. 127–128, 64–90 (2015). doi: 10.1016/j.pneurobio.2015.03.002 CrossRefGoogle Scholar
  37. 37.
    J. Verhagen, L. Engelen, The neurocognitive bases of human multimodal food perception: sensory integration. Neurosci. Biobehav. Rev. 30, 613–650 (2006)CrossRefGoogle Scholar
  38. 38.
    H. Iwata, H. Yano, T. Uemura, T. Moriya, Food Simulator. ICAT’03: Proceedings of the 13th International Conference on Artificial Reality and Telexistence (IEEE Press, 2003)Google Scholar
  39. 39.
    N. Ranasinghe, A. Cheok, R. Nakatsu, E. Yi-Luen Do, in Simulating the Sensation of Taste for Immersive Experiences. ImmersiveMe ’13 (ACM Multimedia, 2013)Google Scholar
  40. 40.
    T. Narumi, M. Sato, T. Tanikawa, M. Hirose, in Evaluating Cross-Sensory Perception of Superimposing Virtual Color onto Real Drink: Toward Realization of Pseudo-gustatory Displays. 1st Augmented Human Interaction Conference (ACM, 2010)Google Scholar
  41. 41.
    T. Narumi, S. Nishizaka, T. Kajinami, T. Tanikawa, M. Hirose, in MetaCookie+. Virtual Reality Conference (IEEE, 2011)Google Scholar
  42. 42.
    G. Wright, Kalabsha: The Preserving of the Temple (Gebr. Mann Verlag, Berlin, 1972)Google Scholar
  43. 43.
    S. Quirke, The Cult of RA: Sun Worship in Ancient Egypt (Thames and Hudson Distributors LTD, New York, NY, 2001)Google Scholar
  44. 44.
    F. Kahnert, Reproducing the optical properties of fine desert dust aerosols using ensembles of simple model particles. J. Quant. Spectrosc. Radiat. Transfer 85, 231–249 (2003)CrossRefGoogle Scholar
  45. 45.
    V. Sundstedt, D. Gutierrez, F. Gomez, A. Chalmers, in Participating Media for High-Fidelity Cultural Heritage. Proceedings of the Conference on Virtual Reality, Archaeology and Cultural Heritage. Eurographics (VAST’05), 2005Google Scholar
  46. 46.
    D.H. Brown, A.G. Chalmers, A. McNamara, in Light and the Culture of Colour in Medieval Pottery. The International Conference of Medieval and Later Archaeology, Brugge, October 1997Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.WMG, University of WarwickCoventryUK

Personalised recommendations