Abstract
The addition of multimodal feedback during navigation in a virtual environment is fundamental when aiming at fully immersive and realistic simulations. Several visual, acoustic, haptic or vibrotactile perceptual cues can be generated when walking over a ground surface. Such sensory feedback can provide crucial and varied information regarding either the ground material itself, the properties of the ground surface such as slope or elasticity, the surrounding environment, the specificities of the foot-floor interaction such as gait phase or forces, or even users’ emotions. This chapter addresses the multimodal rendering of walking over virtual ground surfaces, incorporating haptic, acoustic and graphic rendering to enable truly multimodal walking experiences.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Adjémian F, Evesque P (2002) Different regimes of stick-slip in granular matter: from quasi- periodicity to randomness. In: EGS XXVII general assembly
Alava MJ, Nukala PKVV, Zapperi S (2006) Statistical models of fracture. Adv Phys 55(3):349–476
Avanzini F, Rocchesso D (2001) Modeling collision sounds: non-linear contact force. In: Proceedings of COST-G6 conference on digital audio effects, pp 61–66
Avanzini F, Serafin S, Rocchesso D (2005) Interactive simulation of rigid body interaction with friction-induced sound generation. IEEE Trans Speech Audio Process 13(5):1073–1081
Baldassarri A, Dalton F, Petri A, Zapperi S, Pontuale G, Pietronero L (2006) Brownian forces in sheared granular matter. Phys Rev Lett 96(11):118002
Barrass S, Adcock M (2002) Interactive granular synthesis of haptic contact sounds. In: Proceedings of AES. 22nd international conference on virtual, synthetic and entertainment audio
Blesser B (2001) An interdisciplinary synthesis of reverberation viewpoints. J Audio Eng Soc 49(10):867–903
Bresciani J-P, Ernst MO, Drewing K, Bouyer G, Maury V, Kheddar A (2005) Feeling what you hear: auditory signals can modulate tactile tap perception. Exp Brain Res 162:172–180
Burdea GC, Akay M (1996) Force and touch feedback for virtual reality. Wiley, New York
Christensen RR, Hollerbach JM, Xu Y, Meek SG (2000) Inertial-force feedback for the treadport locomotion interface. Presence: Teleoper Virtual Environ 9(1):1–14
Chueng P (2002) Designing sound canvas: the role of expectation and discrimination. In: Proceedings of ACM CHI extended abstracts on human factors in computing systems. pp 848–849
Cirio G, Marchal M, Hillaire S, Lécuyer A (2011) Six degrees-of-freedom haptic interaction with fluids. IEEE Trans Visual Comput Graph 17(11):1714–1727
Cirio G, Marchal M, Lécuyer A, Cooperstock J (2012) Vibrotactile rendering of splashing fluids. IEEE Trans Haptics 99:1
Cook PR (2002) Modeling Bill’s gait: analysis and parametric synthesis of walking sounds. In: Proceedings of audio engineering society 22nd conference on virtual, synthetic and entertainment audio, Espoo, Finland
Cook PR (2002) Real sound synthesis for interactive applications. AK Peters, Natick
Courtney A, Chow HM (2000) A study of tile design for tactile guide pathways. Int J Ind Ergon 25(6):693–698
Cress DH (1978) Terrain considerations and data base development for the design and testing of devices to detect intruder-induced ground motion. Technical report M-78-1, U.S. Army Engineer Waterways Experiment Station, Vicksburg, Mississippi
Crossan A, Williamson J, Murray-Smith R (2004) Haptic granular synthesis: targeting, visualisation and texturing. In: Proceedings of international symposium on non-visual & multimodal visualization, pp 527–532
Dalton F, Farrelly F, Petri A, Pietronero L, Pitolli L, Pontuale G (2005) Shear stress fluctuations in the granular liquid and solid phases. Phys Rev Lett 95(13):138001
Darken RP, Cockayne WR, Carmein D (1997) The omni-directional treadmill: a locomotion device for virtual worlds. In: Proceedings of the 10th annual ACM symposium on user interface software and technology. pp 213–221
Delle Monache S, Polotti P, Rocchesso D (2010) A toolkit for explorations in sonic interaction design. In: Proceedings of the ACM 5th audio mostly conference. pp 1–7
Dickstein R, Plax M (2012) Metronome rate and walking foot contact time in young adults. Percept Mot Skills 114(1):21–28
Dixon SJ, Collop AC, Batt ME (2000) Surface effects on ground reaction forces and lower extremity kinematics in running. Med Sci Sports Exerc 32(11):1919–1926
Dixon SJ, Cooke A (2004) Shoe-surface interaction in tennis. In: Hung GK, Pallis JM (eds) Biomedical engineering principles in sports. Kluwer Academic/Plenum Publishers, New York, p 125
Ekimov A, Sabatier J (2006) Vibration and sound signatures of human footsteps in buildings. J Acoust Soc Am 120:762
Ekimov A, Sabatier J (2008) A review of human signatures in urban environments using acoustic and seismic methods. In: Proceedings of IEEE technologies for homeland security
Farnell AJ (2007) Marching onwards—procedural synthetic footsteps for video games and animation. In: Pd convention
Farrington SM (1998) Invariant acoustic cues for determining source characteristics of footstep sounds. Master’s thesis, State University of New York
Ferris DP, Louie M, Farley CT (1998) Running in the real world: adjusting leg stiffness for different surfaces. Proc R Soc B: Biol Sci 265(1400):989
Fontana F, Bresin R (2003) Physics-based sound synthesis and control: crushing, walking and running by crumpling sounds. In: Proceedings of colloquium on musical informatics, Florence, Italy, pp 109–114
Fontana F, Morreale F, Regia-Corte T, Lécuyer A, Marchal M (2011) Auditory recognition of floor surfaces by temporal and spectral cues of walking. In: Proceedings of international conference on auditory display
Freeman J, Lessiter J (2001) Hear there & everywhere: the effects of multi-channel audio on presence. In: Proceedings of ICAD 2001, pp 231–234
Fukumoto M, Sugimura T (2001) Active click: tactile feedback for touch panels. In: Proceedings of ACM CHI extended abstracts. pp 121–122
Funkhouser T, Tsingos N, Jot JM (2003) Survey of methods for modeling sound propagation in interactive virtual environment systems. Presence
Galbraith F, Barton M (1970) Ground loading from footsteps. J Acoust Soc Am 48:1288–1292
Gaver WW (1993) What in the world do we hear? An ecological approach to auditory event perception. Ecol Psychol 5(1):1–29
Gibson JJ (1977) The theory of affordances. In: Erlbaum L (eds) Perceiving, Acting, and Knowing. Erlbaum, Hillsdale
Giordano BL, McAdams S, Rocchesso D (2010) Integration of acoustical information in the perception of impacted sound sources: the role of information accuracy and exploitability. J Exp Psychol: Hum Percept Perform 36(2):462–476
Giordano BL, McAdams S, Visell Y, Cooperstock J, Yao H-Y, Hayward V (2008) Non-visual identification of walking grounds. J Acous Soc Am 123(5):3412
Hachisu T, Cirio G, Marchal M, Lécuyer A, Kajimoto H (2011) Pseudo-haptic feedback augmented with visual and tactile vibrations. In: Proceedings of international symposium on VR innovations, pp 331–332
Hardin EC, van den Bogert AJ, Hamill J (2004) Kinematic adaptations during running: effects of footwear, surface, and duration. Med Sci Sports Exerc 36(5):838
Hayward V (2007) Physically-based haptic synthesis. In: Lin M, Otaduy M (eds) Haptic rendering: foundations, algorithms and applications. AK Peters, Natick
Hayward V, Astley OR, Cruz-Hernandez M, Grant D, Robles-De-La-Torre G (2004) Haptic interfaces and devices. Sens Rev 24(1):16–29
Hayward V, Maclean K (2007) Do it yourself haptics, part I. IEEE Robot Autom 14:88–104
Herrmann HJ, Roux S (1990) Statistical models for the fracture of disordered media. North Holland, Amsterdam
Hollerbach J (2008) Locomotion interfaces and rendering. In: Lin M, Otaduy M (eds) Haptic rendering: foundations, algorithms and applications. AK Peters, Natick
Hötting K, Röder B (2004) Hearing cheats touch, but less in congenitally blind than in sighted individuals. Psychol Sci 15:60–64
Iwata H (1999) Walking about virtual environments on an infinite floor. In: Proceedings of IEEE international conference on virtual reality, pp 286–293
Iwata H (2008) Haptic interface. In: Sears A, Jacko JA (eds) The human-computer interaction handbook, 2nd edn. Lawrence Erlbaum Assoc, New York
Kennedy PM, Inglis JT (2002) Distribution and behaviour of glabrous cutaneous receptors in the human foot sole. J Physiol 583(3):731–742
Kinsella-Shaw JM, Shaw B, Turvey MT (1992) Perceiving “walk-on-able” slopes. Ecol Psychol 4(4):223–239
Klatzky RL, Pai DK, Krotkov EP (2000) Perception of material from contact sounds. Presence: Teleoper Virtual Environ 9(4):399–410
Kobayashi Y, Osaka R, Hara T, Fujimoto H (2008) How accurately people can discriminate the differences of floor materials with various elasticities. IEEE Trans Neural Syst Rehabil Eng 16(1):99–105
Kobayashi Y, Takashima T, Hayashi M, Fujimoto H (2005) Gait analysis of people walking on tactile ground surface indicators. IEEE Trans Neural Syst Rehabil Eng 13(1):53–59
Kuchenbecker KJ, Fiene J, Niemeyer G (2006) Improving contact realism through event-based haptic feedback. IEEE Trans Visual Comput Graph 12(2):219–230
LaViola JJ, Feliz DA, Keefe DF, Zeleznik RC (2001) Hands-free multi-scale navigation in virtual environments. In: Proceedings of the ACM symposium on interactive 3D graphics. pp 9–15
Law AW, Ip J, Peck B, Visell Y, Kry PG, Cooperstock JR (2009) Multimodal floor for immersive environments. In: ACM SIGGRAPH emerging technologies
Lécuyer A (2009) Simulating haptic feedback using vision: a survey of research and applications of pseudo-haptic feedback. Presence: Teleoper Virtual Environ 18(1):39–53
Lécuyer A, Burkhardt J-M, Etienne L (2004) Feeling bumps and holes without a haptic interface: the perception of pseudo-haptic textures. In: Proceedings of SIGCHI conference on human factors in computing systems, pp 239–246
Lécuyer A, Burkhardt J-M, Henaff J-M, Donikian S (2006) Camera motions improve sensation of walking in virtual environments. In: Proceedings of IEEE international conference on virtual reality, pp 11–18
Li X, Logan RJ, Pastore RE (1991) Perception of acoustic source characteristics: walking sounds. J Acoust Soc Am 90(6):3036–3049
MacLean KE (2008) Foundations of transparency in tactile information design. IEEE Trans Haptics 1(2):84–95
MacLellan MJ, Patla AE (2006) Adaptations of walking pattern on a compliant surface to regulate dynamic stability. Exp Brain Res 173(3):521–530
Mahvash M, Hayward V (2001) Haptic rendering of cutting: a fracture mechanics approach. Haptics-e 2(3):1–12
Marchal M, Lécuyer A, Cirio G, Bonnet L, Emily M (2010) Walking up and down in immersive virtual worlds: novel interactive techniques based on visual feedback. In: Proceedings of IEEE symposium on 3D user interface, pp 19–26
Marigold DS, Patla AE (2005) Adapting locomotion to different surface compliances: neuromuscular responses and changes in movement dynamics. J Neurophysiol 94(3):1733
Michon JA (ed) (1993) Generic intelligent driver support. CRC Press, Boca Raton
Morioka M, Whitehouse DJ, Griffin MJ (2008) Vibrotactile thresholds at the fingertip, volar forearm, large toe, and heel. Somatosens Mot Res 25(2):101–112
Moritz CT, Farley CT (2004) Passive dynamics change leg mechanics for an unexpected surface during human hopping. J Appl Physiol 97(4):1313
Moritz CT, Greene SM, Farley CT (2004) Neuromuscular changes for hopping on a range of damped surfaces. J Appl Physiol 96(5):1996
Nashel A, Razzaque S (2003) Tactile virtual buttons for mobile devices. In: Proceedings of ACM CHI. pp 854–855
Nasuno S, Kudrolli A, Bak A, Gollub JP (1998) Time-resolved studies of stick-slip friction in sheared granular layers. Phys Rev E 58(2):2161–2171
Nasuno S, Kudrolli A, Gollub JP (1997) Friction in granular layers: hysteresis and precursors. Phys Rev Lett 79(5):949–952
Nordahl R (2006) Increasing the motion of users in photorealistic virtual environments by utilizing auditory rendering of the environment and ego-motion. In: Proceedings of presence, pp 57–62
Norman D (2007) The design of future things. Basic Books, New York
Papetti S, Civolani M, Fontana F (2011) Rhythmnshoes: a wearable foot tapping interface with audio-tactile feedback. In: Proceedings of international conference of new interfaces for musical expression
Papetti S, Fontana F, Civolani M, Berrezag A, Hayward V (2010) Audio-tactile display of ground properties using interactive shoes. In: Haptic and audio interaction design, pp 117–128
Pastore RE, Flint JD, Gaston JR, Solomon MJ (2008) Auditory event perception: the source-perception loop for posture in human gait. Percept Psychophys 70(1):13–29
Perreault S, Gosselin CM (2008) Cable-driven parallel mechanisms: application to a locomotion interface. J Mech Des 130:102301
Poupyrev I, Maruyama S, Rekimoto J (2002) Ambient touch: designing tactile interfaces for handheld devices. In: Proceedings of ACM UIST. pp 51–60
Razzaque S, Kohn Z, Whitton MC (2001) Redirected walking. In: Proceedings of Eurographics, pp 289–294
Rosburg T (2008) Tactile ground surface indicators in public places. In: Grunwald M (ed) Human haptic perception: basics and applications. Birkhäuser, Basel, pp 491–499
Rossignol S, Dubuc R, Gossard JP (2006) Dynamic sensorimotor interactions in locomotion. Physiol Rev 86(1):89
Rovan J, Hayward V (2000) Typology of tactile sounds and their synthesis in gesture-driven computer music performance. In: Wanderley M, Battier M (eds) Trends in gestural control of music. Editions IRCAM, Paris
Rovers AF, van Essen H (2006) Guidelines for haptic interpersonal communication applications. Virtual Real 9:177–191
Sanders RD, Scorgie MA (2002) The effect of sound delivery methods on a user’s sense of presence in a virtual environment
Schafer RM (1977) The tuning of the world. Random House, New York
Serafin S, Fontana F, Turchet L, Papetti S (2012) Auditory rendering and display of interactive floor cues. In: Fontana F, Visell Y (eds) Walking with the senses, Chap 7. Logos Verlag, Berlin, pp 123–152
Serafin S, Turchet L, Nordahl R, Dimitrov S, Berrezag A, Hayward V (2010) Identification of virtual grounds using virtual reality haptic shoes and sound synthesis. In: Proceedings of Eurohaptics, p 61
Slater M, Usoh M, Steed A (1995) Taking steps: the influence of a walking technique on presence in virtual reality. ACM Trans Comput-Hum Inter 2(3):201–219
Srinivasan MA, Basdogan C (1997) Haptics in virtual environments: taxonomy, research status, and challenges. Comput Graph 21(4):393–404
Stiles VH, James IT, Dixon SJ, Guisasola IN (2009) Natural turf surfaces: the case for continued research. Sports Med 39(1):65
Styns F, van Noorden L, Moelants D, Leman M (2007) Walking on music. Hum Movem Sci 26(5):769–785
Swapp D, Williams J, Steed A (2010) The implementation of a novel walking interface within an immersive display. In: Proceedings of IEEE Symposium on 3D user interfaces, pp 71–74
Tax A, van Wezel B, Dietz V (1995) Bipedal reflex coordination to tactile stimulation of the sural nerve during human running. J Neurophysiol 73(5):1947–1964
Templeman JN, Denbrook PS, Sibert LE (1999) Virtual locomotion: walking in place through virtual environments. Presence: Teleoper Virtual Environ 8(6):598–617
Terziman L, Lécuyer A, Hillaire S, Wiener JM (2009) Can camera motions improve the perception of traveled distance in virtual environments? In: Proceedings of IEEE international conference on virtual reality, pp 131–134
Terziman L, Marchal M, Multon F, Arnaldi B, Lécuyer A (2012) The king-kong effects: improving sensation of walking in VR with visual and tactile vibrations at each step. In: Proceedings of IEEE symposium on 3D user interfaces, pp 19–26
Tramberend H, Hasenbrink F, Eckel G, Lechner U, Goebel M (1997) CyberStage—an advanced virtual environment. ERCIM News 31:22–23
Trulsson M (2001) Mechanoreceptive afferents in the human sural nerve. Exp Brain Res 137(1):111–116
Turchet L, Marchal M, Lécuyer A, Serafin S, Nordahl R (2010) Influence of visual feedback for perceiving walking over bumps and holes in desktop VR. In: Proceedings of 17th ACM symposium on virtual reality software and technology, pp 139–142
Turchet L, Serafin S, Nordahl R (2010) Physically based sound synthesis and control of footsteps sounds. In: Proceedings of conference on digital audio effects
van den Doel K, Kry PG, Pai DK (2001) FoleyAutomatic: physically-based sound effects for interactive simulation and animation. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques pp 537–544
van Ulzen NR, Lamoth CJC, Daffertshofer A, Semin GR, Beek PJ (2008) Characteristics of instructed and uninstructed interpersonal coordination while walking side-by-side. Neurosci Lett 432(2):88–93
Velazquez R (2008) On-shoe tactile display. In: Proceedings of the IEEE international workshop on haptic audio visual environment and their applications
Visell Y, Cooperstock J (2010) Design of a vibrotactile display via a rigid surface. In: Proceedings of IEEE haptics symposium, pp 133–140
Visell Y, Cooperstock J, Giordano BL, Franinovic K, Law A, McAdams S, Jathal K, Fontana F (2008) A vibrotactile device for display of virtual ground materials in walking. In: Proceedings of Eurohaptics 2008, pp 420–426
Visell Y, Fontana F, Giordano BL, Nordahl R, Serafin S, Bresin R (2009) Sound design and perception in walking interactions. Int J Hum-Comput Studies 67:947–959
Visell Y, Giordano BL, Millet G, Cooperstock JR (2011) Vibration influences haptic perception of surface compliance during walking. PLoS one 6(3):e17697
Visell Y, Law A, Cooperstock JR (2009) Touch is everywhere: floor surfaces as ambient haptic interfaces. IEEE Trans Haptics 2:148–159
Visell Y, Law A, Smith S, Rajalingham R, Cooperstock JR (2010) Contact sensing and interaction techniques for a distributed multimodal floor display. In: Proceedings of IEEE Symposium on 3D user interfaces. pp 75–78
Visell Y, Law A, Ip J, Smith S, Cooperstock JR (2010) Interaction capture in immersive virtual environments via an intelligent floor surface. In: Proceedings of IEEE international conference on virtual reality, pp 313–314
Watters BG (1965) Impact noise characteristics of female hard-heeled foot traffic. J Acoust Soc Am 37:619–630
Weinstein S (1968) Intensive and extensive aspects of tactile sensitivity as a function of body part, sex, and laterality. In: Kenshalo DR (ed) The skin senses. Thomas, Springfield
Wells C, Ward LM, Chua R, Inglis JT (2003) Regional variation and changes with ageing in vibrotactile sensitivity in the human footsole. J Gerontol A Biol Sci Med Sci 58(8):B680–B686
Zehr EP, Stein RB, Komiyama T (1998) Function of sural nerve reflexes during human walking. J Physiol 507:305–314
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this chapter
Cite this chapter
Marchal, M. et al. (2013). Multimodal Rendering of Walking Over Virtual Grounds. In: Steinicke, F., Visell, Y., Campos, J., Lécuyer, A. (eds) Human Walking in Virtual Environments. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-8432-6_12
Download citation
DOI: https://doi.org/10.1007/978-1-4419-8432-6_12
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-8431-9
Online ISBN: 978-1-4419-8432-6
eBook Packages: EngineeringEngineering (R0)
