Virtual Reality

, Volume 9, Issue 4, pp 234–242

The contribution of virtual reality to research on sensory feedback in remote control

Original Article

Abstract

Here we consider research on the kinds of sensory information most effective as feedback during remote control of machines, and the role of virtual reality and telepresence in that research. We argue that full automation is a distant goal and that remote control deserves continued attention and improvement. Visual feedback to controllers has developed in various ways but autostereoscopic displays have yet to be proven. Haptic force feedback, in both real and virtual settings, has been demonstrated to offer much to the remote control environment and has led to a greater understanding of the kinesthetic and cutaneous components of haptics, and their role in multimodal processes, such as sensory capture and integration. We suggest that many displays using primarily visual feedback would benefit from the addition of haptic information but that much is yet to be learned about optimizing such displays.

Keywords

Haptic Feedback Active Passive Kinesthetic Cutaneous Virtual reality Perception 

References

  1. 1.
    Schmitt VR, Morris JW, Jenney GD (1998) Fly-by-wire. Society of Automotive Engineers PhiladelphiaGoogle Scholar
  2. 2.
    Wallace JC, Vodanovich SJ, Restino R (2003) Predicitng cognitive failures from boredom proneness and daytime sleepiness scores: an investigation within military and undergraduate samples. Pers Individ Dif 34:635–644CrossRefGoogle Scholar
  3. 3.
    Diolaiti N, Melchiorri C (2002) Tele-operation of a mobile robot through haptic feedback. In: HAVE, IEEE international workshop on haptic virtual environments and their applicationsGoogle Scholar
  4. 4.
    Bjelland HV, Roed BK, Hoff T (2005) Studies on throttle sticks in high speed crafts—haptics in mechanical, electronic and haptic feedback interfaces. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 509–510Google Scholar
  5. 5.
    Rastogi A, Milgram P, Drascic D (1996) Telerobotic control with stereoscopic augmented reality. In: Bolas M, Fisher S, Merritt J (eds) Stereoscopic displays and virtual reality systems, vol III. Proc SPIE 2635:115–122Google Scholar
  6. 6.
    Gupta R, Sheridan T, Whitney D (1997) Experiments using multimodal virtual environments in design for assembly analysis. Presence Teleop Virtual Environ 6(3):318–338Google Scholar
  7. 7.
    Hainsworth DW (2001) Teleoperation user interfaces for mining robotics. Auton Robots 11(1):19–28CrossRefMATHGoogle Scholar
  8. 8.
    Elhajj I, Xi N, Fung WK, Liu YH, Li WJ, Kaga T, Fukuda T (2001) Haptic information in internet-based teleoperation. IEEE/ASME Transactions on Mechatronics 6(3): 295–304CrossRefGoogle Scholar
  9. 9.
    Roberts JW, Slattery OT, Swope B, Volker M, Comstock T (2002) Small-scale tactile graphics for virtual reality systems. In: Woods AJ, Merrit JO, Benton SA, Bolas MT (eds) Stereoscopic displays and virtual reality systems, IX. Proc SPIE 4660:422–429Google Scholar
  10. 10.
    Grohn M (2002) Is audio useful in immersive visualization? In: Woods AJ, Merrit JO, Benton SA, Bolas MT (eds) Stereoscopic displays and virtual reality systems, IX. Proc SPIE 4660:411–421Google Scholar
  11. 11.
    Edwards GW, Barfield W, Nussbaum MA (2004) The use of force feedback and auditory cues for performance of an assembly task in an immersive virtual environment. Virtual Real 7:112–119CrossRefGoogle Scholar
  12. 12.
    Sallnas EL Rassmus-Grohn K, Sjostrom C (2001) Supporting presence in collaborative environments by haptic feedback. ACM Trans Comput–Hum Interact 7(4):461–476CrossRefGoogle Scholar
  13. 13.
    Kazi A (2001) Operator performance in surgical telemanipulation. Presence 10:495–510CrossRefGoogle Scholar
  14. 14.
    Raspolli M, Avizzano CA, Facenza G, Bergamasco M (2005) HERMES: an angioplasty surgery simulator. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 148–156Google Scholar
  15. 15.
    Jiang L, Girotra R, Cutkosky MR, Ullrich C (2005) Reducing error rates with low-cost haptic feedback in virtual reality-based training applications. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 420–425Google Scholar
  16. 16.
    White BW, Saunders FA, Scadden L, Bach-y-Rita P, Collins CC (1970) Seeing with the skin. Percept Psychophys 7:23–27Google Scholar
  17. 17.
    Kennedy JM, Richardson BL, Magee LE (1980) The nature of haptics. In: Hagen M (ed) The perception of pictures. Academic, New YorkGoogle Scholar
  18. 18.
    Baier H, Buss M, Freyberger F, Hoogen J, Kammermeier P, Schmidt G (1999) Distributed PC-based haptic, visual and acoustic telepresence system experiments in virtual and remote environments. In: Proceedings of IEEE virtual reality conference, p 118Google Scholar
  19. 19.
    Miner N, Gillespie B, Caudell T (1996) Examining the influence of audio and visual stimuli on a haptic interface. In: Proceedings IMAGE conference, pp 23–35Google Scholar
  20. 20.
    Grane C, Bengtsson P (2005) Menu selection with a rotary device founded on haptic and/or graphic information. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 475–476Google Scholar
  21. 21.
    Palmer S (2002) Vision sciences. Bradford Books CambridgeGoogle Scholar
  22. 22.
    Goldstein BE (1999) Sensation and perception 5th edn. Brooks Cole, Pacific GroveGoogle Scholar
  23. 23.
    Bradshaw MF, Elliot KM, Watt SJ, Davies IR (2002) Do observers exploit binocular disparity information in motor tasks within dynamic telepresence environments? In: Woods AJ, Merrit JO, Benton SA, Bolas MT (eds) Stereoscopic displays and virtual reality systems, IX. Proc SPIE 4660:331–342Google Scholar
  24. 24.
    Schmit A, Grasnik A (2002) Multiviewpoint autostereoscopic displays from 4D-vision. In: Woods AJ, Merrit JO, Benton SA, Bolas MT (eds) Stereoscopic displays and virtual reality systems, IX. Proc SPIE 4660:212–221Google Scholar
  25. 25.
    Perlin K, Paxia S, Kollin J (2000) An autostereoscopic display. In: Proceedings of SIG-GRAPH, ACM conference on computer graphics and interactive techniques, pp 319–326Google Scholar
  26. 26.
    McKnight S, Melder N, Barrow AL, Harwin WS, Wann JP (2005) Perceptual cues for orientation in a two finger haptic grasp task. In: Proceedings of World Haptics Conference. IEEE Computer Society, Los Alamitos, pp 549–550Google Scholar
  27. 27.
    Richardson BL, Wuillemin DB, Symmons MA (2004) Sensory feedback and remote control of machines in mining and extraterrestrial environments. J Aust Inst Mining Metall 2:53–56Google Scholar
  28. 28.
    Woods A (2003) Seeing in depth at depth. Newsletter of the Centre for Marine Science & Technology, Sept. Available at: http://www.cmst.curtin.edu.au/brochures/cmstnewsletter4.pdf
  29. 29.
    Kugah DA (1972) Experiments evaluating compliance and force feedback effect on manipulator performance. Genral Elec Corp NASA – CR 128605 PhiladelphiaGoogle Scholar
  30. 30.
    Gunn C, Hutchins M, Adcock M, Hawkins R (2003) Trans-world Haptic collaboration, In: Proceedings of the SIGGRAPH Conference, Sketches and Applications, p 1Google Scholar
  31. 31.
    Oakley I, O’Modhrain S (2005) Tilt to scroll: evaluating a motion based vibrotactile mobile interface. In: Proceedings of World Haptics Conference. IEEE Computer Society, Los Alamitos, pp 40–49Google Scholar
  32. 32.
    Nojima T, Funabiki K (2005) Cockpit display using tactile sensations. In: Proceedings of World Haptics Conference. IEEE Computer Society, Los Alamitos, pp 501–502Google Scholar
  33. 33.
    Bhargava A, Scott M, Traylor R, Chung R, Mrozek K, Wolter J, Tan HZ (2005) Effect of cognitive load on tactor location identification in zero-g. In: Proceedings of World Haptics Conference. IEEE Computer Society, Los Alamitos, pp 56–62Google Scholar
  34. 34.
    Rauterberg M (1999) New directions in user-system interaction: augmented reality, ubiquitous and mobile computing. In: Proceedings of IEEE symposium on human interfacing, pp 105–133Google Scholar
  35. 35.
    Lindeman RW, Sibert JL, Mendez-Mendez E, Patil S, Phifer D (2005) Effectiveness of directional vibrotactile cuing on a building-clearing task. In: Proceedings of ACM CHI, pp 271–280Google Scholar
  36. 36.
    Richardson BL, Wuillemin DB, Saunders F (1978) Tactile discrimination of competing sounds. Percept Psychophys 24: 546–550PubMedGoogle Scholar
  37. 37.
    Hoffman H, Groen J, Rousseau S, Hollander A, Winn W, Wells M, Furness T (1996) Tactile augmentation: enhancing presence in virtual reality with tactile feedback from real objects. In: Meeting of the American Psychological Society, San Francisco. Available at: http://www.hitl.washington.edu/publications/p-96–1/
  38. 38.
    McGrath BJ, Estrada A, Braithwaite MG, Raj AK, Rupert AH, (2004). Tactile situation awareness system flight demonstration final report USAARL Report 2004–10, MarchGoogle Scholar
  39. 39.
  40. 40.
    Richardson BL, Wuillemin DB, Symmons MA, Accardi R (2005) The Exograsp delivers tactile and kinaesthetic information about virtual objects. In: IEEE Tencon conference, November, MelbourneGoogle Scholar
  41. 41.
    Kammermeier P, Kron A, Hoogen J, Schmit G (2004) Display of holistic sensation by combined tactile and kinaesthetic feedback. Presence 13(1):1–15CrossRefGoogle Scholar
  42. 42.
    Seay AF, Krum DM, Hodges L, Ribarsky W (2001) Simulator sickness and presence in a high FOV virtual environment. In: Proceedings of the virtual reality 2001 conference, pp 299–300Google Scholar
  43. 43.
    Stott R (2002) Interaction between the senses: vision and the vestibular system. In: Roberts D (ed) Signals and perception. Palgrave Macmillan, New YorkGoogle Scholar
  44. 44.
    McGurk H, MacDonald T (1976) Hearing lips and seeing voices. Nature 264:746–748CrossRefPubMedGoogle Scholar
  45. 45.
    Kohlers P, von Grunau M (1976) Shape and color in apparent motion. Vis Res 16:329–335CrossRefPubMedGoogle Scholar
  46. 46.
    Soto-Faraco S, Spence C, Kingstone A (2004) Cross-modal dynamic capture: congruency effects in the perception of motion across sensory modalities. J Exp Psychol Hum Percept Perform 30(2):330–345CrossRefGoogle Scholar
  47. 47.
    Shiffman HR (1996) Sensation and perception, 4th edn. Wiley, New YorkGoogle Scholar
  48. 48.
    Heller MA (1983) Haptic dominance in form perception with blurred vision. Perception 122:607–613CrossRefGoogle Scholar
  49. 49.
    Soto-Faraco S, Kingstone A (2004) Multisensory integration of dynamic information. In: Calvert G, Spence C, Stein B (eds) Handbook of multisensory processes. MIT Press CambridgeGoogle Scholar
  50. 50.
    Caclin A, Soto-Faraco S, Kingstone A, Spence C (2002) Tactile “capture” of audition. Percept Psychophys 18:55–60Google Scholar
  51. 51.
    Richardson BL, Symmons MA, Accardi R (2000) The TDS: A new device for comparing active and passive-guided touch. IEEE Trans Rehabilit Eng 8:414–417CrossRefGoogle Scholar
  52. 52.
    Symmons MA, Richardson BL, Wuillemin DB, VanDoorn GH (2005) Kinaesthetic and cutaneous contributions to raised-line stimulus interpretation. In: World haptics conference, 18–20 March, Pisa. Video clip at http://www-personal.monash.edu.au/~msymmons/images/6_9_qt.mov
  53. 53.
    Magee LE, Kennedy JM (1980) Exploring pictures tactually. Nature (London) 283:287CrossRefGoogle Scholar
  54. 54.
    Richardson BL, Symmons M, Wuillemin DB (2004) The relative importance of cutaneous and kinesthetic cues in raised line drawing exploration. In: Ballesteros S, Heller MA (eds) Touch, blindness, and neuroscience. Universidad Nacional de Educación a Distancia, Madrid, pp 247–250Google Scholar
  55. 55.
    Symmons M, Richardson BL, Wuillemin DB (2004) Active versus passive touch: Superiority depends more on the task than the mode. In: Ballesteros S, Heller MA (eds) Touch, blindness, and neuroscience. Universidad Nacional de Educación a Distancia, Madrid, pp 179–185Google Scholar
  56. 56.
    Wuillemin DB, VanDoorn GH, Richardson BL, Symmons MA (2005) Haptic and visual size judgements in virtual and real environments. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 86–89Google Scholar
  57. 57.
    Sarter N (1998) Turning automation into a teamplayer: The development of multisensory and graded feedback for highly automated (flight deck) systems. Willard Airport Aviation Research Lab, http://www.nsfworkshop.engr.ucf.edu/papers/Sarter.asp/
  58. 58.
    Lee S, Sukhatme GS, Kim GJ, Park C (2005) Haptic teleoperation of a mobile robot: a user study. Presence 14(3):345–365CrossRefGoogle Scholar
  59. 59.
    Bliss JC, Katcher MH, Roger CH, Shepard R (1970) Optical-to-tactile image conversion for the blind. IEEE Trans Man–Mach Syst MMS-11:58–64CrossRefGoogle Scholar
  60. 60.
    Brooks PL, Frost BJ, Mason JL, Gibson DM (1987) Word and feature identification by profoundly deaf teenagers using the Queen’s University Tactile Vocoder. J Speech Hear Res 30:137–141PubMedGoogle Scholar
  61. 61.
    Geldard FA (1966) Cutaneous coding of optical signals: the Optohapt. Percept Psychophys 1:377–381Google Scholar
  62. 62.
    Richardson BL, Frost BJ (1977) Sensory substitution and the design of an artificial ear. J Psychol 96:259–285PubMedGoogle Scholar
  63. 63.
    Gallace A, Tan HZ, Spence C (2005) Tactile change detection. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 12–16Google Scholar
  64. 64.
    van Erp JBF (2005) Vibrotactile spatial acuity on the torso: effects of location and timing parameters. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 80–85Google Scholar

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  • Barry Richardson
    • 1
  • Mark Symmons
    • 1
  • Dianne Wuillemin
    • 1
  1. 1.Bionics and Cognitive Science CentreMonash UniversityChurchillAustralia

Personalised recommendations