Haptic Augmentation in Soft Tissue Interaction

  • Seokhee Jeon
  • Seungmoon Choi
  • Matthias Harders
Part of the Springer Series on Touch and Haptic Systems book series (SSTHS)


Haptic augmented reality is a new paradigm in human—computer interaction. Akin to traditional visual augmented reality, the technique strives to combine real and virtual sensory stimuli to alter perception during object manipulation. In the context of soft tissue interaction the stimuli felt during contact and indentation of a deformable object are overlaid with forces generated with a haptic device. Such a combined rendering can provide a user with an altered percept of object properties and/or shape. This chapter first outlines the general concept of integrating haptics into augmented reality. Thereafter, we will introduce two heuristic algorithms for haptic augmentation—covering stiffness modulation at either one or two contact points. This will address topics of parameter estimation, contact detection and augmentation computation. Finally, an application example is given in the context of tissue palpation. A method for augmenting virtual stiffer inclusions in physical soft tissue samples is presented.


Augmented Reality Real Object Haptic Device Mixed Reality Haptic Interface 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The described research has been supported in parts by Korean government programs, NRL R0A-2008-000-20087-0 from NRF and ITRC (NIPA-2011-)C1090-1111-0008 from NIPA, the Korean-Swiss Science and Technology Cooperation program 2010/2011, and the EU project BEAMING 248620.


  1. Bau O, Poupyrev I, Israr A, Harrison C (2010) Teslatouch: electrovibration for touch surfaces. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology, UIST ’10, pp 283–292Google Scholar
  2. Bayart B, Didier JY, Kheddar A (2008) Force feedback virtual painting on real objects: a paradigm of augmented reality haptics. Lect Notes Comput Sci (EuroHaptics 2008) 5024:776–785Google Scholar
  3. Bayart B, Drif A, Kheddar A, Didier JY (2007) Visuo-haptic blending applied to a tele-touch-diagnosis application. Lect Notes Comput Sci (Eurohaptics 2007) 4563:617–626Google Scholar
  4. Bayart B, Kheddar A (2006) Haptic augmented reality taxonomy: haptic enhancing and enhanced haptics. In: Proceedings of EuroHaptics, pp 641–644Google Scholar
  5. Bennett E, Stevens B (2006) The effect that the visual and haptic problems associated with touching a projection augmented model have on object-presence. Presence: Teleoperators Virtual Environ 15(4):419–437CrossRefGoogle Scholar
  6. Bianchi G, Jung C, Knörlein B, Székely G, Harders M (2006a) High-fidelity visuo-haptic interaction with virtual objects in multi-modal AR systems. In: Proceedings of the IEEE and ACM international symposium on mixed and augmented reality, pp 187–196Google Scholar
  7. Bianchi G, Knörlein B, Székely G, Harders M (2006b) High precision augmented reality haptics. In: Proceedings of EuroHaptics, pp 169–168Google Scholar
  8. Billinghurst M, Kato H, Poupyrev I (2001) The magic book-moving seamlessly between reality and virtuality. IEEE Comput Graph Appl 21(3):6–8Google Scholar
  9. Borst CW, Volz RA (2005) Evaluation of a haptic mixed reality system for interactions with a virtual control panel. Presence: Teleoperators Virtual Environ 14(6):677–696CrossRefGoogle Scholar
  10. Ha T, Chang Y, Woo W (2007) Usability test of immersion for augmented reality based product design. Lect Notes Comput Sci (Edutainment 2007) 4469:152–161Google Scholar
  11. Haddadi A, Hashtrudi-Zaad K (2008) A new method for online parameter estimation of Hunt-Crossley environment dynamic models. In: Proceedings of the IEEE international conference on intelligent robots and systems, pp 981–986Google Scholar
  12. Harders M, Bianchi G, Knörlein B, Székely G (2009) Calibration, registration, and synchronization for high precision augmented reality haptics. IEEE Trans Vis Comput Graph 15(1):138–149CrossRefGoogle Scholar
  13. Hunt K, Crossley F (1975) Coefficient of restitution interpreted as damping in vibroimpact. ASME J Appl Mech 42:440–445CrossRefGoogle Scholar
  14. Janabi-Sharifi F, Hayward V, Chen CSJ (2000) Discrete-time adaptive windowing for velocity estimation. IEEE Trans Control Syst Technol 8(6):1003–1009CrossRefGoogle Scholar
  15. Jeon S, Choi S (2008) Modulating real object stiffness for haptic augmented reality. Lect Notes Comput Sci (EuroHaptics 2008) 5024:609–618Google Scholar
  16. Jeon S, Choi S (2009) Haptic augmented reality: taxonomy and an example of stiffness modulation. Presence: Teleoperators Virtual Environ 18(5):387–408CrossRefGoogle Scholar
  17. Jeon S, Choi S (2010) Stiffness modulation for haptic augmented reality: extension to 3D interaction. In: Proceedings of the haptics symposium, pp 273–280Google Scholar
  18. Jeon S, Choi S (2011) Real stiffness augmentation for haptic augmented reality. Presence: Teleoperators Virtual Environ 20(4):337–370CrossRefMathSciNetGoogle Scholar
  19. Jeon S, Choi S, Harders M (2012) Rendering virtual tumors in real tissue mock-ups using haptic augmented reality. IEEE Trans Haptics 5(1):77–84CrossRefGoogle Scholar
  20. Jeon S, Harders M (2012) Extending haptic augmented reality: modulating stiffness during two-point squeezing. In: Proceedings of the haptics symposium, pp 141–146Google Scholar
  21. Kajimoto H, Kawakami N, Tachi S, Inami M (2004) SmartTouch: electric skin to touch the untouchable. IEEE Comput Graph Appl 24(1):36–43CrossRefGoogle Scholar
  22. Kyung KU, Lee JY (2009) Ubi-Pen: a haptic interface with texture and vibrotactile display. IEEE Comput Graph Appl 29(1):24–32CrossRefGoogle Scholar
  23. Lee C, Adelstein BD, Choi S (2008) Haptic weather. In: Proceedings of the symposium on haptic interfaces for virtual environments and teleoperator systems, pp 473–474Google Scholar
  24. Lee J, Choi S (2010) Effects of haptic guidance and disturbance on motor learning: potential advantage of haptic disturbance. In: Proceedings of the IEEE haptics symposium (HS), pp 335–342Google Scholar
  25. Mahvash M, Okamura AM (2006) Friction compensation for a force-feedback telerobotic system. In: Proceedings of the IEEE international conference on robotics and automation, pp 3268–3273Google Scholar
  26. Milgram P, Colquhoun H Jr (1999) A taxonomy of real and virtual world display integration. In: Tamura Y (ed) Mixed reality-merging real and virtual worlds. Springer, Berlin, pp 1–16Google Scholar
  27. Nojima T, Sekiguchi D, Inami M, Tachi S (2002) The SmartTool: a system for augmented reality of haptics. In Proceedings of the IEEE virtual reality conference, pp 67–72Google Scholar
  28. Ott R, Thalmann D, Vexo F (2007) Haptic feedback in mixed-reality environment. Vis Comput: Int J Comput Graph 23(9):843–849CrossRefGoogle Scholar
  29. Sandor C, Uchiyama S, Yamamoto H (2007) Visuo-haptic systems: half-mirrors considered harmful. In: Proceedings of the world haptics conference, pp 292–297Google Scholar
  30. Scharver C, Evenhouse R, Johnson A, Leigh J (2004) Designing cranial implants in a haptic augmented reality environment. Commun ACM 47(8):32–38CrossRefGoogle Scholar
  31. Vallino JR, Brown CM (1999) Haptics in augmented reality. In: Proceedings of the IEEE international conference on multimedia computing and systems, pp 195–200Google Scholar
  32. Visell Y, Law A, Cooperstock J (2009) Touch is everywhere: floor surfaces as ambient haptic interfaces. IEEE Trans Haptics 2(3):148–159CrossRefGoogle Scholar
  33. Yao H-Y, Hayward V, Ellis RE (2004) A tactile magnification instrument for minimally invasive surgery. Lect Notes Comput Sci (MICCAI) 3217:89–96CrossRefGoogle Scholar
  34. Ye G, Corso J, Hager G, Okamura A (2003) VisHap: augmented reality combining haptics and vision. In: Proceedings of the IEEE international conference on systems, man and cybernetics, pp 3425–3431Google Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • Seokhee Jeon
    • 1
  • Seungmoon Choi
    • 2
  • Matthias Harders
    • 3
  1. 1.Kyung Hee UniversitySeoulSouth Korea
  2. 2.POSTECHPohangSouth Korea
  3. 3.University of InnsbruckInnsbruckAustria

Personalised recommendations