Medical & Biological Engineering & Computing

, Volume 53, Issue 11, pp 1177–1186 | Cite as

Using visual cues to enhance haptic feedback for palpation on virtual model of soft tissue

  • Min LiEmail author
  • Jelizaveta Konstantinova
  • Emanuele L. Secco
  • Allen Jiang
  • Hongbin Liu
  • Thrishantha Nanayakkara
  • Lakmal D. Seneviratne
  • Prokar Dasgupta
  • Kaspar Althoefer
  • Helge A. Wurdemann
Original Article


This paper explores methods that make use of visual cues aimed at generating actual haptic sensation to the user, namely pseudo-haptics. We propose a new pseudo-haptic feedback-based method capable of conveying 3D haptic information and combining visual haptics with force feedback to enhance the user’s haptic experience. We focused on an application related to tumor identification during palpation and evaluated the proposed method in an experimental study where users interacted with a haptic device and graphical interface while exploring a virtual model of soft tissue, which represented stiffness distribution of a silicone phantom tissue with embedded hard inclusions. The performance of hard inclusion detection using force feedback only, pseudo-haptic feedback only, and the combination of the two feedbacks was compared with the direct hand touch. The combination method and direct hand touch had no significant difference in the detection results. Compared with the force feedback alone, our method increased the sensitivity by 5 %, the positive predictive value by 4 %, and decreased detection time by 48.7 %. The proposed methodology has great potential for robot-assisted minimally invasive surgery and in all applications where remote haptic feedback is needed.


Haptic feedback Pseudo-haptic feedback Rigid tool/soft tissue interaction Tumor identification 



The research leading to these results has received funding from the National Natural Science Foundation of China (Approval No. 51175412), the China Scholarship Council, the GSTT charity, the National Institute for Health Research (NIHR) Biomedical Research Centre based at Guy’s and St Thomas’ NHS Foundation Trust and King’s College London, and the European Commission’s Seventh Framework Programme under grant agreement 287728 in the framework of EU project STIFF-FLOP. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health.


  1. 1.
    Altman DG, Bland J (1994) Diagnostic test 1: sensitivity and specificity. BMJ 308:1552PubMedCentralCrossRefPubMedGoogle Scholar
  2. 2.
    Bibin L, Anatole L, Bonnet M, Delbos A, Dillon C (2008) SAILOR: a 3-D medical simulator of loco-regional anaesthesia based on desktop virtual reality and pseudo-haptic feedback. In: ACM symposium on virtual reality software and technology (VRST) 2008, pp 97–100Google Scholar
  3. 3.
    Conover WJ (1980) Practical nonparametric statistics, 2nd edn. Wiley, UKGoogle Scholar
  4. 4.
    De Gersem G (2005) Reliable and enhanced stiffness perception in soft-tissue telemanipulation. Int J Robot Res 24(10):805–822CrossRefGoogle Scholar
  5. 5.
    Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870):429–433CrossRefPubMedGoogle Scholar
  6. 6.
    Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27(8):861–874CrossRefGoogle Scholar
  7. 7.
    Gwilliam JC, Mahvash M, Vagvolgyi B, Vacharat A, Yuh DD, Okamura AM (2009) Effects of haptic and graphical force feedback on teleoperated palpation. In: Proceedings of IEEE international conference on robotics and automation 2009, pp 677–682Google Scholar
  8. 8.
    Hachisu T, Cirio G, Marchal M, Luyer A (2011) Pseudo-haptic feedback augmented with visual and tactile vibrations. IEEE international symposium on virtual reality innovation 2011, pp 327–328Google Scholar
  9. 9.
    Hayward V, Astley O, Cruz-Hernandez M, Grant D, Robles-De-La-Torre G (2004) Haptic interfaces and devices. Sens Rev 24(1):16–29CrossRefGoogle Scholar
  10. 10.
    Hayward V (2008) A brief taxonomy of tactile illusions and demonstrations that can be done in a hardware store. Brain Res Bull 75(6):742–752CrossRefPubMedGoogle Scholar
  11. 11.
    Kim SY, Kyung KU, Park J, Kwon DS (2007) Real-time area-based haptic rendering and the augmented tactile display device for a palpation simulator. Adv Robot 21(9):961–981CrossRefGoogle Scholar
  12. 12.
    Kimura T, Nojima T (2012) Pseudo-haptic feedback on softness induced by grasping motion. In: Isokoski P, Springare J (eds): EuroHaptics 2012, pp 202–205Google Scholar
  13. 13.
    Klatzky RL, Lederman SJ, Langseth S (2003) Watching a cursor distorts haptically guided reproduction of mouse movement. J Exp Psychol Appl 9(4):228–235CrossRefPubMedGoogle Scholar
  14. 14.
    Lecuyer A, Burkhardt JM, Tan CH (2008) A study of the modification of the speed and size of the cursor for simulating pseudo-haptic bumps and holes. ACM Trans Appl Percept 5(3):1–21CrossRefGoogle Scholar
  15. 15.
    Lecuyer A, Burkhardt JM, Coquillart S, Coiffet P (2001) Boundary of illusion: an experiment of sensory integration with a pseudo-haptic system. In: Proceedings of the 2001 IEEE virtual reality conference, pp 115–122Google Scholar
  16. 16.
    Li M, Faragasso A, Konstantinova J, Aminzadeh V, Seneviratne LD, Dasgupta P, Althoefer K (2014) A novel tumor localization method using haptic palpation based on soft tissue probing data. In: Proceedings of IEEE international conference on robotics and automation 2014, pp 4188–4193Google Scholar
  17. 17.
    Li M, Liu H, Seneviratne LD, Althoefer K (2012) Tissue stiffness simulation and abnormality localization using pseudo-haptic feedback. In: Proceedings of IEEE international conference on robotics and automation 2012, pp 5359–5364Google Scholar
  18. 18.
    Liu H, Noonan DP, Challacombe BJ, Dasgupta P, Seneviratne LD, Althoefer K (2010) Rolling mechanical imaging for tissue abnormality localization during minimally invasive surgery. IEEE Trans Biomed Eng 57(2):404–414CrossRefPubMedGoogle Scholar
  19. 19.
    Liu H, Li J, Song X, Seneviratne LD, Althoefer K (2011) Rolling indentation probe for tissue abnormality identification during minimally invasive surgery. IEEE Trans Robot 27(3):450–460CrossRefGoogle Scholar
  20. 20.
    Liu H, Sangpradit K, Li M, Dasgupta P, Althoefer K, Seneviratne LD (2014) Inverse finite-element modeling for tissue parameter identification using a rolling indentation probe. Med Biol Eng Comput 52(1):17–28CrossRefPubMedGoogle Scholar
  21. 21.
    Masuzaki R, Tateishi R, Yoshida H, Sato T, Ohki T, Goto T, Yoshida H, Sato S, Sugioka Y, Ikeda H, Shiina S, Kawabe T, Omata M (2007) Assessing liver tumor stiffness by transient elastography. Hepatol Int 1(3):394–397PubMedCentralCrossRefPubMedGoogle Scholar
  22. 22.
    Megumi N, Kuroda T, Komori M, Oyama H (2003) Evaluation and user study of haptic simulator for learning palpation in cardiovascular surgery. In: Proceedings of international conference on artificial reality and telexistence 2003.Google Scholar
  23. 23.
    Mensvoort K, Vos P, Hermes DJ, Liere RV (2010) Perception of mechanically and optically simulated bumps and holes. ACM Trans Appl Percept 7(2):10:1–24CrossRefGoogle Scholar
  24. 24.
    Nedel LP, Thalmann D (1998) Real-time muscle deformations using mass-spring systems. In: Proceedings computer graphics international 1998, pp 156–165Google Scholar
  25. 25.
    Netti PA, Berk DA, Swartz MA, Grodzinsky AJ, Jain RK (2000) Role of extracellular matrix assembly in interstitial transport in solid tumors. Cancer Res 60(9):2497–2503PubMedGoogle Scholar
  26. 26.
    Salomon G, Kollerman J, Thederan I, Chun FKH, Budaus L, Schlomm T, Isbarn H, Heinzer H, Huland H, Graefen M (2008) Evaluation of prostate cancer detection with ultrasound real-time elastography: a comparison with step section pathological analysis after radical prostatectomy. Eur Urol 54(6):1354–1362CrossRefPubMedGoogle Scholar
  27. 27.
    Sangpradit K, Liu H, Dasgupta P, Althoefer K, Seneviratne LD (2011) Finite-element modeling of soft tissue rolling indentation. IEEE Trans Biomed Eng 58(12):3319–3327CrossRefPubMedGoogle Scholar
  28. 28.
    Venkatesh SK, Yin M, Glockner JF, Takahashi N, Araoz PA, Talwalkar JA, Ehman RL (2008) MR elastography of liver tumors: preliminary results. AJR Am J Roentgenol 190(6):1534–1540PubMedCentralCrossRefPubMedGoogle Scholar
  29. 29.
    Wallis S (2013) Binomial confidence intervals and contingency tests: mathematical fundamentals and the evaluation of alternative methods. J Quant Linguist 20(3):178–208CrossRefGoogle Scholar
  30. 30.
    Wellman P, Howe R (1999) Breast tissue stiffness in compression is correlated to histological diagnosis. Harv BioRob Lab Tech RepGoogle Scholar
  31. 31.
    Wilcoxon F (1946) Individual comparisons of grounded data by ranking methods. J Econ Entomol 39:269CrossRefPubMedGoogle Scholar
  32. 32.
    Wilson EB (1927) Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 22:209–212CrossRefGoogle Scholar
  33. 33.
    Woodward W, Strom E, Tucker SL, McNeese MD, Perkins GH, Schechter NR, Singletary SE, Theriault RL, Hortobagyi GN, Hunt KK, Buchholz T (2003) Changes in the 2003 American joint committee on cancer staging for breast cancer dramatically affect stage-specific survival. J Clin Oncol 21(17):324–348CrossRefGoogle Scholar
  34. 34.
    Yamamoto T, Abolhassani N (2012) Augmented reality and haptic interfaces for robot assisted surgery. Int J Med Robot Comput Assist Surg 8:45–56CrossRefGoogle Scholar
  35. 35.
    Yau C (2009) R tutoral eBook. Accessed 20 Mar 2014

Copyright information

© International Federation for Medical and Biological Engineering 2015

Authors and Affiliations

  • Min Li
    • 1
    Email author
  • Jelizaveta Konstantinova
    • 2
  • Emanuele L. Secco
    • 2
    • 3
  • Allen Jiang
    • 2
  • Hongbin Liu
    • 2
  • Thrishantha Nanayakkara
    • 2
  • Lakmal D. Seneviratne
    • 2
    • 4
  • Prokar Dasgupta
    • 5
  • Kaspar Althoefer
    • 2
  • Helge A. Wurdemann
    • 2
  1. 1.School of Mechanical EngineeringXi’an Jiaotong UniversityXi’anChina
  2. 2.Department of InformaticsKings College LondonLondonUK
  3. 3.Department of Mathematics and Computer ScienceHope UniversityLiverpoolUK
  4. 4.College of EngineeringKhalifa University of Science, Technology and ResearchAbu DhabiUAE
  5. 5.Medical Research Council (MRC) Centre for TransplantationKing’s College London, Kings Health Partners, Guys HospitalLondonUK

Personalised recommendations