Augmented reality haptic (ARH): an approach of electromagnetic tracking in minimally invasive surgery

  • J. B. Pagador
  • L. F. Sánchez
  • J. A. Sánchez
  • P. Bustos
  • J. Moreno
  • F. M. Sánchez-Margallo
Original Article



Minimally Invasive Surgery (MIS) is a widely used surgical technique that requires a long training process due to its difficulty and complexity. We developed an Augmented Reality Haptic (ARH) System based on electromagnetic tracking devices for use in creation training models (computer-enhanced trainers), in computer-assisted surgery or telemanipulation applications.


The ARH system consists currently in a Linux driver and a calibration protocol to acquire the tooltip position of conventional laparoscopic tools in real time. A Polhemus Isotrack® II was used to track surgical endoscopic tooltip movements. The receiver was mounted on the tool handle in order to measure laparoscopic tools positions without complex modifications. Two validation tests were done to guarantee the proper functioning of the ARH system in a MIS environment. The first one checks the driver operation and the second measures the accuracy and reliability of the tooltip pose estimation process.


Jitter and orientation errors for the first test were 2.00±0.10 and 2.00±0.09 mm, respectively. Relative position error of 0.25±0.06 cm for a distance of 5 cm was found. Jitter error for the second test was 127 ± 60, 117 ± 40 and 122 ± 39 mm in Z, Y and X rotations, respectively.


Results obtained with the ARH system are sufficiently accurate for use in MIS training. A supplementary correction procedure would be necessary to use this ARH system in computer-assisted surgery or telemanipulation.


Electromagnetic device tracking Minimally invasive surgery Surgical assessment Computer-assisted surgery Surgical training 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Usón J, Sánchez F, Pascual S, Climent S (2007) Formación en Cirugía Laparoscópica Paso a Paso. Centro de Cirugía de Mínima Invasión, Cáceres, SpainGoogle Scholar
  2. 2.
    Fried GM (2008) FLS assessment of competency using simulated laparoscopic task. J Gastrointest Surg 12(2): 210–212PubMedCrossRefGoogle Scholar
  3. 3.
    Kanumuri P, Ganai S, Wohaibi EM, Bush RW, Grow DR, Seymour NE (2008) Virtual reality and computer-enhanced training devices equally improve laparoscopic surgical skill in novices. JSLS 12(3): 219–226PubMedGoogle Scholar
  4. 4.
    Fichera A, Prachand V, Kives S, Levine R, Hasson H (2005) Physical reality simulation for training of laparoscopists in the 21st century. A multispecialty, multi-institutional study. JSLS 9(2): 125–129PubMedGoogle Scholar
  5. 5.
    Madan AK, Frantzides CT, Tebbit C, Shervin N (2005) Self-reported vs observed scores in laparoscopic skills training. Surg Endosc 19: 670–672PubMedCrossRefGoogle Scholar
  6. 6.
    Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A (2004) Laparoscopic virtual reality and box trainers. Is one superior to the other?. Surgical Endoscopic 18: 485–494CrossRefGoogle Scholar
  7. 7.
    Fellinger EK, Ganey ME, Gallagher AG, Scott DJ, Bush RW, Seymour NE (2006) Complex laparoscopic task performance on two new computer-based skills training devices. Surg Endosc 20: S363Google Scholar
  8. 8.
    Botden SM, Buzink SN, Schijven MP, Jakimowicz JJ (2007) Augmented versus virtual reality laparoscopic simulation: what is the difference? a comparison of the promis augmented reality laparoscopic simulator versus lapsim virtual reality laparoscopic simulator. World J Surg 31(4): 764–772PubMedCrossRefGoogle Scholar
  9. 9.
    Madan AK, Frantzides CT, Tebbit CL, Quiros RM (2005) Participant’s opinions of laparoscopic trainers during basic laparoscopic training courses. Am J Surg 189: 758–761PubMedCrossRefGoogle Scholar
  10. 10.
    Waxberg SL, Goodell KH, Avgerinos DV Schwaitzberg SD, Cao CGL (2004) Evaluation of physical versus virtual surgical training simulators. Proce HFES 48th Annu Meet 48: 1675–1679Google Scholar
  11. 11.
    Young D, Slevin F, Cassidy D, Ryan D (2007) Objective measurement of FLS precission cutting task. Society of American Gastrointestinal and Endoscopic Surgeons (SAGES), ETP057Google Scholar
  12. 12.
    English J, Chang CY, Tardella N, Hu J (2005) A vision-based surgical tool tracking approach for untethered surgery simulation and training. Stud Health Technol Inform 111: 126–132PubMedGoogle Scholar
  13. 13.
    Dewan M, Marayong P, Okamura AM, Hager GD (2004) Vision-based assistance for ophthalmic microsurgery. Med Image Comput Comput Assist Interv 2: 49–57Google Scholar
  14. 14.
    Fischer J, Eichler M, Bartz D, Straser W (2006) Model-based hybrid tracking for medical augmented reality. Eurographics Symp Virtual EnvironGoogle Scholar
  15. 15.
    Doignon C, Nageotte F, Maurin B, Krupa A (2008) Pose estimation and feature tracking for robot assisted surgery with medical imaging. In: Kragic D, Kyrik V (eds) Unifying perspectives in computational and robot vision. Springer, USGoogle Scholar
  16. 16.
    Lizarraga CA, Ortega LR, Urbina BJ, Rodríguez O (2008) Visión artificial para detección y ubicación espacial de instrumental laparoscópico. In: Cardona A, Storti M, Zuppa C (eds) Mecánica Computacional vol XXVII. San Luis, Argentina, pp 3627–3637Google Scholar
  17. 17.
    Chmarra MK, Grimbergen CA, Dankelman J (2007) Systems for tracking minimally invasive surgical instruments. Minim Invasive Ther Allied Technol 16(6): 328–340PubMedCrossRefGoogle Scholar
  18. 18.
    Gunther S, Rosen J, Hannaford B, Sinanan M (2007) The red DRAGON: a multi-modality system for simulation and training in minimally invasive surgery. Stud Health Technol Inform 125: 149–154PubMedGoogle Scholar
  19. 19.
    Feng C, Haniffa H, Rozenblit J, Hamilton A, Salkini M (2006) Surgical training and performance assessment using a motion tracking system. Int Med Model Multiconference Proc 647–652Google Scholar
  20. 20.
    Diaz CA, Posada D, Trefftz H, Bernal J (2008) Development of a surgical simulator to training laparoscopic procedures. Int J Educa Inf Technol 2(1): 95–110Google Scholar
  21. 21.
    Martinez H, Mucha D, Kosmecki B, Kruger T (2008) Calibration method for electromagnetic tracked instruments in clinical applications. Int J Comput Assist Radiol Surg 3(1): 111–113Google Scholar
  22. 22.
    Birch DW, Bonjer HJ, Crossley C, Burnett G, Gara C, Gomes A, Hagen J, Maciver AG, Mercer CD, Panton ON, Schlachta CM, Smith AJ, Warnock GL (2009) Canadian consensus conference on the development of training and practice standards in advanced minimally invasive surgery. Can J Surg 52(4): 321–327PubMedGoogle Scholar
  23. 23.
    Detmer PR, Bashein G, Hodges T, Beach KW, Filer EP, Burns DH, Strandness DE (1994) 3D ultrasound image feature localization based on magnetic scan head tracking: in vitro calibration and validation. Ultrasound Med Biol 20: 923–936PubMedCrossRefGoogle Scholar
  24. 24.
    Hartov A, Eisner SD, David MS, Roberts W, Paulsen KD, Platenik BS, Miga MI (1999) Error analysis for a free-hand three dimensional ultrasound system for Navigation. Neurosurg Focus 6(3): E7CrossRefGoogle Scholar
  25. 25.
    Jannin P, Korb W (2008) Assessment of image-guided interventions. In: Peters T, Cleary K (eds) Image-guided interventions. Springer, US, pp 531–549CrossRefGoogle Scholar
  26. 26.
    Hummel JB, Figl ML, Kang Y, Maurer C, Birkfellner WW, Bergmann H, Shahidi R (2005) Design and application of an assessment protocol for electromagnetic tracking systems. Med Phys 32(7): 2371–2379PubMedCrossRefGoogle Scholar
  27. 27.
    Schneider M, Stevens C (2007) Development and testing of a new magnetic-tracking device for image guidance. Medical imaging 2007: visualization and image-guided procedures. In: Cleary K, Miga M (eds) Proceedings of the SPIE 6509:650901Google Scholar
  28. 28.
    Schicho K, Figl M, Donat M, Birkfellner W, Seemann R, Wagner A, Bergmann H, Ewers R (2005) Stability of miniature electromagnetic tracking systems. Phys Med Biol 50(9): 2089–2098PubMedCrossRefGoogle Scholar
  29. 29.
    Xeroulis G, Dubrowski A, Leslie K (2009) Simulation in laparoscopic surgery: a concurrent validity study for FLS. Surg Endosc 3(1): 161–165CrossRefGoogle Scholar
  30. 30.
    Hayter MA, Friedman Z, Bould MD, Hanlon JG, Katznelson R, Borges B, Naik VN (2009) Validation of the Imperial College Surgical Assessment Device (ICSAD) for labour epidural placement. Can J Anaesth 56(6): 419–426PubMedCrossRefGoogle Scholar
  31. 31.
    Ezra DG, Aggarwal R, Michaelides M, Okhravi N, Verma S, Benjamin L, Bloom P, Darzi A, Sullivan P (2009) Skills acquisition and assessment after a microsurgical skills course for ophthalmology residents. Ophthalmology 116(2): 257–262PubMedCrossRefGoogle Scholar
  32. 32.
    Grober ED, Roberts M, Shin EJ, Mahdi M, Bacal V (2010) Intraoperative assessment of technical skills on live patients using economy of hand motion: establishing learning curves of surgical compentence. Am J Surg 199: 81–85PubMedCrossRefGoogle Scholar
  33. 33.
    Moorthy K, Munz Y, Sarker SK, Darzi A (2003) Objective assessment of technical skills in surgery. Br Med J 327: 1032–1037CrossRefGoogle Scholar
  34. 34.
    McClusky DA, Smith CD (2008) Design and development of a surgical skills simulation curriculum. World J Surg 32: 171–181PubMedCrossRefGoogle Scholar
  35. 35.
    Lombardo JC, Cani MP, Neyret F (1999) Real-time collision detection for virtual surgery. In: Computer Animation. p 82Google Scholar
  36. 36.
    Teschner M, Kimmerle S, Heidelberger B, Zachmann G, Raghupathi L, Fuhrmann A, Cani MP, Faure F, Magnenat-Thalmann N, Strasser W, Volino P (2004) Collision detection for deformable objects. Comput Graph 23(3): 1–22Google Scholar
  37. 37.
    Kockara S, Halic T, Bayrak C, Iqbal K, Rowe RA (2009) Contact detection algorithms. J comput 4(10): 1053–1063Google Scholar
  38. 38.
    Fried GM, Feldman LS (2008) Objective assessment of technical performance. World J Surg 32(2): 156–160PubMedCrossRefGoogle Scholar
  39. 39.
    Nistor V, Allen B, Dutson E, Faloutsos P, Carman GP (2007) Immersive training and mentoring for laparoscopic surgery. Nanosensors, microsensors, and biosensors and systems 2007. Proc SPIE 6528:65280Q1–65280Q11Google Scholar
  40. 40.
    Sánchez-Margallo FM, Sánchez MA, Pérez FJ, Pagador JB, Mateos J, Bustos P, Díaz-Güemes I, Moyano JL, Correa L, Usón J (2009) Dispositivo Electrónico para el Entrenamiento de Técnicas Endoquirúrgicas (DEETE)®: validación clínica subjetiva, vol XVII. Reunión Nacional de CirugíaGoogle Scholar

Copyright information

© CARS 2010

Authors and Affiliations

  • J. B. Pagador
    • 1
  • L. F. Sánchez
    • 1
  • J. A. Sánchez
    • 1
  • P. Bustos
    • 2
  • J. Moreno
    • 2
  • F. M. Sánchez-Margallo
    • 1
  1. 1.Jesús Usón Minimally Invasive Surgery CentreCáceresSpain
  2. 2.Laboratory of Robotics and Artificial VisionUniversity of ExtremaduraCáceresSpain

Personalised recommendations