A real-time navigation system for laparoscopic surgery based on three-dimensional ultrasound using magneto-optic hybrid tracking configuration

  • Kozo Konishi
  • Masahiko Nakamoto
  • Yoshihiro Kakeji
  • Kazuo Tanoue
  • Hirofumi Kawanaka
  • Shohei Yamaguchi
  • Satoshi Ieiri
  • Yoshinobu Sato
  • Yoshihiko Maehara
  • Shinichi Tamura
  • Makoto Hashizume
Original Article


Objectives In laparoscopic liver surgery, intraoperative navigation is strongly recommended. We developed a navigation system based on intraoperative ultrasound (IOUS). The purpose of this study was to evaluate the usefulness and accuracy of this system using an animate model.

Materials and methods Augmented reality (AR) visualization superimposing three-dimensional ultrasound (3D-US) images onto captured laparoscopic live images was constructed. We employed magneto-optic hybrid tracking configuration and a rapid method of magnetic distortion correction. Twelve pigs and liver tumor mimics were used, and effects of magnetic distortion correction and accuracy of 3D-US navigation were evaluated.

Results Using magnetic distortion correction, tracking error was significantly reduced. Each ultrasound scanning time was within 30 s, and the time to generate 3D-US images was within 3 min. All tumor mimics were successfully puncture-guided with navigation. Registration accuracy was significantly improved from 17.2 ± 5.27 to 1.96 ± 0.87 mm.

Conclusion We developed an AR navigation system based on IOUS. Experimental results showed that the proposed method was effective, and could be used in clinical settings. 3D-US, as an imaging modality allows real-time imaging regardless of organ shifts or distortion.


Augmented reality (AR) Navigation 3D-US Laparoscopic surgery Image-guided surgery (IGS) 



intraoperative ultrasound


augmented reality


three-dimensional ultrasound


degrees of freedom


region of interest


computed tomography


root mean square


image-guided surgery


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cherqui D, Husson E, Hammoud R, Malassagne B, Stephan F, Bensaid S, Rotman N et al (2000) Laparoscopic liver resections: a feasibility study in 30 patients. Ann Surg 232:753–762PubMedCrossRefGoogle Scholar
  2. 2.
    Chung MH, Wood TF, Tsioulias GJ, Rose DM and Bilchik AJ (2001). Laparoscopic radiofrequency ablation of unresectable hepatic malignancies: a phase 2 trial. Surg Endosc 15: 1020–1026 PubMedCrossRefGoogle Scholar
  3. 3.
    Gigot JF, Glineur D, Santiago Azagra J, Goergen M, Camrick M, Morino M and Etienne J (2002). Laparoscopic liver resection for malignant liver tumors: preliminary results of a multicenter European study. Ann Surg 236: 90–97 PubMedCrossRefGoogle Scholar
  4. 4.
    Lesurtel M, Cherqui D, Laurent A, Tayar C and Fagniez PL (2003). Laparoscopic versus open left lateral hepatic lobectomy: a case–control study. J Am Coll Surg 196: 236–242 PubMedCrossRefGoogle Scholar
  5. 5.
    Wallace JR, Christians KK, Quiroz FA, Foley WD, Pitt HA and Quebbeman EJ (2001). Ablation of liver metastasis: is preoperative imaging sufficiently accurate. J Gastrointest Surg 5(1): 98–107 PubMedCrossRefGoogle Scholar
  6. 6.
    Hata N, Dohi T, Iseki H and Takakura K (1997). Development of a frameless and armless stereotactic neuronavigation system with ultrasonographic registration. Neurosurgery 41: 608–613 PubMedCrossRefGoogle Scholar
  7. 7.
    Cartellieri M, Kremser J and Vorbeck F (2001). Comparison of different 3D navigation systems by a clinical user. Eur Arch Otorhinolaryngol 258: 38–41 PubMedCrossRefGoogle Scholar
  8. 8.
    Gumprecht HK, Widenka DC and Lumenta CB (1999). BrainLab Vector Vision neuronavigation system: technology and clinical experiences in 131 cases. Neurosurgery 44: 97–105 PubMedCrossRefGoogle Scholar
  9. 9.
    Strong AJ, Chandler CL and Gleeson MJ (2000). Design and evaluation of a system for microscope-assisted guided interventions (MAGI). IEEE Trans Med Imaging 19(11): 1082–1093 PubMedCrossRefGoogle Scholar
  10. 10.
    Akatsuka Y, Kawamata T, Fujii M, Furuhashi Y, Saito A, Shibasaki T, Isaki H, Hori T (2000) AR navigation system for neurosurgery. In: Lecture notes in computer science, vol. 1935, Proceedings of the 3rd international conference on medical image computing and computer-assisted intervention (MICCAI 2000), Pittsburgh, PA, pp 833–838Google Scholar
  11. 11.
    Sato Y, Nakamoto M, Tamaki Y, Sasama T, Sakita I, Nakajima Y, Tamura S and Monden M (1998). Image guidance of breast cancer surgery using 3-D ultrasound images and augmented reality visualization. IEEE Trans Med Imaging 17(5): 681–693 PubMedCrossRefGoogle Scholar
  12. 12.
    Inoue T, Tamaki Y, Sato Y, Nakamoto M, Tamura S, Tanji Y, Taguchi T and Noguchi S (2005). Three-dimensional ultrasound imaging of breast cancer by a real-time intraoperative navigation system. Breast Cancer 12(2): 122–129 PubMedCrossRefGoogle Scholar
  13. 13.
    Nakamoto M, Sato Y et al (2000) Magneto-optic hybrid 3-D sensor for surgical navigation. In: Lecture notes in computer science (LNCS), vol 1935 (MICCAI 2000), pp 839–848Google Scholar
  14. 14.
    Nakamoto M, Sato Y, Miyamoto M et al. (2002) 3D ultrasound system using a magneto-optic hybrid tracker for augmented reality visualization in laparoscopic liver surgery. In: Lecture notes in computer science(LNCS), vol. 2489 (MICCAI 2002), pp 148–155Google Scholar
  15. 15.
    Nakada K, Nakamoto M, Sato Y, Konishi K, Hashizume M, Tamura S (2003) A rapid method for magnetic tracker calibration using a magneto-optic hybrid tracker. In: Lecture notes on computer science, vol. 2879, Proceeding of MICCAI 2003, pp 285–293Google Scholar
  16. 16.
    Yamaguchi T, Nakamoto M, Sato Y, Konishi K, Hashizume M, Sugano N, Yoshikawa H and Tamura S (2004). Development of a camera model and calibration procedure for oblique-viewing endoscopes(dagger). Comput Aided Surg 9(5): 203–214 PubMedCrossRefGoogle Scholar
  17. 17.
    Nomori H (2005). Radiofrequency abation of pulmonary tumors and normal lung tissue in Swine and rabbits. Chest 127(3): 973–977 PubMedCrossRefGoogle Scholar
  18. 18.
    Hernes T, Ommedal S, Lie T, Lindseth F, Lango T and Unsgaard G (2002). Stereoscopic navigation-controlled display of preoperative MRI and intraoperative 3D ultrasound in planning and guidance of neurosurgery: new technology for minimally invasive image guided surgery approaches. Minim Invasive Neurosurg 46: 129–137 Google Scholar
  19. 19.
    Kaspersen JH, So1ie E, Wesche J, Å sland J, Lundbom J, Odegård A, Lindseth F and Hernes T (2003). 3D ultrasound based navigation combined with preoperative CT during abdominal interventions: a feasability study. Cardiovasc Intervent Radiol 26: 347–356 PubMedCrossRefGoogle Scholar
  20. 20.
    Harms J, Feussner H, Baumgartner M, Schneider A, Donhauser M and Wessels G (2001). Three-dimensional navigated laparoscopic ultrasonography: first experiences with a new minimally invasive diagnostic device. Surg Endosc 15(12): 1459–1462 PubMedCrossRefGoogle Scholar
  21. 21.
    Ellsmere J, Stoll J, Rattner D, Brooks D, Kane R, Wells W, Kikinis R, Vosburgh K (2003) A navigation system for augmenting laparoscopic ultrasound. In: Lecture notes in computer science, vol. 2879, Proceedings of the 6th international conference on medical image computing and computer-assisted intervention (MICCAI 2003), Montreal, Canada, pp 184–191Google Scholar
  22. 22.
    Birkfellner W, Watzinger F, Wanschitz F, Enislidis G, Killmann C, Rafolt D, Nowotny R, Ewers R and Bergmann H (1998). Systematic distortions in magnetic position digitizers. Med Phys 25(11): 2242–2248 PubMedCrossRefGoogle Scholar
  23. 23.
    Hummel JB, Bax MR, Figl ML, Kang Y, Calvin Maurer J, Birkfellner W, Bergmann H and Shahidi R (2005). Design and application of an assessment protocol for electromagnetic tracking systems. Med Phys 32(7): 2371–2379 PubMedCrossRefGoogle Scholar
  24. 24.
    Bao P, Warmath J, Galloway R and Herline A (2005). Ultrasound-to-computer-tomography registration for image-guided laparoscopic liver surgery. Surg Endosc 19(3): 424–429 PubMedCrossRefGoogle Scholar
  25. 25.
    Kleemann M, Hildebrand P, Birth M, Bruch HP (2006) Laparoscopic ultrasound navigation in liver surgery: technical aspects and accuracy. Surg Endosc 20(5):726–729. Epub 2006 March 16Google Scholar
  26. 26.
    Hautmann H, Schneider A, Pinkau T, Peltz F and Feussner H (2005). Electromagnetic catheter navigation during bronchoscopy: validation of a novel method by conventional fluoroscopy. Chest 128(1): 382–387 PubMedCrossRefGoogle Scholar
  27. 27.
    Turcza P and Duplaga M (2004). Navigation systems based on registration of endoscopic and CT-derived virtual images for bronchofiberoscopic procedures. Stud Health Technol Inform 105: 253–263 PubMedGoogle Scholar
  28. 28.
    Konishi K, Nakamoto M, Miyamoto M, Hashizume M, Sato Y, Tamura S (2002) Development of AR navigation system for laparoscopic surgery using magneto-optic hybrid sensor: experiences with 3 cases. In: Proceedings of CARS2002, p 1093Google Scholar
  29. 29.
    Foroutani A, Garland AM, Berber E, String A, Engle K, Ryan TL, Pearl JM and Siperstein AE (2000). Laparoscopic ultrasound vs triphasic computed tomography for detecting liver tumors. Arch Surg 135(8): 933–938 PubMedCrossRefGoogle Scholar
  30. 30.
    Kane RA (2004). Intraoperative ultrasonography: history, current state of the art and future directions. J Ultrasound Med 23(11): 1407–1420 PubMedGoogle Scholar

Copyright information

© CARS 2007

Authors and Affiliations

  • Kozo Konishi
    • 1
    • 2
  • Masahiko Nakamoto
    • 5
  • Yoshihiro Kakeji
    • 2
  • Kazuo Tanoue
    • 4
  • Hirofumi Kawanaka
    • 2
  • Shohei Yamaguchi
    • 2
    • 3
  • Satoshi Ieiri
    • 3
  • Yoshinobu Sato
    • 5
  • Yoshihiko Maehara
    • 2
  • Shinichi Tamura
    • 5
  • Makoto Hashizume
    • 3
    • 4
    • 6
  1. 1.Department of Innovative Medical TechnologyKyushu UniversityFukuokaJapan
  2. 2.Department of Surgery and Science, Graduate School of Medical SciencesKyushu UniversityFukuokaJapan
  3. 3.Department of Disaster and Emergency Medicine, Graduate School of Medical SciencesKyushu UniversityFukuokaJapan
  4. 4.Center for Integration of Advanced Medicine, Life Science and Innovative TechnologyKyushu UniversityFukuokaJapan
  5. 5.Division of Interdisciplinary Image AnalysisOsaka University Graduate School of MedicineOsakaJapan
  6. 6.FukuokaJapan

Personalised recommendations