Robust bronchoscope motion tracking using sequential Monte Carlo methods in navigated bronchoscopy: dynamic phantom and patient validation

  • Xióngbiāo LuóEmail author
  • Marco Feuerstein
  • Takayuki Kitasaka
  • Kensaku Mori
Original Article



Accurate and robust estimates of camera position and orientation in a bronchoscope are required for navigation. Fusion of pre-interventional information (e.g., CT, MRI, or US) and intra-interventional information (e.g., bronchoscopic video) were incorporated into a navigation system to provide physicians with an augmented reality environment for bronchoscopic interventions.


Two approaches were used to predict bronchoscope movements by incorporating sequential Monte Carlo (SMC) simulation including (1) image-based tracking techniques and (2) electromagnetic tracking (EMT) methods. SMC simulation was introduced to model ambiguities or uncertainties that occurred in image- and EMT-based bronchoscope tracking. Scale invariant feature transform (SIFT) features were employed to overcome the limitations of image-based motion tracking methods. Validation was performed on five phantom and ten human case datasets acquired in the supine position.


For dynamic phantom validation, the EMT–SMC simulation method improved the tracking performance of the successfully registered bronchoscopic video frames by 12.7% compared with a hybrid-based method. In comparisons between tracking results and ground truth, the accuracy of the EMT–SMC simulation method was 1.51 mm (positional error) and 5.44° (orientation error). During patient assessment, the SIFT–SMC simulation scheme was more stable or robust than a previous image-based approach for bronchoscope motion estimation, showing 23.6% improvement of successfully tracked frames. Comparing the estimates of our method to ground truth, the position and orientation errors are 3.72 mm and 10.2°, while those of our previous image-based method were at least 7.77 mm and 19.3°. The computational times of our EMT– and SIFT–SMC simulation methods were 0.9 and 1.2 s per frame, respectively.


The SMC simulation method was developed to model ambiguities that occur in bronchoscope tracking. This method more stably and accurately predicts the bronchoscope camera position and orientation parameters, reducing uncertainties due to problematic bronchoscopic video frames and airway deformation during intra-bronchoscopy navigation.


Sequential Monte Carlo methods Bronchoscope motion tracking Virtual bronchoscopy Navigated bronchoscopy 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    American Cancer Society (2010) Global cancer facts & figures 2010, American Cancer Society, Atlanta.
  2. 2.
    Mori K, Urano A, Hasegawa J, Toriwaki J, Anno H, Katada K (1996) Virtualized endoscope system—an application of virtual reality technology to diagnostic aid. IEICE Trans Inf Syst E 79-D(6): 809–819Google Scholar
  3. 3.
    Kiraly AP, Higgins WE, McLennan G, Hoffman EA, Reinhardt JM (2002) Three-dimensional human airway segmentation methods for cinical virtual bronchoscopy. Acad Radiol 9: 1153–1168PubMedCrossRefGoogle Scholar
  4. 4.
    Mayer D, Bartz D, Fischer J, Ley S, del Rio A, Thust S, Kauczor H-U, Heussel CP (2004) Hybrid segmentation and virtual bronchoscopy based on CT images. Acad Radiol 11: 551–565PubMedCrossRefGoogle Scholar
  5. 5.
    Deligianni F, Chung AJ, Yang GZ (2006) Nonrigid 2-D/3-D registration for patient specific bronchoscopy simulation with statistical shape modeling: phantom validation. IEEE Trans Med Imaging 25(11): 1462–1471PubMedCrossRefGoogle Scholar
  6. 6.
    Helferty J, Sherbondy A, Kiraly A, Higgins W (2007) Computer-based system for the virtual-endoscopic guidance of bronchoscopy. Comput Vis Image Underst 108: 171–187PubMedCrossRefGoogle Scholar
  7. 7.
    Suter MJ, Reinhardt JM, McLennan G (2008) Integrated CT/bronchoscopy in the central airways: preliminary results. Acad Radiol 15: 786–798PubMedCrossRefGoogle Scholar
  8. 8.
    Deguchi D, Mori K, Feuerstein M, Kitasaka T, Maurer CR Jr, Suenaga Y, Takabatake H, Mori M, Natori H (2009) Selective image similarity measure for bronchoscope tracking based on image registration. Med Image Anal 13(4): 621–633PubMedCrossRefGoogle Scholar
  9. 9.
    Solomon SB, White JP, Wiener CM, Orens JB, Wang KP (2000) Three-dimensionsal CT-guided bronchoscopy with a real-time electromagnetic position sensor: a comparison of two image registration methods. Chest 118(6): 1783–1787PubMedCrossRefGoogle Scholar
  10. 10.
    Schwarz Y, Greif J, Becker HD, Ernst A, Mehta A (2006) Real-time electromagnetic navigation bronchoscopy to peripheral lung lesions using overlaid CT images: the first human study. Chest 129(4): 988–994PubMedCrossRefGoogle Scholar
  11. 11.
    Levy EB, Tang J, Lindisch D, Glossop N, Banovac F, Cleary K (2007) Implementation of an electromagnetic tracking system for accurate intrahepatic puncture needle guidance: accuracy results in an in vitro model. Acad Radiol 14: 344–354PubMedCrossRefGoogle Scholar
  12. 12.
    Yaniv Z, Cheng P, Wilson E, Popa T, Lindisch D, Campos-Nanez E, Abeledo H, Watson V, Cleary K, Banovac F (2010) Needle-based interventions with the image-guided surgery toolkit (IGSTK): from phantoms to clinical trials. IEEE Trans Biomed Eng 57(4): 922–933PubMedCrossRefGoogle Scholar
  13. 13.
    Muller SA, Maier-Hein L, Tekbas A, Seitel A, Ramsauer S, Radeleff B, Franz AM, Tetzlaff R, Mehrabi A, Wolf I, Kauczor H-U, Meinzer H-P, Schmied BM (2010) Navigated liver biopsy using a novel soft tissue navigation system versus CT-guided liver biopsy in a porcine model: a prospective randomized trial. Acad Radiol 17: 1282–1287PubMedCrossRefGoogle Scholar
  14. 14.
    Mori K, Deguchi D, Akiyama K, Kitasaka T, Maurer CR Jr, Suenaga Y, Takabatake H, Mori M, Natori H (2005) Hybrid bronchoscope tracking using a magnetic tracking sensor and image registration. In: Proceedings of MICCAI 2005, vol LNCS 3750. pp 543–550Google Scholar
  15. 15.
    Soper TD, Haynor DR, Glenny RW, Seibel EJ (2010) In vivo validation of a hybrid tracking system for navigation of an ultrathin bronchoscope within peripheral airways. IEEE Trans Biomed Eng 57(3): 736–745PubMedCrossRefGoogle Scholar
  16. 16.
    Kindratenko VV (2000) A survey of electromagnetic position tracker calibration techniques. Virtual Real 5(3): 169–182CrossRefGoogle Scholar
  17. 17.
    Feuerstein M, Reichl T, Vogel J, Traub J, Navab N (2009) Magneto-optical tracking of flexible laparoscopic ultrasound: model-based online detection and correction of magnetic tracking errors. IEEE Trans Med Imaging 28(6): 951–967PubMedCrossRefGoogle Scholar
  18. 18.
    Luo X, Reichl T, Feuerstein M, Kitasaka T, Mori K (2010) Modified hybrid bronchoscope tracking based on sequential monte carlo sampler: dynamic phantom validation. In: Proceedings of ACCV 2010, vol 3. pp 1722–1733Google Scholar
  19. 19.
    Arnaud Doucet CA, Godsill S (2000) On sequential Monte Carlo sampling methods for Bayesian filtering. Stat Comput 10(3): 197–208CrossRefGoogle Scholar
  20. 20.
    Doucet A, de Freitas N, Gordon N (2001) Sequential Monte Carlo methods in practice. Springer, BerlinGoogle Scholar
  21. 21.
    Liu JS, Chen R (1998) Sequential Monte Carlo methods for dynamic systems. J Am Stat Assoc 93(443): 1032–1044Google Scholar
  22. 22.
    Arulampalam M et al (2002) A tutorial on particle filters for nonlinear/non-gaussian Bayesian tracking. IEEE Trans Signal Process 50: 174–188CrossRefGoogle Scholar
  23. 23.
    Hartley R, Zisserman A (2004) Multiple view geometry in computer vision. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  24. 24.
    Hart JC, Francis GK, Kauffman LH (1994) Visualizing quaternion rotation. ACM Trans Graph 13(3): 256–276CrossRefGoogle Scholar
  25. 25.
    Hanson AJ (2006) Visualizing quaternions. Morgan-Kaufmann/ Elsevier, Los AltosGoogle Scholar
  26. 26.
    Isard M, Blake A (1998) Condensation—conditional density propagation for visual tracking. Int J Comput Vis 29(1): 5–28CrossRefGoogle Scholar
  27. 27.
    Luo X, Feuerstein M, Deguchi D, Kitasaka T, Takabatake H, Mori K Development and comparison of new hybrid motion tracking for bronchoscopic navigation. Med Image Anal (in press)Google Scholar
  28. 28.
    Mori K, Suenaga Y, Toriwaki J (2003) Fast software-based volume rendering using multimedia instructions on PC platforms and its application to virtual endoscopy. In: Proceedings of SPIE, vol 5031. pp 111–122Google Scholar
  29. 29.
    Luo X, Feuerstein M, Sugiura T, Kitasaka T, Imaizumi K, Hasegawa Y, Mori K (2010) Towards hybrid bronchoscope tracking under respiratory motion: evaluation on a dynamic motion phantom. In: Proceedings of SPIE, vol 7625. p 76251BGoogle Scholar
  30. 30.
    Schneider M, Stevens C (2007) Development and testing of a new magnetic-tracking device for image guidance. In: Proceedings of SPIE, vol 6509. p 65090IGoogle Scholar

Copyright information

© CARS 2011

Authors and Affiliations

  • Xióngbiāo Luó
    • 1
    Email author
  • Marco Feuerstein
    • 2
  • Takayuki Kitasaka
    • 3
  • Kensaku Mori
    • 1
    • 4
  1. 1.Graduate School of Information ScienceNagoya UniversityNagoyaJapan
  2. 2.Computer Aided Medical Procedures (CAMP)Technische Universität MünchenMunichGermany
  3. 3.Faculty of Information ScienceAichi Institute of TechnologyToyotaJapan
  4. 4.Information and Communications HeadquartersNagoya UniversityNagoyaJapan

Personalised recommendations