Skip to main content
SpringerLink
Log in

Pose-aware C-arm for automatic re-initialization of interventional 2D/3D image registration

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

In minimally invasive interventions assisted by C-arm imaging, there is a demand to fuse the intra-interventional 2D C-arm image with pre-interventional 3D patient data to enable surgical guidance. The commonly used intensity-based 2D/3D registration has a limited capture range and is sensitive to initialization. We propose to utilize an opto/X-ray C-arm system which allows to maintain the registration during intervention by automating the re-initialization for the 2D/3D image registration. Consequently, the surgical workflow is not disrupted and the interaction time for manual initialization is eliminated.

Methods

We utilize two distinct vision-based tracking techniques to estimate the relative poses between different C-arm arrangements: (1) global tracking using fused depth information and (2) RGBD SLAM system for surgical scene tracking. A highly accurate multi-view calibration between RGBD and C-arm imaging devices is achieved using a custom-made multimodal calibration target.

Results

Several in vitro studies are conducted on pelvic-femur phantom that is encased in gelatin and covered with drapes to simulate a clinically realistic scenario. The mean target registration errors (mTRE) for re-initialization using depth-only and RGB \(+\) depth are 13.23 mm and 11.81 mm, respectively. 2D/3D registration yielded 75% success rate using this automatic re-initialization, compared to a random initialization which yielded only 23% successful registration.

Conclusion

The pose-aware C-arm contributes to the 2D/3D registration process by globally re-initializing the relationship of C-arm image and pre-interventional CT data. This system performs inside-out tracking, is self-contained, and does not require any external tracking devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bay H, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. In: European conference on computer vision. Springer, Berlin, pp 404–417

  2. Van der Bom M, Bartels L, Gounis M, Homan R, Timmer J, Viergever M, Pluim J (2010) Robust initialization of 2D–3D image registration using the projection-slice theorem and phase correlation. Med Phys 37(4):1884–1892

    Article  PubMed  Google Scholar 

  3. Endres F, Hess J, Engelhard N, Sturm J, Cremers D, Burgard W (2012) An evaluation of the RGB-D SLAM system. In: 2012 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1691–1696 (2012)

  4. Euler E, Heining S, Riquarts C, Mutschler W (2003) C-arm-based three-dimensional navigation: a preliminary feasibility study. Comput Aided Surg 8(1):35–41

    Article  PubMed  Google Scholar 

  5. Fischer M, Fuerst B, Lee SC, Fotouhi J, Habert S, Weidert S, Euler E, Osgood G, Navab N (2016) Preclinical usability study of multiple augmented reality concepts for K-wire placement. Int J Comput Assist Radiol Surg 11(6):1007–1014

    Article  PubMed  Google Scholar 

  6. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  Google Scholar 

  7. Fotouhi J, Fuerst B, Lee SC, Keicher M, Fischer M, Weidert S, Euler E, Navab N, Osgood G (2016) Interventional 3D augmented reality for orthopedic and trauma surgery. In: 16th annual meeting of the international society for computer assisted orthopedic surgery (CAOS)

  8. Fotouhi J, Fuerst B, Wein W, Navab N (2017) Can real-time rgbd enhance intraoperative cone-beam CT? Int J Comput Assist Radiol Surg 1–9. doi:10.1007/s11548-017-1572-y

  9. Fuerst B, Fotouhi J, Navab N (2015) Vision-based intraoperative cone-beam ct stitching for non-overlapping volumes. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 387–395

  10. Geiger A, Moosmann F, Car Ö, Schuster B (2012) Automatic camera and range sensor calibration using a single shot. In: 2012 IEEE international conference on robotics and automation (ICRA). IEEE, pp 3936–3943

  11. Gendrin C, Furtado H, Weber C, Bloch C, Figl M, Pawiro SA, Bergmann H, Stock M, Fichtinger G, Georg D (2012) Monitoring tumor motion by real time 2D/3D registration during radiotherapy. Radiother Oncol 102(2):274–280

    Article  PubMed  Google Scholar 

  12. Glassner AS (1989) An introduction to ray tracing. Elsevier, Amsterdam

    Google Scholar 

  13. Goitein M, Abrams M, Rowell D, Pollari H, Wiles J (1983) Multi-dimensional treatment planning: Ii. beam’s eye-view, back projection, and projection through ct sections. Int J Radiat Oncol Biol Phys 9(6):789–797

    Article  CAS  PubMed  Google Scholar 

  14. Gong RH, Güler Ö, Kürklüoglu M, Lovejoy J, Yaniv Z (2013) Interactive initialization of 2D/3D rigid registration. Med Phys 40(12):121911

    Article  PubMed  Google Scholar 

  15. Grisetti G, Kümmerle R, Stachniss C, Frese U, Hertzberg C (2010) Hierarchical optimization on manifolds for online 2D and 3D mapping. In: 2010 IEEE international conference on robotics and automation (ICRA). IEEE, pp 273–278

  16. Jain A, Fichtinger G (2006) C-arm tracking and reconstruction without an external tracker. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 494–502 (2006)

  17. Knaan D, Joskowicz L (2003) Effective intensity-based 2D/3D rigid registration between fluoroscopic X-ray and CT. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 351–358

  18. Kümmerle R, Grisetti G, Strasdat H, Konolige K, Burgard W (2011) g2o: a general framework for graph optimization. In: 2011 IEEE international conference on robotics and automation (ICRA). IEEE, pp 3607–3613

  19. Lee SC, Fuerst B, Fotouhi J, Fischer M, Osgood G, Navab N (2016) Calibration of RGBD camera and cone-beam CT for 3D intra-operative mixed reality visualization. Int J Comput Assist Radiol Surg 11(6):967–975

    Article  PubMed  Google Scholar 

  20. Lemieux L, Jagoe R, Fish D, Kitchen N, Thomas D (1994) A patient-to-computed-tomography image registration method based on digitally reconstructed radiographs. Med Phys 21(11):1749–1760

    Article  CAS  PubMed  Google Scholar 

  21. Mitrović U, Pernuš F, Likar B, Špiclin Ž (2015) Simultaneous 3D–2D image registration and c-arm calibration: application to endovascular image-guided interventions. Med Phys 42(11):6433–6447

    Article  PubMed  Google Scholar 

  22. Navab N, Bani-Hashemi A, Nadar MS, Wiesent K, Durlak P, Brunner T, Barth K, Graumann R (1998) 3D reconstruction from projection matrices in a c-arm based 3D-angiography system. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 119–129

  23. Newcombe RA, Izadi S, Hilliges O, Molyneaux D, Kim D, Davison AJ, Kohi P, Shotton J, Hodges S, Fitzgibbon A (2011) Kinectfusion: real-time dense surface mapping and tracking. In: 2011 10th IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, pp 127–136

  24. Otake Y, Armand M, Armiger RS, Kutzer MD, Basafa E, Kazanzides P, Taylor RH (2012) Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery: incorporation of fiducial-based c-arm tracking and gpu-acceleration. IEEE Trans Med Imaging 31(4):948–962

    Article  PubMed  Google Scholar 

  25. Otake Y, Armand M, Sadowsky O, Armiger RS, Kutzer MD, Mears SC, Kazanzides P, Taylor, RH (2010) An image-guided femoroplasty system: development and initial cadaver studies. In: SPIE medical imaging. International Society for Optics and Photonics, pp 76250P–76250P (2010)

  26. Otake Y, Schafer S, Stayman J, Zbijewski W, Kleinszig G, Graumann R, Khanna A, Siewerdsen J (2012) Automatic localization of vertebral levels in X-ray fluoroscopy using 3D–2D registration: a tool to reduce wrong-site surgery. Phys Med Biol 57(17):5485

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Powell MJ (2009) The BOBYQA algorithm for bound constrained optimization without derivatives. Cambridge NA Report NA2009/06, University of Cambridge, Cambridge (2009)

  28. Rusinkiewicz S, Levoy M (2001) Efficient variants of the ICP algorithm. In: Proceedings. Third international conference on 3-D digital imaging and modeling, 2001. IEEE, pp 145–152

  29. Sadowsky O, Chintalapani G, Taylor RH (2007) Deformable 2D-3D registration of the pelvis with a limited field of view, using shape statistics. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 519–526 (2007)

  30. Uneri A, Otake Y, Wang A, Kleinszig G, Vogt S, Khanna AJ, Siewerdsen J (2013) 3D–2D registration for surgical guidance: effect of projection view angles on registration accuracy. Phys Med Biol 59(2):271

    Article  PubMed  PubMed Central  Google Scholar 

  31. Varnavas A, Carrell T, Penney G (2013) Increasing the automation of a 2D–3D registration system. IEEE Trans Med Imaging 32(2):387–399

    Article  PubMed  Google Scholar 

  32. Varnavas A, Carrell T, Penney G (2015) Fully automated 2D–3D registration and verification. Med Image Anal 26(1):108–119

    Article  PubMed  Google Scholar 

  33. Wein W (2003) Intensity based rigid 2D-3D registration algorithms for radiation therapy. Master’s thesis, Technische Universität München

  34. Yang L, Wang J, Kobayashi E, Ando T, Yamashita H, Sakuma I, Chiba T (2015) Image mapping of untracked free-hand endoscopic views to an ultrasound image-constructed 3D placenta model. Int J Med Robot Comput Assist Surg 11(2):223–234

    Article  Google Scholar 

  35. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334

    Article  Google Scholar 

Download references

Acknowledgements

The authors want to thank Wolfgang Wein and his team from ImFusion GmbH, Munich, for the opportunity of using the ImFusion Suite, and Gerhard Kleinzig and Sebastian Vogt from SIEMENS Healthineers for their support and making a SIEMENS ARCADIS Orbic 3D available.

Funding Research reported in this publication was partially supported by NIH/NIAMS under Award Number T32AR067708, NIH/NIBIB under the Award Numbers R01EB016703 and R21EB020113, and Johns Hopkins University internal funding sources.

Author information

Authors and Affiliations

  1. Computer Aided Medical Procedures, Johns Hopkins University, Baltimore, MD, USA

    Javad Fotouhi, Bernhard Fuerst, Sing Chun Lee & Nassir Navab

  2. Department of Orthopaedic Surgery, Johns Hopkins Hospital, Baltimore, MD, USA

    Alex Johnson, Greg Osgood & Mehran Armand

  3. Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

    Russell Taylor

  4. Computer Aided Medical Procedures, Technische Universität München, Munich, Germany

    Nassir Navab

  5. Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD, USA

    Mehran Armand

  6. Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA

    Mehran Armand

Authors
  1. Javad Fotouhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernhard Fuerst
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alex Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sing Chun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Russell Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Greg Osgood
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nassir Navab
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mehran Armand
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Javad Fotouhi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Javad Fotouhi and Bernhard Fuerst contributed equally and should be considered joint first authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fotouhi, J., Fuerst, B., Johnson, A. et al. Pose-aware C-arm for automatic re-initialization of interventional 2D/3D image registration. Int J CARS 12, 1221–1230 (2017). https://doi.org/10.1007/s11548-017-1611-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-017-1611-8

Keywords

Search

Navigation