Skip to main content

Advertisement

SpringerLink
Log in

Real-time 6DoF pose recovery from X-ray images using library-based DRR and hybrid optimization

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Real-time 6 degrees of freedom (6DoF) pose recovery and tracking from X-ray images is a key enabling technology for many interventional imaging applications. However, real-time 2D/3D registration is a very challenging problem because of the heavy computation in iterative digitally reconstructed radiograph (DRR) generation. In this paper, we propose a real-time 2D/3D registration framework using library-based DRRs to achieve high computational efficiency.

Method

The proposed method pre-computes a library of canonical DRRs and reconstructs library-based DRRs (libDRRs) during registration without online rendering. The transformation parameters are decoupled to 2 geometry-relevant and 4 geometry-irrelevant ones so that canonical DRRs only need to cover the variation of 2 geometry-relevant parameters, making it practical to be pre-computed and stored. The 2D/3D registration using libDRRs is then solved as a hybrid optimization problem, i.e., continuous in geometry-irrelevant parameters while discrete in geometry-relevant parameters.

Results

On 5 fluoroscopic sequences with 246 frames acquired during animal studies with a transesophageal echocardiography (TEE) probe in the field of view, 6DoF tracking of the TEE probe using the proposed method achieved a mean target registration error in the projection direction (mTREproj) of 0.81 mm, a success rate of 100 % (defined as mTREproj \(<\)2.5 mm), and a registration frame rate of 23.1 fps on a pure CPU-based implementation executed in a single thread.

Conclusion

Using libDRRs with a hybrid optimization can significantly improve the computational efficiency (up to tenfold) for 6DoF pose recovery and tracking with little degradation in robustness and accuracy, compared to conventional intensity-based 2D/3D registration using ray casting DRRs with a continuous optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Vetter SY, Mühlhäuser I, von Recum J, Grützner P-A, Franke J (2014) Validation of a virtual implant planning system (vips) in distal radius fractures. Bone Joint J Orthop Proc Suppl 96(SUPP 16):50–50

    Google Scholar 

  2. Mountney P, Ionasec R, Kaizer M, Mamaghani S, Wu W, Chen T, John M, Boese J, Comaniciu D (2012) Ultrasound and fluoroscopic images fusion by autonomous ultrasound probe detection In: Ayache N, Delingette H, Golland P, Mori K (eds) Medical image computing and computer-assisted intervention—MICCAI 2012: Proceedings of 15th International Conference, Part II, Nice, France, 1–5 October 2012. Springer, Berlin, pp 544–551

  3. Markelj P, Tomaževič D, Likar B, Pernuš F (2012) A review of 3D/2D registration methods for image-guided interventions. Med Image Anal 16(3):642–661

    Article  CAS  PubMed  Google Scholar 

  4. Zöllei L, Grimson E, Norbash A, Wells W (2001) 2D-3D rigid registration of X-ray fluoroscopy and CT images using mutual information and sparsely sampled histogram estimators. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001, vol 2. CVPR 2001. IEEE, pp 2–696

  5. Birkfellner W, Stock M, Figl M, Gendrin C, Hummel J, Dong S, Kettenbach J, Georg D, Bergmann H (2009) Stochastic rank correlation: a robust merit function for 2D/3D registration of image data obtained at different energies. Med Phys 36(8):3420–3428

    Article  PubMed  PubMed Central  Google Scholar 

  6. Westover L (1990) Footprint evaluation for volume rendering. In: ACM siggraph computer graphics, vol 24. ACM, pp 367–376

  7. Hatt Charles R, Speidel Michael A, Raval Amish N (2015) Robust 5D of transesophageal echo probe tracking at fluoroscopic frame rates. In: Medical image computing and computer-assisted intervention–MICCAI 2015. Springer, pp 290–297

  8. Kaiser M, John M, Borsdorf A, Mountney P, Ionasec R, Nöttling A, Kiefer P, Seeburger J, Neumuth T (2013) Significant acceleration of 2D-3D registration-based fusion of ultrasound and X-ray images by mesh-based DRR rendering. In: SPIE medical imaging. International Society for Optics and Photonics, pp 867111–867111

  9. Miao S, Huynh T, Adnet C, Pfister M, Liao R (2013a) Intensity-based 3D-2D mesh-to-image registration using mesh-based digitally reconstructed radiography. In: Augmented reality environments for medical imaging and computer-assisted interventions. Springer, pp 86–96

  10. Mitrović U, Pernuš F, Likar B, Špiclin Ž (2015) Simultaneous 3D–2D image registration and c-arm calibration: application to endovascular image-guided interventions. Med Phys 42(11):6433–6447

    Article  PubMed  Google Scholar 

  11. Banks SA, Hodge WA (June 1996) Accurate measurement of three-dimensional knee replacement kinematics using single-plane fluoroscopy. Biomed Eng IEEE Trans 43(6): 638–649. ISSN 0018-9294. doi:10.1109/10.495283

  12. Aksoy Timur, Unal Gozde, Demirci Stefanie, Navab Nassir, Degertekin Muzaffer (2013) Template-based CTA to X-ray angio rigid registration of coronary arteries in frequency domain with automatic X-ray segmentation. Med Phys 40(10):101903

    Article  PubMed  Google Scholar 

  13. Miao S, Liao R, Lucas J, Chefd’hotel C (2013b) Toward accurate and robust 2-D/3-D registration of implant models to single-plane fluoroscopy. In: Liao H, Linte CA, Masamune K, Peters TM, Zheng G (eds) Augmented reality environments for medical imaging and computer-assisted interventions. Proceedings of 6th International Workshop, MIAR 2013 and 8th International Workshop, AE-CAI 2013, Held in Conjunction with MICCAI 2013, Nagoya, Japan, 22 September 2013. Springer, Berlin, pp 97–106

  14. Gouveia AR, Metz C, Freire L, Almeida P, Klein S (2015) Registration-by-regression of coronary CTA and X-ray angiography. Comput Methods Biomech Biomed Eng Imaging Vis. doi:10.1080/21681163.2015.1054520

  15. Russell S, Norvig P (2009) Artificial intelligence: a modern approach. Prentice Hall Press, Upper Saddle River

    Google Scholar 

  16. Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313

    Article  Google Scholar 

  17. Powell Michael JD (2009) The BOBYQA algorithm for bound constrained optimization without derivatives. Technical report, Centre for Mathematical Sciences, Department of Applied Mathematics and Theoretical Physics

  18. Schumann S, Thelen B, Ballestra S, Nolte L-P, Bchler P, Zheng G (2014) X-ray image calibration and its application to clinical orthopedics. Med Eng Phys 36(7):968–974

    Article  PubMed  Google Scholar 

  19. Gao G, Penney G, Ma Y, Gogin N, Cathier P, Arujuna A, Morton G, Caulfield D, Gill J, Rinaldi CA, Hancock J, Redwood S, Thomas M, Razavi R, Gijsbers G, Rhode K (2012) Registration of 3d trans-esophageal echocardiography to X-ray fluoroscopy using image-based probe tracking. Med Image Anal 16(1):38–49

    Article  PubMed  Google Scholar 

  20. De Kraats EB, Penney GP, Tomaževič D, Van Walsum T, Niessen WJ (2005) Standardized evaluation methodology for 2-D-3-D registration. Med Imaging IEEE Trans 24(9):1177–1189

    Article  Google Scholar 

  21. Kaiser M, John M, Heimann T, Brost A, Neumuth T, Rose G (2014) 2D/3D registration of TEE probe from two non-orthogonal c-arm directions. In: Golland P, Hata N, Barrilot C, Hornegger J, Howe R (eds) Medical image computing and computer-assisted intervention—MICCAI 2014: Proceedings of 17th International Conference, Part I, Boston, USA, 14–18 September 2014. Springer, cham, pp 283–290

  22. Schmid J, Chênes C (2014) Segmentation of X-ray images by 3D-2D registration based on multibody physics. In: Computer vision–ACCV 2014. Springer, pp 674–687

Download references

Author information

Authors and Affiliations

  1. Medical Imaging Technologies, Siemens Healthcare, Princeton, NJ, 08540, USA

    S. Miao, A. Tuysuzoglu & R. Liao

  2. Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, V6T 1Z4, Canada

    Z. J. Wang

Authors
  1. S. Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Tuysuzoglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Z. J. Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Miao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.

Funding

No funding was received for this study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Miao, S., Tuysuzoglu, A., Wang, Z.J. et al. Real-time 6DoF pose recovery from X-ray images using library-based DRR and hybrid optimization. Int J CARS 11, 1211–1220 (2016). https://doi.org/10.1007/s11548-016-1387-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-016-1387-2

Keywords

Search

Navigation