Abstract
Traditional intensity-based 2D/3D registration requires near-perfect initialization in order for image similarity metrics to yield meaningful updates of X-ray pose and reduce the likelihood of getting trapped in a local minimum. The conventional approaches strongly depend on image appearance rather than content, and therefore, fail in revealing large pose offsets that substantially alter the appearance of the same structure. We complement traditional similarity metrics with a convolutional neural network-based (CNN-based) registration solution that captures large-range pose relations by extracting both local and contextual information, yielding meaningful X-ray pose updates without the need for accurate initialization. To register a 2D X-ray image and a 3D CT scan, our CNN accepts a target X-ray image and a digitally reconstructed radiograph at the current pose estimate as input and iteratively outputs pose updates in the direction of the pose gradient on the Riemannian Manifold. Our approach integrates seamlessly with conventional image-based registration frameworks, where long-range relations are captured primarily by our CNN-based method while short-range offsets are recovered accurately with an image similarity-based method. On both synthetic and real X-ray images of the human pelvis, we demonstrate that the proposed method can successfully recover large rotational and translational offsets, irrespective of initialization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Berger, M., et al.: Marker-free motion correction in weight-bearing cone-beam CT of the knee joint. Med. Phys. 43(3), 1235–1248 (2016)
Bier, B., et al.: Learning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views. Int. J. Comput. Assist. Radiol. Surg. 1–11 (2019)
Bier, B., et al.: X-ray-transform invariant anatomical landmark detection for pelvic trauma surgery. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 55–63. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_7
Clark, K., et al.: The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26(6), 1045–1057 (2013)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Esteban, J., Grimm, M., Unberath, M., Zahnd, G., Navab, N.: Towards fully automatic X-ray to CT registration. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, pp. 631–639. Springer, Heidelberg (2019). https://doi.org/10.1007/978-3-030-32226-7_70
Fotouhi, J., et al.: Pose-aware C-arm for automatic re-initialization of interventional 2D/3D image registration. Int. J. Comput. Assist. Radiol. Surg. 12(7), 1221–1230 (2017)
Gong, R.H., Stewart, J., Abolmaesumi, P.: Multiple-object 2-D-3-D registration for noninvasive pose identification of fracture fragments. IEEE Trans. Biomed. Eng. 58(6), 1592–1601 (2011)
Grupp, R., et al.: Automatic annotation of hip anatomy in fluoroscopy for robust and efficient 2D/3D registration. arXiv preprint arXiv:1911.07042 (2019)
Grupp, R.B., et al.: Pose estimation of periacetabular osteotomy fragments with intraoperative X-ray navigation. arXiv preprint arXiv:1903.09339 (2019)
Hajek, J., Unberath, M., Fotouhi, J., Bier, B., Lee, S.C., Osgood, G., Maier, A., Armand, M., Navab, N.: Closing the calibration loop: an inside-out-tracking paradigm for augmented reality in orthopedic surgery. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 299–306. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_35
Hou, B., et al.: Deep pose estimation for image-based registration. AR (2018)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Ketcha, M., et al.: Multi-stage 3D–2D registration for correction of anatomical deformation in image-guided spine surgery. Phys. Med. Biol. 62(11), 4604 (2017)
Lemieux, L., Jagoe, R., Fish, D., Kitchen, N., Thomas, D.: A patient-to-computed-tomography image registration method based on digitally reconstructed radiographs. Med. Phys. 21(11), 1749–1760 (1994)
Liao, H., Lin, W.A., Zhang, J., Zhang, J., Luo, J., Zhou, S.K.: Multiview 2D/3D rigid registration via a point-of-interest network for tracking and triangulation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12638–12647 (2019)
Liu, L., Ecker, T., Schumann, S., Siebenrock, K., Nolte, L., Zheng, G.: Computer assisted planning and navigation of periacetabular osteotomy with range of motion optimization. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8674, pp. 643–650. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10470-6_80
Liu, R., et al.: An intriguing failing of convolutional neural networks and the CoordConv solution. In: Advances in Neural Information Processing Systems, pp. 9628–9639 (2018)
Markelj, P., Tomaževič, D., Likar, B., Pernuš, F.: A review of 3D/2D registration methods for image-guided interventions. Med. Image Anal. 16(3), 642–661 (2012)
Miao, S., Wang, Z.J., Liao, R.: A CNN regression approach for real-time 2D/3D registration. IEEE Trans. Med. Imaging 35(5), 1352–1363 (2016)
Miolane, N., Mathe, J., Donnat, C., Jorda, M., Pennec, X.: Geomstats: a python package for Riemannian geometry in machine learning (2018)
Murray, R.M.: A Mathematical Introduction to Robotic Manipulation. CRC Press, Boca Raton (2017)
Otake, Y., et al.: Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery: incorporation of fiducial-based C-arm tracking and GPU-acceleration. IEEE Trans. Med. Imaging 31(4), 948–962 (2012)
Ruijters, D., ter Haar Romeny, B.M., Suetens, P.: Vesselness-based 2D–3D registration of the coronary arteries. Int. J. Comput. Assist. Radiol. Surg. 4(4), 391–397 (2009)
Troelsen, A., Elmengaard, B., Søballe, K.: A new minimally invasive transsartorial approach for periacetabular osteotomy. JBJS 90(3), 493–498 (2008)
Unberath, M., et al.: Enabling machine learning in X-ray-based procedures via realistic simulation of image formation. Int. J. Comput. Assist. Radiol. Surg. 1–12 (2019)
Unberath, M., et al.: DeepDRR – a catalyst for machine learning in fluoroscopy-guided procedures. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 98–106. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_12
Yaniv, Z.: Registration for orthopaedic interventions. In: Zheng, G., Li, S. (eds.) Computational Radiology for Orthopaedic Interventions. LNCVB, vol. 23, pp. 41–70. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-23482-3_3
Yao, J., et al.: A C-arm fluoroscopy-guided progressive cut refinement strategy using a surgical robot. Comput. Aided Surg.: Official J. Int. Soc. Comput. Aided Surg. (ISCAS) 5(6), 373–390 (2000)
Acknowledgement
This work is supported in part by NIH grant (R21EB028505).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Gu, W., Gao, C., Grupp, R., Fotouhi, J., Unberath, M. (2020). Extended Capture Range of Rigid 2D/3D Registration by Estimating Riemannian Pose Gradients. In: Liu, M., Yan, P., Lian, C., Cao, X. (eds) Machine Learning in Medical Imaging. MLMI 2020. Lecture Notes in Computer Science(), vol 12436. Springer, Cham. https://doi.org/10.1007/978-3-030-59861-7_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-59861-7_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59860-0
Online ISBN: 978-3-030-59861-7
eBook Packages: Computer ScienceComputer Science (R0)
