Metric-Driven Learning of Correspondence Weighting for 2-D/3-D Image Registration

  • Roman SchaffertEmail author
  • Jian Wang
  • Peter Fischer
  • Anja Borsdorf
  • Andreas Maier
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11269)


Registration of pre-operative 3-D volumes to intra-operative 2-D X-ray images is important in minimally invasive medical procedures. Rigid registration can be performed by estimating a global rigid motion that optimizes the alignment of local correspondences. However, inaccurate correspondences challenge the registration performance. To minimize their influence, we estimate optimal weights for correspondences using PointNet. We train the network directly with the criterion to minimize the registration error. We propose an objective function which includes point-to-plane correspondence-based motion estimation and projection error computation, thereby enabling the learning of a weighting strategy that optimally fits the underlying formulation of the registration task in an end-to-end fashion. For single-vertebra registration, we achieve an accuracy of \(0.74\pm 0.26\) mm and highly improved robustness. The success rate is increased from 79.3% to 94.3% and the capture range from 3 mm to 13 mm.


Medical image registration 2-D/3-D registration Deep learning Point-to-plane correspondence model 

Supplementary material

Supplementary material 1 (mp4 11977 KB)


  1. 1.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–698 (1986)CrossRefGoogle Scholar
  2. 2.
    Elliott, D.L.: A better activation function for artificial neural networks. Technical report (1993)Google Scholar
  3. 3.
    Feng, Y., Huang, X., Shi, L., Yang, Y., Suykens, J.A.: Learning with the maximum correntropy criterion induced losses for regression. J. Mach. Learn. Res. 16, 993–1034 (2015)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn, p. 200. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  5. 5.
    van de Kraats, E.B., Penney, G.P., Tomaževič, D., van Walsum, T., Niessen, W.J.: Standardized evaluation methodology for 2-D-3-D registration. IEEE Trans. Med. imaging 24(9), 1177–1189 (2005)CrossRefGoogle Scholar
  6. 6.
    Kubias, A., Deinzer, F., Feldmann, T., Paulus, D., Schreiber, B., Brunner, T.: 2D/3D image registration on the GPU. Pattern Recogn. Image Anal. 18(3), 381–389 (2008)CrossRefGoogle Scholar
  7. 7.
    Maier, A., et al.: Precision learning: towards use of known operators in neural networks. arXiv preprint arXiv:1712.00374v3 (2017)
  8. 8.
    Markelj, P., Tomaževič, D., Likar, B., Pernuš, F.: A review of 3D/2D registration methods for image-guided interventions. Med. Image Anal. 16(3), 642–661 (2012)CrossRefGoogle Scholar
  9. 9.
    Miao, S., et al.: Dilated FCN for multi-agent 2D/3D medical image registration. In: AAAI Conference on Artificial Intelligence (AAAI), pp. 4694–4701 (2018)Google Scholar
  10. 10.
    Mitrović, U., Špiclin, Ž., Likar, B., Pernuš, F.: 3D–2D registration of cerebral angiograms: a method and evaluation on clinical images. IEEE Trans. Med. Imaging 32(8), 1550–1563 (2013)CrossRefGoogle Scholar
  11. 11.
    Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77–85 (2017)Google Scholar
  12. 12.
    Schaffert, R., Wang, J., Fischer, P., Borsdorf, A., Maier, A.: Multi-view depth-aware rigid 2-D/3-D registration. In: IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC) (2017)Google Scholar
  13. 13.
    Schmid, J., Chênes, C.: Segmentation of X-ray images by 3D-2D registration based on multibody physics. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9004, pp. 674–687. Springer, Cham (2015). Scholar
  14. 14.
    Wang, J., Borsdorf, A., Heigl, B., Köhler, T., Hornegger, J.: Gradient-based differential approach for 3-D motion compensation in interventional 2-D/3-D image fusion. In: International Conference on 3D Vision (3DV), pp. 293–300 (2014)Google Scholar
  15. 15.
    Wang, J., et al.: Dynamic 2-D/3-D rigid registration framework using point-to-plane correspondence model. IEEE Trans. Med. Imaging 36(9), 1939–1954 (2017)CrossRefGoogle Scholar
  16. 16.
    Yi, K.M., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., Fua, P.: Learning to find good correspondences. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2666–2674 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Pattern Recognition Lab, Friedrich-Alexander Universität Erlangen-NürnbergErlangenGermany
  2. 2.Siemens Healthineers AGForchheimGermany
  3. 3.Graduate School in Advanced Optical Technologies (SAOT)ErlangenGermany

Personalised recommendations