Non-rigid Craniofacial 2D-3D Registration Using CNN-Based Regression

  • Yuru Pei
  • Yungeng Zhang
  • Haifang Qin
  • Gengyu Ma
  • Yuke Guo
  • Tianmin Xu
  • Hongbin Zha
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10553)


The 2D-3D registration is a cornerstone to align the inter-treatment X-ray images with the available volumetric images. In this paper, we propose a CNN regression based non-rigid 2D-3D registration method. An iterative refinement scheme is introduced to update the reference volumetric image and the digitally-reconstructed-radiograph (DRR) for convergence to the target X-ray image. The CNN-based regressor represents the mapping between an image pair and the in-between deformation parameters. In particular, the short residual connections in the convolution blocks and long jump connections for the multi-scale feature map fusion facilitate the information propagation in training the regressor. The proposed method has been applied to 2D-3D registration of synthetic X-ray and clinically-captured CBCT images. Experimental results demonstrate the proposed method realizes an accurate and efficient 2D-3D registration of craniofacial images.



This work was supported by National Natural Science Foundation of China under Grant 61272342.


  1. 1.
    Groher, M., Bender, F., Hoffmann, R.T., Navab, N.: Segmentation-driven 2D–3D registration for abdominal catheter interventions. In: Medical image computing and computer-assisted intervention-MICCAI, pp. 527–535 (2007)Google Scholar
  2. 2.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  3. 3.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: ACM Multimedia, pp. 675–678 (2014)Google Scholar
  4. 4.
    Mahfouz, M.R., Hoff, W.A., Komistek, R.D., Dennis, D.A.: Effect of segmentation errors on 3D-to-2D registration of implant models in X-ray images. J. Biomech. 38(2), 229–239 (2005)CrossRefGoogle Scholar
  5. 5.
    Miao, S., Wang, Z.J., Liao, R.: A CNN regression approach for real-time 2D/3D registration. IEEE Trans. Med. imaging 35(5), 1352–1363 (2016)CrossRefGoogle Scholar
  6. 6.
    Pei, Y., Dai, F., Xu, T., Zha, H., Ma, G.: Volumetric reconstruction of craniofacial structures from 2D lateral cephalograms by regression forest. In: IEEE ICIP, pp. 4052–4056 (2016)Google Scholar
  7. 7.
    Perona, P., Shiota, T., Malik, J.: Anisotropic diffusion. In: Geometry-Driven Diffusion in Computer Vision, pp. 73–92 (1994)Google Scholar
  8. 8.
    Shen, D., Wu, G., Suk, H.I.: Deep learning in medical image analysis. Ann. Rev. Biomed. Eng. 19, 221–248 (2017)CrossRefGoogle Scholar
  9. 9.
    Steininger, P., Neuner, M., Birkfellner, W., Gendrin, C., Mooslechner, M., Bloch, C., Pawiro, S., Sedlmayer, F., Deutschmann, H.: An ITK-based implementation of the stochastic rank correlation (SRC) metric. Insight J. 11, 5 (2010)Google Scholar
  10. 10.
    Whitmarsh, T., Humbert, L., De Craene, M., Luis, M., Fritscher, K., Schubert, R., Eckstein, F., Link, T., Frangi, A.F.: 3D bone mineral density distribution and shape reconstruction of the proximal femur from a single simulated DXA image: an in vitro study. In: SPIE Medical Imaging, pp. 76234U–76234U (2010)Google Scholar
  11. 11.
    Yu, W., Tannast, M., Zheng, G.: Non-rigid free-form 2D–3D registration using a B-spline-based statistical deformation model. Pattern Recogn. 63, 689–699 (2017)CrossRefGoogle Scholar
  12. 12.
    Zheng, G.: Effective incorporating spatial information in a mutual information based 3D–2D registration of a CT volume to X-ray images. Comput. Med. Imaging Graph. 34(7), 553–562 (2010)CrossRefGoogle Scholar
  13. 13.
    Zheng, G.: Personalized X-ray reconstruction of the proximal femur via intensity-based non-rigid 2D–3D registration. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI, pp. 598–606 (2011)Google Scholar
  14. 14.
    Zheng, G.: 3D volumetric intensity reconsturction from 2D X-ray images using partial least squares regression. In: IEEE ISBI, pp. 1268–1271 (2013)Google Scholar
  15. 15.
    Zöllei, L.: 2D–3D rigid-body registration of X-ray Fluoroscopy and CT images. Ph.D. thesis, Harvard Medical School (2001)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yuru Pei
    • 1
  • Yungeng Zhang
    • 1
  • Haifang Qin
    • 1
  • Gengyu Ma
    • 2
  • Yuke Guo
    • 3
  • Tianmin Xu
    • 4
  • Hongbin Zha
    • 1
  1. 1.Key Laboratory of Machine Perception (MOE), Department of Machine IntelligencePeking UniversityBeijingChina
  2. 2.USens Inc.San JoseUSA
  3. 3.Luoyang Institute of Science and TechnologyLuoyangChina
  4. 4.School of StomatologyPeking UniversityBeijingChina

Personalised recommendations