End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network

  • Bob D. de VosEmail author
  • Floris F. Berendsen
  • Max A. Viergever
  • Marius Staring
  • Ivana Išgum
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10553)


In this work we propose a deep learning network for deformable image registration (DIRNet). The DIRNet consists of a convolutional neural network (ConvNet) regressor, a spatial transformer, and a resampler. The ConvNet analyzes a pair of fixed and moving images and outputs parameters for the spatial transformer, which generates the displacement vector field that enables the resampler to warp the moving image to the fixed image. The DIRNet is trained end-to-end by unsupervised optimization of a similarity metric between input image pairs. A trained DIRNet can be applied to perform registration on unseen image pairs in one pass, thus non-iteratively. Evaluation was performed with registration of images of handwritten digits (MNIST) and cardiac cine MR scans (Sunnybrook Cardiac Data). The results demonstrate that registration with DIRNet is as accurate as a conventional deformable image registration method with short execution times.


Deep learning Deformable image registration Convolution neural network Spatial transformer Cardiac cine MRI 



This study was funded by the Netherlands Organization for Scientific Research (NWO): project 12726.


  1. 1.
    Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). In: ICLR (2016)Google Scholar
  2. 2.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: PMLR, vol. 37, pp. 448–456 (2015)Google Scholar
  3. 3.
    Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 2017–2025. Curran Associates, Inc., Red Hook (2015)Google Scholar
  4. 4.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)Google Scholar
  5. 5.
    LeCun, Y., Cortes, C.: The MNIST database of handwritten digits (1998)Google Scholar
  6. 6.
    Liao, R., Miao, S., de Tournemire, P., Grbic, S., Kamen, A., Mansi, T., Comaniciu, D.: An artificial agent for robust image registration. arXiv preprint arXiv:1611.10336 (2016)
  7. 7.
    Marstal, K., Berendsen, F., Staring, M., Klein, S.: SimpleElastix: a user-friendly, multi-lingual library for medical image registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2016)Google Scholar
  8. 8.
    Miao, S., Wang, Z.J., Liao, R.: A CNN regression approach for real-time 2D/3D registration. IEEE Trans. Med. Imaging 35(5), 1352–1363 (2016)CrossRefGoogle Scholar
  9. 9.
    Radau, P., Lu, Y., Connelly, K., Paul, G., Dick, A., Wright, G.: Evaluation framework for algorithms segmenting short axis cardiac MRI (2009)Google Scholar
  10. 10.
    Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes, D.J.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18(8), 712–721 (1999)CrossRefGoogle Scholar
  11. 11.
    Wu, G., Kim, M., Wang, Q., Munsell, B.C., Shen, D.: Scalable high performance image registration framework by unsupervised deep feature representations learning. IEEE Trans. Biomed. Eng. 63(7), 1505–1516 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Bob D. de Vos
    • 1
    Email author
  • Floris F. Berendsen
    • 2
  • Max A. Viergever
    • 1
  • Marius Staring
    • 2
  • Ivana Išgum
    • 1
  1. 1.Image Sciences InstituteUniversity Medical Center UtrechtUtrechtThe Netherlands
  2. 2.Division of Image ProcessingLeiden University Medical CenterLeidenThe Netherlands

Personalised recommendations