Roto-Translation Covariant Convolutional Networks for Medical Image Analysis

  • Erik J. BekkersEmail author
  • Maxime W. LafargeEmail author
  • Mitko Veta
  • Koen A. J. Eppenhof
  • Josien P. W. Pluim
  • Remco Duits
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11070)


We propose a framework for rotation and translation covariant deep learning using SE(2) group convolutions. The group product of the special Euclidean motion group SE(2) describes how a concatenation of two roto-translations results in a net roto-translation. We encode this geometric structure into convolutional neural networks (CNNs) via SE(2) group convolutional layers, which fit into the standard 2D CNN framework, and which allow to generically deal with rotated input samples without the need for data augmentation.

We introduce three layers: a lifting layer which lifts a 2D (vector valued) image to an SE(2)-image, i.e., 3D (vector valued) data whose domain is SE(2); a group convolution layer from and to an SE(2)-image; and a projection layer from an SE(2)-image to a 2D image. The lifting and group convolution layers are SE(2) covariant (the output roto-translates with the input). The final projection layer, a maximum intensity projection over rotations, makes the full CNN rotation invariant.

We show with three different problems in histopathology, retinal imaging, and electron microscopy that with the proposed group CNNs, state-of-the-art performance can be achieved, without the need for data augmentation by rotation and with increased performance compared to standard CNNs that do rely on augmentation.


Group convolutional network Roto-translation group Mitosis detection Vessel segmentation Cell boundary segmentation 



The research leading to these results has received funding from the ERC council under the EC’s 7th Framework Programme (FP7/2007–2013)/ERC grant agr. No. 335555.


  1. 1.
    Worrall, D.E., Garbin, S.J., Turmukhambetov, D., Brostow, G.J.: Harmonic networks: deep translation and rotation equivariance. In: CVPR, pp. 5028–5037 (2017)Google Scholar
  2. 2.
    Sohn, K., Lee, H.: Learning invariant representations with local transformations. In: CVPR, pp. 1339–1346. Omnipress (2012)Google Scholar
  3. 3.
    Gens, R., Domingos, P.M.: Deep symmetry networks. In: Advances in Neural Information Processing Systems, pp. 2537–2545 (2014)Google Scholar
  4. 4.
    Sifre, L., Mallat, S.: Rotation, scaling and deformation invariant scattering for texture discrimination. In: CVPR, pp. 1233–1240. IEEE (2013)Google Scholar
  5. 5.
    Oyallon, E., Mallat, S., Sifre, L.: Generic deep networks with wavelet scattering. arXiv preprint arXiv:1312.5940 (2013)
  6. 6.
    Henriques, J.F., Vedaldi, A.: Warped convolutions: efficient invariance to spatial transformations. In: International Conference on Machine Learning, pp. 1461–1469 (2017)Google Scholar
  7. 7.
    Cohen, T., Welling, M.: Group equivariant convolutional networks. In: International Conference on Machine Learning, pp. 2990–2999 (2016)Google Scholar
  8. 8.
    Dieleman, S., De Fauw, J., Kavukcuoglu, K.: Exploiting cyclic symmetry in convolutional neural networks. arXiv preprint arXiv:1602.02660 (2016)
  9. 9.
    Weiler, M., Hamprecht, F.A., Storath, M.: Learning steerable filters for rotation equivariant CNNs. arXiv preprint arXiv:1711.07289 (2017)
  10. 10.
    Bekkers, E.J., Loog, M., ter Haar Romeny, B.M., Duits, R.: Template matching via densities on the roto-translation group. IEEE tPAMI 40(2), 452–466 (2018)CrossRefGoogle Scholar
  11. 11.
    Duits, R., Felsberg, M., Granlund, G.H., ter Haar Romeny, B.M.: Image analysis and reconstruction using a wavelet transform constructed from a reducible representation of the Euclidean motion group. IJCV 72(1), 79–102 (2007)CrossRefGoogle Scholar
  12. 12.
    Lafarge, M.W., Pluim, J.P.W., Eppenhof, K.A.J., Moeskops, P., Veta, M.: Domain-adversarial neural networks to address the appearance variability of histopathology images. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 83–91. Springer, Cham (2017). Scholar
  13. 13.
    Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Mitosis detection in breast cancer histology images with deep neural networks. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 411–418. Springer, Heidelberg (2013). Scholar
  14. 14.
    Veta, M., et al.: Assessment of algorithms for mitosis detection in breast cancer histopathology images. MEDIA 20(1), 237–248 (2015)Google Scholar
  15. 15.
    Staal, J., et al.: Ridge-based vessel segmentation in color images of the retina. IEEE TMI 23(4), 501–509 (2004)Google Scholar
  16. 16.
    Arganda-Carreras, I., et al.: Crowdsourcing the creation of image segmentation algorithms for connectomics. Front. Neuroanat. 9, 142 (2015)CrossRefGoogle Scholar
  17. 17.
    Rand, W.M.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66(336), 846–850 (1971)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Erik J. Bekkers
    • 1
    Email author
  • Maxime W. Lafarge
    • 2
    Email author
  • Mitko Veta
    • 2
  • Koen A. J. Eppenhof
    • 2
  • Josien P. W. Pluim
    • 2
  • Remco Duits
    • 1
  1. 1.Department of Mathematics and Computer ScienceEindhoven University of TechnologyEindhovenThe Netherlands
  2. 2.Department of Biomedical EngineeringEindhoven University of TechnologyEindhovenThe Netherlands

Personalised recommendations