Abstract
Regular group convolutional neural networks (G-CNNs) have been shown to increase model performance and improve equivariance to different geometrical symmetries. This work addresses the problem of SE(3), i.e., roto-translation equivariance, on volumetric data. Volumetric image data is prevalent in many medical settings. Motivated by the recent work on separable group convolutions, we devise a SE(3) group convolution kernel separated into a continuous SO(3) (rotation) kernel and a spatial kernel. We approximate equivariance to the continuous setting by sampling uniform SO(3) grids. Our continuous SO(3) kernel is parameterized via RBF interpolation on similarly uniform grids. We demonstrate the advantages of our approach in volumetric medical image analysis. Our SE(3) equivariant models consistently outperform CNNs and regular discrete G-CNNs on challenging medical classification tasks and show significantly improved generalization capabilities. Our approach achieves up to a 16.5% gain in accuracy over regular CNNs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Our codebase can be accessed at: https://github.com/ThijsKuipers1995/gconv.
References
Armato III, S.G., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. Phys. 38(2), 915–931 (2011)
Bekkers, E.J., B-spline \(\{\rm CNN\}\)s on lie groups. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=H1gBhkBFDH
Bekkers, E.J., Lafarge, M.W., Veta, M., Eppenhof, K.A.J., Pluim, J.P.W., Duits, R.: Roto-translation covariant convolutional networks for medical image analysis. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 440–448. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_50
Bilic, P., et al.: The liver tumor segmentation benchmark (lits). Med. Image Anal. 84, 102680 (2023)
Bogoni, L., et al.: Impact of a computer-aided detection (CAD) system integrated into a picture archiving and communication system (PACS) on reader sensitivity and efficiency for the detection of lung nodules in thoracic CT exams. J. Digit. Imaging 25(6), 771–781 (2012)
Cohen, T.: Learning transformation groups and their invariants. Ph.D. thesis. University of Amsterdam (2013)
Cohen, T., Welling, M.: Group equivariant convolutional networks. In: International Conference on Machine Learning, pp. 2990–2999. PMLR (2016)
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Jin, L., et al.: Deep-learning-assisted detection and segmentation of rib fractures from CT scans: development and validation of FracNet. EBioMedicine 62, 103106 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference for Learning Representations (2015)
Knigge, D.M., Romero, D.W., Bekkers, E.J.: Exploiting redundancy: separable group convolutional networks on lie groups. In: International Conference on Machine Learning, pp. 11359–11386. PMLR (2022)
Kondor, R., Trivedi, S.: On the generalization of equivariance and convolution in neural networks to the action of compact groups. In: International Conference on Machine Learning, pp. 2747–2755. PMLR (2018)
LeCun, Y., Bengio, Y., et al.: Convolutional networks for images, speech, and time series. Handb. Brain Theory Neural Netw. 3361(10), 1995 (1995)
Qi, C.R., et al.: Volumetric and multi-view CNNs for object classification on 3D data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5648–5656 (2016)
Renaud, N., et al.: DeepRank: a deep learning framework for data mining 3D protein-protein interfaces. Nat. Commun. 12(1), 1–8 (2021)
Romero, D.W., Lohit, S.: Learning Equivariances and Partial Equivariances From Data (2022). https://openreview.net/forum?id=jFfRcKVut98
Sifre, L., Mallat, S.: Rotation, scaling and deformation invariant scattering for texture discrimination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1233–1240 (2013)
Thomas, N., et al.: Tensor field networks: rotation-and translation equivariant neural networks for 3d point clouds. arXiv preprint arXiv:1802.08219 (2018)
Weiler, M., et al.: 3D steerable CNNs: learning rotationally equivariant features in volumetric data. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Winkels, M., Cohen, T.S.: 3D G-CNNs for pulmonary nodule detection. arXiv preprint arXiv:1804.04656 (2018)
Winkels, M., Cohen, T.S.: 3D G-CNNs for pulmonary nodule detection. Med. Imaging Deep Learn. (2018). https://openreview.net/forum?id=H1sdHFiif
Worrall, D., Brostow, G.: CubeNet: equivariance to 3D rotation and translation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11209, pp. 585–602. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01228-1_35
Worrall, D.E., et al.: Harmonic networks: deep translation and rotation equivariance. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5028–5037 (2017)
Wu, W., Qi, Z., Fuxin, L.: PointConv: deep convolutional networks on 3D point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9621–9630 (2019)
Xu, X., et al.: Efficient multiple organ localization in CT image using 3D region proposal network. IEEE Trans. Med. Imaging 38(8), 1885–1898 (2019)
Yang, J., Shi, R., Ni, B.: MedMNIST classification decathlon: a lightweight AutoML benchmark for medical image analysis. In: IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 191–195 (2021)
Yang, J., et al.: MedMNIST v2-a large-scale lightweight benchmark for 2D and 3D biomedical image classification. Sci. Data 10(1), 41 (2023)
Acknowledgments
This work was part of the research program VENI with project “context-aware AI in medical image analysis” with number 17290, financed by the Dutch Research Council (NWO). We want to thank SURF for the use of the National Supercomputer Snellius. For providing financial support, we want to thank ELLIS and Qualcomm, and the University of Amsterdam.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kuipers, T.P., Bekkers, E.J. (2023). Regular SE(3) Group Convolutions for Volumetric Medical Image Analysis. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14222. Springer, Cham. https://doi.org/10.1007/978-3-031-43898-1_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-43898-1_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43897-4
Online ISBN: 978-3-031-43898-1
eBook Packages: Computer ScienceComputer Science (R0)
