Abstract
Recent advances in machine learning have highlighted the importance of using group equivariant non-expansive operators for building neural networks in a more transparent and interpretable way. An operator is called equivariant with respect to a group if the action of the group commutes with the operator. Group equivariant non-expansive operators can be seen as multi-level components that can be joined and connected in order to form neural networks by applying the operations of chaining, convex combination and direct product. In this paper we prove that each linear G-equivariant non-expansive operator (GENEO) can be produced by a weighted summation associated with a suitable permutant measure, provided that the group G transitively acts on a finite signal domain. This result is based on the Birkhoff–von Neumann decomposition of doubly stochastic matrices and some well known facts in group theory. Our theorem makes available a new method to build all linear GENEOs with respect to a transitively acting group in the finite setting. This work is part of the research devoted to develop a good mathematical theory of GENEOs, seen as relevant components in machine learning.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Anselmi, F., Evangelopoulos, G., Rosasco, L., Poggio, T.: Symmetry-adapted representation learning. Pattern Recogn. 86, 201–208 (2019). https://doi.org/10.1016/j.patcog.2018.07.025. http://www.sciencedirect.com/science/article/pii/S0031320318302620
Anselmi, F., Rosasco, L., Poggio, T.: On invariance and selectivity in representation learning. Information and Inference: A Journal of the IMA 5(2), 134–158 (2016). https://doi.org/10.1093/imaiai/iaw009
Aschbacher, M.: Finite group theory, 2 edn. Cambridge Studies in Advanced Mathematics Cambridge University Press. https://doi.org/10.1017/CBO9781139175319 (2000)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Bergomi, M.G., Frosini, P, Giorgi, D, Quercioli, N: Towards a topological–geometrical theory of group equivariant non-expansive operators for data analysis and machine learning. Nature Machine Intelligence 1(9), 423–433 (2019). https://doi.org/10.1038/s42256-019-0087-3
Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A., Vandergheynst, P.: Geometric deep learning: going beyond Euclidean data. IEEE Signal Process. Mag. 34 (4), 18–42 (2017). https://doi.org/10.1109/MSP.2017.2693418
Camporesi, F., Frosini, P., Quercioli, N.: On a new method to build group equivariant operators by means of permutants. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) 2nd International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MAKE), Machine Learning and Knowledge Extraction, vol. LNCS-11015, pp. 265–272. Springer International Publishing, Hamburg, Germany . https://doi.org/10.1007/978-3-319-99740-7_18. https://hal.inria.fr/hal-02060057. Part 4: MAKE-Topology (2018)
Cohen, T., Welling, M.: Group equivariant convolutional networks. In: International Conference on Machine Learning, pp. 2990–2999 (2016)
Conti, F., Frosini, P., Quercioli, N.: On the construction of group equivariant non-expansive operators via permutants and symmetric functions. Frontiers in Artificial Intelligence vol. 5. https://doi.org/10.3389/frai.2022.786091. https://www.frontiersin.org/articles/10.3389/frai.2022.786091 (2022)
Donatini, P., Frosini, P.: Natural pseudodistances between closed manifolds. Forum Math. 16(5), 695–715 (2004). https://doi.org/10.1515/form.2004.032
Donatini, P., Frosini, P.: Natural pseudodistances between closed surfaces. J. Eur. Math. Soc. (JEMS) 9(2), 331–353 (2007)
Donatini, P., Frosini, P.: Natural pseudo-distances between closed curves. Forum Math. 21(6), 981–999 (2009). https://doi.org/10.1515/FORUM.2009.049
Dufossé, F., Kaya, K., Panagiotas, I., Uçar, B.: Further notes on Birkhoff–von Neumann decomposition of doubly stochastic matrices. Linear Algebra Appl. 554, 68–78 (2018). https://doi.org/10.1016/j.laa.2018.05.017. http://www.sciencedirect.com/science/article/pii/S0024379518302568
Frosini, P.: Does intelligence imply contradiction? Cogn. Syst. Res. 10, 297–315 (2009)
Frosini, P.: Towards an observer-oriented theory of shape comparison. In: Ferreira, A., Giachetti, A., Giorgi, D. (eds.) Eurographics Workshop on 3D Object Retrieval. The Eurographics Association. https://doi.org/10.2312/3dor.20161080 (2016)
Frosini, P., Jabłoński, G.: Combining persistent homology and invariance groups for shape comparison. Discrete Comput. Geom 55(2), 373–409 (2016). https://doi.org/10.1007/s00454-016-9761-y
Frosini, P., Mulazzani, M.: Size homotopy groups for computation of natural size distances. Bull. Belg. Math. Soc. Simon Stevin 6(3), 455–464 (1999). http://projecteuclid.org.ezproxy.unibo.it/euclid.bbms/1103065863
Frosini, P., Quercioli, N.: Some remarks on the algebraic properties of group invariant operators in persistent homology. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) 1st International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MAKE), Machine Learning and Knowledge Extraction, vol. LNCS-10410, pp. 14–24. Springer International Publishing, Reggio, Italy. https://doi.org/10.1007/978-3-319-66808-6_2. https://hal.inria.fr/hal-01677132. Part 1: MAKE Topology (2017)
Kondor, R., Trivedi, S.: On the generalization of equivariance and convolution in neural networks to the action of compact groups. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 80, pp. 2747–2755. PMLR, Stockholmsmässan, Stockholm Sweden. http://proceedings.mlr.press/v80/kondor18a.html (2018)
Mallat, S.: Group invariant scattering. Commun. Pure Appl. Math. 65(10), 1331–1398 (2012). https://doi.org/10.1002/cpa.21413
Mallat, S.: Understanding deep convolutional networks. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 374(2065), 20150203 (2016). https://doi.org/10.1098/rsta.2015.0203
Worrall, D.E., Garbin, S.J., Turmukhambetov, D., Brostow, G.J.: Harmonic networks: deep translation and rotation equivariance. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), vol. 2 (2017)
Zhang, C., Voinea, S., Evangelopoulos, G., Rosasco, L., Poggio, T.: Discriminative template learning in group-convolutional networks for invariant speech representations. In: INTERSPEECH-2015. International Speech Communication Association (ISCA), International Speech Communication Association (ISCA), Dresden, Germany. http://www.isca-speech.org/archive/interspeech_2015/i15_3229.html (2015)
Acknowledgements
This research has been partially supported by INdAM-GNSAGA. The authors thank Alessandro Achille, Fabio Anselmi and Giovanni Paolini for their helpful advice. This paper is dedicated to the memory of Dr. Mohamed Mashally.
Funding
Open access funding provided by Alma Mater Studiorum - Università di Bologna within the CRUI-CARE Agreement.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no conflict of interest. P.F. devised the project. All authors contributed to the manuscript. All authors read and approved the final manuscript. The authors of this paper have been listed in alphabetical order. Data available on request from the authors.
Additional information
Open Access
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This research has been partially supported by INdAM-GNSAGA.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Bocchi, G., Botteghi, S., Brasini, M. et al. On the finite representation of linear group equivariant operators via permutant measures. Ann Math Artif Intell 91, 465–487 (2023). https://doi.org/10.1007/s10472-022-09830-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10472-022-09830-1