Abstract
Neural networks have traditionally relied on mostly linear models, such as the multiply-accumulate architecture of a linear perceptron that remains the dominant paradigm of neuronal computation. However, from a biological standpoint, neuron activity may as well involve inherently nonlinear and competitive operations. Mathematical morphology and minimax algebra provide the necessary background in the study of neural networks made up from these kinds of nonlinear units. This paper deals with such a model, called the morphological perceptron. We study some of its geometrical properties and introduce a training algorithm for binary classification. We point out the relationship between morphological classifiers and the recent field of tropical geometry, which enables us to obtain a precise bound on the number of linear regions of the maxout unit, a popular choice for deep neural networks introduced recently. Finally, we present some relevant numerical results.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The term “tropical” was playfully introduced by French mathematicians in honor of the Brazilian theoretical computer scientist, Imre Simon. Another example of a tropical semiring is the \((\max , \times )\) semiring, also referred to as the subtropical semiring.
- 2.
The matrix \(-{\varvec{A}}^T\), often denoted by \({\varvec{A}}^{\sharp }\) in the tropical geometry community, is sometimes called the Cuninghame-Green inverse of \({\varvec{A}}\).
References
Allamigeon, X., Benchimol, P., Gaubert, S., Joswig, M.: Tropicalizing the simplex algorithm. SIAM J. Discret. Math. 29(2), 751–795 (2015)
Araújo, R.D.A., Oliveira, A.L., Meira, S.R: A hybrid neuron with gradient-based learning for binary classification problems. In: Encontro Nacional de Inteligência Artificial-ENIA (2012)
Barrera, J., Dougherty, E.R., Tomita, N.S.: Automatic programming of binary morphological machines by design of statistically optimal operators in the context of computational learning theory. J. Electron. Imaging 6(1), 54–67 (1997)
Butkovič, P.: Max-linear Systems: Theory and Algorithms. Springer Science & Business Media, Heidelberg (2010)
Cuninghame-Green, R.A.: Minimax Algebra. Lecture Notes in Economics and Mathematical Systems, vol. 166. Springer, Heidelberg (1979)
Davidson, J.L., Hummer, F.: Morphology neural networks: an introduction with applications. Circ. Syst. Sig. Process. 12(2), 177–210 (1993)
Diamond, S., Boyd, S.: CVXPY: a Python-embedded modeling language for convex optimization. J. Mach. Learn. Res. 17(83), 1–5 (2016)
Gärtner, B., Jaggi, M.: Tropical support vector machines. Technical report ACS-TR-362502-01 (2008)
Gaubert, S., Katz, R.D.: Minimal half-spaces and external representation of tropical polyhedra. J. Algebraic Comb. 33(3), 325–348 (2011)
Gondran, M., Minoux, M.: Graphs, Dioids and Semirings: New Models and Algorithms, vol. 41. Springer Science & Business Media, Heidelberg (2008)
Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A.C., Bengio, Y.: Maxout networks. ICML 3(28), 1319–1327 (2013)
LeCun, Y., Cortes, C., Burges, C.J.: The MNIST database of handwritten digits (1998)
Maclagan, D., Sturmfels, B.: Introduction to Tropical Geometry, vol. 161. American Mathematical Society, Providence (2015)
Maragos, P.: Morphological filtering for image enhancement and feature detection. In: Bovik, A.C. (ed.) The Image and Video Processing Handbook, 2nd edn, pp. 135–156. Elsevier Academic Press, Amsterdam (2005)
Maragos, P.: Dynamical systems on weighted lattices: general theory. arXiv preprint arXiv:1606.07347 (2016)
Montufar, G.F., Pascanu, R., Cho, K., Bengio, Y.: On the number of linear regions of deep neural networks. In: Advances in Neural Information Processing Systems, pp. 2924–2932 (2014)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML 2010, pp. 807–814 (2010)
Ripley, B.D.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (2007)
Ritter, G.X., Sussner, P.: An introduction to morphological neural networks. In: 1996 Proceedings of the 13th International Conference on Pattern Recognition, vol. 4, pp. 709–717. IEEE (1996)
Ritter, G.X., Urcid, G.: Lattice algebra approach to single-neuron computation. IEEE Trans. Neural Netw. 14(2), 282–295 (2003)
Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386 (1958)
Serra, J.: Image Analysis and Mathematical Morphology, vol. 1. Academic Press, Cambridge (1982)
Shen, X., Diamond, S., Gu, Y., Boyd, S.: Disciplined convex-concave programming. arXiv preprint arXiv:1604.02639 (2016)
Sussner, P., Esmi, E.L.: Morphological perceptrons with competitive learning: lattice-theoretical framework and constructive learning algorithm. Inf. Sci. 181(10), 1929–1950 (2011)
Sussner, P., Valle, M.E.: Gray-scale morphological associative memories. IEEE Trans. Neural Netw. 17(3), 559–570 (2006)
Vanderbei, R.J., et al.: Linear Programming. Springer, Heidelberg (2015)
Wolberg, W.H., Mangasarian, O.L.: Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proc. Nat. Acad. Sci. 87(23), 9193–9196 (1990)
Xu, L., Crammer, K., Schuurmans, D.: Robust support vector machine training via convex outlier ablation. In: AAAI, vol. 6, pp. 536–542 (2006)
Yang, P.F., Maragos, P.: Min-max classifiers: learnability, design and application. Pattern Recogn. 28(6), 879–899 (1995)
Yuille, A.L., Rangarajan, A.: The concave-convex procedure. Neural Comput. 15(4), 915–936 (2003)
Ziegler, G.M.: Lectures on Polytopes, vol. 152. Springer Science & Business Media, Heidelberg (1995)
Acknowledgements
This work was partially supported by the European Union under the projects BabyRobot with grant H2020-687831 and I-SUPPORT with grant H2020-643666.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Charisopoulos, V., Maragos, P. (2017). Morphological Perceptrons: Geometry and Training Algorithms. In: Angulo, J., Velasco-Forero, S., Meyer, F. (eds) Mathematical Morphology and Its Applications to Signal and Image Processing. ISMM 2017. Lecture Notes in Computer Science(), vol 10225. Springer, Cham. https://doi.org/10.1007/978-3-319-57240-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-57240-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-57239-0
Online ISBN: 978-3-319-57240-6
eBook Packages: Computer ScienceComputer Science (R0)