Morphological Perceptrons: Geometry and Training Algorithms

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10225)


Neural networks have traditionally relied on mostly linear models, such as the multiply-accumulate architecture of a linear perceptron that remains the dominant paradigm of neuronal computation. However, from a biological standpoint, neuron activity may as well involve inherently nonlinear and competitive operations. Mathematical morphology and minimax algebra provide the necessary background in the study of neural networks made up from these kinds of nonlinear units. This paper deals with such a model, called the morphological perceptron. We study some of its geometrical properties and introduce a training algorithm for binary classification. We point out the relationship between morphological classifiers and the recent field of tropical geometry, which enables us to obtain a precise bound on the number of linear regions of the maxout unit, a popular choice for deep neural networks introduced recently. Finally, we present some relevant numerical results.


Mathematical morphology Neural networks Machine learning Tropical geometry Optimization 



This work was partially supported by the European Union under the projects BabyRobot with grant H2020-687831 and I-SUPPORT with grant H2020-643666.


  1. 1.
    Allamigeon, X., Benchimol, P., Gaubert, S., Joswig, M.: Tropicalizing the simplex algorithm. SIAM J. Discret. Math. 29(2), 751–795 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Araújo, R.D.A., Oliveira, A.L., Meira, S.R: A hybrid neuron with gradient-based learning for binary classification problems. In: Encontro Nacional de Inteligência Artificial-ENIA (2012)Google Scholar
  3. 3.
    Barrera, J., Dougherty, E.R., Tomita, N.S.: Automatic programming of binary morphological machines by design of statistically optimal operators in the context of computational learning theory. J. Electron. Imaging 6(1), 54–67 (1997)CrossRefGoogle Scholar
  4. 4.
    Butkovič, P.: Max-linear Systems: Theory and Algorithms. Springer Science & Business Media, Heidelberg (2010)CrossRefzbMATHGoogle Scholar
  5. 5.
    Cuninghame-Green, R.A.: Minimax Algebra. Lecture Notes in Economics and Mathematical Systems, vol. 166. Springer, Heidelberg (1979)zbMATHGoogle Scholar
  6. 6.
    Davidson, J.L., Hummer, F.: Morphology neural networks: an introduction with applications. Circ. Syst. Sig. Process. 12(2), 177–210 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Diamond, S., Boyd, S.: CVXPY: a Python-embedded modeling language for convex optimization. J. Mach. Learn. Res. 17(83), 1–5 (2016)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Gärtner, B., Jaggi, M.: Tropical support vector machines. Technical report ACS-TR-362502-01 (2008)Google Scholar
  9. 9.
    Gaubert, S., Katz, R.D.: Minimal half-spaces and external representation of tropical polyhedra. J. Algebraic Comb. 33(3), 325–348 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Gondran, M., Minoux, M.: Graphs, Dioids and Semirings: New Models and Algorithms, vol. 41. Springer Science & Business Media, Heidelberg (2008)zbMATHGoogle Scholar
  11. 11.
    Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A.C., Bengio, Y.: Maxout networks. ICML 3(28), 1319–1327 (2013)Google Scholar
  12. 12.
    LeCun, Y., Cortes, C., Burges, C.J.: The MNIST database of handwritten digits (1998)Google Scholar
  13. 13.
    Maclagan, D., Sturmfels, B.: Introduction to Tropical Geometry, vol. 161. American Mathematical Society, Providence (2015)zbMATHGoogle Scholar
  14. 14.
    Maragos, P.: Morphological filtering for image enhancement and feature detection. In: Bovik, A.C. (ed.) The Image and Video Processing Handbook, 2nd edn, pp. 135–156. Elsevier Academic Press, Amsterdam (2005)CrossRefGoogle Scholar
  15. 15.
    Maragos, P.: Dynamical systems on weighted lattices: general theory. arXiv preprint arXiv:1606.07347 (2016)
  16. 16.
    Montufar, G.F., Pascanu, R., Cho, K., Bengio, Y.: On the number of linear regions of deep neural networks. In: Advances in Neural Information Processing Systems, pp. 2924–2932 (2014)Google Scholar
  17. 17.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML 2010, pp. 807–814 (2010)Google Scholar
  18. 18.
    Ripley, B.D.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (2007)zbMATHGoogle Scholar
  19. 19.
    Ritter, G.X., Sussner, P.: An introduction to morphological neural networks. In: 1996 Proceedings of the 13th International Conference on Pattern Recognition, vol. 4, pp. 709–717. IEEE (1996)Google Scholar
  20. 20.
    Ritter, G.X., Urcid, G.: Lattice algebra approach to single-neuron computation. IEEE Trans. Neural Netw. 14(2), 282–295 (2003)CrossRefGoogle Scholar
  21. 21.
    Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386 (1958)CrossRefGoogle Scholar
  22. 22.
    Serra, J.: Image Analysis and Mathematical Morphology, vol. 1. Academic Press, Cambridge (1982)zbMATHGoogle Scholar
  23. 23.
    Shen, X., Diamond, S., Gu, Y., Boyd, S.: Disciplined convex-concave programming. arXiv preprint arXiv:1604.02639 (2016)
  24. 24.
    Sussner, P., Esmi, E.L.: Morphological perceptrons with competitive learning: lattice-theoretical framework and constructive learning algorithm. Inf. Sci. 181(10), 1929–1950 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Sussner, P., Valle, M.E.: Gray-scale morphological associative memories. IEEE Trans. Neural Netw. 17(3), 559–570 (2006)CrossRefGoogle Scholar
  26. 26.
    Vanderbei, R.J., et al.: Linear Programming. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  27. 27.
    Wolberg, W.H., Mangasarian, O.L.: Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proc. Nat. Acad. Sci. 87(23), 9193–9196 (1990)CrossRefzbMATHGoogle Scholar
  28. 28.
    Xu, L., Crammer, K., Schuurmans, D.: Robust support vector machine training via convex outlier ablation. In: AAAI, vol. 6, pp. 536–542 (2006)Google Scholar
  29. 29.
    Yang, P.F., Maragos, P.: Min-max classifiers: learnability, design and application. Pattern Recogn. 28(6), 879–899 (1995)CrossRefGoogle Scholar
  30. 30.
    Yuille, A.L., Rangarajan, A.: The concave-convex procedure. Neural Comput. 15(4), 915–936 (2003)CrossRefzbMATHGoogle Scholar
  31. 31.
    Ziegler, G.M.: Lectures on Polytopes, vol. 152. Springer Science & Business Media, Heidelberg (1995)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of ECENational Technical University of AthensAthensGreece

Personalised recommendations