Morphological Perceptrons: Geometry and Training Algorithms
Neural networks have traditionally relied on mostly linear models, such as the multiply-accumulate architecture of a linear perceptron that remains the dominant paradigm of neuronal computation. However, from a biological standpoint, neuron activity may as well involve inherently nonlinear and competitive operations. Mathematical morphology and minimax algebra provide the necessary background in the study of neural networks made up from these kinds of nonlinear units. This paper deals with such a model, called the morphological perceptron. We study some of its geometrical properties and introduce a training algorithm for binary classification. We point out the relationship between morphological classifiers and the recent field of tropical geometry, which enables us to obtain a precise bound on the number of linear regions of the maxout unit, a popular choice for deep neural networks introduced recently. Finally, we present some relevant numerical results.
KeywordsMathematical morphology Neural networks Machine learning Tropical geometry Optimization
This work was partially supported by the European Union under the projects BabyRobot with grant H2020-687831 and I-SUPPORT with grant H2020-643666.
- 2.Araújo, R.D.A., Oliveira, A.L., Meira, S.R: A hybrid neuron with gradient-based learning for binary classification problems. In: Encontro Nacional de Inteligência Artificial-ENIA (2012)Google Scholar
- 8.Gärtner, B., Jaggi, M.: Tropical support vector machines. Technical report ACS-TR-362502-01 (2008)Google Scholar
- 11.Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A.C., Bengio, Y.: Maxout networks. ICML 3(28), 1319–1327 (2013)Google Scholar
- 12.LeCun, Y., Cortes, C., Burges, C.J.: The MNIST database of handwritten digits (1998)Google Scholar
- 15.Maragos, P.: Dynamical systems on weighted lattices: general theory. arXiv preprint arXiv:1606.07347 (2016)
- 16.Montufar, G.F., Pascanu, R., Cho, K., Bengio, Y.: On the number of linear regions of deep neural networks. In: Advances in Neural Information Processing Systems, pp. 2924–2932 (2014)Google Scholar
- 17.Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML 2010, pp. 807–814 (2010)Google Scholar
- 19.Ritter, G.X., Sussner, P.: An introduction to morphological neural networks. In: 1996 Proceedings of the 13th International Conference on Pattern Recognition, vol. 4, pp. 709–717. IEEE (1996)Google Scholar
- 23.Shen, X., Diamond, S., Gu, Y., Boyd, S.: Disciplined convex-concave programming. arXiv preprint arXiv:1604.02639 (2016)
- 28.Xu, L., Crammer, K., Schuurmans, D.: Robust support vector machine training via convex outlier ablation. In: AAAI, vol. 6, pp. 536–542 (2006)Google Scholar