Abstract
Under consideration are the classification problem and application of simple neural networks for solving the problem. A robust modification of the error backpropagation algorithm is proposed and used for training neural networks. Some proposition is proved that allows us to construct the proposed modification with the Huber loss-function. In order to study the properties of the so-obtained neural network, a number of computational experiments are carried out. We consider various values of the outliers’ fraction, noise level, and training and test samples sizes. Inspection of the results shows that the proposed modification can significantly increase the classification accuracy and learning rate of a neural network when working with noisy data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
Yu. P. Lankin, T. F. Baskanova, and T. I. Lobova, “Neural Network Analysis of Complicated Ecological Data,” in Modern Problems of Science and Education, No. 4 (2012).
V. G. Manzhula and D. S. Fedyashov, “Kohonen Neural Networks and Fuzzy Neural Networks in Data Mining,” in Basic Research, No 4, pp. 108–115 (2011).
Deep neural networks. Part 1: Data Preprocessing URL: https://www.mql5.com/ru/articles/3486.
J. Fan and I. Gijbels, Local Polynomial Modelling and Its Applications (Chapman & Hall, London, 1996).
S. Fujimoto, D. Meger, and D. Precup, “An Equivalence between Loss Functions and Nonuniform Sampling in Experience Replay” (2020); URL: https://papers.nips.cc/paper/2020/file/a3bf6e4db673b6449 c2f7d13 ee6ec9c0-Paper.pdf.
J. T. Barron, A General and Adaptive Robust Loss Function (2017); https://arxiv.org/abs/1701.03077.
P. Andreou, C. Charalambous, and S. Martzoukos, “Robust Artificial Neural Networks for Pricing of European Options,” Comput. Econom. 2 (27), 329–351 (2006).
F. Sebastiani, “Text Categorization,” in Text Mining and Its Applications (WIT Press, Southampton, 2005), pp. 109–129.
C. Bishop, Neural Networks for Pattern Recognition (Oxford Univ. Press, New York, 1995).
D. M. Himmelblau, Applied Nonlinear Programming (McGraw-Hill, 1972; Mir, Moscow, 1975).
J. P. Huber, Robust Statistics (Wiley, Hoboken, New Jersey, 2009).
UCI Machine Learning Repository URL: http://www.ics.uci.edu/ mlearn/MLRepository.html.
A. G. Ivahnenko and V. S. Stepashko, Noise-Resistant Modeling (Naukova Dumka, Kiev, 1985) [in Russian].
Funding
The authors were supported by the Russian Foundation for Basic Research (project no. 20–37–90077).
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Timofeev, V.S., Sivak, M.A. A Robust Neural Network with Simple Architecture. J. Appl. Ind. Math. 15, 670–678 (2021). https://doi.org/10.1134/S1990478921040104
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1990478921040104