Abstract
Outliers and gross errors in training data sets can seriously deteriorate the performance of traditional supervised feedforward neural networks learning algorithms. This is why several learning methods, to some extent robust to outliers, have been proposed. In this paper we present a new robust learning algorithm based on the iterative Least Median of Squares, that outperforms some existing solutions in its accuracy or speed. We demonstrate how to minimise new non-differentiable performance function by a deterministic approximate method. Results of simulations and comparison with other learning methods are demonstrated. Improved robustness of our novel algorithm, for data sets with varying degrees of outliers, is shown.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Chen DS, Jain RC (1994) A robust back propagation learning algorithm for function approximation. IEEE Trans Neural Netw 5: 467–479
Chuang C, Su S, Hsiao C (2000) The annealing robust backpropagation (ARBP) learning algorithm. IEEE Trans Neural Netw 11: 1067–1076
Chuang CC, Jeng JT, Lin PT (2004) Annealing robust radial basis function networks for function approximation with outliers. Neurocomputing 56: 123–139
David Sanchez VA (1995) Robustization of a learning method for RBFnetworks. Neurocomputing 9: 85–94
Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7: 130
El-Melegy MT, Essai MH, Ali AA (2009) Robust training of artificial feedforward neural networks. Found Comput Intell 1: 217–242
Hagan MT, Demuth HB, Beale MH (1996) Neural network design. PWS Publishing, Boston
Hampel FR, Ronchetti EM, Rousseeuw PJ, Stahel WA (1986) Robust statistics the approach based on influence functions. Wiley, New York
Haykin S (1999) Neural networks—a comprehensive foundation, 2nd edn. Prentice Hall, Upper Saddle River
Hornik K, Stinchconbe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2: 359–366
Huber PJ (1981) Robust statistics. Wiley, New York
Igel C, Husken M (2003) Empirical evaluation of the improved Rprop learning algorithms. Neurocomputing 50: 105–123
Jing X (2012) Robust adaptive learning of feedforward neural networks via LMI optimizations. Neural Netw 31: 33–45
Liano K (1996) Robust error measure for supervised neural network learning with outliers. IEEE Trans Neural Netw 7: 246–250
Olive DJ, Hawkins DM (2007) Robustifying robust estimators. Springer, New York
Pernia-Espinoza AV, Ordieres-Mere JB, Martinez-de-Pison FJ, Gonzalez-Marcos A (2005) TAO-robust backpropagation learning algorithm. Neural Netw 18: 191–204
Riedmiller M, Braun H (1993) A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: Proceedings of the IEEE International Conference on Neural Networks. ICNN, San Francisco, pp 586–591
Rousseeuw PJ (1984) Least median of squares regression. J Am Stat Assoc 79: 871–880
Rousseeuw AM, Leroy PJ (1987) Robust regression and outlier detection. Wiley, New York
Rusiecki AL (2005) Fault tolerant feedforward neural network with median neuron input function. Electron Lett 41(10): 603–605
Rusiecki AL et al (2008) Robust MCD-based backpropagation learning algorithm. In: Rusiecki AL (eds) ICAISC 2008. LNCS (LNAI), vol 5097. Springer, New York, pp 154–163
Sun S, Jin F (2011) Robust co-training. Int J Pattern Recognit Artif Intell 25(7): 1113–1126
Open Access
This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Rusiecki, A. Robust Learning Algorithm Based on Iterative Least Median of Squares. Neural Process Lett 36, 145–160 (2012). https://doi.org/10.1007/s11063-012-9227-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-012-9227-z