Robust LTS Backpropagation Learning Algorithm

  • Andrzej Rusiecki
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4507)


Training data sets containing outliers are often a problem for supervised neural networks learning algorithms. They may not always come up with acceptable performance and build very inaccurate models. In this paper new, robust to outliers, learning algorithm based on the Least Trimmed Squares (LTS) estimator is proposed. The LTS learning algorithm is simultaneously the first robust learning algorithm that takes into account not only gross errors but also leverage data points. Results of simulations of networks trained with the new algorithm are presented and the robustness against outliers is demonstrated.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chen, D.S., Jain, R.C.: A robust back propagation learning algorithm for function approximation. IEEE Transactions on Neural Networks 5, 467–479 (1994)CrossRefGoogle Scholar
  2. 2.
    Chuang, C., Su, S., Hsiao, C.: The Annealing Robust Backpropagation (ARBP) Learning Algorithm. IEEE Transactions on Neural Networks 11, 1067–1076 (2000)CrossRefGoogle Scholar
  3. 3.
    Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural Network Design. PWS Publishing, Boston (1996)Google Scholar
  4. 4.
    Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust Statistics the Approach Based on Influence Functions. John Wiley and Sons, New York (1986)zbMATHGoogle Scholar
  5. 5.
    Hornik, K., Stinchconbe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989)CrossRefGoogle Scholar
  6. 6.
    Huber, P.J.: Robust Statistics. Wiley, New York (1981)zbMATHGoogle Scholar
  7. 7.
    Jacobs, R.A.: Increased rates of convergence through learning rate adaptation. Neural Networks 1, 295–307 (1988)CrossRefGoogle Scholar
  8. 8.
    Liano, K.: Robust error measure for supervised neural network learning with outliers. IEEE Transactions on Neural Networks 7, 246–250 (1996)CrossRefGoogle Scholar
  9. 9.
    Pernia-Espinoza, A.V., Ordieres-Mere, J.B., Martinez-de-Pison, F.J., Gonzalez-Marcos, A.: TAO-robust backpropagation learning algorithm. Neural Networks 18, 191–204 (2005)CrossRefGoogle Scholar
  10. 10.
    Rousseeuw, P.J.: Least median of squares regression. Journal of the American Statistical Association 79, 871–880 (1984)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Rousseeuw, P.J.: Multivariate Estimation with High Breakdown Point. In: Mathematical Statistics and Applications, vol. B, Reidel, Dordrecht (1985)Google Scholar
  12. 12.
    Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. Wiley, New York (1987)zbMATHGoogle Scholar
  13. 13.
    Rusiecki, A.L.: Robust Learning Algorithm with the Variable Learning Rate. In: ICAISC 2006, Artificial Intelligence and Soft Computing, pp. 83–90 (2006)Google Scholar
  14. 14.
    Stromberg, A.J., Ruppert, D.: Breakdown in nonlinear regression. J. Amer. Statist. Assoc. 87, 991–997 (1992)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Vogl, T.P., et al.: Accelerating the convergence of the backpropagation method. Biological Cybernetics 59, 256–264 (1988)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Andrzej Rusiecki
    • 1
  1. 1.Institute of Computer Engineering, Control and Robotics, WroclawPoland

Personalised recommendations