Abstract
The optimal linear associative memory (OLAM) proposed by Kohonen and Ruohonen [16] is a classic neural network model widely used as a standalone pattern classifier or as a fundamental component of multilayer nonlinear classification approaches, such as the extreme learning machine (ELM) [10] and the echo-state network (ESN) [6]. In this paper, we develop an extension of OLAM which is robust to labeling errors (outliers) in the data set. The proposed model is robust to label noise not only near the class boundaries, but also far from the class boundaries which can result from mistakes in labelling or gross errors in measuring the input features. To deal with this problem, we propose the use of M-estimators, a parameter estimation framework widely used in robust regression, to compute the weight matrix operator, instead of using the ordinary least squares solution. We show the usefulness of the proposed classification approach through simulation results using synthetic and real-world data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baek, D., Oh, S.Y.: Improving optimal linear associative memory using data partitioning. In: Proceedings of the 2006 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2006), vol. 3, pp. 2251–2256 (2006)
Barreto, G.A., Frota, R.A.: A unifying methodology for the evaluation of neural network models on novelty detection tasks. Pattern Analysis and Applications 16(1), 83–972 (2013)
Cherkassky, V., Fassett, K., Vassilas, N.: Linear algebra approach to neural associative memories and noise performance of neural classifiers. IEEE Transactions on Computers 40(12), 1429–1435 (1991)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons (2006)
Eichmann, G., Kasparis, T.: Pattern classification using a linear associative memory. Pattern Recognition 22(6), 733–740 (1989)
Emmerich, C., Reinhart, R.F., Steil, J.J.: Recurrence enhances the spatial encoding of static inputs in reservoir networks. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds.) ICANN 2010, Part II. LNCS, vol. 6353, pp. 148–153. Springer, Heidelberg (2010)
Fox, J.: Applied Regression Analysis, Linear Models, and Related Methods. Sage Publications (1997)
Frank, A., Asuncion, A.: UCI machine learning repository (2010), http://archive.ics.uci.edu/ml
Horata, P., Chiewchanwattana, S., Sunat, K.: Robust extreme learning machine. Neurocomputing 102, 31–44 (2012)
Huang, G.B., Wang, D.H., Lan, Y.: Extreme learning machines: a survey. International Journal of Machine Learning and Cybernetics 2, 107–122 (2011)
Huber, P.J.: Robust estimation of a location parameter. Annals of Mathematical Statistics 35(1), 73–101 (1964)
Huber, P.J., Ronchetti, E.M.: Robust Statistics. John Wiley & Sons, LTD. (2009)
Hunt, B., Nadar, M., Keller, P., VonColln, E., Goyal, A.: Synthesis of a nonrecurrent associative memory model based on a nonlinear transformation in the spectral domain. IEEE Transactions on Neural Networks 4(5), 873–878 (1993)
Kim, H.-C., Ghahramani, Z.: Outlier robust gaussian process classification. In: da Vitoria Lobo, N., Kasparis, T., Roli, F., Kwok, J.T., Georgiopoulos, M., Anagnostopoulos, G.C., Loog, M. (eds.) SSPR&SPR 2008. LNCS, vol. 5342, pp. 896–905. Springer, Heidelberg (2008)
Kohonen, T., Oja, E.: Fast adaptive formation of orthogonalizing filters and associative memory in recurrent networks of neuron-like elements. Biological Cybernetics 25, 85–95 (1976)
Kohonen, T., Ruohonen, M.: Representation of associated data by matrix operators. IEEE Transactions on Computers 22(7), 701–702 (1973)
Li, D., Han, M., Wang, J.: Chaotic time series prediction based on a novel robust echo state network. IEEE Transactions on Neural Networks and Learning Systems 23(5), 787–799 (2012)
Poggio, T., Girosi, F.: Networks for approximation and learning. Proceedings of the IEEE 78(9), 1481–1497 (1990)
Stiles, G.S., Denq, D.: On the effect of noise on the Moore-Penrose generalized inverse associative memory. IEEE Transactions on Pattern Analysis and Machine Intelligence 7(3), 358–360 (1985)
Stiles, G., Denq, D.L.: A quantitative comparison of the performance of three discrete distributed associative memory models. IEEE Transactions on Computers 36(3), 257–263 (1987)
Webb, A.: Statistical Pattern Recognition, 2nd edn. John Wiley & Sons, LTD. (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
de Paula Barros, A.L.B., Barreto, G.A. (2013). Improving the Classification Performance of Optimal Linear Associative Memory in the Presence of Outliers. In: Rojas, I., Joya, G., Gabestany, J. (eds) Advances in Computational Intelligence. IWANN 2013. Lecture Notes in Computer Science, vol 7902. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38679-4_63
Download citation
DOI: https://doi.org/10.1007/978-3-642-38679-4_63
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38678-7
Online ISBN: 978-3-642-38679-4
eBook Packages: Computer ScienceComputer Science (R0)