Abstract
To generalize the Fisher Discriminant Analysis (FDA) algorithm to the case of discriminant functions belonging to a nonlinear, finite dimensional function space \(\mathcal{F}\) (Nonlinear FDA or NFDA), it is sufficient to expand the input data by computing the output of a basis of \(\mathcal{F}\) when applied to it [1,2,3,4]. The solution to NFDA can then be found like in the linear case by solving a generalized eigenvalue problem on the between- and within-classes covariance matrices (see e.g.[5]). The goal of NFDA is to find linear projections of the expanded data (i.e., nonlinear transformations of the original data) that minimize the variance within a class and maximize the variance between different classes. Such a representation is of course ideal to perform classification. The application of NFDA to pattern recognition is particularly appealing, because for a given input signal and a fixed function space it has no parameters and it is easy to implement and apply. Moreover, given C classes only Cā1 projections are relevant [5]. As a consequence, the feature space is very small and the algorithm has low memory requirements and high speed during recognition.
This work has been supported by a grant from the Volkswagen Foundation.
An erratum to this chapter can be found at http://dx.doi.org/10.1007/11550907_163 .
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Mika, S., RƤtsch, G., Weston, J., Schƶlkopf, B., MĆ¼ller, K.R.: Fisher discriminant analysis with kernels. In: Hu, Y.H., Larsen, J., Wilson, E., Douglas, S. (eds.) Proceedings of the IEEE Signal Processing Society Workshop. Neural Networks for Signal Processing, vol.Ā IX, pp. 41ā48 (1999)
Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural ComputationĀ 12, 2385ā2404 (2000)
Mika, S., RƤtsch, G., Weston, J., Schƶlkopf, B., Smola, A., MĆ¼ller, K.R.: Invariant feature extraction and classification in kernel spaces. In: Solla, S., Leen, T., MĆ¼ller, K.R. (eds.) Advances in Neural Information Processing Systems, vol.Ā 12, pp. 526ā532 (2000)
Mika, S., Smola, A., Schƶlkopf, B.: An improved training algorithm for kernel Fisher discriminants. In: Jaakkola, T., Richardson, T. (eds.): Proceedings AISTATS 2001, pp. 98ā104 (2001)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEEĀ 86, 2278ā2324 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Berkes, P. (2005). Handwritten Digit Recognition with Nonlinear Fisher Discriminant Analysis. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds) Artificial Neural Networks: Formal Models and Their Applications ā ICANN 2005. ICANN 2005. Lecture Notes in Computer Science, vol 3697. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11550907_45
Download citation
DOI: https://doi.org/10.1007/11550907_45
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28755-1
Online ISBN: 978-3-540-28756-8
eBook Packages: Computer ScienceComputer Science (R0)