Abstract
Recently the kernel discriminant analysis (KDA) has been successfully applied in many applications. However, kernel functions are usually defined a priori and it is not known what the optimum kernel function for nonlinear discriminant analysis is. Otsu derived the optimum nonlinear discriminant analysis (ONDA) by assuming the underlying probabilities similar with the Bayesian decision theory. Kurita derived discriminant kernels function (DKF) as the optimum kernel functions in terms of the discriminant criterion by investigating the optimum discriminant mapping constructed by the ONDA. The derived kernel function is defined by using the Bayesian posterior probabilities. We can define a family of DKFs by changing the estimation method of the Bayesian posterior probabilities. In this paper, we propose a novel discriminant kernel function based on L1-regularized regression, called L1 DKF. L1 DKF is given by using the Bayesian posterior probabilities estimated by L1 regression. Since L1 regression yields a sparse representation for given samples, we can naturally introduce the sparseness into the discriminant kernel function. To introduce the sparseness into LDA, we use L1 DKF as the kernel function of LDA. In experiments, we show sparseness and classification performance of L1 DKF.
Chapter PDF
Similar content being viewed by others
Keywords
- Kernel Function
- Linear Discriminant Analysis
- Sparse Representation
- Bayesian Posterior Probability
- Generalize Eigenvalue Problem
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural Computation 12(10), 2385–2404 (2000)
Chow, C.K.: An optimum character recognition system using decision functions. IRE Trans. EC-6, 247–254 (1957)
Clemmensen, L., Hastie, T., Witten, D., Ersboll, B.: Sparse discriminant analysis (2011)
Fisher, R.A.: The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics 7, 179–188 (1936)
Frank, A., Asuncion, A.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, http://archive.ics.uci.edu/ml
Hidaka, A., Kurita, T.: Discriminant Kernels based Support Vector Machine. In: The First Asian Conference on Pattern Recognition (ACPR 2011), Beijing, China, November 28-30, pp. 159–163 (2011)
Kurita, T.: “Discriminant Kernels derived from the Optimum Nonlinear Discriminant Analysis. In: Proc. of 2011 International Joint Conference on Neural Networks, San Jose, California, USA, July 31-August 5 (2011)
Mika, S., Ratsch, G., Weston, J., Scholkopf, B., Smola, A., Muller, K.: Fisher discriminant analysis with kernels. In: Proc. IEEE Neural Networks for Signal Processing Workshop, pp. 41–48 (1999)
Otsu, N.: Nonlinear discriminant analysis as a natural extension of the linear case. Behavior Metrika 2, 45–59 (1975)
Otsu, N.: Mathemetical Studies on Feature Extraction In Pattern Recognition. Researches on the Electrotechnical Laboratory 818 (1981) (in Japanease)
Otsu, N.: Optimal linear and nonlinear solutions for least-square discriminant feature extraction. In: Proceedings of the 6th International Conference on Pattern Recognition, pp. 557–560 (1982)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Royal. Statist. Soc B. 58(1), 267–288 (1996)
Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. Journal of Computational and Graphical Statistics 15(2), 262–286 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hidaka, A., Kurita, T. (2012). Sparse Discriminant Analysis Based on the Bayesian Posterior Probability Obtained by L1 Regression. In: Gimel’farb, G., et al. Structural, Syntactic, and Statistical Pattern Recognition. SSPR /SPR 2012. Lecture Notes in Computer Science, vol 7626. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34166-3_71
Download citation
DOI: https://doi.org/10.1007/978-3-642-34166-3_71
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34165-6
Online ISBN: 978-3-642-34166-3
eBook Packages: Computer ScienceComputer Science (R0)