Learning Kernel Subspace Classifier

  • Bailing Zhang
  • Hanseok Ko
  • Yongsheng Gao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4642)


Subspace classifiers are well-known in pattern recognition, which represent pattern classes by linear subspaces spanned by the class specific basis vectors through simple mathematical operations like SVD. Recently, kernel based subspace methods have been proposed to extend the functionalities by directly applying the Kernel Principal Component Analysis (KPCA). The projection variance in kernel space as applied in these earlier proposed kernel subspace methods, however, is not a trustworthy criteria for class discrimination and they simply fail in many recognition problems as we encountered in biometrics research. We address this issue by proposing a learning kernel subspace classifier which attempts to reconstruct data in input space through the kernel subspace projection. While the pre-image methods aiming at finding an approximate pre-image for each input by minimization of the reconstruction error in kernel space, we emphasize the problem of how to estimate a kernel subspace as a model for a specific class. Using the occluded face recognition as examples, our experimental results demonstrated the efficiency of the proposed method.


Face Recognition Recognition Accuracy Input Space Probe Image Reconstruction Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Oja, E.: Subspace Methods of Pattern Recognition. Research Studies Press, Letchworth and J.Wiley (1983)Google Scholar
  2. 2.
    Laaksonen, J., Oja, E.: Subspace Dimension Selection and Averaged Learning Subspace Method in Handwritten Digit Recognition. In: Vorbrüggen, J.C., von Seelen, W., Sendhoff, B. (eds.) Artificial Neural Networks - ICANN 96. LNCS, vol. 1112, pp. 227–232. Springer, Heidelberg (1996)Google Scholar
  3. 3.
    Rosipal, R., Girolami, M., Trejo, L., Cichocki, A.: Kernel PCA for Feature Extraction and De-Noising in Non-linear Regression. Neural Computing & Applications 10, 231–243 (2001)zbMATHCrossRefGoogle Scholar
  4. 4.
    Tsuda, K.: Subspace classifier in the Hilbert Space. Pattern Recognition Letters 20, 513–519 (1999)CrossRefGoogle Scholar
  5. 5.
    Maeda, E., et al.: Multi-category Classification by Kernel based Nonlinear Subspace Method. ICASSP 1999 2, 1025–1028 (1999)Google Scholar
  6. 6.
    Scholkopf, B., Smola, A.: Muller K-R. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)CrossRefGoogle Scholar
  7. 7.
    Bakir, G.H., Weston, J., Scholkopf, B.: Learning to Find Pre-Images. In: Advances in Neural Information Processing Systems, vol. 16, pp. 449–456. MIT Press, Cambridge, MA, USA (2004)Google Scholar
  8. 8.
    Mika, S., Scholkopf, B., Smola, A., Muller, K., Scholz, M., Ratsch, G.: Kernel PCA and De-Noising in Feature Spaces. In: Proc.1998 conference on Advances in neural information processing systems II, pp. 536–542. MIT Press, Cambridge, MA, USA (1998)Google Scholar
  9. 9.
    Scholkopf, B., Smola, A., Muller, K.: Kernel principal component analysis. In: Scholkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods - SV Learning, pp. 327–352. MIT Press, Cambridge, MA, USA (1999)Google Scholar
  10. 10.
    Rosipal, R., Trejo, L.: Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space. Journal of Machine Learning Research 2, 97–123 (2001)CrossRefGoogle Scholar
  11. 11.
    Kim, J., Choi, J., Yi, J., Turk, M.: Effective Representation Using ICA for Face Recognition Robust to Local Distortion and Partial Occlusion. IEEE Trans. PAMI 27, 1977–1981 (2005)Google Scholar
  12. 12.
    Martinez, A., Kak, A.: PCA versus LDA. IEEE Trans. PAMI 23, 228–233 (2001)Google Scholar
  13. 13.
    Park, B., Lee, K., Lee, S.: Face Recognition Using Face-ARG Matching. IEEE Trans. PAMI 27, 1982–1988 (2005)Google Scholar
  14. 14.
    Fidler, S., Skocaj, D., Leonardis, A.: Combining Reconstructive and Discriminative Subspace Methods for Robust Classification and Regression by Subsampling. IEEE Trans. PAMI 28, 337–350 (2006)Google Scholar
  15. 15.
    Oh, H., Lee, K., Lee, S.: Occlusion invariant face recognition using selective LNMF basis images. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3851, pp. 120–129. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    UPC Face Database:
  17. 17.
    Zhang, B.: Cancer Classification by Kernel Principal Component Self-regression. In: Australian Conf. on Artificial Intelligence 2006, Horbat, Australia, pp. 719–728 (2006)Google Scholar
  18. 18.
    Zhang, B.: Kernel Auto-associator from Kernel Principal Component Autoregression with Application to Face Recognition. In: CIMCA 2005. Proc. Int’l Conf. Comput. Inte. for Modeling, Control & Automation, Vienna, pp. 15–19 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Bailing Zhang
    • 1
  • Hanseok Ko
    • 2
  • Yongsheng Gao
    • 3
  1. 1.School of Computer Science and Mathematics, Victoria University, VIC 3011Australia
  2. 2.School of Electronics and Computer Engineering, Korea University, Seoul, 136-713Korea
  3. 3.School of Engineering, Griffith University, QLD 4111Australia

Personalised recommendations