Journal of Mathematical Imaging and Vision

, Volume 60, Issue 4, pp 512–524 | Cite as

Fast Kernel Generalized Discriminative Common Vectors for Feature Extraction

  • Katerine Diaz-Chito
  • Jesús Martínez del Rincón
  • Aura Hernández-Sabaté
  • Marçal Rusiñol
  • Francesc J. Ferri


This paper presents a supervised subspace learning method called Kernel Generalized Discriminative Common Vectors (KGDCV), as a novel extension of the known Discriminative Common Vectors method with Kernels. Our method combines the advantages of kernel methods to model complex data and solve nonlinear problems with moderate computational complexity, with the better generalization properties of generalized approaches for large dimensional data. These attractive combination makes KGDCV specially suited for feature extraction and classification in computer vision, image processing and pattern recognition applications. Two different approaches to this generalization are proposed: a first one based on the Kernel Trick and a second one based on the Nonlinear Projection Trick (NPT) for even higher efficiency. Both methodologies have been validated on four different image datasets containing faces, objects and handwritten digits and compared against well-known nonlinear state-of-the-art methods. Results show better discriminant properties than other generalized approaches both linear or kernel. In addition, the KGDCV-NPT approach presents a considerable computational gain, without compromising the accuracy of the model.


Nonlinear feature extraction Kernel Discriminative Common Vectors Kernel Trick Nonlinear Projection Trick Computational efficiency 


  1. 1.
    Arlandis, J., Perez-Cortes, J.C., Llobet, R.: Handwritten character recognition using the continuous distance transformation, pp. 940–943 (2000)Google Scholar
  2. 2.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)CrossRefGoogle Scholar
  3. 3.
    Cai, D., He, X., Han, J.: Speed up kernel discriminant analysis. VLDB J. 20(1), 21–33 (2011)CrossRefGoogle Scholar
  4. 4.
    Cevikalp, H., Neamtu, M., Barkana, A.: The kernel common vector method: a novel nonlinear subspace classifier for pattern recognition. IEEE Trans. Syst. Man Cybern. B 37(4), 937–951 (2007)CrossRefGoogle Scholar
  5. 5.
    Cevikalp, H., Neamtu, M., Wilkes, M.: Discriminative common vector method with kernels. IEEE Trans. Neural Netw. 17(6), 1550–1565 (2006)CrossRefGoogle Scholar
  6. 6.
    Cevikalp, H., Neamtu, M., Wilkes, M., Barkana, A.: Discriminative common vectors for face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 4–13 (2005)CrossRefGoogle Scholar
  7. 7.
    Chen, G., Tsai, W.: An incremental-learning-by-navigation approach to vision-based autonomous land vehicle guidance in indoor environments using vertical line information and multiweighted generalized hough transform technique. Syst. Man Cybern. B: Cybern. IEEE Trans. 28(5), 740–748 (1998)Google Scholar
  8. 8.
    Chen, L.F., Liao, H.Y., Ko, M.T., Lin, J.C., Yu, G.J.: A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognit. 33(10), 1713–1726 (2000)CrossRefGoogle Scholar
  9. 9.
    Cun, Y.L., Boser, B., Denker, J.S., Howard, R.E., Habbard, W., Jackel, L.D., Henderson, D.: Advances in neural information processing systems 2. Chap. Handwritten Digit Recognition with a Back-Propagation Network, pp. 396–404 (1990)Google Scholar
  10. 10.
    Ferri, F., Diaz-Chito, K., Diaz-Villanueva, W.: Fast approximated discriminative common vectors using rank-one svd updates. Neural Inf. Process. 8228, 368–375 (2013)Google Scholar
  11. 11.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, Cambridge (1990)zbMATHGoogle Scholar
  12. 12.
    Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001)CrossRefGoogle Scholar
  13. 13.
    Howland, P., Wang, J., Park, H.: Solving the small sample size problem in face recognition using generalized discriminant analysis. Pattern Recognit. 39(2), 277–287 (2006)CrossRefGoogle Scholar
  14. 14.
    Kovacs, Z., Guerrieri, R.: Computer recognition of hand-written characters using the distance transform. Electron. Lett. 28(19), 1825–1827 (1992)CrossRefGoogle Scholar
  15. 15.
    Kwak, N.: Nonlinear projection trick in kernel methods: an alternative to the kernel trick. IEEE Trans. Neural Netw. Learn. Syst. 24(12), 2113–2119 (2013)CrossRefGoogle Scholar
  16. 16.
    Liu, Q., Cheng, J., Lu, H., Ma, S.: Modeling face appearance with nonlinear independent component analysis. In: Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004, pp. 761–766 (2004)Google Scholar
  17. 17.
    Martinez, A., Benavente, R.: The ar face database. Technical Report 24, Computer Vision Center CVC (1998)Google Scholar
  18. 18.
    Nene, S., Nayar, S., Murase, H.: Columbia object image library (coil-100). Technical Report CUCS-006-96, Department of Computer Science, Columbia University (1996)Google Scholar
  19. 19.
    Samaria, F., Harter, A.: Parameterisation of a stochastic model for human face identification. In: WACV94, pp. 138–142 (1994)Google Scholar
  20. 20.
    Schlkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)Google Scholar
  21. 21.
    Schlkopf, B., Smola, A., Mller, K.: Nonlinear component analysis as a kernel eigenvalue problem. Technical Report 44, Max Planck Institute for Biological Cybernetics, Tbingen, Germany (1996)Google Scholar
  22. 22.
    Schlkopf, B., Smola, A., Mller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)CrossRefGoogle Scholar
  23. 23.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, New York (2004)CrossRefzbMATHGoogle Scholar
  24. 24.
    Tamura, A., Zhao, Q.: Rough common vector: A new approach to face recognition. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 2366–2371 (2007)Google Scholar
  25. 25.
    van der Heijden, F., Duin, R.P.W., de Ridder, D., Tax, D.M.: Classification, Parameter Estimation and State Estimation: An Engineering Approach Using Matlab. Wiley, Hoboken (2004)CrossRefzbMATHGoogle Scholar
  26. 26.
    Wechsler, H., Phillips, P., Bruce, V., Fogelman, F., Huang, T.E.: Face recognition: From theory to applications. NATO ASI Ser. F Comput. Syst. Sci. 163, 446–456 (1998)Google Scholar
  27. 27.
    Xiong, T., Ye, J., Li, Q., Cherkassky, V., Janardan, R.: Efficient kernel discriminant analysis via qr decomposition. In: Proceedings of the 17th International Conference on Neural Information Processing Systems, NIPS’04, pp. 1529–1536 (2004)Google Scholar
  28. 28.
    Yang, M.: Kernel eigenfaces vs. kernel fisherfaces: Face recognition using kernel methods. In: Proceedings of the 5th International Conference on Automatic Face Gesture Recognition, pp. 215–220 (2002)Google Scholar
  29. 29.
    Zheng, J., Huang, Q., Chen, S., Wang, W.: Efficient kernel discriminative common vectors for classification. Vis. Comput. 31(5), 643–655 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Katerine Diaz-Chito
    • 1
  • Jesús Martínez del Rincón
    • 2
  • Aura Hernández-Sabaté
    • 1
  • Marçal Rusiñol
    • 1
  • Francesc J. Ferri
    • 3
  1. 1.Centre de Visió per ComputadorUniversitat Autònoma de BarcelonaBarcelonaSpain
  2. 2.Centre for Secure Information TechnologiesQueen’s University BelfastBelfastUK
  3. 3.Departament d’InformàticaUniversitat de ValenciaValenciaSpain

Personalised recommendations