Discriminative Locality Alignment

  • Tianhao Zhang
  • Dacheng Tao
  • Jie Yang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5302)


Fisher’s linear discriminant analysis (LDA), one of the most popular dimensionality reduction algorithms for classification, has three particular problems: it fails to find the nonlinear structure hidden in the high dimensional data; it assumes all samples contribute equivalently to reduce dimension for classification; and it suffers from the matrix singularity problem. In this paper, we propose a new algorithm, termed Discriminative Locality Alignment (DLA), to deal with these problems. The algorithm operates in the following three stages: first, in part optimization, discriminative information is imposed over patches, each of which is associated with one sample and its neighbors; then, in sample weighting, each part optimization is weighted by the margin degree, a measure of the importance of a given sample; and finally, in whole alignment, the alignment trick is used to align all weighted part optimizations to the whole optimization. Furthermore, DLA is extended to the semi-supervised case, i.e., semi-supervised DLA (SDLA), which utilizes unlabeled samples to improve the classification performance. Thorough empirical studies on the face recognition demonstrate the effectiveness of both DLA and SDLA.


Linear Discriminant Analysis Face Image Locality Preserve Projection Unlabeled Sample Neighbor Sample 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces: Recognition using Class Specific Linear Projection. IEEE Trans. Pattern Analysis and Machine Intelligence 19(7), 711–720 (1997)CrossRefGoogle Scholar
  2. 2.
    Belkin, M., Niyogi, P.: Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering. Neural Information Processing Systems 14, 585–591Google Scholar
  3. 3.
    Belkin, M., Niyogi, P., Sindhwani, V.: On Manifold Regularization. In: Proc. Int’l Workshop on Artificial Intelligence and Statistics (2005)Google Scholar
  4. 4.
    Cai, D., He, X., Han, J.: Using Graph Model for Face Analysis. Technical report, Computer Science Department, UIUC, UIUCDCS-R-2005-2636 (2005)Google Scholar
  5. 5.
    Chen, L.F., Liao, H.Y., Ko, M.T., Lin, J.C., Yu, G.J.: A New LDA-based Face Recognition System Which Can Solve the Small Sample Size Problem. Pattern Recognition 33(10), 1713–1726 (2000)CrossRefGoogle Scholar
  6. 6.
    Fisher, R.A.: The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics 7, 179–188 (1936)CrossRefGoogle Scholar
  7. 7.
    Graham, D.B., Allinson, N.M.: Characterizing Virtual Eigensignatures for General Purpose Face Recognition. In: Face Recognition: From Theory to Applications. NATO ASI Series F, Computer and Systems Science, vol. 163, pp. 446–456 (2006)Google Scholar
  8. 8.
    He, X., Niyogi, P.: Locality Preserving Projections. In: Advances in Neural Information Processing Systems, vol. 16 (2004)Google Scholar
  9. 9.
    Hotelling, H.: Analysis of A Complex of Statistical Variables into Principal Components. Journal of Educational Psychology 24, 417–441 (1933)CrossRefzbMATHGoogle Scholar
  10. 10.
    Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET Evaluation Methodology for Face-Recognition Algorithms. IEEE Trans. Pattern Analysis and Machine Intelligence 22(10), 1090–1104 (2000)CrossRefGoogle Scholar
  11. 11.
    Roweis, S.T., Saul, L.K.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  12. 12.
    Song, M., Dong, Z., Theobalt, C., Wang, H., Liu, Z., Seidel, H.-P.: A Generic Framework for Efficient 2-D and 3-D Facial Expression Analogy. IEEE Trans. Multimedia 9(7), 1384–1395 (2007)CrossRefGoogle Scholar
  13. 13.
    Song, M., You, M., Li, N., Chen, C.: A robust multimodal approach for emotion recognition. Neurocomputing 7(10-12), 1913–1920 (2008)CrossRefGoogle Scholar
  14. 14.
    Tao, D., Li, X., Wu, X., Maybank, S.: Geometric Mean for Subspace Selection in Multiclass Classification. IEEE Trans. Pattern Analysis and Machine Intelligence 30 (2008)Google Scholar
  15. 15.
    Turk, M., Pentland, A.: Face Recognition using Eigenfaces. In: Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition, pp. 586–591 (1991)Google Scholar
  16. 16.
    Yan, S., Xu, D., Zhang, B., Zhang, H.J., Yang, Q., Lin, S.: Graph Embedding and Extensions: A General Framework for Dimensionality Reduction. IEEE Trans. Pattern Analysis and Machine Intelligence 29(1), 40–51 (2007)CrossRefGoogle Scholar
  17. 17.
    Yan, S., Wang, H., Fu, Y., Yan, J., Tang, X., Huang, T.: Synchronized Submanifold Embedding for Person-Independent Pose Estimation and Beyond. IEEE Trans. Image Processing (2008)Google Scholar
  18. 18.
    Yu, H., Yang, J.: A Direct LDA Algorithm for High-dimensional Data with Application to Face Recognition. Pattern Recognition 34(12), 2067–2070 (2001)CrossRefzbMATHGoogle Scholar
  19. 19.
    Zhang, Z., Zha, H.: Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment. SIAM J. Scientific Computing 26(1), 313–338 (2005)CrossRefzbMATHGoogle Scholar
  20. 20.
    Zhao, D., Lin, Z., Tang, X.: Laplacian PCA and Its Applications. In: Proc. IEEE Int’l Conf. Computer Vision, pp. 1–8 (2007)Google Scholar
  21. 21.
    Zhang, T., Tao, D., Li, X., Yang, J.: A Unifying Framework for Spectral Analysis based Dimensionality Reduction. In: Proc. Int’l J. Conf. Neural Networks (2008)Google Scholar
  22. 22.
    Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised Learning using Gaussian Fields and Harmonic Functions. In: Proc. Int’l Conf. Machine Learning (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Tianhao Zhang
    • 1
  • Dacheng Tao
    • 2
    • 3
  • Jie Yang
    • 1
  1. 1.Institute of Image Processing and Pattern RecognitionShanghai Jiao Tong UniversityShanghaiChina
  2. 2.School of Computer EngineeringNanyang Technological UniversitySingapore
  3. 3.College of Computer ScienceZhejiang UniversityChina

Personalised recommendations