Supervised Learning for Classification

  • Hongyu Li
  • Wenbin Chen
  • I-Fan Shen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3614)


Supervised local tangent space alignment is proposed for data classification in this paper. It is an extension of local tangent space alignment, for short, LTSA, from unsupervised to supervised learning. Supervised LTSA is a supervised dimension reduction method. It make use of the class membership of each data to be trained in the case of multiple classes, to improve the quality of classification. Furthermore we present how to determine the related parameters for classification and apply this method to a number of artificial and realistic data. Experimental results show that supervised LTSA is superior for classification to other popular methods of dimension reduction when combined with simple classifiers such as the k-nearest neighbor classifier.


Feature Space Dimension Reduction Locally Linear Embedding Linear Embedding Membership Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Devijver, P., Kittler, J.: Pattern recognition, a statistical approach. Prentice-Hall, London (1982)zbMATHGoogle Scholar
  2. 2.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern classification, 2nd edn. John Wiley & Sons, New York (2001)zbMATHGoogle Scholar
  3. 3.
    Jain, A., Duin, R., Mao, J.: Statistical pattern recognition: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 4–37 (2000)CrossRefGoogle Scholar
  4. 4.
    Kohonen, T.: Self-organizing Maps, 3rd edn. Springer, Heidelberg (2000)Google Scholar
  5. 5.
    Bishop, C., Svensén, M., Williams, C.: Gtm: The generative topographic mapping. Neural Computation 10, 215–234 (1998)CrossRefGoogle Scholar
  6. 6.
    Hastie, T., Stuetzle, W.: Principal curves. J. Am. Statistical Assoc. 84 (1988)Google Scholar
  7. 7.
    DeMers, D., Cottrell, G.: Non-linear dimensionality reduction. In: Giles, C.L., Hanson, S.J., Cowan, J.D. (eds.) Advances in Neural Information Processing Systems 5, San Mateo, CA, pp. 580–587. Morgan Kaufmann, San Francisco (1993)Google Scholar
  8. 8.
    Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal component analyzers. Neural Computation 11, 443–482 (1999)CrossRefGoogle Scholar
  9. 9.
    Zhang, Z., Zha, H.: Principal manifolds and nonlinear dimension reduction via local tangent space alignment. SIAM Journal of Scientific Computing 26, 313–338 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Roweis, S., Saul, L.: Nonlinear dimension reduction by locally linear embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  11. 11.
    Li, H., Teng, L., Chen, W., Shen, I.-F.: Supervised learning on local tangent space. In: Wang, J., Liao, X.-F., Yi, Z. (eds.) ISNN 2005. LNCS, vol. 3496, pp. 546–551. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Li, H., Chen, W., Shen, I.-F.: Supervised local tangent space alignment for classification. In: IJCAI 2005, poster paper (2005) (to appear) Google Scholar
  13. 13.
    Blake, C., Merz, C.: Uci repository of machine learning databases (1998)Google Scholar
  14. 14.
    Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 13, 71–86 (1991)CrossRefGoogle Scholar
  15. 15.
    de Ridder, D., Duin, R.: Locally linear embedding for classification. Technical Report PH-2002-01, Pattern Recogniion Group, Dept. of Imaging Science and Technology, Delft University of Technology, Delft, The Netherlands (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Hongyu Li
    • 1
  • Wenbin Chen
    • 2
  • I-Fan Shen
    • 1
  1. 1.Department of Computer Science and EngineeringFudan UniversityShanghaiChina
  2. 2.Department of MathematicsFudan UniversityShanghaiChina

Personalised recommendations