Learning from Labeled and Unlabeled Data Using Random Walks

  • Dengyong Zhou
  • Bernhard Schölkopf
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3175)


We consider the general problem of learning from labeled and unlabeled data. Given a set of points, some of them are labeled, and the remaining points are unlabeled. The goal is to predict the labels of the unlabeled points. Any supervised learning algorithm can be applied to this problem, for instance, Support Vector Machines (SVMs). The problem of our interest is if we can implement a classifier which uses the unlabeled data information in some way and has higher accuracy than the classifiers which use the labeled data only. Recently we proposed a simple algorithm, which can substantially benefit from large amounts of unlabeled data and demonstrates clear superiority to supervised learning methods. Here we further investigate the algorithm using random walks and spectral graph theory, which shed light on the key steps in this algorithm.


Support Vector Machine Random Walk Unlabeled Data Neural Information Processing System Supervise Learning Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aldous, D., Fill, J.: Reversible Markov Chains and Random Walks on Graphs (in Preparation),
  2. 2.
    Belkin, M., Matveeva, I., Niyogi, P.: Regression and regularization on large graphs. Technical report, University of Chicago (2003)Google Scholar
  3. 3.
    Chan, T., Shen, J.: Variational restoration of non-flat image features: Models and algorithms. SIAM Journal of Applied Mathematics 61(4), 1338–1361 (2000)MATHMathSciNetGoogle Scholar
  4. 4.
    Chung, F.: Spectral Graph Theory. Regional Conference Series in Mathematics, vol. 92. American Mathematical Society, Providence (1997)MATHGoogle Scholar
  5. 5.
    Devijver, P.A., Kittier, J.: Pattern Recognition: A Statistical Approach. Prentice-Hall, London (1982)MATHGoogle Scholar
  6. 6.
    Ham, J., Lee, D.D., Mika, S., Schölkopf, B.: A kernel view of the dimensionality reduction of manifolds. In: Proceedings of the 21st International Conference on Machine Learning (2004)Google Scholar
  7. 7.
    Ng, Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems 14, MIT Press, Cambridge (2002)Google Scholar
  8. 8.
    Schölkopf, B., Smola, A.J.: Learning with kernels. MIT Press, Cambridge (2002)Google Scholar
  9. 9.
    Vapnik, V.N.: Statistical learning theory. Wiley, NY (1998)MATHGoogle Scholar
  10. 10.
    Zhang, T., Oles, F.: A probability analysis on the value of unlabeled data for classification problems. In: Proceedings of the 17th International Conference on Machine Learning (2000)Google Scholar
  11. 11.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems 16, MIT Press, Cambridge (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Dengyong Zhou
    • 1
  • Bernhard Schölkopf
    • 1
  1. 1.Max Planck Institute for Biological CyberneticsTuebingenGermany

Personalised recommendations