Learning from Labeled and Unlabeled Data Using Random Walks
We consider the general problem of learning from labeled and unlabeled data. Given a set of points, some of them are labeled, and the remaining points are unlabeled. The goal is to predict the labels of the unlabeled points. Any supervised learning algorithm can be applied to this problem, for instance, Support Vector Machines (SVMs). The problem of our interest is if we can implement a classifier which uses the unlabeled data information in some way and has higher accuracy than the classifiers which use the labeled data only. Recently we proposed a simple algorithm, which can substantially benefit from large amounts of unlabeled data and demonstrates clear superiority to supervised learning methods. Here we further investigate the algorithm using random walks and spectral graph theory, which shed light on the key steps in this algorithm.
KeywordsSupport Vector Machine Random Walk Unlabeled Data Neural Information Processing System Supervise Learning Method
Unable to display preview. Download preview PDF.
- 1.Aldous, D., Fill, J.: Reversible Markov Chains and Random Walks on Graphs (in Preparation), http://stat-www.berkeley.edu/users/aldous/RWG/book.html
- 2.Belkin, M., Matveeva, I., Niyogi, P.: Regression and regularization on large graphs. Technical report, University of Chicago (2003)Google Scholar
- 6.Ham, J., Lee, D.D., Mika, S., Schölkopf, B.: A kernel view of the dimensionality reduction of manifolds. In: Proceedings of the 21st International Conference on Machine Learning (2004)Google Scholar
- 7.Ng, Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems 14, MIT Press, Cambridge (2002)Google Scholar
- 8.Schölkopf, B., Smola, A.J.: Learning with kernels. MIT Press, Cambridge (2002)Google Scholar
- 10.Zhang, T., Oles, F.: A probability analysis on the value of unlabeled data for classification problems. In: Proceedings of the 17th International Conference on Machine Learning (2000)Google Scholar
- 11.Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems 16, MIT Press, Cambridge (2004)Google Scholar