Kernel-Based Transductive Learning with Nearest Neighbors

  • Liangcai Shu
  • Jinhui Wu
  • Lei Yu
  • Weiyi Meng
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5446)


In the k-nearest neighbor (KNN) classifier, nearest neighbors involve only labeled data. That makes it inappropriate for the data set that includes very few labeled data. In this paper, we aim to solve the classification problem by applying transduction to the KNN algorithm. We consider two groups of nearest neighbors for each data point — one from labeled data, and the other from unlabeled data. A kernel function is used to assign weights to neighbors. We derive the recurrence relation of neighboring data points, and then present two solutions to the classification problem. One solution is to solve it by matrix computation for small or medium-size data sets. The other is an iterative algorithm for large data sets, and in the iterative process an energy function is minimized. Experiments show that our solutions achieve high performance and our iterative algorithm converges quickly.


KNN transductive learning semi-supervised learning kernel function 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Webb, A.R.: Statistical Pattern Recognition. John Wiley and Sons Ltd., Chichester (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    Bennett, K.P., Demiriz, A.: Semi-supervised support vector machines. In: NIPS (1998)Google Scholar
  3. 3.
    Kulis, B., Basu, S., Dhillon, I., Mooney, R.: Semi-supervised graph clustering: A kernel approach. In: ICML (2005)Google Scholar
  4. 4.
    Nigam, K., McCallum, A., Thrun, S., Mitchell, T.: Learning to classify text from labeled and unlabeled documents. In: AAAI (1998)Google Scholar
  5. 5.
    Nigam, K., McCallum, A.K., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using em. Machine Learning 1(34) (1999)Google Scholar
  6. 6.
    Seeger, M.: Learning with labeled and unlabeled data. Inst. for Adaptive and Neural Computation, technical report (2001)Google Scholar
  7. 7.
    Sindhwani, V., Niyogi, P., Belkin, M.: Beyond the point cloud: from transductive to semi-supervised learning. In: ICML (2005)Google Scholar
  8. 8.
    Tang, W., Xiong, H., Zhong, S., Wu, J.: Enhancing semi-supervised clustering: A feature projection perspective. In: KDD (2007)Google Scholar
  9. 9.
    Tong, W., Jin, R.: Semi-supervised learning by mixed label propagation. In: AAAI (2007)Google Scholar
  10. 10.
    Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using gaussian fields and harmonic functions. In: ICML (2003)Google Scholar
  11. 11.
    Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning. MIT Press, Cambridge (2006)Google Scholar
  12. 12.
    Tran, T.N., Wehrensa, R., Buydens, L.M.: Knn-kernel density-based clustering for high-dimensional multivariate data. Computational Statistics & Data Analysis 51(2), 513–525 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Zhou, D., Bousquet, O., Lal, T., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: NIPS (2003)Google Scholar
  14. 14.
    Blum, A., Chawla, S.: Learning from labeled and unlabeled data using graph mincuts. In: ICML (2001)Google Scholar
  15. 15.
    Driessens, K., Reutemann, P., Pfahringer, B., Leschi, C.: Using weighted nearest neighbor to benefit from unlabeled data. In: Ng, W.-K., Kitsuregawa, M., Li, J., Chang, K. (eds.) PAKDD 2006. LNCS, vol. 3918, pp. 60–69. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Liangcai Shu
    • 1
  • Jinhui Wu
    • 1
  • Lei Yu
    • 1
  • Weiyi Meng
    • 1
  1. 1.Dept. of Computer ScienceSUNY at Binghamton, BinghamtonNew YorkU.S.A.

Personalised recommendations