Posterior Distribution Learning (PDL): A Novel Supervised Learning Framework

  • Enmei Tu
  • Jie Yang
  • Zhenghong Jia
  • Nicola Kasabov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8834)


In order to obtain a robust supervised model with good generalization ability, traditional supervised learning method has to be trained with sufficient well labeled and uniformly distributed samples. However, in many real applications, the cost of labeled samples is generally very expensive. How to make use of ample easily available unlabeled samples to remedy the insufficiency of labeled samples to train a supervised model is of great interest and practical significance. In this paper we propose a new supervised learning framework, Posterior Distribution Learning (PDL), which could train a robust supervised model with very a few labeled samples by including those unlabeled samples into training stage. Experimental results on both synthetic and real world data sets are presented to demonstrate the effectiveness of the proposed framework.


distribution learning nonlinear regression manifold classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: COLT, pp. 92–100. ACM (1998)Google Scholar
  2. 2.
    Nigam, K., McCallum, A.K., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using EM. Machine Learning 39, 103–134 (2000)CrossRefzbMATHGoogle Scholar
  3. 3.
    Amini, M.-R., Gallinari, P.: The use of unlabeled data to improve supervised learning for text summarization. In: SIGIR, pp. 105–112. ACM (2002)Google Scholar
  4. 4.
    Goldman, S., Zhou, Y.: Enhancing supervised learning with unlabeled data. In: ICML, pp. 327–334. Citeseer (2000)Google Scholar
  5. 5.
    Denis, F.: PAC learning from positive statistical queries. In: Richter, M.M., Smith, C.H., Wiehagen, R., Zeugmann, T. (eds.) ALT 1998. LNCS (LNAI), vol. 1501, pp. 112–126. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  6. 6.
    Li, X., Liu, B.: Learning to classify texts using positive and unlabeled data. In: IJCAI, vol. 3, pp. 587–592 (2003)Google Scholar
  7. 7.
    Liu, B., Lee, W.S., Yu, P.S., Li, X.: Partially supervised classification of text documents. In: ICML, vol. 2, pp. 387–394. Citeseer (2002)Google Scholar
  8. 8.
    Chi, M., Bruzzone, L.: A semilabeled-sample-driven bagging technique for ill-posed classification problems. IEEE Geoscience and Remote Sensing Letters 2, 69–73 (2005)CrossRefGoogle Scholar
  9. 9.
    Lee, W.S., Liu, B.: Learning with positive and unlabeled examples using weighted logistic regression. In: ICML, vol. 3, pp. 448–455 (2003)Google Scholar
  10. 10.
    Shrager, J., Hogg, T., Huberman, B.A.: Observation of phase transitions in spreading activation networks. Science 236, 1092–1094 (1987)CrossRefGoogle Scholar
  11. 11.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: NIPS, pp. 595–602 (2004)Google Scholar
  12. 12.
    Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Processing Letters 9, 293–300 (1999)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Zhou, Z.-H., Li, M.: Tri-training: Exploiting unlabeled data using three classifiers. IEEE Transactions on Knowledge and Data Engineering 17, 1529–1541 (2005)CrossRefGoogle Scholar
  14. 14.
    Nie, F., Xu, D., Li, X., Xiang, S.: Semisupervised dimensionality reduction and classification through virtual label regression. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 41, 675–685 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Enmei Tu
    • 1
  • Jie Yang
    • 1
  • Zhenghong Jia
    • 2
  • Nicola Kasabov
    • 3
  1. 1.Institute of Image Processing and Pattern RecognitionShanghai Jiao Tong UniversityChina
  2. 2.School of Information Science and EngineeringXinjiang UniversityUrumqiChina
  3. 3.The Knowledge Engineering and Discovery Research InstituteAuckland University of TechnologyAucklandNew Zealand

Personalised recommendations