Abstract
In supervised learning, the learning algorithm uses labeled training examples from every class to generate a classification function. One of the drawbacks of this classic paradigm is that a large number of labeled examples are needed in order to learn accurately. Since labeling is often done manually, it can be very labor intensive and time consuming. In this chapter, we study two partially supervised learning tasks. As their names suggest, these two learning tasks do not need full supervision, and thus are able to reduce the labeling effort. The first is the task of learning from labeled and unlabeled examples, which is commonly known as semisupervised learning. In this chapter, we also call it LU learning (L and U stand for “labeled” and “unlabeled” respectively). In this learning setting, there is a small set of labeled examples of every class, and a large set of unlabeled examples. The objective is to make use of the unlabeled examples to improve learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Bibliography
Barbará, D., C. Domeniconi, and N. Kang. Classifying documents without labels. In Proceedings of SIAM International Conference on Data Mining (SDM-2004), 2004.
Blum, A. and S. Chawla. Learning from Labeled and Unlabeled Data Using Graph Mincuts. In Proceedings of International Conference on Machine Learning (ICML-2001), 2001.
Blum, A. and T. Mitchell. Combining labeled and unlabeled data with cotraining. In Proceedings of Conference on Computational Learning Theory, 1998.
Buckley, C., G. Salton, and J. Allan. The effect of adding relevance information in a relevance feedback environment. In Proceedings of ACM SIGIR Conf. on Research and Development in Information Retrieval (SIGIR-1994), 1994.
Castelli, V. and T. Cover. Classification rules in the unknown mixture parameter case: relative value of labeled and unlabeled samples. In Proceedings of IEEE International Symp. Information Theory, 1994.
Chapelle, O., B. Schölkopf, and A. Zien. Semi-supervised learning. Vol. 2. 2006: MIT Press.
Collins, M. and Y. Singer. Unsupervised models for named entity classification. In Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-1999), 1999.
Cong, G., W. Lee, H. Wu, and B. Liu. Semi-supervised text classification using partitioned EM. In Proceedings of Conference of Database Systems for Advanced Applications (DASFAA 2004), 2004.
Cormen, T., C. Leiserson, R. Rivest, and C. Stein. Introduction to Algorithms. 2001: MIT Press.
Dasgupta, S., M. Littman, and D. McAllester. PAC generalization bounds for co-training. In Proceedings of Advances in Neural Information Processing Systems (NIPS-2001), 2001.
Dempster, A., N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 1977, 39(1): p. 1–38.
Deng, L., X. Chai, Q. Tan, W. Ng, and D. Lee. Spying out real user preferences for metasearch engine personalization. In Proceedings of Workshop on WebKDD, 2004.
Denis, F. PAC learning from positive statistical queries. In Proceedings of Intl. Conf. on Algorithmic Learning Theory (ALT-1998), 1998.
Elkan, C. and K. Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2008), 2008.
Fung, G., J. Yu, H. Lu, and P. Yu. Text classification without labeled negative documents. In Proceedings of IEEE International Conference on Data Engingeering (ICDE-2005), 2005.
Ghahramani, Z. and K. Heller. Bayesian sets. Advances in Neural Information Processing Systems, 2006, 18: p. 435.
Goldman, S. and Y. Zhou. Enhanced Supervised Learning with Unlabeled Data. In Proceedings of International Conference on Machine Learning (ICML-2000), 2000.
Heckman, J. Sample selection bias as a specification error. Econometrica: Journal of the econometric society, 1979: p. 153–161.
Huang, J., A. Smola, A. Gretton, K. Borgwardt, and B. Scholkopf. Correcting sample selection bias by unlabeled data. Advances in Neural Information Processing Systems, 2007, 19: p. 601.
Joachims, T. Transductive inference for text classification using support vector machines. In Proceedings of International Conference on Machine Learning (ICML-1999), 1999.
Joachims, T. Transductive learning via spectral graph partitioning. In Proceedings of International Conference on Machine Learning (ICML-2003), 2003.
Kearns, M. Efficient noise-tolerant learning from statistical queries. Journal of the ACM (JACM), 1998, 45(6): p. 983–1006.
Lee, L. Measures of distributional similarity. In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL-1999), 1999.
Lee, W. and B. Liu. Learning with positive and unlabeled examples using weighted logistic regression. In Proceedings of International Conference on Machine Learning (ICML-2003), 2003.
Letouzey, F., F. Denis, and R. Gilleron. Learning from positive and unlabeled examples. In Proceedings of Intl. Conf. on Algorithmic Learning Theory (ALT-200), 2000.
Li, X. and B. Liu. Learning to classify texts using positive and unlabeled data. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI-2003), 2003.
Li, X., B. Liu, and S. Ng. Negative Training Data can be Harmful to Text Classification. In Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-2010), 2010.
Li, X., L. Zhang, B. Liu, and S. Ng. Distributional similarity vs. PU learning for entity set expansion. In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL-2010), 2010.
Liu, B., Y. Dai, X. Li, W. Lee, and P. Yu. Building text classifiers using positive and unlabeled examples. In Proceedings of IEEE International Conference on Data Mining (ICDM-2003), 2003.
Liu, B., W. Lee, P. Yu, and X. Li. Partially supervised classification of text documents. In Proceedings of International Conference on Machine Learning (ICML-2002), 2002.
Luigi, C., E. Charles, and C. Michele. Learning gene regulatory networks from only positive and unlabeled data. BMC Bioinformatics, 2010, 11.
Manevitz, L. and M. Yousef. One-class svms for document classification. The Journal of Machine Learning Research, 2002, 2.
Nigam, K. and R. Ghani. Analyzing the effectiveness and applicability of cotraining. In Proceedings of ACM International Conference on Information and Knowledge Management (CIKM-2000), 2000.
Nigam, K., A. McCallum, S. Thrun, and T. Mitchell. Text classification from labeled and unlabeled documents using EM. Machine Learning, 2000, 39(2): p. 103–134.
Niu, Z., D. Ji, and C. Tan. Word sense disambiguation using label propagation based semi-supervised learning. In Proceedings of Meeting of the Association for Computational Linguistics (ACL-2005), 2005.
Pantel, P., E. Crestan, A. Borkovsky, A. Popescu, and V. Vyas. Web-scale distributional similarity and entity set expansion. In Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-2009), 2009.
Pham, T., H. Ng, and W. Lee. Word sense disambiguation with semisupervised learning. In Proceedings of National Conference on Artificial Intelligence (AAAI-2005), 2005.
Platt, J.C. Probabilities for SV machines. In Advances in Large Margin Classifiers, A. J. Smola, P. Bartlett, B. Schölkopf, and D. Schuurmans, Editors. 1999, MIT Press. p. 61–73.
Schölkopf, B., J. Platt, J. Shawe-Taylor, A. Smola, and R. Williamson. Estimating the support of a high-dimensional distribution. Neural computation, 2001, 13(7): p. 1443–1471.
Shimodaira, H. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 2000, 90(2): p. 227–244.
Vapnik, V. and V. Vapnik. Statistical learning theory. Vol. 2. 1998: Wiley New York.
Yu, H. General MC: Estimating boundary of positive class from small positive data. In Proceedings of IEEE International Conference on Data Mining (ICDM-2003), 2003: IEEE.
Yu, H., J. Han, and K. Chang. PEBL: positive example based learning for Web page classification using SVM. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2002), 2002.
Zadrozny, B. Learning and evaluating classifiers under sample selection bias. In Proceedings of International Conference on Machine Learning (ICML- 2004), 2004.
Zhang, D. and W. Lee. A simple probabilistic approach to learning from positive and unlabeled examples. In Proceedings of 5th Annual UK Workshop on Computational Intelligence, 2005.
Zhu, X., Z. Ghahramani, and J. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of International Conference on Machine Learning (ICML-2003), 2003.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Liu, B., Lee, W.S. (2011). Partially Supervised Learning. In: Web Data Mining. Data-Centric Systems and Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19460-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-19460-3_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19459-7
Online ISBN: 978-3-642-19460-3
eBook Packages: Computer ScienceComputer Science (R0)