Abstract
A rather simple semi-supervised version of the equally simple nearest mean classifier is presented. However simple, the proposed approach is of practical interest as the nearest mean classifier remains a relevant tool in biomedical applications or other areas dealing with relatively high-dimensional feature spaces or small sample sizes. More importantly, the performance of our semi-supervised nearest mean classifier is typically expected to improve over that of its standard supervised counterpart and typically does not deteriorate with increasing numbers of unlabeled data. This behavior is achieved by constraining the parameters that are estimated to comply with relevant information in the unlabeled data, which leads, in expectation, to a more rapid convergence to the large-sample solution because the variance of the estimate is reduced. In a sense, our proposal demonstrates that it may be possible to properly train a known classification scheme such that it can benefit from unlabeled data, while avoiding the additional assumptions typically made in semi-supervised learning.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Abney, S.: Understanding the Yarowsky algorithm. Computational Linguistics 30(3), 365–395 (2004)
Asuncion, A., Newman, D.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Basu, S., Banerjee, A., Mooney, R.: Semi-supervised clustering by seeding. In: Proceedings of the Nineteenth International Conference on Machine Learning, pp. 19–26 (2002)
Ben-David, S., Lu, T., Pál, D.: Does unlabeled data provably help? worst-case analysis of the sample complexity of semi-supervised learning. In: Proceedings of COLT 2008, pp. 33–44 (2008)
Castelli, V., Cover, T.: On the exponential value of labeled samples. Pattern Recognition Letters 16(1), 105–111 (1995)
Chapelle, O., Schölkopf, B., Zien, A.: Introduction to semi-supervised learning. In: Semi-Supervised Learning, ch. 1. MIT Press, Cambridge (2006)
Chapelle, O., Schölkopf, B., Zien, A.: Semi-Supervised Learning. MIT Press, Cambridge (2006)
Cohen, I., Cozman, F., Sebe, N., Cirelo, M., Huang, T.: Semisupervised learning of classifiers: Theory, algorithms, and their application to human-computer interaction. IEEE Transactions on Pattern Analysis and Machine Intelligence pp. 1553–1567 (2004)
Cozman, F., Cohen, I.: Risks of semi-supervised learning. In: Semi-Supervised Learning, chap. 4. MIT Press, Cambridge (2006)
Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39(1), 1–38 (1977)
Duda, R., Hart, P.: Pattern classification and scene analysis. John Wiley & Sons, Chichester (1973)
Hastie, T., Buja, A., Tibshirani, R.: Penalized discriminant analysis. The Annals of Statistics 23(1), 73–102 (1995)
Lafferty, J., Wasserman, L.: Statistical analysis of semi-supervised regression. In: Advances in Neural Information Processing Systems, vol. 20, pp. 801–808 (2007)
Liu, Q., Sung, A., Chen, Z., Liu, J., Huang, X., Deng, Y.: Feature selection and classification of MAQC-II breast cancer and multiple myeloma microarray gene expression data. PLoS ONE 4(12), e8250 (2009)
Liu, W., Laitinen, S., Khan, S., Vihinen, M., Kowalski, J., Yu, G., Chen, L., Ewing, C., Eisenberger, M., Carducci, M., Nelson, W., Yegnasubramanian, S., Luo, J., Wang, Y., Xu, J., Isaacs, W., Visakorpi, T., Bova, G.: Copy number analysis indicates monoclonal origin of lethal metastatic prostate cancer. Nature Medicine 15(5), 559–565 (2009)
McLachlan, G.: Iterative reclassification procedure for constructing an asymptotically optimal rule of allocation in discriminant analysis. Journal of the American Statistical Association 70(350), 365–369 (1975)
McLachlan, G.: Discriminant analysis and statistical pattern recognition. John Wiley & Sons, Chichester (1992)
McLachlan, G., Ganesalingam, S.: Updating a discriminant function on the basis of unclassified data. Communications in Statistics - Simulation and Computation 11(6), 753–767 (1982)
Nigam, K., McCallum, A., Thrun, S., Mitchell, T.: Learning to classify text from labeled and unlabeled documents. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 792–799 (1998)
Noguchi, S., Nagasawa, K., Oizumi, J.: The evaluation of the statistical classifier. In: Watanabe, S. (ed.) Methodologies of Pattern Recognition, pp. 437–456. Academic Press, London (1969)
Roepman, P., Jassem, J., Smit, E., Muley, T., Niklinski, J., van de Velde, T., Witteveen, A., Rzyman, W., Floore, A., Burgers, S., Giaccone, G., Meister, M., Dienemann, H., Skrzypski, M., Kozlowski, M., Mooi, W., van Zandwijk, N.: An immune response enriched 72-gene prognostic profile for early-stage non-small-cell lung cancer. Clinical Cancer Research 15(1), 284 (2009)
Schölkopf, B.: The kernel trick for distances. In: Advances in Neural Information Processing Systems, vol. 13, p. 301. The MIT Press, Cambridge (2001)
Seeger, M.: A taxonomy for semi-supervised learning methods. In: Semi-Supervised Learning, ch. 2. MIT Press, Cambridge (2006)
Singh, A., Nowak, R., Zhu, X.: Unlabeled data: Now it helps, now it doesn’t. In: Advances in Neural Information Processing Systems, vol. 21 (2008)
Sokolovska, N., Cappé, O., Yvon, F.: The asymptotics of semi-supervised learning in discriminative probabilistic models. In: Proceedings of the 25th International Conference on Machine Learning, pp. 984–991 (2008)
Titterington, D.: Updating a diagnostic system using unconfirmed cases. Journal of the Royal Statistical Society. Series C (Applied Statistics) 25(3), 238–247 (1976)
Vittaut, J., Amini, M., Gallinari, P.: Learning classification with both labeled and unlabeled data. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 69–78. Springer, Heidelberg (2002)
Wessels, L., Reinders, M., Hart, A., Veenman, C., Dai, H., He, Y., Veer, L.: A protocol for building and evaluating predictors of disease state based on microarray data. Bioinformatics 21(19), 3755 (2005)
Yarowsky, D.: Unsupervised word sense disambiguation rivaling supervised methods. In: Proceedings of the 33rd annual meeting on Association for Computational Linguistics, pp. 189–196 (1995)
Zhu, X., Goldberg, A.: Introduction to Semi-Supervised Learning. Morgan & Claypool Publishers, San Francisco (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Loog, M. (2010). Constrained Parameter Estimation for Semi-supervised Learning: The Case of the Nearest Mean Classifier. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2010. Lecture Notes in Computer Science(), vol 6322. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15883-4_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-15883-4_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15882-7
Online ISBN: 978-3-642-15883-4
eBook Packages: Computer ScienceComputer Science (R0)