Abstract
One of the popular techniques of active learning for data annotations is uncertainty sampling, however, which often presents problems when outliers are selected. To solve this problem, this paper proposes a density-based re-ranking technique, in which a density measure is adopted to determine whether an unlabeled example is an outlier. The motivation of this study is to prefer not only the most informative example in terms of uncertainty measure, but also the most representative example in terms of density measure. Experimental results of active learning for word sense disambiguation and text classification tasks using six real-world evaluation data sets show that our proposed density-based re-ranking technique can improve uncertainty sampling.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Eduard, H., Marcus, M., Palmer, M., Ramshaw, L., Weischedel, R.: Ontonotes: The 90% Solution. In: Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL, poster session (2006)
David, C., Atlas, L., Ladner, R.: Improving generalization with active learning. Machine Learning 15(2), 201–221 (1994)
Seung, H.S., Opper, M., Sompolinsky, H.: Query by committee. In: Proceedings of the Fifth Annual ACM Conference on Computational Learning Theory, pp. 287–294. ACM Press, New York (1992)
Lewis David, D., Gale, W.A.: A sequential algorithm for training text classifiers. In: Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 3–12 (1994)
Jinying, C., Schein, A., Ungar, L., Palmer, M.: An empirical study of the behavior of active learning for word sense disambiguation. In: Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pp. 120–127 (2006)
Seng, C.Y., Ng, H.T.: Domain adaptation with active learning for word sense disambiguation. In: Proceedings of the 45th annual meeting on Association for Computational Linguistics, pp. 49–56 (2007)
Jingbo, Z., Wang, H., Yao, T., Tsou, B.: Active Learning with Sampling by Uncertainty and Density for Word Sense Disambiguation and Text Classification. In: Proceedings of the 22nd International Conference on Computational Linguistics, pp. 1137–1144 (2008)
Min, T., Luo, X., Roukos, S.: Active learning for statistical natural language parsing. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 120–127 (2002)
Dan, S., Zhang, J., Su, J., Zhou, G., Tan, C.-L.: Multi-criteria-based active learning for named entity recognition. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics (2004)
Nicholas, R., McCallum, A.: Toward optimal active learning through sampling estimation of error reduction. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 441–448 (2001)
Cohn David, A., Ghahramani, Z., Jordan, M.I.: Active learning with statistical models. Journal of Artificial Intelligence Research 4, 129–145 (1996)
Jingbo, Z., Hovy, E.: Active learning for word sense disambiguation with methods for addressing the class imbalance problem. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 783–790 (2007)
Ying, Z., Hildebrand, A.S., Vogel, S.: Distributed language modeling for N-best list reranking. In: Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pp. 216–223 (2006)
Michael, C., Koo, T.: Discriminative reranking for natural language parsing. In: Proceedings of 17th International Conference on Machine Learning, pp. 175–182 (2000)
Eduard, H., Lin, C.-Y.: Automated Text Summarization in SUMMARIST. In: Maybury, M., Mani, I. (eds.) Advances in Automatic Text Summarization. MIT Press, Cambridge (1998)
Schein Andrew, I., Ungar, L.H.: Active learning for logistic regression: an evaluation. Machine Learning 68(3), 235–265 (2007)
Tou, N.H., Lee, H.B.: Integrating multiple knowledge sources to disambiguate word sense: an exemplar-based approach. In: Proceedings of the Thirty-Fourth Annual Meeting of the Association for Computational Linguistics, pp. 40–47 (1996)
Claudia, L., Towell, G., Voorhees, E.: Corpus-Based Statistical Sense Resolution. In: Proceedings of the ARPA Workshop on Human Language Technology, pp. 260–265 (1993)
Andrew, P., Hovy, E., Pantel, P.: The Omega Ontology. In: Proceedings of OntoLex 2005 - Ontologies and Lexical Resources, pp. 59–66 (2005)
Andrew, M., Nigam, K.: A comparison of event models for naïve bayes text classification. In: AAAI 1998 workshop on learning for text categorization (1998)
Berger Adam, L., Vincent, J., Della Pietra, S.A.D.: A maximum entropy approach to natural language processing. Computational Linguistics 22(1), 39–71 (1996)
Keok, L.Y., Ng, H.T.: An empirical evaluation of knowledge sources and learning algorithm for word sense disambiguation. In: Proceedings of the ACL 2002 conference on Empirical methods in natural language processing, pp. 41–48 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhu, J., Wang, H., Tsou, B.K. (2009). A Density-Based Re-ranking Technique for Active Learning for Data Annotations. In: Li, W., Mollá-Aliod, D. (eds) Computer Processing of Oriental Languages. Language Technology for the Knowledge-based Economy. ICCPOL 2009. Lecture Notes in Computer Science(), vol 5459. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00831-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-00831-3_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-00830-6
Online ISBN: 978-3-642-00831-3
eBook Packages: Computer ScienceComputer Science (R0)