Abstract
This paper presents a machine-learning approach for ranking web documents according to the proportion of procedural text they contain. By ‘procedural text’ we refer to ordered lists of steps, which are very common in some instructional genres such as online manuals. Our initial training corpus is built up by applying some simple heuristics to select documents from a large collection and contains only a few documents with a large proportion of procedural texts. We adapt the Naive Bayes classifier to better fit this less than ideal training corpus. This adapted model is compared with several other classifiers in ranking procedural texts using different sets of features and is shown to perform well when only highly distinctive features are used.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Clarke, C., Cormack, G., Laszlo, M., Lynam, T., Terra, E.: The Impact of Corpus Size on Question Answering Performance. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in IR, Tampere, Finland (1998)
Freund, Y., Mason, L.: The Alternating Decision Tree Learning Algorithm. In: Proceeding of the Sixteenth International Conference on Machine Learning, Bled, Slovenia, pp. 124–133 (1999)
John, G., Langley, P.: Estimating Continuous Distributions in Bayesian Classifiers. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338–345. Morgan Kaufmann, San Mateo (1995)
Kelly, D., Murdock, V., Yuan, X.J., Croft, W.B., Belkin, N.J.: Features of Documents Relevant to Task- and Fact-Oriented Questions. In: Proceedings of the Eleventh International Conference on Information and Knowledge Management (CIKM 2002), McLean, VA, pp. 645–647 (2002)
Santini, M.: A Shallow Approach to Syntactic Feature Extraction for Genre Classification. In: Proceedings of the 7th Annual Colloquium for the UK Special Interest Group for Computational Linguistics (CLUK 2004) (2004)
Schwitter, R., Rinaldi, F., Clematide, S.: The Importance of How-Questions in Technical Domains. In: Question-Answering workshop of TALN 2004, Fez, Morocco (2004)
Sebastiani, F.: Machine Learning in Automated Text Categorisation. ACM Computing Surveys 34(1), 1–47 (1999)
Stricker, M., Vichot, F., Dreyfus, G., Wolinski, F.: Two Steps Feature Selection and Neural Network Classification for the TREC-8 Routing. CoRR cs. CL/0007016 (2000)
Takechi, M., Tokunaga, T., Matsumoto, Y., Tanaka, H.: Feature Selection in Categorizing Procedural Expressions. In: The Sixth International Workshop on Information Retrieval with Asian Languages (IRAL 2003), pp. 49–56 (2003)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Mateo (2000)
Yang, Y.: An Evaluation of Statistical Approaches to Text Categorization. Journal of Information Retrieval 1(1/2), 67–88 (1999)
Yang, Y., Liu, X.: A Re-Examination of Text Categorization Methods. In: Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1999), pp. 42–49 (1999)
Yin, L.: A Two-Stage Approach to Retrieve Answers for How-To Questions. In: Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, Student session, Trento, Italy (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yin, L., Power, R. (2006). Adapting the Naive Bayes Classifier to Rank Procedural Texts. In: Lalmas, M., MacFarlane, A., Rüger, S., Tombros, A., Tsikrika, T., Yavlinsky, A. (eds) Advances in Information Retrieval. ECIR 2006. Lecture Notes in Computer Science, vol 3936. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11735106_17
Download citation
DOI: https://doi.org/10.1007/11735106_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33347-0
Online ISBN: 978-3-540-33348-7
eBook Packages: Computer ScienceComputer Science (R0)