Abstract
In Information Retrieval (IR), stemming enables a matching of query and document terms which are related to a same meaning but which can appear in different morphological variants. In this paper we will propose and evaluate a statistical graph-based algorithm for stemming. Considering that a word is formed by a stem (prefix) and a derivation (suffix), the key idea is that strongly interlinked prefixes and suffixes form a community of sub-strings. Discovering these communities means searching for the best word splits which give the best word stems. We conducted some experiments on CLEF 2001 test subcollections for Italian language. The results show that stemming improve the IR effectiveness. They also show that effectiveness level of our algorithm is comparable to that of an algorithm based on a-priori linguistic knowledge. This is an encouraging result, particularly in a multi-lingual context.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
M. Agosti, M. Bacchin, N. Ferro and M. Melucci. University of Padua at CLEF 2002: Experiments to evaluate a statistical stemming algorithm. In Cross-Language Information Retrieval and Evaluation: Proceedings of the CLEF 2002 workshop, Lecture Notes in Computer Science series, Springer Verlag (forthcoming).
C. Cleverdon. The Cranfield Tests on Index Language Devices. In K. Sparck Jones and P. Willett (Eds.). Readings in Information Retrieval, pages 47–59, Morgan Kaufmann, 1997.
W.B. Frakes and R. Baeza-Yates. Information Retrieval: data structures and algorithms. Prentice Hall, 1992.
J. Goldsmith. Unsupervised learning of the morphology of a natural language. Computational Linguistics, 27(2):154–198, 2001. au]7._M. Hafer and S. Weiss. Word segmentation by letter successor varieties. Information Storage and Retrieval, 10:371–385, 1994.
D. Harman. How effective is suffixing? Journal of the American Society for Information Science, 42(1):7–15, 1991.
J. Kleinberg. Authoritative sources in a hyperlinked environment. Journal of the ACM, 46(5):604–632, September 1999.
R. Krovetz. Viewing Morphology as an Inference Process,. In Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), 1993.
J. Lovins. Development of a stemming algorithm. Mechanical Translation and Computational Linguistics, 11:22–31, 1968.
C.D. Manning and H. Schütze. Foundations of statistical natural language processing. The MIT Press, 1999.
C.D. Paice. Another Stemmer. In ACM SIGIR Forum, 24, 56–61, 1990.
M. Popovic and P. Willett. The effectiveness of stemming for natural-language access to sloven textual data. Journal of the American Society for Information Science, 43(5):383–390, 1992.
C. Peters and M. Braschler. Cross-Language System Evaluation: the CLEF Campaigns. Journal of the American Society for Information Science and Technology, 52(12):1067–1072, 2001.
M. Porter. Snowball: A language for stemming algorithms. http://snowball.sourceforge.net, 2001.
M.F. Porter. An algorithm for suffix stripping. Program, 14(3):130–137, 1980.
G. Salton and M. McGill. Introduction to modern Information Retrieval. McGraw-Hill, New York, NY, 1983.
G. Salton and C. Buckley. Term weighting approaches in automatic text retrieval. Information Processing & Management, 24(5):513–523, 1988.
Institut interfacultaire d’informatique. CLEF and Multilingual information retrieval. University of Neuchatel. http://www.unine.ch/info/clef/, 2002.
C. Buckley. Trec eval. ftp://ftp.cs.cornell.edu/pub/smart/, 2002.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bacchin, M., Ferro, N., Melucci, M. (2002). The Effiectiveness of a Graph-Based Algorithm for Stemming. In: Lim, E.P., et al. Digital Libraries: People, Knowledge, and Technology. ICADL 2002. Lecture Notes in Computer Science, vol 2555. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36227-4_12
Download citation
DOI: https://doi.org/10.1007/3-540-36227-4_12
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00261-1
Online ISBN: 978-3-540-36227-2
eBook Packages: Springer Book Archive