Statistical Stemmers: A Reproducibility Study

  • Gianmaria Silvello
  • Riccardo Bucco
  • Giulio Busato
  • Giacomo Fornari
  • Andrea Langeli
  • Alberto Purpura
  • Giacomo Rocco
  • Alessandro Tezza
  • Maristella Agosti
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10772)

Abstract

Statistical stemmers are important components of Information Retrieval (IR) systems, especially for text search over languages with few linguistic resources. In recent years, research on stemmers produced relevant results, especially in 2011 when three language-independent stemmers were published in relevant venues. In this paper, we describe our efforts for reproducing these three stemmers. We also share the code as open-source and an extended version of Terrier system integrating the developed stemmers.

References

  1. 1.
    Di Nunzio, G.M., Ferro, N., Mandl, T., Peters, C.: CLEF 2007: ad hoc track overview. In: Peters, C., et al. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 13–32. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-85760-0_2 CrossRefGoogle Scholar
  2. 2.
    Dietz, F., Petras, V.: A component-level analysis of an academic search test collection. In: Jones, G.J.F., et al. (eds.) CLEF 2017. LNCS, vol. 10456, pp. 29–42. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-65813-1_3 CrossRefGoogle Scholar
  3. 3.
    Dolamic, L., Savoy, J.: Indexing and stemming approaches for the Czech language author links open overlay panel. Inf. Proces. Manage. 45(6), 714–720 (2009)CrossRefGoogle Scholar
  4. 4.
    Ferro, N., Silvello, G.: CLEF 15th birthday: what can we learn from ad hoc retrieval? In: Kanoulas, E., Lupu, M., Clough, P., Sanderson, M., Hall, M., Hanbury, A., Toms, E. (eds.) CLEF 2014. LNCS, vol. 8685, pp. 31–43. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-11382-1_4 Google Scholar
  5. 5.
    Krovetz, R.: Viewing morphology as an inference process. In: Proceedings of 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1993), pp. 191–202. ACM Press (1993)Google Scholar
  6. 6.
    Lovins, J.B.: Development of a Stemming algorithm. Mech. Transl. Comput. Linguist. 11(1/2), 22–31 (1968)Google Scholar
  7. 7.
    Macdonald, C., McCreadie, R., Santos, R.L.T., Ounis, I.: From puppy to maturity: experiences in developing terrier. In: Proceedings of OSIR at SIGIR, pp. 60–63 (2012)Google Scholar
  8. 8.
    Paik, J.H., Mitra, M., Parui, S.K., Järvelin, K.: GRAS: an effective and efficient stemming algorithm for information retrieval. ACM Trans. Inf. Syst. 29(4), 19 (2011)CrossRefGoogle Scholar
  9. 9.
    Paik, J.H., Pal, D., Parui, S.K.: A novel corpus-based stemming algorithm using co-occurrence statistics. In: Proceedings of 34th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2011), pp. 863–872. ACM Press (2011)Google Scholar
  10. 10.
    Paik, J.H., Parui, S.K.: A fast corpus-based stemmer. ACM Trans. Asian Lang. Inf. Process. 10(2), 1–16 (2011)CrossRefGoogle Scholar
  11. 11.
    Porter, M.F.: An algorithm for suffix stripping. Program 14(3), 130–137 (1980)CrossRefGoogle Scholar
  12. 12.
    Savoy, J.: Searching strategies for the Hungarian language. Inf. Process. Manage. 44(1), 310–324 (2008)CrossRefGoogle Scholar
  13. 13.
    Singh, J., Gupta, V.: Text stemming: approaches, applications, and challenges. ACM Comput. Surv. (CSUR) 49(3), 45:1–45:46 (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Gianmaria Silvello
    • 1
  • Riccardo Bucco
    • 1
  • Giulio Busato
    • 1
  • Giacomo Fornari
    • 1
  • Andrea Langeli
    • 1
  • Alberto Purpura
    • 1
  • Giacomo Rocco
    • 1
  • Alessandro Tezza
    • 1
  • Maristella Agosti
    • 1
  1. 1.Department of Information EngineeringUniversity of PaduaPaduaItaly

Personalised recommendations