On the Evaluation of Snippet Selection for WebCLEF

  • Arnold Overwijk
  • Dong Nguyen
  • Claudia Hauff
  • Dolf Trieschnigg
  • Djoerd Hiemstra
  • Franciska de Jong
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5706)

Abstract

WebCLEF is about supporting a user who is an expert in writing a survey article on a specific topic with a clear goal and audience by generating a ranked list with relevant snippets. This paper focuses on the evaluation methodology of WebCLEF. We show that the evaluation method and test set used for WebCLEF 2007 cannot be used to evaluate new systems and give recommendations how to improve the evaluation.

Keywords

Measurement Performance Experimentation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Rose, D.E., Levinson, D.: Understanding user goals in web search. In: Proceedings of the 13th international conference on World Wide Web. ACM, New York (2004)Google Scholar
  2. 2.
    Jijkoun, V., de Rijke, M.: Overview of WebCLEF 2007. In: Peters, C., Jijkoun, V., Mandl, T., Müller, H., Oard, D.W., Peñas, A., Petras, V., Santos, D. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 725–731. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Jijkoun, V., de Rijke, M.: The University of Amsterdam at Web CLEF 2007: Using Centrality to Rank Web Snippets. In: CLEF 2007, Budapest, Hungary (2007)Google Scholar
  4. 4.
    Voorhees, E.M., Tice, D.M.: The TREC-8 question answering track evaluation. In: Text Retrieval Conference TREC-8, pp. 83–105 (1999)Google Scholar
  5. 5.
    Pehcevski, J., Thom, J.A.: HiXEval: Highlighting XML Retrieval Evaluation. In: Fuhr, N., Lalmas, M., Malik, S., Kazai, G. (eds.) INEX 2005. LNCS, vol. 3977, pp. 43–57. Springer, Heidelberg (2006)Google Scholar
  6. 6.
    Lin, C.-Y.: ROUGE: a Package for Automatic Evaluation of Summaries. In: Proceedings of Workshop on Text Summarization, Barcelona, Spain (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Arnold Overwijk
    • 1
  • Dong Nguyen
    • 1
  • Claudia Hauff
    • 1
  • Dolf Trieschnigg
    • 1
  • Djoerd Hiemstra
    • 1
  • Franciska de Jong
    • 1
  1. 1.University of TwenteThe Netherlands

Personalised recommendations