AutoEval: An Evaluation Methodology for Evaluating Query Suggestions Using Query Logs

  • M-Dyaa Albakour
  • Udo Kruschwitz
  • Nikolaos Nanas
  • Yunhyong Kim
  • Dawei Song
  • Maria Fasli
  • Anne De Roeck
Conference paper

DOI: 10.1007/978-3-642-20161-5_60

Part of the Lecture Notes in Computer Science book series (LNCS, volume 6611)
Cite this paper as:
Albakour MD. et al. (2011) AutoEval: An Evaluation Methodology for Evaluating Query Suggestions Using Query Logs. In: Clough P. et al. (eds) Advances in Information Retrieval. ECIR 2011. Lecture Notes in Computer Science, vol 6611. Springer, Berlin, Heidelberg

Abstract

User evaluations of search engines are expensive and not easy to replicate. The problem is even more pronounced when assessing adaptive search systems, for example system-generated query modification suggestions that can be derived from past user interactions with a search engine. Automatically predicting the performance of different modification suggestion models before getting the users involved is therefore highly desirable. AutoEval is an evaluation methodology that assesses the quality of query modifications generated by a model using the query logs of past user interactions with the system. We present experimental results of applying this methodology to different adaptive algorithms which suggest that the predicted quality of different algorithms is in line with user assessments. This makes AutoEval a suitable evaluation framework for adaptive interactive search engines.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • M-Dyaa Albakour
    • 1
  • Udo Kruschwitz
    • 1
  • Nikolaos Nanas
    • 2
  • Yunhyong Kim
    • 3
  • Dawei Song
    • 3
  • Maria Fasli
    • 1
  • Anne De Roeck
    • 4
  1. 1.University of EssexColchesterUK
  2. 2.Centre for Research and TechnologyThessalyGreece
  3. 3.Robert Gordon UniversityAberdeenUK
  4. 4.Open UniversityMilton KeynesUK

Personalised recommendations