Reference Work Entry

Encyclopedia of Database Systems

pp 961-963

Effectiveness Involving Multiple Queries

  • Eric C. JensenAffiliated withTwitter, Inc.
  • , Steven M. BeitzelAffiliated withTelcordia Technologies
  • , Ophir FriederAffiliated withGeorgetown University


Relevance evaluation of IR systems


In information retrieval (IR), effectiveness is defined as the relevance of retrieved information to a given query. System effectiveness evaluation typically focuses on the problem of document retrieval: retrieving a ranked list of documents for each input query. Effectiveness is then measured with respect to an environment of interest consisting of the populations of documents, queries, and relevance judgments defining which of these documents are relevant to which queries. Sampling methodologies are employed for each of these populations to estimate a relevance metric, typically a function of precision (the ratio of relevant documents retrieved to the total number of documents retrieved) and recall (the ratio of relevant documents retrieved to the total number of relevant documents for that query). Conclusions about which systems outperform others are drawn from common experimental design, typically focusing on a r ...

This is an excerpt from the content