Web Information Systems Engineering - WISE 2012

Volume 7651 of the series Lecture Notes in Computer Science pp 440-454

Evaluating Web Archive Search Systems

  • Miguel CostaAffiliated withFoundation for National Scientific ComputingLaSIGE, Faculty of Science, University of Lisbon
  • , Mário J. SilvaAffiliated withIST/INESC-ID

* Final gross prices may vary according to local VAT.

Get Access


The information published on the web, a representation of our collective memory, is rapidly vanishing. At least 77 web archives have been developed to cope with the web’s transience problem, but despite their technology having achieved a good maturity level, the retrieval effectiveness of the search services they provide still presents unsatisfactory results. In this work, we propose an evaluation methodology for web archive search systems based on a list of requirements compiled from previous characterizations of web archives and their users. The methodology includes the design of a test collection and the selection of evaluation measures to support realistic and reproducible experiments. The test collection enabled, for the first time, to measure the effectiveness of state-of-the-art IR technology employed in web archives. Results confirm the poor quality of search results retrieved with such technology. However, we show how to combine temporal features, along with the regular topical features, to improve the search effectiveness on web archives. The test collection is available to the research community.