Advertisement

Overview of the INEX 2008 Ad Hoc Track

  • Jaap Kamps
  • Shlomo Geva
  • Andrew Trotman
  • Alan Woodley
  • Marijn Koolen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5631)

Abstract

This paper gives an overview of the INEX 2008 Ad Hoc Track. The main goals of the Ad Hoc Track were two-fold. The first goal was to investigate the value of the internal document structure (as provided by the XML mark-up) for retrieving relevant information. This is a continuation of INEX 2007 and, for this reason, the retrieval results are liberalized to arbitrary passages and measures were chosen to fairly compare systems retrieving elements, ranges of elements, and arbitrary passages. The second goal was to compare focused retrieval to article retrieval more directly than in earlier years. For this reason, standard document retrieval rankings have been derived from all runs, and evaluated with standard measures. In addition, a set of queries targeting Wikipedia have been derived from a proxy log, and the runs are also evaluated against the clicked Wikipedia pages. The INEX 2008 Ad Hoc Track featured three tasks: For the Focused Task a ranked-list of non-overlapping results (elements or passages) was needed. For the Relevant in Context Task non-overlapping results (elements or passages) were returned grouped by the article from which they came. For the Best in Context Task a single starting point (element start tag or passage start) for each article was needed. We discuss the results for the three tasks, and examine the relative effectiveness of element and passage retrieval. This is examined in the context of content only (CO, or Keyword) search as well as content and structure (CAS, or structured) search. Finally, we look at the ability of focused retrieval techniques to rank articles, using standard document retrieval techniques, both against the judged topics as well as against queries and clicks from a proxy log.

Keywords

Context Task Focus Task Passage Retrieval Content Only Generalize Precision 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Callan, J.P.: Passage-level evidence in document retrieval. In: Proceedings of the 17th Annual International ACM SIGIR Conference, pp. 302–310 (1994)Google Scholar
  2. 2.
    Clarke, C.L.A.: Range results in XML retrieval. In: Proceedings of the INEX 2005 Workshop on Element Retrieval Methodology, pp. 4–5 (2005)Google Scholar
  3. 3.
    Denoyer, L., Gallinari, P.: The Wikipedia XML Corpus. SIGIR Forum 40, 64–69 (2006)CrossRefGoogle Scholar
  4. 4.
    Huang, W., Trotman, A., O’Keefe, R.A.: Element retrieval using a passage retrieval approach. In: Proceedings of the 11th Australasian Document Computing Symposium (ADCS 2006), pp. 80–83 (2006)Google Scholar
  5. 5.
    Itakura, K.Y., Clarke, C.L.A.: From passages into elements in XML retrieval. In: Proceedings of the SIGIR 2007 Workshop on Focused Retrieval, pp. 17–22 (2007)Google Scholar
  6. 6.
    Kamps, J., Koolen, M.: On the relation between relevant passages and XML document structure. In: Proceedings of the SIGIR 2007 Workshop on Focused Retrieval, pp. 28–32 (2007)Google Scholar
  7. 7.
    Kamps, J., Marx, M., de Rijke, M., Sigurbjörnsson, B.: The importance of morphological normalization for XML retrieval. In: Proceedings of the First INEX Workshop, pp. 41–48 (2003)Google Scholar
  8. 8.
    Kamps, J., Marx, M., de Rijke, M., Sigurbjörnsson, B.: Articulating information needs in XML query languages. Transactions on Information Systems 24, 407–436 (2006)CrossRefGoogle Scholar
  9. 9.
    Kekäläinen, J., Järvelin, K.: Using graded relevance assessments in IR evaluation. Journal of the American Society for Information Science and Technology 53, 1120–1129 (2002)CrossRefGoogle Scholar
  10. 10.
    Thom, J.A., Pehcevski, J.: How well does best in context reflect ad hoc XML retrieval. In: Pre-Proceedings of INEX 2007, pp. 124–125 (2007)Google Scholar
  11. 11.
    Trotman, A., Geva, S.: Passage retrieval and other XML-retrieval tasks. In: Proceedings of the SIGIR 2006 Workshop on XML Element Retrieval Methodology, pp. 43–50 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jaap Kamps
    • 1
  • Shlomo Geva
    • 2
  • Andrew Trotman
    • 3
  • Alan Woodley
    • 2
  • Marijn Koolen
    • 1
  1. 1.University of AmsterdamAmsterdamThe Netherlands
  2. 2.Queensland University of TechnologyBrisbaneAustralia
  3. 3.University of OtagoDunedinNew Zealand

Personalised recommendations