Skip to main content
SpringerLink
Log in

A Retrospective Evaluation Method for Exact-Match and Best-Match Queries Applying an Interactive Query Performance Analyser

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2291))

Included in the following conference series:

Abstract

A retrospective method for the performance comparison of Boolean and best-match queries is introduced. The method is based on the interactive optimisation of queries by a group of test searchers using a query performance analyser. The case experiment focused on comparing the maximum effectiveness of Boolean exact-match queries, and structured and unstructured best-match queries. The experiment verified the problems in maintaining precision of Boolean queries at high recall levels. Interesting similarities were also observed between structured and unstructured best-match queries giving new light on the results of earlier studies. The case experiment showed that the proposed evaluation method yields more elaborated results in comparisons than earlier query-centred methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Allan, J., Callan, J., Croft, W.B., Ballestros, L., Broglio, J., Xu, J. & Shu, H. (1997). INQUERY at TREC 5. In: Harman, D.K. & Voorhees, E.M. (Eds.) Information technology: The Fifth Text REtrieval Conference (TREC-5). Gaithersburg, National Institute of Standards and Technology, 119–132.

    Google Scholar 

  2. Blair, D.C. & Maron, M.E. (1985). An evaluation of retrieval effectiveness for a full-text document retrieval system. Communications of the ACM (28)3, 289–299.

    Article  Google Scholar 

  3. Feldman, S. (1996). Testing Natural Language: Comparing DIALOG, TARGET, and DRLINK. Online 20(6), 71–79.

    Google Scholar 

  4. Frants, V.I., Shapiro, J., Taksa, I. & Voiskunskii, V.G. (1999). Boolean Search: Current State and Perspectives. Journal of the American Society of Information Science 50(1), 86–95.

    Article  Google Scholar 

  5. Harter, S.P. (1986). Online Information retrieval. Orlando: Academic Press.

    Google Scholar 

  6. Harter, S.P. (1990). Search Term Combinations and Retrieval Overlap: A Proposed Methodology and Case Study. Journal of the American Society for Information Science 41(2), 132–146.

    Article  Google Scholar 

  7. Hersh, W.R. & Hickam, D.H. (1995). An Evaluation of Interactive Boolean and Natural Language Searching with Online Medical Textbook. Journal of the American Society for Information Science 48(7), 478–489.

    Article  Google Scholar 

  8. Hersh, W., Turpin, A., et al. (2001). Challenging conventional assumptions of automated information retrieval with real users: Boolean searching and batch retrieval evaluation. Information Processing & Management 37(), 383–402.

    Article  MATH  Google Scholar 

  9. Keen, E.M. (1992). Presenting results of experimental retrieval comparisons. Information Processing & Management 28(4), 491–502.

    Article  Google Scholar 

  10. Kekäläinen, J. & Järvelin, K. (1998). The impact of query structure and query expansion on retrieval performance. In: Croft, W.B., Moffat, A., et al. (Eds.), Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (ACM SIGIR’ 98). New York, NY: ACM Press, pp. 130–137.

    Google Scholar 

  11. Lu, X.A., Holt, J.D. & Miller, D.J. (1996). Boolean System Revisited: Its Performance and its Behaviour. In: Harman, D.K. (Ed.) The Fourth Text REtrieval Conference (TREC-4). Gaithersburg, National Institute of Standards and Technology, 459–473.

    Google Scholar 

  12. Paris, L.A.H. & Tibbo, H.R. (1998). Freestyle vs. Boolean: A comparison of partial and exact match retrieval systems. Information Processing & Management 34(2/3), 175–190.

    Article  Google Scholar 

  13. Pirkola A. (1998). The effects of query structure and dictionary setups in dictionary-based cross-language information retrieval. In: Croft, W.B., Moffat, A. et al. (Eds.), Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (ACM SIGIR’ 98). New York, NY: ACM Press, pp. 55–63.

    Google Scholar 

  14. Robertson, S.E. (1996). Letter to the Editor. Information Processing & Management 32(5), 635–636.

    Article  Google Scholar 

  15. Robertson, S.E. & Thompson, C.L. (1990). Weighted searching: The CIRT experiment. In: Jones, K.P. (Ed.), Informatics 10-Prospects for Intelligent retrieval. London: Aslib, p. 153–165.

    Google Scholar 

  16. Salton, G., Fox, E.A. & Wu, H. (1983). Extended Boolean Information Retrieval. Communications of the ACM 26(11), 1022–1036.

    Article  MATH  MathSciNet  Google Scholar 

  17. Shaw, W.M. (1995). Term-relevance computations and perfect retrieval performance. Information Processing and Management 31(4), 491–498.

    Article  Google Scholar 

  18. Sormunen, E. (2000a). A method for measuring wide range performance of Boolean queries in full-text databases. Doctoral Thesis. Acta Electronica Universitatis Tamperensis, URL: http://acta.uta.fi/pdf/951-44-4732-8.pdf. Tampere: University of Tampere, 2000.

  19. Sormunen, E. (2000b). A novel method for the evaluation of Boolean query effectiveness across a wide operational range. In: Belkin, N.J., Ingwersen, P. and Leong, M.-K: eds. Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY: ACM, 2000, 25–32.

    Google Scholar 

  20. Sormunen, E. (2001) Extensions to the STAIRS Study-Empirical Evidence for the Hypothesised Ineffectiveness of Boolean Queries in Large Full-Text Databases. Information Retrieval 4(3/4):257–274.

    Article  MATH  Google Scholar 

  21. Sormunen, E., Keskustalo, H. & Halttunen, K. (2001a). Query Performance Analyser-a interactive tool for bridging information retrieval research and education. Submitted for publication in Information Retrieval.

    Google Scholar 

  22. Sormunen, E., Kekäläinen, J., Koivisto, J. and Järvelin, K. (2001b). Document text characteristics affect the ranking of the most relevant documents by expanded structured queries. Journal of Documentation 57(3):358–374.

    Article  Google Scholar 

  23. Sperer, R. & Oard, D.W. (2000). Structured translation for cross-language information retrieval. In: Belkin, N.J., Ingwersen, P. and Leong, M.-K: eds. Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY: ACM, 2000, 120–127.

    Google Scholar 

  24. Tague-Sutcliffe, J. (1992). The pragmatics of information retrieval experimentation, revisited. Information Processing and Management 28(4), 467–490.

    Article  Google Scholar 

  25. Tenopir, C. & Cahn, P. (1994). TARGET & FREESTYLE: Dialog and Mead join the relevance ranks. Online 18(3), 31–47.

    Google Scholar 

  26. Tenopir, C. & Shu, M.E. (1989). Magazines in full text: uses and search strategies. Online Review 13 (2), 107–118.

    Article  Google Scholar 

  27. Turtle, H. R. (1990). Inference networks for document retrieval. Ph.D. dissertation. Computer and information Science Department, University of Massachusetts. COINS Technical Report 90-92.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Studies, University of Tampere, 33014, Tampere, FIN, Finland

    Eero Sormunen

Authors
  1. Eero Sormunen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer and Information Sciences, University of Strathclyde, 26 Richmond Street, G1 1XH, Glasgow, UK

    Fabio Crestani

  2. School of Information and Communication Technologies, University of Paisley, High Street, PA1 2BE, Paisley, UK

    Mark Girolami

  3. Computing Science Department, University of Glasgow, 17 Lilybank Gardens, G12 8RZ, Glasgow, UK

    Cornelis Joost van Rijsbergen

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Sormunen, E. (2002). A Retrospective Evaluation Method for Exact-Match and Best-Match Queries Applying an Interactive Query Performance Analyser. In: Crestani, F., Girolami, M., van Rijsbergen, C.J. (eds) Advances in Information Retrieval. ECIR 2002. Lecture Notes in Computer Science, vol 2291. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45886-7_22

Download citation

  • DOI: https://doi.org/10.1007/3-540-45886-7_22

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43343-9

  • Online ISBN: 978-3-540-45886-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation