Skip to main content
SpringerLink
Log in

CLIR System Evaluation at the Second NTCIR Workshop

  • Conference paper
  • First Online:
Evaluation of Cross-Language Information Retrieval Systems (CLEF 2001)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2406))

Included in the following conference series:

Abstract

This paper introduces the NTCIR Workshop, a series of evaluation workshops, which are designed to enhance research in information retrieval and related text-processing techniques, such as summarization and extraction, by providing large-scale test collections and a forum for researchers. The paper gives a brief history of the Workshop, describes the tasks, participants, test collections and CLIR evaluation at the second workshop, and discusses the plans for the third workshop. In conclusion, some thoughts on future directions are suggested.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. NTCIR Project: http://research.nii.ac.jp/ntcir/

  2. NTCIR Workshop 1: Proceedings of the First NTCIR Workshop on Research in Japanese Text Retrieval and Term Recognition, 30 Aug.-1 Sept., 1999, Tokyo, (ISBN4-924600-77-6. http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings/).

  3. IREX URL:http://cs.nyu.edu/cs/projects/proteus/irex/

  4. NTCIR Workshop 2: Proceedings of the Second NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization, Tokyo, June 2000-March 2001 ISBN4-924600-96-2)

    Google Scholar 

  5. Kando, N.: Cross-Linguistic Scholarly Information Transfer and Database Services in Japan. Annual Meeting of the ASIS, Washington DC. Nov. 1, 1997

    Google Scholar 

  6. Kuriyama, K., Yoshioka, M., Kando, N.: Effect of Cross-Lingual Pooling. In NTCIR Workshop 2: Proceedings of the Second NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization, Tokyo, June 2000-March 2001 ISBN: 4-924600-96-2)

    Google Scholar 

  7. Spink, A., Bateman, J. From highly relevant to not relevant: Examining different regions of relevance. Information Processing and Management, Vol.34, No.5, pp.599–622, 1998

    Article  Google Scholar 

  8. Dunlop, M.D. Reflections on Mira, Journal of the American Society for Information Sciences, Vol.51, No.14, pp.1269–1274, 2000

    Article  Google Scholar 

  9. Spink, A., Greisdorf, H. Regions and levels: Measuring and mapping users’ relevance judgments. Journal of the American Society for Information Sciences, Vol.52, No.2, pp.161–173, 2001

    Article  Google Scholar 

  10. Kando, N., Kuriyama, K., Yoshioka, M. Evaluation based on multi-grade relevance judgements. IPSJ SIG Notes, Vol.2001-FI-63, pp.105–112, July 2001.

    Google Scholar 

  11. Sung, H.M. “HANTEC Collection”. Presented at the panel on IR Evaluation in the 4th IRAL, Hong Kong, 30 Sept.-3 Oct. 2000.

    Google Scholar 

  12. Yoshioka, M., Kuriyama, K., Kando, N.: Analysis on the Usage of Japanese Segmented Texts in the NTCIR Workshop 2. In NTCIR Workshop 2: Proceedings of the Second NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization, Tokyo, June 2000-March 2001 ISBN: 4-924600-96-2).

    Google Scholar 

  13. Kando, N, Nozue, T., Kuriyama, K., Oyama, K.: NTCIR-1: Its Policy and Practice, IPSJSIG Notes, Vol.99, No.20, pp.33–40, 1999 [in Japanese].

    Google Scholar 

  14. Kuriyama, K., Nozue, T., Kando, N., Oyama, K.: Pooling for a Large Scale Test Collection: Analysis of the Search Results for the Pre-test of the NTCIR-1 Workshop, IPSJ SIG Notes, Vol.99-FI-54, pp.25–32 May, 1999 [in Japanese].

    Google Scholar 

  15. Kuriyama, K., Kando, K.: Construction of a Large Scale Test Collection: Analysis of the Training Topics of the NTCIR-1, IPSJ SIG Notes, Vol.99-FI-55, pp.41–48, July 1999 [in Japanese].

    Google Scholar 

  16. Kando, N., Eguchi, K., Kuriyama, K.: Construction of a Large Scale Test Collection: Analysis of the Test Topics of the NTCIR-1, In Proceedings of IPSJ Annual Meeting [in Japanese]. pp.3–107 — 3–108, 30 Sept.-3 Oct. 1999.

    Google Scholar 

  17. Voorhees, E.M.: Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness, In Proceedings of 21st Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval. pp.315–323, Melbourne, Australia, August 1998.

    Google Scholar 

  18. Chen, K.H., Chen, H.H.: The Chinese Text Retrieval Tasks of NTCIR Workshop In NTCIR Workshop 2: Proceedings of the Second NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization, Tokyo, June 2000-March 2001 (ISBN: 4-924600-96-2).

    Google Scholar 

  19. Kando, N., Kuriyama, K., Yoshioka, M.: Overview of Japanese and English Information Retrieval Tasks (JEIR) at the Second NTCIR Workshop. In NTCIR Workshop 2: Proceedings of the Second NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization, Tokyo, June 2000-March 2001 ISBN: 4-924600-96-2).

    Google Scholar 

  20. CLEF: Cross-Language Evaluation Forum, http://www.iei.pi.cnr.it/DELOS/CLEF

Download references

Author information

Authors and Affiliations

  1. National Institute of Informatics (NII), Tokyo, 101-8430, Japan

    Noriko Kando

Authors
  1. Noriko Kando
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Consiglio Nazionale delle Ricerche, Istituto di Scienza e Tecnologie, Via G. Moruzzi 1, 56124, Pisa, Italy

    Carol Peters

  2. Eurospider Information Technology AG, Schaffhauserstrasse 18, 8006, Zürich, Switzerland

    Martin Braschler

  3. E.T.S.I. Industriales, Universidad Nacional de Educación a Distancia, Ciudad Universitaria s/n, 28040, Madrid, Spain

    Julio Gonzalo

  4. Informations Zentrum Sozialwissenschaften, Lennestr. 30, 53113, Bonn, Germany

    Michael Kluck

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kando, N. (2002). CLIR System Evaluation at the Second NTCIR Workshop. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds) Evaluation of Cross-Language Information Retrieval Systems. CLEF 2001. Lecture Notes in Computer Science, vol 2406. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45691-0_35

Download citation

  • DOI: https://doi.org/10.1007/3-540-45691-0_35

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-44042-0

  • Online ISBN: 978-3-540-45691-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation