Skip to main content
SpringerLink
Log in

Evaluating Natural Language Processing Techniques in Information Retrieval

A TREC Perspective

  • Chapter
Natural Language Information Retrieval

Part of the book series: Text, Speech and Language Technology ((TLTB,volume 7))

Abstract

Natural language processing techniques may hold a tremendous potential for overcoming the inadequacies of purely quantitative methods of text information retrieval, but the empirical evidence to support such predictions has thus far been inadequate, and appropriate scale evaluations have been slow to emerge. In this chapter, we report on the progress of the Natural Language Information Retrieval project, a joint effort of several sites led by GE Research, and its evaluation in a series of Text Retrieval Conferences (TREC) conducted since 1992 under the auspices of the National Institute of Standards and Technology (NIST) and the Defense Advanced Research Projects Agency (DARPA).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Brill, Eric. (1992). “A Simple Rule-Based Part Of Speech Tagger.” Proceedings of the Third Conference on Applied Computational Linguistics (ANLP).

    Google Scholar 

  • Buckley, Chris, Amit Singhal, Mandar Mitra, Gerard Salton. (1995). “New Retrieval Approches Using SMART: TREC 4”. Proceedings of the Fourth Text REtrieval Conference (TREC-4), NIST Special Publication 500–236, pp. 25–48.

    Google Scholar 

  • Buckley, Chris. (1993). “The Importance of Proper Weighting Methods.” Human Language Technology, Proceedings of the workshop, Princeton, NJ. Morgan-Kaufmann, pp. 349–352.

    Google Scholar 

  • Callan, Jamie. (1994). “Passage-Level Evidence in Document Retrieval.” Proceedings of ACM SIGIR’94. pp. 302–310.

    Google Scholar 

  • Callan, Jamie, Zhihong Lu, and W. Bruce Croft. (1995). “Searching Distributed Collections with Inference Networks.” Proceedings of ACM SIGIR’95. pp. 21–29.

    Google Scholar 

  • Dillon, Martin, and Ann S. Gray. (1983). “FASIT: A fully automatic syntactically-based indexing system.” Journal of the American Society for Information Science, 34 (2), pp. 99–108.

    Article  Google Scholar 

  • Evans, David, and Robert G. Lefferts. (1994). “Design and Evaluation of the CLAR.ITTR.EC-2 System.” Proceedings of the Second Text REtrieval Conference (TREC-2), NIST Special Publication 500–215, National Institute of Standards and Technology, Gaithersburg, MD. pp. 137–150.

    Google Scholar 

  • Fagan, Joel L. (1987). Experiments in Automated Phrase Indexing for Document Retrieval: A Comparison of Syntactic and Non-Syntactic Methods. Ph.D. Thesis. Department of Computer Science, Cornell University.

    Google Scholar 

  • Fox, Ed, M. Koushik, J. Shaw, R.. Modlin, and D. Rao. (1993). “Comining Evidence from Multiple Searches.” Proceedings of First Text Retrieval Conference (TREC-1), NIST Special Publication 500–207, National Institute of Standards and Technology, Gaithersburg, MD. pp. 319–328.

    Google Scholar 

  • Harman, Donna K. (ed.). (1993). The First Text REtrieval Conference (TREC-1). NIST Special Publication 500–207, National Institute of Standards and Technology, Gaithersburg, MD.

    Google Scholar 

  • Harman, Donna K. (ed.). (1994). The Second Text REtrieval Conference (TREC-2). NIST Special Publication 500–215, National Institute of Standards and Technology, Gaithersburg, MD.

    Google Scholar 

  • Harman, Donna. (1988). “Towards interactive query expansion.” Proceedings of ACM SIGIR-88, pp. 321–331.

    Google Scholar 

  • Harman, Donna and Gerald Candela. (1989). “Retrieving Records from a Gigabyte of text on a Minicomputer Using Statistical Ranking.” Journal of the American Society for Information Science, 41 (8), pp. 581–589.

    Article  Google Scholar 

  • Krovetz, Robert and W. Bruce Croft. (1992). “Lexical ambiguity and information retrieval.” ACM Transactions on Information Systems, 10 (2), pp. 113–141.

    Article  Google Scholar 

  • Kwok, K.L., L. Papadopoulos and Kathy Y.Y. Kwan. (1993). “Retrieval Experiments with a Large Collection using PIR.CS.” Proceedings of TREC-1 conference, NIST special publication 500–207, pp. 153–172.

    Google Scholar 

  • Lewis, David D. and W. Bruce Croft. (1990). “Term Clustering of Syntactic Phrases”. Proceedings of ACM SIGIR-90, pp. 385–405.

    Google Scholar 

  • Liddy, Elizabeth D., and Sung H. Myaeng. (1994). “DR.-LINK: A System Update for TREC-2.” Proceedings of the Second Text REtrieval Conference (TREC-2), NIST Special Publication 500–215, National Institute of Standards and Technology, Gaithersburg, MD. pp. 85–99.

    Google Scholar 

  • Mauldin, Michael. (1991). “Retrieval Performance in Ferret: A Conceptual Information Retrieval System.” Proceedings of ACM SIGIR-91, pp. 347–355.

    Google Scholar 

  • Metzler, Douglas P., Stephanie W. Haas, Cynthia L. Cosic, and Leslie H. Wheeler. (1989). “Constituent Object Parsing for Information Retrieval and Similar Text Processing Problems.” Journal of the ASIS, 40 (6), pp. 398–423.

    Google Scholar 

  • Rocchio, J. J. (1971). “Relevance Feedback in Informatio Retrieval.” In Salton, G. (Ed.), The SMART Retrieval System, pp. 313–323. Prentice Hall, Inc., Englewood Cliffs, NJ.

    Google Scholar 

  • Ruge, Gerda. (1991). “Experiments on Linguistically Based Term Associations.” Pro-ceedings of RIAO’91, Barcelona, Spain. pp. 528–545.

    Google Scholar 

  • Ruge, Gerda, Christoph Schwarz, Amy J. Warner. (1991). “Effectiveness and Efficiency in Natural Language Processing for Large Amounts of Text.” Journal of the American Society for information Science42(6), pp. 450–456.

    Google Scholar 

  • Sager, Naomi. (1981). Natural Language Information Processing. Addison-Wesley. Salton, Gerald. (1989). Automatic Text Processing: the transformation analysis and retrieval of information by computer. Addison-Wesley, Reading, MA.

    Google Scholar 

  • Saracevic, T., Kantor, P. (1988). “A Study of Information Seeking and Retrieving. III. Searchers, Searches, and Overlap.” Journal of the American Society for information Science39(3), pp. 197–216.

    Google Scholar 

  • Smeaton, Alan F. and C. J. van Rijsbergen. (1988). “Experiments on incorporating syntactic processing of user queries into a document retrieval strategy.” Proceedings of ACM SIGIR-88, pp. 31–51.

    Google Scholar 

  • Sparck Jones, K. and J. I. Tait. (1984). “Automatic search term variant generation.” Journal of Documentation, 40 (1), pp. 50–66.

    Article  Google Scholar 

  • Strzalkowski, Tomek and Barbara Vauthey. (1992). “Information Retrieval Using Robust Natural Language Processing.” Proceedings of the 30th ACL Meeting, Newark, DE, pp. 104–111.

    Google Scholar 

  • Strzalkowski, Tomek. (1992). “TTP: A Fast and Robust Parser for Natural Language.” Proceedings of the 14th International Conference on Computational Linguistics (COLING), Nantes, France, pp. 198–204.

    Google Scholar 

  • Strzalkowski, Tomek. (1993). “Natural Language Processing in Large-Scale Text Retrieval Tasks.” Proceedings of the First Text REtrieval Conference (TREC-1), NIST Special Publication 500–207, National Institute of Standards and Technology, Gaithersburg, MD. pp. 173–187.

    Google Scholar 

  • Strzalkowski, Tomek. (1993). “Robust Text Processing in Automated Information Retrieval.” Proceedings of ACL-sponsored workshop on Very Large Corpora. Ohio State Univ. Columbus.

    Google Scholar 

  • Strzalkowski, Tomek, Louise Guthrie, Jussi Karlgren, Jim Leistensnider, Fang Lin, Jose Perez-Carballo, Troy Straszheim, Jin Wang, and Jon Wilding. (1997). “Natural Language Information Retrieval: TREC-5 Report.” Proceedings of The Fifth Text Retrieval Conference (TREC-5), NIST Special Publication 500–238. pp. 291–314.

    Google Scholar 

  • Strzalkowski, Tomek, and Peter Scheyen. (1993). “An Evaluation of TTP Parser: a preliminary report.” Proceedings of International Workshop on Parsing Technologies (IWPT-93), Tilburg, Netherlands and Durbuy, Belgium.

    Google Scholar 

  • Strzalkowski, Tomek and Jose Perez Carballo. (1994). “Recent Developments in Natural Language Text Retrieval.” Proceedings of the First Text REtrieval Conference (TREC-2); NIST Special Publication 500–215, National Institute of Standards and Technology, Gaithersburg, MD. pp. 123–136.

    Google Scholar 

  • Strzalkowski, Tomek, Jose Perez-Carballo and Mihnea Marinescu. (1995). “Natural Language Information Retrieval: TREC-3 Report.” Proceedings of the Third Text REtrieval Conference (TREC-3), NIST Special Publication 500–225, pp. 39–53.

    Google Scholar 

  • Strzalkowski, Tomek, Jose Perez-Carballo and Mihnea Marinescu. (1996). “Natural Language Information Retirieval: TREC-4 Report.” Proceedings of the Fourth Text REtrieval Conference (TREC-4), NIST Special Publication 500–236.

    Google Scholar 

  • Strzalkowski, Tomek. (1995). “Natural Language Information Retrieval” Information Processing and Management31(3), pp. 397–417, Pergamon/Elsevier.

    Google Scholar 

  • Strzalkowski, Tomek, and Peter Scheyen. (1996). “An Evaluation of TTP Parser: a preliminary report.” In H. Bunt, M. Tomita (eds) Recent Advances in Parsing TechnologyKluwer Academic Publishers, pp. 201–220.

    Google Scholar 

  • Voorhees, Ellen M. (1994). “Query Expansion Using Lexical-Semantic Relations.” Proceedings of ACM SIGIR.’94, pp. 61–70.

    Google Scholar 

  • Voorhees, Ellen M. (1993). “Using WordNet to Disambiguate Word Senses for Text Retrieval.” Proceedings of ACM SIGIR’93, pp. 171–180.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. GE Corporate Research & Development, Schenectady, NY, 12301, USA

    Tomek Strzalkowski, Fang Lin & Jin Wang

  2. School of Communication, Information and Library Studies, Rutgers University, New Brunswick, NJ, 04612, USA

    Jose Perez-Carballo

Authors
  1. Tomek Strzalkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fang Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jose Perez-Carballo
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. General Electric, Research & Development, 12301, Schenectady, NY, USA

    Tomek Strzalkowski

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Strzalkowski, T., Lin, F., Wang, J., Perez-Carballo, J. (1999). Evaluating Natural Language Processing Techniques in Information Retrieval. In: Strzalkowski, T. (eds) Natural Language Information Retrieval. Text, Speech and Language Technology, vol 7. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-2388-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-94-017-2388-6_5

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-90-481-5209-4

  • Online ISBN: 978-94-017-2388-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation