Skip to main content
SpringerLink
Log in

The Power of the TSNLP: Lessons from a Diagnostic Evaluation of a Broad-Coverage Parser

  • Conference paper
  • First Online:
Advances in Artificial Intelligence (Canadian AI 2000)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 1822))

Abstract

We show a diagnostic evaluation of DIPETT, a broad-coverage parser of English sentences. We consider the TSNLP suite as a diagnostic tool, and propose an alternative broader-coverage test suite of test sentences extracted from Quirk et al. We compare the diagnostic effectiveness of the two suites, and draw a few general conclusions. The evaluation results were used to make significant improvements to DIPETT.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Balkan, L., Meijer, S., Arnold, D., Dauphin, E., Estival, D., Falkedal, K., Lehmann, S., Regnier-Prost, S. (1994). Test Suite Design Guidelines and Methodology. Report to LRE 62-089 (D-WP2.1). University of Essex, UK.

    Google Scholar 

  2. Balkan, L., Meijer, S., Arnold, D., Dauphin, E., Estival, D., Falkedal, K., Lehmann, S., Netter, K., Regnier-Prost, S. (1994). Issues in Test Suite Design. Report to LRE 62-089 (DWP2.1). University of Essex, UK.

    Google Scholar 

  3. Balkan, L., Netter, K., Arnold, D., Meijer, S. (1994). TSNLP: Test Suites for Natural Language Processing. In Proceedings of the Language Engineering Convention, ELSNET, Centre for Cognitive Science, University of Edinburgh (pp. 17–22). Edinburgh, Scotland.

    Google Scholar 

  4. Barker, K., Copeck, T., Delisle, S. and Szpakowicz, S. (1997) “Systematic Construction of a Versatile Case System”, J. Natural Language Engineering, vol. 3, no. 4, 279–315.

    Article  Google Scholar 

  5. Barker, K., Delisle, S., Szpakowicz, S. (1998). Test-Driving Tanka: Evaluating a Semi-Automatic System of Text Analysis for Knowledge Acquisition. In Proceedings of the Twelfth Canadian Conference on Artificial Intelligence (LNAI 1418). Vancouver, Canada.

    Google Scholar 

  6. Carroll, J., Briscoe, E. & Sanfilippo, A. (1998). Parser Evaluation: a Survey and a New Proposal. In Proceedings of the International Conference on Language Resources and Evaluation. Granada, Spain. ftp://cogs.susx.ac.uk/pub/users/johnca/lre98-final.ps.

  7. Cole, R., Mariani, J., Uszkoreit, H., Zaenen, A. & Zue, V. (1996). Survey of the state of the art in human language technology. http://www.cse.ogi.edu/CSLU/HLTsurvey/HLTsurvey.html

  8. Delisle, S. (1994). “Text Processing without A-Priori Domain Knowledge: Semi-Automatic Linguistic Analysis for Incremental Knowledge Acquisition”, Ph.D. thesis, TR-94-02, Department of Computer Science, University of Ottawa.

    Google Scholar 

  9. Delisle, S. and Szpakowicz, S. (1995) “Realistic Parsing: Practical Solutions of Difficult Problems”, Proc PACLING’95, 59–68.

    Google Scholar 

  10. Delisle, S., Barker, K., Copeck, T., and Szpakowicz, S. (1996) “Interactive Semantic Analysis of Technical Texts”, Computational Intelligence vol. 12, no. 2, 273–306.

    Article  Google Scholar 

  11. Delisle, S., Szpakowicz, S. (1997) “Extraction of Predicate-Argument Structures from Texts”, Proc Conf on Recent Advances in Natural Language Processing RANLP97, Bulgaria, September 1997, 318–323.

    Google Scholar 

  12. Estival, D., Falkedal, K., Lehmann, S., Balkan, L.,., Meijer, S., Arnold, D., Regnier-Prost, S., Dauphin, E., Netter, K., Oepen, S. (1994). Test Suite Design: Annotation Scheme. Report to LRE 62-089 (D-WP2.2). University of Essex, UK.

    Google Scholar 

  13. Lehmann, S., Oopen, S. Regnier-Prost, S., Netter, K., Lux, V., Klein, J., Falkedal, L., Fouvry, F., Estival, D., Dauphin, E., Compagnion, H., Baur, J., Balkan, L. & Arnold, D. (1996). TSNLP — test suites for natural language processing. In Proceedings of the International Conference on Computational Linguistics, COLING-96 (pp. 711–716). Copenhagen, Denmark.

    Google Scholar 

  14. Quirk, R. & Greenbaum, S. (1973). A University Grammar of English, Longman.

    Google Scholar 

  15. Quirk, R., Greenbaum, S., Leech, G., Svartvik, J. (1985). A Comprehensive Grammar of the English Language, Longman.

    Google Scholar 

  16. Soames, S. & Permutter, D. (1979). Syntactic Argumentation and the structure of English. University of California Press, Berkeley. Los Angeles, 1979.

    Google Scholar 

  17. Srinivas, B., Sarkar, A., Doran, C. & Hockey, B. (1998). Grammar & Parser Evaluation in the XTAG Project. In Proceedings of the International Conference on Language Resources and Evaluation. Granada, Spain.

    Google Scholar 

  18. Winograd, T. (1983). Language as a Cognitive Process (Volume 1: Syntax), Addison-Wesley.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Technology and Engineering, University of Ottawa, Ottawa, Ontario, Canada, K1N 6N5

    Elizabeth Scarlett & Stan Szpakowicz

Authors
  1. Elizabeth Scarlett
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stan Szpakowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Regina, Regina, SK, S4S 0A2, Canada

    Howard J. Hamilton

Rights and permissions

Reprints and permissions

Copyright information

© 2000 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Scarlett, E., Szpakowicz, S. (2000). The Power of the TSNLP: Lessons from a Diagnostic Evaluation of a Broad-Coverage Parser. In: Hamilton, H.J. (eds) Advances in Artificial Intelligence. Canadian AI 2000. Lecture Notes in Computer Science(), vol 1822. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45486-1_12

Download citation

  • DOI: https://doi.org/10.1007/3-540-45486-1_12

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-67557-0

  • Online ISBN: 978-3-540-45486-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation