Abstract
This chapter describes the metrics for the evaluation of information retrieval and natural language processing systems, the annotation techniques and evaluation metrics and the concepts of training, development and evaluations sets for information retrieval systems.
Download chapter PDF
References
Artstein, R., & Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4), 555–596.
Cleverdon, C. (1967). The Cranfield tests on index language devices. In Aslib Proceedings (pp. 173–194). MCB UP Ltd.
Hripcsak, G., & Rothschild, A. S. (2005). Agreement, the F-measure, and reliability in information retrievas. Journal of the American Medical Informatics Association, 12(3), 296–298.
Japkowicz, N., & Shah, M. (2011). Evaluating Learning Algorithms: A Classification Perspective. Cambridge: Cambridge University Press.
Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In International Joint Conference on Artificial Intelligence (IJCAI) (pp. 1137–1145).
Neves, M., & Leser, U. (2012). A survey on annotation tools for the biomedical literature. Briefings in Bioinformatics, 15(2), 327–340.
Pustejovsky, J., & Stubbs, A. (2012). Natural Language Annotation for Machine Learning. O’Reilly Media, Inc. Beijing.
Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., & Tsujii, J. (2012). BRAT: A web-based tool for NLP-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (pp. 102–107). Association for Computational Linguistics.
Van Rijsbergen, C. J. (1979). Information Retrieval. Butterworth & Co. http://www.dcs.glasgow.ac.uk/Keith/Preface.html. Accessed 11 Jan 2018.
Voorhees, E. M. (2001). The philosophy of information retrieval evaluation. In Evaluation of Cross-Language Information Retrieval Systems (pp. 355–370). Berlin: Springer.
Author information
Authors and Affiliations
Rights and permissions
This chapter is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
Copyright information
© 2018 The Author(s)
About this chapter
Cite this chapter
Dalianis, H. (2018). Evaluation Metrics and Evaluation. In: Clinical Text Mining. Springer, Cham. https://doi.org/10.1007/978-3-319-78503-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-78503-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-78502-8
Online ISBN: 978-3-319-78503-5
eBook Packages: Computer ScienceComputer Science (R0)