Language Resources and Evaluation

, Volume 50, Issue 4, pp 705–727 | Cite as

A corpus for research in text processing for evidence based medicine

  • Diego MolláEmail author
  • María Elena Santiago-Martínez
  • Abeed Sarker
  • Cécile Paris
Original Paper


Evidence based medicine (EBM) urges the medical doctor to incorporate the latest available clinical evidence at point of care. A major stumbling block in the practice of EBM is the difficulty to keep up to date with the clinical advances. In this paper we describe a corpus designed for the development and testing of text processing tools for EBM, in particular for tasks related to the extraction and summarisation of answers and corresponding evidence related to a clinical query. The corpus is based on material from the Clinical Inquiries section of The Journal of Family Practice. It was gathered and annotated by a combination of automated information extraction, crowdsourcing tasks, and manual annotation. It has been used for the original summarisation task for which it was designed, as well as for other related tasks such as the appraisal of clinical evidence and the clustering of the results. The corpus is available at SourceForge (


Corpus Evidence based medicine Annotation Crowdsourcing Text summarization 



Parts of this research were funded by the Oak Ridge Institute for Science and Education (ORISE), Macquarie University, and the Commonwealth Scientific and Industrial Research Organisation (CSIRO).


  1. Afantenos, S., Karkaletsis, V., & Stamatopoulos, P. (2005). Summarization from medical documents: A survey. Artificial Intelligence in Medicine, 33(2), 157–177. doi: 10.1016/j.artmed.2004.07.017.CrossRefGoogle Scholar
  2. Athenikos, S. J., & Han, H. (2010). Biomedical question answering: A survey. Computer Methods and Programs in Biomedicine, 99(1), 1–24. doi: 10.1016/j.cmpb.2009.10.003.CrossRefGoogle Scholar
  3. Demner-Fushman, D., & Lin, J. J. (2007). Answering clinical questions with knowledge-based and statistical techniques. Computational Linguistics, 33(1), 63–103.CrossRefGoogle Scholar
  4. Ebell, M. H., Siwek, J., Weiss, B. D., Woolf, S. H., Susman, J., Ewigman, B., et al. (2004). Strength of recommendation taxonomy (SORT): A patient-centered approach to grading evidence in the medical literature. The Journal of the American Board of Family Practice/American Board of Family Practice, 17(1), 59–67.CrossRefGoogle Scholar
  5. Ekbal, A., Saha, S., Mollá, D., & Ravikumar, K. (2013). Multiobjective optimization for clustering of medical publications. In Proceedings ALTA.Google Scholar
  6. Elhadad, N., Kan, M. Y., Klavans, J. L., & McKeown, K. R. (2005). Customization in a unified framework for summarizing medical literature. Artificial Intelligence in Medicine, 33(2), 179–198. doi: 10.1016/j.artmed.2004.07.018.CrossRefGoogle Scholar
  7. Fiszman, M., Rindflesch, T.C., & Kilicoglu, H. (2004). Abstraction summarization for managing the biomedical research literature. In Proceedings of HLT-NAACL workshop on computational lexical semantics (pp. 76–83).Google Scholar
  8. Kim, S. N., Martinez, D., Cavedon, L., & Yencken, L. (2011). Automatic classification of sentences to support evidence based medicine. BMC Bioinformatics, 12(Suppl 2), S5. doi: 10.1186/1471-2105-12-S2-S5.CrossRefGoogle Scholar
  9. Mollá, D. (2010). A corpus for evidence based medicine summarisation. In Proceedings of the Australasian language technology workshop (Vol. 8, pp. 76–80).Google Scholar
  10. Mollá, D., & Santiago-Martínez, M. E. (2011). Development of a corpus for evidence based medicine summarisation. In Proceedings of the Australasian language technology workshop.Google Scholar
  11. Mollá, D., & Santiago-Martínez, M. E. (2012). Creation of a corpus for evidence based medicine summarisation. Australasian Medical Journal, 5(9), 503–506.CrossRefGoogle Scholar
  12. Mollá, D., & Sarker, A. (2011). Automatic grading of evidence: The 2011 ALTA Shared Task. In Proceedings of the 2011 Australasian language technology workshop (ALTA 2011). doi: 10.13140/2.1.1706.1444.
  13. Mounsey, A. L., & Henry, S. L. (2009). Clinical inquiries. Which treatments work best for hemorrhoids? The Journal of Family Practice, 58(9), 492–493.Google Scholar
  14. Niu, Y., Hirst, G., McArthur, G., & Rodriguez-Gianolli, P. (2003). Answering clinical questions with role identification. In Proceedings of the ACL, workshop on natural language processing in biomedicine.
  15. Sackett, D. L., Rosenberg, W. M., Gray, J., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn’t. BMJ, 312(7023), 71–72.CrossRefGoogle Scholar
  16. Sarker, A., & Molla, D. (2012). Towards two-step multi-document summarisation for evidence based medicine: A quantitative analysis. In Proceedings of the 2012 Australasian language technology workshop (ALTA 2012).Google Scholar
  17. Sarker, A., Molla, D., & Paris, C. (2013). An approach for query-focused text summarisation for evidence based medicine. In Proceedings of the 14th Conference on artificial intelligence in medicine (AIME 2013) (pp. 295–304), Springer, Murcia, Spain.
  18. Shash, S. F., & Mollá, D. (2013). Clustering of medical publications for evidence based medicine summarisation. In Artificial intelligence in medicine (pp. 305–309). doi: 10.1007/978-3-642-38326-7_42.
  19. Yu, H., & Cao, Y. G. (2008). Automatically extracting information needs from ad hoc clinical questions. In AMIA annual symposium proceedings, American medical informatics association (Vol. 2008, p. 96).Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  • Diego Mollá
    • 1
    Email author
  • María Elena Santiago-Martínez
    • 1
  • Abeed Sarker
    • 1
  • Cécile Paris
    • 2
  1. 1.Department of ComputingMacquarie UniversitySydneyAustralia
  2. 2.CSIRO MarsfieldSydneyAustralia

Personalised recommendations