Advertisement

Journal of Digital Imaging

, Volume 31, Issue 1, pp 84–90 | Cite as

Using Natural Language Processing of Free-Text Radiology Reports to Identify Type 1 Modic Endplate Changes

  • Hannu T. Huhdanpaa
  • W. Katherine Tan
  • Sean D. Rundell
  • Pradeep Suri
  • Falgun H. Chokshi
  • Bryan A. Comstock
  • Patrick J. Heagerty
  • Kathryn T. James
  • Andrew L. Avins
  • Srdjan S. Nedeljkovic
  • David R. Nerenz
  • David F. Kallmes
  • Patrick H. Luetmer
  • Karen J. Sherman
  • Nancy L. Organ
  • Brent Griffith
  • Curtis P. Langlotz
  • David Carrell
  • Saeed Hassanpour
  • Jeffrey G. Jarvik
Article

Abstract

Electronic medical record (EMR) systems provide easy access to radiology reports and offer great potential to support quality improvement efforts and clinical research. Harnessing the full potential of the EMR requires scalable approaches such as natural language processing (NLP) to convert text into variables used for evaluation or analysis. Our goal was to determine the feasibility of using NLP to identify patients with Type 1 Modic endplate changes using clinical reports of magnetic resonance (MR) imaging examinations of the spine. Identifying patients with Type 1 Modic change who may be eligible for clinical trials is important as these findings may be important targets for intervention. Four annotators identified all reports that contained Type 1 Modic change, using N = 458 randomly selected lumbar spine MR reports. We then implemented a rule-based NLP algorithm in Java using regular expressions. The prevalence of Type 1 Modic change in the annotated dataset was 10%. Results were recall (sensitivity) 35/50 = 0.70 (95% confidence interval (C.I.) 0.52–0.82), specificity 404/408 = 0.99 (0.97–1.0), precision (positive predictive value) 35/39 = 0.90 (0.75–0.97), negative predictive value 404/419 = 0.96 (0.94–0.98), and F1-score 0.79 (0.43–1.0). Our evaluation shows the efficacy of rule-based NLP approach for identifying patients with Type 1 Modic change if the emphasis is on identifying only relevant cases with low concern regarding false negatives. As expected, our results show that specificity is higher than recall. This is due to the inherent difficulty of eliciting all possible keywords given the enormous variability of lumbar spine reporting, which decreases recall, while availability of good negation algorithms improves specificity.

Keywords

Natural language processing Radiology reporting Lumbar spine imaging Modic classification 

Notes

Acknowledgements

This work is supported by the National Institutes of Health (NIH) Common Fund, through a cooperative agreement (5UH3AR06679) from the Office of Strategic Coordination within the Office of the NIH Director. The views presented here are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health.

BOLD funding through AHRQ grant no. 1R01HS022972.

References

  1. 1.
    Cai T et al.: Natural Language Processing Technologies in Radiology Research and Clinical Applications. Radiographics 36:176–191, 2016CrossRefPubMedPubMedCentralGoogle Scholar
  2. 2.
    Langlotz CP: Structured radiology reporting: are we there yet? Radiology 253:23–25, 2009CrossRefPubMedGoogle Scholar
  3. 3.
    Burnside ES et al.: The ACR BI-RADS experience: learning from history. J Am Coll Radiol 6:851–860, 2009CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Hirschberg J, Manning CD: Advances in natural language processing. Science 349:261–266, 2015CrossRefPubMedGoogle Scholar
  5. 5.
    Chapman WW, Bridewell W, Hanbury P, Cooper GF, Buchanan BG: A simple algorithm for identifying negated findings and diseases in discharge summaries. J Biomed Inform 34:301–310, 2001CrossRefPubMedGoogle Scholar
  6. 6.
    Harkema H, Dowling JN, Thornblade T, Chapman WW: ConText: an algorithm for determining negation, experiencer, and temporal status from clinical reports. J Biomed Inform 42:839–851, 2009CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Modic MT, Steinberg PM, Ross JS, Masaryk TJ, Carter JR: Degenerative disk disease: assessment of changes in vertebral body marrow with MR imaging. Radiology 166:193–199, 1988CrossRefPubMedGoogle Scholar
  8. 8.
    Jensen TS, Karppinen J, Sorensen JS, Niinimäki J, Leboeuf-Yde C: Vertebral endplate signal changes (Modic change): a systematic literature review of prevalence and association with non-specific low back pain. Eur Spine J 17:1407–1422, 2008CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Jarvik JG et al.: Back pain in seniors: the back pain outcomes using longitudinal data (BOLD) cohort baseline data. BMC Musculoskelet Disord 15:134, 2014CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Jarvik JG et al.: Lumbar imaging with reporting of epidemiology (LIRE)—protocol for a pragmatic cluster randomized trial. Contemp Clin Trials 45:157–163, 2015CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG: Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:377–381, 2009CrossRefPubMedGoogle Scholar
  12. 12.
    Pons E, Braun LM, Hunink MG, Kors JA: Natural language processing in radiology: a systematic review. Radiology 279:329–343, 2016CrossRefPubMedGoogle Scholar
  13. 13.
    R Core Team: R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing, 2013Google Scholar
  14. 14.
    Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 33:159–174, 1977CrossRefPubMedGoogle Scholar
  15. 15.
    Fardon DF, Williams AL, Dohring EJ, Murtagh FR, Gabriel Rothman SL, Sze GK: Lumbar disc nomenclature: version 2.0: Recommendations of the combined task forces of the North American Spine Society, the American Society of Spine Radiology and the American Society of Neuroradiology. Spine J 14:2525–2545, 2014CrossRefPubMedGoogle Scholar
  16. 16.
    Cheng LT, Zheng J, Savova GK, Erickson BJ: Discerning tumor status from unstructured MRI reports—completeness of information in existing reports and utility of automated natural language processing. J Digit Imaging 23:119–132, 2010CrossRefPubMedGoogle Scholar
  17. 17.
    Lakhani P, Kim W, Langlotz CP: Automated detection of critical results in radiology reports. J Digit Imaging 25:30–36, 2012CrossRefPubMedGoogle Scholar
  18. 18.
    Wei Q, Dunbrack RL: The role of balanced training and testing data sets for binary classifiers in bioinformatics. PLoS One 8:e67863, 2013CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Society for Imaging Informatics in Medicine 2017

Authors and Affiliations

  • Hannu T. Huhdanpaa
    • 1
  • W. Katherine Tan
    • 2
    • 3
  • Sean D. Rundell
    • 4
    • 5
  • Pradeep Suri
    • 4
    • 5
    • 6
  • Falgun H. Chokshi
    • 7
  • Bryan A. Comstock
    • 2
    • 3
  • Patrick J. Heagerty
    • 2
    • 3
  • Kathryn T. James
    • 5
    • 8
  • Andrew L. Avins
    • 9
  • Srdjan S. Nedeljkovic
    • 10
  • David R. Nerenz
    • 11
  • David F. Kallmes
    • 12
  • Patrick H. Luetmer
    • 12
  • Karen J. Sherman
    • 13
  • Nancy L. Organ
    • 2
    • 3
  • Brent Griffith
    • 14
  • Curtis P. Langlotz
    • 15
  • David Carrell
    • 13
  • Saeed Hassanpour
    • 16
  • Jeffrey G. Jarvik
    • 5
    • 8
    • 17
    • 18
  1. 1.Radia, Inc.LynwoodUSA
  2. 2.Department of BiostatisticsUniversity of WashingtonSeattleUSA
  3. 3.Center for Biomedical StatisticsUniversity of WashingtonSeattleUSA
  4. 4.Department of Rehabilitation MedicineUniversity of WashingtonSeattleUSA
  5. 5.Comparative Effectiveness, Cost and Outcomes Research CenterUniversity of WashingtonSeattleUSA
  6. 6.Division of Rehabilitation Care Services, Seattle Epidemiologic Research and Information Center, VA Puget Sound Health Care SystemSeattleUSA
  7. 7.Department of Radiology and Imaging SciencesEmory University School of MedicineAtlantaUSA
  8. 8.Department of RadiologyUniversity of WashingtonSeattleUSA
  9. 9.Division of Research, Kaiser Permanente Northern CaliforniaOaklandUSA
  10. 10.Department of Anesthesiology, Perioperative and Pain Medicine, Harvard Vanguard Medical AssociatesBrigham and Women’s Hospital and Spine UnitBostonUSA
  11. 11.Henry Ford HospitalNeuroscience InstituteDetroitUSA
  12. 12.Department of RadiologyMayo ClinicRochesterUSA
  13. 13.Kaiser Permanente of Washington Research InstituteSeattleUSA
  14. 14.Department of RadiologyHenry Ford HospitalDetroitUSA
  15. 15.Department of RadiologyStanford UniversityPalo AltoUSA
  16. 16.Department of Biomedical Data ScienceDartmouth CollegeLebanonUSA
  17. 17.Department of Neurological SurgeryUniversity of WashingtonSeattleUSA
  18. 18.Department of Health ServicesUniversity of WashingtonSeattleUSA

Personalised recommendations