Skip to main content

Simplified Readability Metric Drives Improvement of Radiology Reports: an Experiment on Ultrasound Reports at a Pediatric Hospital


Highly complex medical documents, including ultrasound reports, are greatly mismatched with patient literacy levels. While improving radiology reports for readability is a longstanding concern, few articles objectively measure the effectiveness of physician training for readability improvement. We hypothesized that writing styles may be evaluated using an objective two-dimensional measure and writing training could improve the writing styles of radiologists. To test it, a simplified “grade vs. length” readability metric is developed based on results from factor analysis of ten readability metrics applied to more than 500,000 radiology reports. To test the short-term effectiveness of a writing workshop, we measured the writing style improvement before and after the training. Statistically significant writing style improvement occurred as a result of the training. Although the degree of improvement varied for different measures, it is evident that targeted training could provide potential benefits to improve readability due to our statistically significant results. The simplified grade vs. length metric enables future clinical decision support systems to quantitatively guide physicians to improve writing styles through writing workshops.

This is a preview of subscription content, access via your institution.

Fig. 1


  1. 1.

    Weiner SJ, Schwartz A, Sharma G, et al. Patient-centered decision making and health care outcomes: an observational study. Annals of internal medicine 2013;158(8):573–79

    Article  PubMed  Google Scholar 

  2. 2.

    Woods SS, Schwartz E, Tuepker A, et al. Patient experiences with full electronic access to health records and clinical notes through the My HealtheVet Personal Health Record Pilot: qualitative study. Journal of medical Internet research 2013;15(3):e65

    Article  PubMed  PubMed Central  Google Scholar 

  3. 3.

    Landro L Radiologists push for medical reports patients can understand. Wall Street Journal 2014

  4. 4.

    Safeer RS, Keenan J. Health literacy: the gap between physicians and patients. Am Fam Physician 2005;72(3):463–68

    PubMed  Google Scholar 

  5. 5.

    Abujudeh H, Pyatt RS, Bruno MA, et al. Radpeer peer review: relevance, use, concerns, challenges, and direction forward. Journal of the American College of Radiology 2014;11(9):899–904

    Article  PubMed  Google Scholar 

  6. 6.

    Kirsch IS Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey: ERIC, 1993

  7. 7.

    Svider PF, Agarwal N, Choudhry OJ, et al. Readability assessment of online patient education materials from academic otolaryngology–head and neck surgery departments. American journal of otolaryngology 2013;34(1):31–35

    Article  PubMed  Google Scholar 

  8. 8.

    Eloy JA, Li S, Kasabwala K, et al. Readability assessment of patient education materials on major otolaryngology association websites. Otolaryngology--Head and Neck Surgery 2012;147(5):848–54

    Article  PubMed  Google Scholar 

  9. 9.

    Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Education & Behavior 2006;33(3):352–73

    Article  Google Scholar 

  10. 10.

    Friedman DB, Kim S-H, Tanner A, Bergeron CD, Foster C, General K. How are we communicating about clinical trials?: an assessment of the content and readability of recruitment resources. Contemporary clinical trials 2014;38(2):275–83

    Article  PubMed  Google Scholar 

  11. 11.

    Valentini M, Daniela D, Alonzo MCP, Lucisano G, Nicolucci A: Application of a readability score in informed consent forms for clinical studies. Journal of Clinical Research & Bioethics 2013:2013

  12. 12.

    Okuhara T, Ishikawa H, Okada H, Kiuchi T. Readability, suitability and health content assessment of cancer screening announcements in municipal newspapers in Japan. Asian Pacific journal of cancer prevention: APJCP 2014;16(15):6719–27

    Article  Google Scholar 

  13. 13.

    Zeng-Treitler Q, Goryachev S, Kim H, Keselman A, Rosendale D. Making texts in electronic health records comprehensible to consumers: a prototype translator. AMIA Annu Symp Proc 2007:846–50

  14. 14.

    Leroy G, Helmreich S, Cowie JR, Miller T, Zheng W. Evaluating online health information: beyond readability formulas. AMIA Annu Symp Proc 2008:394–8

  15. 15.

    Savova GK, Masanz JJ, Ogren PV, et al. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assn 2010;17(5):507–13

    Article  Google Scholar 

  16. 16.

    Meystre SM, Savova GK, Kipper-Schuler KC, Hurdle JF. Extracting information from textual documents in the electronic health record: a review of recent research. Yearb Med Inform 2008:128–44

  17. 17.

    Osborne H.: Health Literacy from A to Z: Jones & Bartlett Publishers, 2012

  18. 18.

    Applying multiple methods to assess the readability of a large corpus of medical documents. 14th World Congress on Medical and Health Informatics, MEDINFO 2013; 2013

  19. 19.

    Bruno MA, Petscavage-Thomas JM, Mohr MJ, Bell SK, Brown SD. The “open letter”: radiologists’ reports in the era of patient web portals. Journal of the American College of Radiology 2014;11(9):863–67

    Article  PubMed  Google Scholar 

  20. 20.

    Flanders AE, Lakhani P. Radiology reporting and communications: a look forward. Neuroimaging Clinics of North America 2012;22(3):477–96

    Article  PubMed  Google Scholar 

  21. 21.

    Penniston A, Harley E: Attempts to verify written english. Proceedings of The Fourth International Conference on Computer Science and Software Engineering 121–28,2011

  22. 22.

    Comrey AL, Lee HB. A First Course in Factor Analysis: Psychology Press, 2013

  23. 23.

    Kline P An easy guide to factor analysis: Routledge, 2014.

  24. 24.

    Cliff N. The eigenvalues-greater-than-one rule and the reliability of components. Psychol Bull 1988;103(2):276–79

    Article  Google Scholar 

  25. 25.

    Hansberry DR, John A, John E, Agarwal N, Gonzales SF, Baker SR. A critical review of the readability of online patient education resources from RadiologyInfo. Org. American Journal of Roentgenology 2014;202(3):566–75

    Article  PubMed  Google Scholar 

  26. 26.

    AHRQ: Health Literacy. Secondary Health Literacy 2013.

  27. 27.

    Leong EK, Ewing MT, Pitt LF. E-comprehension: evaluating B2B websites using readability formulae. Industrial Marketing Management 2002;31(2):125–31

    Article  Google Scholar 

  28. 28.

    Allen ED, Bernhardt EB, Berry MT, Demel M. Comprehension and text genre: an analysis of secondary school foreign language readers. The Modern Language Journal 1988;72(2):163–72

    Article  Google Scholar 

  29. 29.

    Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults. Journal of general internal medicine 1995;10(10):537–41

    CAS  Article  PubMed  Google Scholar 

  30. 30.

    Klare GR. The role of word frequency in readability. Elementary English 1968;45(1):12–22

    Google Scholar 

  31. 31.

    Marks CB, Doctorow MJ, Wittrock MC. Word frequency and reading comprehensiony1. The Journal of Educational Research 1974;67(6):259–62

    Article  Google Scholar 

Download references


We would like to thank Richard Hoyt and Megan Reynolds from Nationwide Children’s Hospital’s Research Information Solutions and Innovation (RISI) center for their assistance with retrieving data from Epic. We also thank Tran Bourgeois for using REDCap (Research Electronic Data Capture) to set up user experiments, Katherine Strohm for project management, Dr. Jeffrey Hoffman for discussion of EPIC integration, and Dr. Michael Bruno from the Milton S. Hershey Medical Center at the Penn State College of Medicine for inspiring the study.

Author information



Corresponding author

Correspondence to Wei Chen.

Ethics declarations


This study was supported by grant UL1TR001070 from the Center for Clinical and Translational Science at the Ohio State University to Dr. Brent Adler and MDSR Roessler Scholarship from the Ohio State University medical center to Claire Durkin.

Competing Interests

The author’s declare that they have no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chen, W., Durkin, C., Huang, Y. et al. Simplified Readability Metric Drives Improvement of Radiology Reports: an Experiment on Ultrasound Reports at a Pediatric Hospital. J Digit Imaging 30, 710–717 (2017).

Download citation


  • Readability metrics
  • Radiology reports
  • Ultrasound reports
  • Writing styles
  • Factor analysis