Highly complex medical documents, including ultrasound reports, are greatly mismatched with patient literacy levels. While improving radiology reports for readability is a longstanding concern, few articles objectively measure the effectiveness of physician training for readability improvement. We hypothesized that writing styles may be evaluated using an objective two-dimensional measure and writing training could improve the writing styles of radiologists. To test it, a simplified “grade vs. length” readability metric is developed based on results from factor analysis of ten readability metrics applied to more than 500,000 radiology reports. To test the short-term effectiveness of a writing workshop, we measured the writing style improvement before and after the training. Statistically significant writing style improvement occurred as a result of the training. Although the degree of improvement varied for different measures, it is evident that targeted training could provide potential benefits to improve readability due to our statistically significant results. The simplified grade vs. length metric enables future clinical decision support systems to quantitatively guide physicians to improve writing styles through writing workshops.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Weiner SJ, Schwartz A, Sharma G, et al. Patient-centered decision making and health care outcomes: an observational study. Annals of internal medicine 2013;158(8):573–79
Woods SS, Schwartz E, Tuepker A, et al. Patient experiences with full electronic access to health records and clinical notes through the My HealtheVet Personal Health Record Pilot: qualitative study. Journal of medical Internet research 2013;15(3):e65
Landro L Radiologists push for medical reports patients can understand. Wall Street Journal 2014
Safeer RS, Keenan J. Health literacy: the gap between physicians and patients. Am Fam Physician 2005;72(3):463–68
Abujudeh H, Pyatt RS, Bruno MA, et al. Radpeer peer review: relevance, use, concerns, challenges, and direction forward. Journal of the American College of Radiology 2014;11(9):899–904
Kirsch IS Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey: ERIC, 1993
Svider PF, Agarwal N, Choudhry OJ, et al. Readability assessment of online patient education materials from academic otolaryngology–head and neck surgery departments. American journal of otolaryngology 2013;34(1):31–35
Eloy JA, Li S, Kasabwala K, et al. Readability assessment of patient education materials on major otolaryngology association websites. Otolaryngology--Head and Neck Surgery 2012;147(5):848–54
Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Education & Behavior 2006;33(3):352–73
Friedman DB, Kim S-H, Tanner A, Bergeron CD, Foster C, General K. How are we communicating about clinical trials?: an assessment of the content and readability of recruitment resources. Contemporary clinical trials 2014;38(2):275–83
Valentini M, Daniela D, Alonzo MCP, Lucisano G, Nicolucci A: Application of a readability score in informed consent forms for clinical studies. Journal of Clinical Research & Bioethics 2013:2013
Okuhara T, Ishikawa H, Okada H, Kiuchi T. Readability, suitability and health content assessment of cancer screening announcements in municipal newspapers in Japan. Asian Pacific journal of cancer prevention: APJCP 2014;16(15):6719–27
Zeng-Treitler Q, Goryachev S, Kim H, Keselman A, Rosendale D. Making texts in electronic health records comprehensible to consumers: a prototype translator. AMIA Annu Symp Proc 2007:846–50
Leroy G, Helmreich S, Cowie JR, Miller T, Zheng W. Evaluating online health information: beyond readability formulas. AMIA Annu Symp Proc 2008:394–8
Savova GK, Masanz JJ, Ogren PV, et al. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assn 2010;17(5):507–13
Meystre SM, Savova GK, Kipper-Schuler KC, Hurdle JF. Extracting information from textual documents in the electronic health record: a review of recent research. Yearb Med Inform 2008:128–44
Osborne H.: Health Literacy from A to Z: Jones & Bartlett Publishers, 2012
Applying multiple methods to assess the readability of a large corpus of medical documents. 14th World Congress on Medical and Health Informatics, MEDINFO 2013; 2013
Bruno MA, Petscavage-Thomas JM, Mohr MJ, Bell SK, Brown SD. The “open letter”: radiologists’ reports in the era of patient web portals. Journal of the American College of Radiology 2014;11(9):863–67
Flanders AE, Lakhani P. Radiology reporting and communications: a look forward. Neuroimaging Clinics of North America 2012;22(3):477–96
Penniston A, Harley E: Attempts to verify written english. Proceedings of The Fourth International Conference on Computer Science and Software Engineering 121–28,2011
Comrey AL, Lee HB. A First Course in Factor Analysis: Psychology Press, 2013
Kline P An easy guide to factor analysis: Routledge, 2014.
Cliff N. The eigenvalues-greater-than-one rule and the reliability of components. Psychol Bull 1988;103(2):276–79
Hansberry DR, John A, John E, Agarwal N, Gonzales SF, Baker SR. A critical review of the readability of online patient education resources from RadiologyInfo. Org. American Journal of Roentgenology 2014;202(3):566–75
AHRQ: Health Literacy. Secondary Health Literacy 2013. http://nnlm.gov/outreach/consumer/hlthlit.html.
Leong EK, Ewing MT, Pitt LF. E-comprehension: evaluating B2B websites using readability formulae. Industrial Marketing Management 2002;31(2):125–31
Allen ED, Bernhardt EB, Berry MT, Demel M. Comprehension and text genre: an analysis of secondary school foreign language readers. The Modern Language Journal 1988;72(2):163–72
Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults. Journal of general internal medicine 1995;10(10):537–41
Klare GR. The role of word frequency in readability. Elementary English 1968;45(1):12–22
Marks CB, Doctorow MJ, Wittrock MC. Word frequency and reading comprehensiony1. The Journal of Educational Research 1974;67(6):259–62
We would like to thank Richard Hoyt and Megan Reynolds from Nationwide Children’s Hospital’s Research Information Solutions and Innovation (RISI) center for their assistance with retrieving data from Epic. We also thank Tran Bourgeois for using REDCap (Research Electronic Data Capture) to set up user experiments, Katherine Strohm for project management, Dr. Jeffrey Hoffman for discussion of EPIC integration, and Dr. Michael Bruno from the Milton S. Hershey Medical Center at the Penn State College of Medicine for inspiring the study.
This study was supported by grant UL1TR001070 from the Center for Clinical and Translational Science at the Ohio State University to Dr. Brent Adler and MDSR Roessler Scholarship from the Ohio State University medical center to Claire Durkin.
The author’s declare that they have no competing interests.
About this article
Cite this article
Chen, W., Durkin, C., Huang, Y. et al. Simplified Readability Metric Drives Improvement of Radiology Reports: an Experiment on Ultrasound Reports at a Pediatric Hospital. J Digit Imaging 30, 710–717 (2017). https://doi.org/10.1007/s10278-017-9972-7
- Readability metrics
- Radiology reports
- Ultrasound reports
- Writing styles
- Factor analysis