Skip to main content

Advertisement

SpringerLink
Log in

Readability Analysis of Online Breast Cancer Surgery Patient Education Materials from National Cancer Institute-Designated Cancer Centers Compared with Top Internet Search Results

  • Global Health Services Research
  • Published:
Annals of Surgical Oncology Aims and scope Submit manuscript

Abstract

Background

The National Institutes of Health (NIH) recommends patient education materials reflect the average reading grade level of the US population. Due to the importance of shared decision-making in breast cancer surgery, this study evaluates the reading level of patient education materials from National Cancer Institute-designated cancer centers (NCI-DCC) compared with top Internet search results.

Methods

Online materials from NCI-DCC and top Internet search results on breast cancer, staging, surgical options, and pre- and postoperative expectations were analyzed using three validated readability algorithms: Simplified Measure of Gobbledygook Readability Formula, Coleman–Liau index, and Flesch–Kincaid grade level. Mean readability was compared across source groups and information subcategories using an unpaired t-test with statistical significance set at p < 0.05. Mean readability was compared using a one-way analysis of variance.

Results

Mean readability scores from NCI-DCC and Internet groups ranged from a 9th–12th grade level, significantly above the NIH recommended reading level of 6th–7th grade. There was no significant difference between reading levels from the two sources. The discrepancy between actual and recommended reading level was most pronounced for “surgical options” at a 10th–12th grade level from both sources.

Conclusions

Patient education materials on breast cancer from both NCI-DCC and top Internet search results were written several reading grade levels higher than the NIH recommendation. Materials should be revised to enhance patient comprehension of breast cancer surgical treatment and guide patients in this important decision-making process to ultimately improve health outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Rowlands G. Health literacy. Hum Vaccin Immunother. 2014;10(7):2130–5.

    PubMed  PubMed Central  Google Scholar 

  2. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155(2):97–107.

    PubMed  Google Scholar 

  3. De Oliveira GS, McCarthy RJ, Wolf MS, Holl J. The impact of health literacy in the care of surgical patients: a qualitative systematic review. BMC Surg. 2015;15(1):86.

    PubMed  PubMed Central  Google Scholar 

  4. Rikard RV, Thompson MS, McKinney J, Beauchamp A. Examining health literacy disparities in the United States: a third look at the National Assessment of Adult Literacy (NAAL). BMC Public Health. 2016;16(1):975.

    CAS  PubMed  PubMed Central  Google Scholar 

  5. Williams DR, Collins C. US Socioeconomic and racial differences in health: patterns and explanations. Ann Rev Sociol. 1995;21(1):349–86.

    Google Scholar 

  6. Jeppesen KM, Coyle JD, Miser WF. Screening questions to predict limited health literacy: a cross-sectional study of patients with diabetes mellitus. Ann Fam Med. 2009;7(1):24–31.

    PubMed  PubMed Central  Google Scholar 

  7. Gazmararian JA, Baker DW, Williams MV, Parker RM, Scott TL, Green C, et al. Health literacy among Medicare enrollees in a managed care organization. JAMA. 1999;281(6):545–51.

    CAS  PubMed  Google Scholar 

  8. Ayotte BJ, Allaire JC, Bosworth H. The associations of patient demographic characteristics and health information recall: the mediating role of health literacy. Aging Neuropsychol Cogn. 2009;16(4):419–32.

    Google Scholar 

  9. Sudore RL, Yaffee K, Satterfield S, Harris TB, Mehta KM, Simonsick EM, et al. Limited literacy and mortality in the elderly: the health, aging, and body composition study. J Gen Intern Med. 2006;21(8):806–12.

    PubMed  PubMed Central  Google Scholar 

  10. Siegel RL, Miller KD, Wagle NS, Jemal A. Cancer statistics 2023 CA. Cancer J Clin. 2023;73(1):17–48.

    Google Scholar 

  11. Guy GP Jr, Richardson LC. Visit duration for outpatient physician office visits among patients with cancer. J Oncol Pract. 2012;8(3S):2–8.

    Google Scholar 

  12. Cohen RA. Use of the Internet for health information: United States. Natl Center Health Stat. 2011;66:1–8.

    Google Scholar 

  13. Hutchinson N, Baird GL, Garg M. Examining the reading level of internet medical information for common internal medicine diagnoses. Am J Med. 2016;129(6):637–9.

    PubMed  Google Scholar 

  14. Powell LE, Andersen ES, Pozez AL. Assessing readability of patient education materials on breast reconstruction by major US academic hospitals as compared with nonacademic sites. Ann Plast Surg. 2021;86(6):610–4.

    CAS  PubMed  Google Scholar 

  15. Mc Laughlin GH. SMOG Grading-a new readability formula. J Reading. 1969;12(8):639–46.

    Google Scholar 

  16. Wang LW, Miller MJ, Schmitt MR, Wen FK. Assessing readability formula differences with written health information materials: application, results, and recommendations. Res Social Adm Pharm. 2013;9(5):503–16.

    PubMed  Google Scholar 

  17. Betschart P, Abt D, Schmid HP, Viktorin P, Langenauer J, Zumstein V. Readability assessment of commonly used urological questionnaires. Investig Clin Urol. 2018;59(5):297–304.

    PubMed  PubMed Central  Google Scholar 

  18. Coleman M, Liau TL. A computer readability formula designed for machine scoring. J Appl Psychol. 1975;60:283–4.

    Google Scholar 

  19. Jindal P, MacDermid J. Assessing reading levels of health information: uses and limitations of flesch formula. Educ Health. 2017;30(1):84–8.

    Google Scholar 

  20. Swanson CE, Fox H. Validity of readability formulas. J Appl Psychol. 1953;37:114–8.

    Google Scholar 

  21. DuBay WH. The principles of readability. Costa Mesa: Impact Information, 2004.

  22. Grabeel KL, Russomanno J, Oelschlegel S, Tester E, Heidel RE. Computerized versus hand-scored health literacy tools: a comparison of Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid in printed patient education materials. J Med Libr Assoc. 2018;106(1):38–45.

    PubMed  PubMed Central  Google Scholar 

  23. Rudd RE, Anderson JE. The health literacy environment of hospitals and health centers. partners for action: making your healthcare facility literacy-friendly. Health and Adult Literacy and Learning Initiative. Harvard School of Public Health. 2006; https://www.hsph.harvard.edu/healthliteracy/practice/environmental-barriers/. Accessed 4/15/2023.

  24. Brangan S. Development of SMOG-Cro readability formula for healthcare communication and patient education. Coll Antropol. 2015;39:11–20.

    PubMed  Google Scholar 

  25. Kue J, Klemanski DL, Browning KK. Evaluating readability scores of treatment summaries and cancer survivorship care plans. JCO Oncol Pract. 2021;17(10):615–21.

    PubMed  Google Scholar 

  26. Sand-Jecklin K. The impact of medical terminology on readability of patient education materials. J Community Health Nurs. 2007;24(2):119–29.

    PubMed  Google Scholar 

  27. Agarwal N, Chaudhari A, Hansberry DR, Tomei KL, Prestigiacomo CJ. A comparative analysis of neurosurgical online education materials to assess patient comprehension. J Clin Neurosci. 2013;20(10):1357–61.

    PubMed  Google Scholar 

  28. Ayyaswami V, Padmanabhan D, Patel M, Prabhu AV, Hansberry DR, Agarwal N, et al. A readability analysis of online cardiovascular disease-related health education materials. HLRP: Health Lit Res Pract. 2019;3(2):e75–80.

    PubMed Central  Google Scholar 

  29. Eltorai AEM, Sharma P, Wang J, Daniels AH. Most american academy of orthopaedic surgeons’ online patient education material exceeds average patient reading level. Clin Orthop Relat Res. 2015;473(4):1181–6.

    PubMed  Google Scholar 

  30. Kaye DR, Richardson CR, Ye Z, Herrel LA, Ellimoottil C, Miller DC. Association between patient satisfaction and short-term outcomes after major cancer surgery. Ann Surg Oncol. 2017;24(12):3486–93. https://doi.org/10.1245/s10434-017-6049-2

    Article  PubMed  PubMed Central  Google Scholar 

  31. Chen Q, Beal EW, Okunrintemi V, Cerier E, Paredes A, Sun S, et al. The association between patient satisfaction and patient-reported health outcomes. J Patient Exp. 2019;6(3):201–9.

    PubMed  Google Scholar 

  32. Rademakers J, Delnoji D, Nijman J, de Boer D. Educational inequalities in patient-centred care: patients’ preferences and experiences. BMC Health Serv Res. 2012;12:261.

    PubMed  PubMed Central  Google Scholar 

  33. Humphrey KE, Sundberg M, Milliren CE, Graham DA, Landrigan CP. Frequency and nature of communication and handoff failures in medical malpractice claims. J Patient Saf. 2022;18(2):130–7.

    PubMed  Google Scholar 

  34. Wong C, Harrison C, Britt H, Henderson J. Patient use of the internet for health information. Aust Fam Phys. 2014;43(12):875–7.

    Google Scholar 

  35. Diaz JA, Griffith RA, Ng JJ, Reinert SE, Friedmann PD, Moulton AW. Patients’ use of the Internet for medical information. J Gen Intern Med. 2002;17(3):180–5.

    PubMed  PubMed Central  Google Scholar 

  36. Bujnowska-Fedak MM, Waligóra J, Mastalerz-Migas A. The internet as a source of health information and services. Adv Exp Med Biol. 2019;1211:1–16.

    CAS  PubMed  Google Scholar 

  37. van Eenbergen MC, Vromans RD, Boll D, Kil PJM, Vos CM, Krahmer EJ, Mols F, van de Poll-Franse LV. Changes in internet use and wishes of cancer survivors: a comparison between 2005 and 2017. Cancer. 2020;126(2):408–15.

    PubMed  Google Scholar 

  38. Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, et al. Health information on the internet: accessibility, quality, and readability in English and Spanish. JAMA. 2001;285(20):2612–21.

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Early J, Hernandez A. Digital Disenfranchisement and COVID-19: Broadband internet access as a social determinant of health. Health Promot Pract. 2021;22(5):605–10.

    PubMed  Google Scholar 

  40. Miller LM, Bell RA. Online health information seeking: the influence of age, information trustworthiness, and search challenges. J Aging Health. 2012;24(3):525–34.

    PubMed  Google Scholar 

  41. Houts PS, Doak CC, Doak LG, Loscalzo MJ. The role of pictures in improving health communication: a review of research on attention, comprehension, recall, and adherence. Patient Educ Couns. 2006;61(2):173–90.

    PubMed  Google Scholar 

  42. Sudore RL, Schillinger D. Interventions to improve care for patients with limited health literacy. J Clin Outcomes Manag. 2009;16(1):20–9.

    PubMed  PubMed Central  Google Scholar 

  43. Delp C, Jones J. Communicating information to patients: the use of cartoon illustrations to improve comprehension of instructions. Acad Emerg Med. 1996;3(3):264–70.

    CAS  PubMed  Google Scholar 

  44. DeWalt DA, Broucksou KA, Hawk V, Brach C, Hink A, Rudd R, et al. Developing and testing the health literacy universal precautions toolkit. Nurs Outlook. 2011;59(2):85–94.

    PubMed  PubMed Central  Google Scholar 

  45. Norris EM. The constructive use of images in medical teaching: a literature review. JRSM Short Rep. 2012;3(5):33.

    PubMed  PubMed Central  Google Scholar 

  46. Dowse R, Ramela T, Barford KL, Browne S. Developing visual images for communicating information aboutantiretroviral side effects to a low-literate population. Afr J AIDS Res. 2010;9(3):213–24.

    PubMed  Google Scholar 

  47. Hung YL, Stones C. Visual design in healthcare for low-literate users - A case study of healthcare leaflets for new immigrants in Taiwan. Interacción. 2014.

  48. Smith RM, Anderson ES, Powell LE, Schuth OA, Mountziaris PM, Feldman MJ. It’s not all white: implicit racial bias in imagery used in plastic surgery resident education. J Surg Educ. 2022;79(4):943–9.

    PubMed  Google Scholar 

  49. Hirko KA, Rocque G, Reasor E, Taye A, Daly A, Cutress RI, et al. The impact of race and ethnicity in breast cancer-disparities and implications for precision oncology. BMC Med. 2022;20(1):72.

    PubMed  PubMed Central  Google Scholar 

  50. Eltorai AE, Ghanian S, Adams CA Jr, Born CT, Daniels AH. Readability of patient education materials on the American Orthopaedic Society for Sports Medicine website. Phys Sportsmed. 2014;42(4):125–30.

    PubMed  Google Scholar 

  51. Stocco F, Kwan JY, Sood M, Scott DJA, Bailey MA, Coughlin PA. Assessment of available online website and youtube resources for patients with abdominal aortic aneurysms. Ann Vasc Surg. 2023;S0890–5096(23):00252–62.

    Google Scholar 

  52. Eltorai AE, Ghanian S, Adams CA Jr, Born CT, Daniels AH. Readability of patient education materials on the american association for surgery of trauma website. Arch Trauma Res. 2014;3(2):e18161.

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

The authors have no acknowledgements.

Author information

Authors and Affiliations

  1. University of Minnesota School of Medicine, Minneapolis, MN, USA

    Anna Rauzi BS

  2. Division of Plastic Surgery, Department of Surgery, University of Minnesota, Minneapolis, MN, USA

    Lauren E. Powell MD

  3. Division of Surgical Oncology, Department of Surgery, University of Minnesota, Minneapolis, MN, USA

    McKenzie White MD, Saranya Prathibha MD & Jane Yuet Ching Hui MD, MS

Authors
  1. Anna Rauzi BS
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lauren E. Powell MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. McKenzie White MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saranya Prathibha MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jane Yuet Ching Hui MD, MS
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jane Yuet Ching Hui MD, MS.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Presentation: This study was presented at the Society of Surgical Oncology 2023 International Conference on Surgical Cancer Care in Boston, MA (3/23/23).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rauzi, A., Powell, L.E., White, M. et al. Readability Analysis of Online Breast Cancer Surgery Patient Education Materials from National Cancer Institute-Designated Cancer Centers Compared with Top Internet Search Results. Ann Surg Oncol 30, 8061–8066 (2023). https://doi.org/10.1245/s10434-023-14279-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1245/s10434-023-14279-5

Search

Navigation