Abstract
Study design
Cross-sectional analysis of patient educational materials from top pediatric orthopedic hospital websites.
Objective
To assess the readability of online educational materials of top pediatric orthopedic hospital websites for pediatric spinal deformity.
Summary of background data
The internet has become an increasingly popular source of health information for patients and their families. Healthcare experts recommend that the readability of online education materials be at or below a 6th-grade reading level. However, previous studies have demonstrated that the readability of online education materials on various orthopedic topics is too advanced for the average patient. To date, the readability of online education materials for pediatric spinal deformity has not been analyzed.
Methods
Online patient education materials from the top 25 pediatric orthopedic institutions, as ranked by the U.S. News and World Report hospitals for pediatric orthopedics, were accessed utilizing the following readability assessments: Flesch–Kincaid (FK), Flesch Reading Ease, Gunning Fog Index, Coleman–Liau Index, Simple Measure of the Gobbledygook Index (SMOG), Automated Readability Index, FORCAST, and the New Dale and Chall Readability. Correlations between academic institutional ranking, geographic location, and the use of concomitant multi-media modalities with FK scores were evaluated using a Spearman regression.
Results
Only 48% (12 of 25) of top pediatric orthopedic hospitals provided online information regarding pediatric spinal deformity at or below a 6th-grade reading level. The mean FK score was 9.0 ± 2.7, Flesch Reading Ease 50.8 ± 15.6, Gunning Fog Score 10.6 ± 3.1, Coleman–Liau Index 11.6 ± 2.6, SMOG index 11.7 ± 2.0, Automated Readability Index 8.6 ± 2.8, and Dale–Chall Readability Score 6.4 ± 1.4. There was no significant correlation between institutional ranking, geographic location, or use of multimedia with FK scores.
Conclusion
Online educational material for pediatric spinal deformity from top pediatric orthopedic institutional websites are associated with poor readability.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
The internet has become a widely used resource for patients seeking orthopedic information. Over 8 million Americans use the internet each day to learn about their health conditions and treatment options [1,2,3,4,5,6]. Parents, in particular, rate online health information as one of their most valuable tools, with an estimated 98% having used the internet to search for information regarding the health of their child [7]. However, the use of medical terminology and complex writing styles may limit comprehension of online educational materials amongst patients and parents of different educational backgrounds. Recent recommendations from the National Institutes of Health and the National Academy of Medicine recommend that patient educational materials should be written at the sixth-grade reading level or below [1, 2, 8,9,10,11,12].
Readability is a numerical value that is determined by systematic formulas, reflecting the grade level of reading necessary to understand the information [13, 14]. The Flesch-Kincaid reading level (FK) is a popular readability score that was created by the U.S. military and has been validated in previous studies [15, 16]. Other reading scores that emphasize different metrics compared to FK include the Coleman–Liau Index, New Dale–Chall Readability Formula, FORCAST Readability Formula, Gunning Fog Index, Simple Measure of Gobbledygook (SMOG), and Automated Readability Index. The formulas for these scores are found in Table 1. Investigators have studied the readability of patient education materials across various orthopedic subspecialties including adult reconstruction [13], foot and ankle [14], shoulder and elbow [11, 12], spine [3], hand [4], arthroscopy [17], sports medicine [10, 18], pediatric orthopedics, and the American Academy of Orthopedics website itself [1]. However, to our knowledge, no study has evaluated the readability of online patient education materials regarding pediatric spinal deformity.
The purpose of our study was to assess the readability of patient education materials related to pediatric spinal deformity available from leading pediatric orthopedic centers. We hypothesized that, on average, pediatric spinal deformity-related patient education materials from the top children’s hospitals for orthopedic surgery would be written at greater than the sixth-grade reading level.
Materials and methods
In December 2021, we searched for spine-related patient education materials from the leading pediatric orthopedic institutions based on US News & World Reports rankings for pediatric orthopedic surgery [19]. We searched each institution’s website for patient information and assessed all web pages pertinent to spinal deformity. In centers that had a specific spine section of patient education, articles with keywords including “scoliosis”, “kyphosis”, “lordosis”, and “deformity” were included. On those websites that did not have a specific spine section, all articles were screened for their relevance and spine pathology by one of the senior authors (CM). The patient education resources were then converted into a text-only format to exclude figures, disclaimers, acknowledgements, citations, references and hyperlinks. Reformatted patient education files were then analyzed using ReadablePro 20201 (Readable, Added Bytes Ltd.; Horsham, UK).
Statistical analysis
Using this software, the following readability scores were calculated: Flesch–Kincaid (FK), Flesch Reading Ease, Gunning Fog Index, Coleman–Liau Index, Simple Measure of the Gobbledygook Index (SMOG), Automated Readability Index, FORCAST, and the New Dale and Chall Readability. Equations used to calculate these scores are listed in Table 1. All of the aforementioned scores, with the exception of the Flesch Reading Ease, provide a score which correlates with the grade reading level associated with the article (e.g. score of 7 equates to 7th grade reading level). A linear regression analysis was employed to generate variance inflation factors, with values ≥ 10 indicating collinearity between various readability scores [20].
Continuous variables were presented as means and standard deviations. Correlations between institutional ranking and FK scores were assessed using a spearman regression. Additional factors including geographic location (urban versus rural), private versus public institution, and use of concomitant multimedia modalities (pictures or videos present on institutions web site versus no media) that may impact institutional readability scores (as determined by FK) were analyzed with independent t test and Mann–Whitney tests for parametric and non-parametric continuous variables, respectively. All tests were 2-sided. Analyses were performed with RStudio 2021.09.1 (RStudio, Boston, Mass., USA).
Results
In total, all 25 of the top 25 pediatric orthopedic institutions listed on the U.S. News and World Report’s website contained online resources for pediatric spinal deformity. We included 57 web pages in our final analysis. Readability scores were calculated for all web pages that were found in our search. A wide spectrum of FK scores was observed, ranging from 5 to 15.2. Notably, only 11 of the 57 web pages assessed included patient information at or below a 6th-grade reading level. These web pages were contributed by only 12 of the top 25 (48%) institutions for pediatric orthopedics as listed by U.S. News and World Report. Overall, the mean composite scores were FK score was 9.0 ± 2.7, Flesch Reading Ease 50.8 ± 15.6, Gunning Fog Score 10.6 ± 3.1, Coleman–Liau Index 11.6 ± 2.6, SMOG Index 11.7 ± 2.0, Automated Readability Index 8.6 ± 2.8, FORCAST 11.1 ± 1.1, Dale–Chall Readability Index 6.4 ± 1.4. Average readability scores for each institution can be found in Table 2, sorted by U.S. News and World Report ranking.
When assessing multicollinearity, it was determined that all demonstrated a high rate of collinearity with FK scores (variance inflation factor for each score: Flesch Reading Ease = 47.5, Gunning Fog Index = 23.5, Coleman–Liau = 11.8, SMOG Index 66.3, Automated Readability Index 20.8, FORCAST 22.7, and Dale–Chall Readability Score 13.0). As a result, FK scores were used to analyze the relationship between readability and other institutional factors including ranking, geographic location, and presence of figures or videos.
The average FK score of the 12 institutions that included some form of illustration, figure or video is 9.069 and the average of those that did not was 9.65. Regionally, the average FK score of the 5 institutions from the Northeast is 9.07, the 6 from the South is 10.16, the 8 from the West is 9.50 and the 6 from the Midwest is 8.58. There was no correlation between ranking and FK Scores (rho = − 0.08, p = 0.565). There was no significant relationship found for institutional online resource readability based on FK scores and geographic location (rho 0.15, p = 0.261), use of pictures (rho = − 0.14, p = 0.2938) or videos (rho = − 0.07, p = 0.6212).
24 webpages of the 57 (42%) included in the study contained pictures or illustrations and 3 of 57 included videos (5%). 12 of the top 25 U.S. News and World Report ranked institutions featured pictures or illustrations while only 3 included videos in their patient education materials pertaining to pediatric spinal deformity. Every institution has at least one webpage discussing scoliosis. 15 of the 25 intuitions (60%) has some discussion on kyphosis and only 5 of the 25 (20%) has information regarding lordosis.
Discussion
The internet has become a vital part of daily life, and its role in disseminating health care information to patients is expanding at a rapid pace [13]. With this expansion, it is important to ensure patient education is not only accurate but also at an appropriate reading level. The U.S. adult population is composed of 5% that are illiterate in English, 14% who have below-basic literacy skills, and 29% with basic literacy skills [21, 22]. Further, a similar distribution was found concerning health literacy, with only 22% of patients having basic health literacy [23]. Health literacy has been an ongoing discussion in the orthopedic community, with previous analyses showing that many materials including those from the AAOS itself were written at reading levels too complex for most patients to understand [1]. The goal of the current study was to assess the readability of parent-facing educational materials on pediatric spinal deformity from top pediatric orthopedic institutions.
The number of institutions featuring educational materials regarding pediatric spinal conditions at a suitable reading level was 12 out of 25 (48%). Furthermore, the average FK reading level of the web pages included in our study was 9.0. Of the 57 web pages included, only 11 (19%) were at or below a 6th-grade reading level. We found a high degree of collinearity in all readability scores, indicating there would be little difference when assessing different scores compared to the FK scores that were used. The U.S. News & World Report ranking of these institutions did not have any association with the readability of patient educational materials based on our statistical analysis (rho = − 0.08, p = 0.565). These findings are demonstrated in Fig. 1. Geographic location and use of concomitant pictures or videos were also not associated with a statistically significant change in readability levels. Additionally, a severe lack in the inclusion of multimedia in patient educational materials was noted with only 24 of 57 web pages (42%) including figures or illustrations and 3 of 57 (5%) including videos. This represents only 12 of 25 and 3 of 25 institutions that included figures and videos respectively.
Our findings were in concordance with previous studies that assessed the readability of orthopedic patient educational materials [2, 9, 11, 13, 17]. Many of these studies concluded that the majority of orthopedic patient educational materials are at or beyond a 6th-grade reading level and have the propensity to confuse both patients and their families. In addition, there were few others who were able to establish an association between institution rank and readability was the study by Parsa et al. which assessed the readability of hip-preservation-related educational materials. This study concluded a weakly negative association between institution rank and readability scores. Institutional-related factors were not found to be statistically significant.
The lack of institutions including multimedia may also be a detriment to patient understanding and comprehension. Videos have been shown in the past to be high quality in content and enhance patient understanding [18]. In the setting of a large differential between readability scores on these web pages and reading level, a video or picture represents a simple way to supplement understanding of pediatric spinal conditions and may be helpful to increase the readability of orthopedic information.
Some other ways to improve the readability of patient resources include choosing words with a single definition, using familiar words, avoiding unnecessary abbreviations or acronyms, and shorter words with decreased complexity [10, 24]. The use of medical jargon and the description of specific anatomical or procedural details can decrease readability. We found the articles with the lowest readability scores overused medical jargon and long complex sentences. Some terms that may be used in place of medical jargon are shown in Table 3. It has been well studied that patient education significantly improves adherence, follow-up and compliance [25]. The internet has become an invaluable educational resource for patients and their families. Top institutions, which are often associated with large academic centers, could improve tremendously by investing in resources and expert educators to help develop patient educational material. Future studies can investigate how the formation of inter-departmental committee boards and communication reviewers at top institutions can elevate the quality of healthcare.
Limitations
There are several factors that limited our study. While the readability of patient educational materials does not entirely indicate the quality of patient educational materials, it may increase the patient's ability to comprehend medical information. In addition, while there was a high degree of concordance between the readability metrics that were used, they were not entirely in agreement. There is no clear winner in terms of the best readability metric to use and as a result, we used FK scores as that is what other papers have used in the past to assess the readability of patient educational materials.
Conclusion
This study demonstrates the readability of educational materials meant for pediatric patients or parents provided by the nation’s top pediatric orthopedic institutions has poor readability. It is concerning that so few institutions have resources at an 6th-grade reading level or below. We recommend avoiding the use of medical jargon and removing detailed explanations regarding procedures from these web pages as well as the inclusion of supplemental pictures or videos to enhance readability and patient understanding. This will ensure a higher degree of health literacy and help to tailor patient expectations and ultimately improve outcomes.
References
Badarudeen S, Sabharwal S (2008) Readability of patient education materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America web sites. J Bone Jt Surg Am 90(1):199–204. https://doi.org/10.2106/JBJS.G.00347
Ryu JH, Yi PH (2016) Readability of spine-related patient education materials from leading orthopedic academic centers. Spine 41(9):E561-565. https://doi.org/10.1097/BRS.0000000000001321
Vives M, Young L, Sabharwal S (2009) Readability of spine-related patient education materials from subspecialty organization and spine practitioner websites. Spine 34(25):2826–2831. https://doi.org/10.1097/BRS.0b013e3181b4bb0c
Wang SW, Capo JT, Orillaza N (2009) Readability and comprehensibility of patient education material in hand-related web sites. J Hand Surg 34(7):1308–1315. https://doi.org/10.1016/j.jhsa.2009.04.008
Cline RJW, Haynes KM (2001) Consumer health information seeking on the Internet: the state of the art. Health Educ Res 16(6):671–692. https://doi.org/10.1093/her/16.6.671
Sulmasy DP, Lehmann LS, Levine DM et al (1994) Patients’ perceptions of the quality of informed consent for common medical procedures. J Clin Ethics 5(3):189–194
Peterlein CD, Bosch M, Timmesfeld N et al (2019) Parental internet search in the field of pediatric orthopedics. Eur J Pediatr 178(6):929–935. https://doi.org/10.1007/s00431-019-03369-w
Eltorai AEM, Sharma P, Wang J et al (2015) Most American Academy of Orthopaedic Surgeons’ online patient education material exceeds average patient reading level. Clin Orthop 473(4):1181–1186. https://doi.org/10.1007/s11999-014-4071-2
Feghhi DP, Komlos D, Agarwal N et al (2014) Quality of online pediatric orthopaedic education materials. J Bone Jt Surg Am Vol 96(23):e194(1). https://doi.org/10.2106/JBJS.N.00043
Kakazu R, Schumaier A, Minoughan C et al (2018) Poor readability of aossm patient education resources and opportunities for improvement. Orthop J Sports Med 6(11):2325967118805386. https://doi.org/10.1177/2325967118805386
Schumaier AP, Kakazu R, Minoughan CE et al (2018) Readability assessment of American Shoulder and Elbow Surgeons patient brochures with suggestions for improvement. JSES Open Access 2(2):150–154. https://doi.org/10.1016/j.jses.2018.02.003
Stelzer JW, Wellington IJ, Trudeau MT et al (2021) Readability assessment of patient educational materials for shoulder arthroplasty from top academic orthopaedic institutions. JSES Int. https://doi.org/10.1016/j.jseint.2021.08.004 (Published online September 28, 2021)
Polishchuk DL, Hashem J, Sabharwal S (2012) Readability of online patient education materials on adult reconstruction Web sites. J Arthroplasty 27(5):716–719. https://doi.org/10.1016/j.arth.2011.08.020
Bluman EM, Foley RP, Chiodo CP (2009) Readability of the patient education section of the AOFAS website. Foot Ankle Int 30(4):287–291. https://doi.org/10.3113/FAI.2009.0287
Ley P, Florio T (1996) The use of readability formulas in health care. Psychol Health Med. https://doi.org/10.1080/13548509608400003
Friedman DB, Hoffman-Goetz L (2006) A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav Off Publ Soc Public Health Educ 33(3):352–373. https://doi.org/10.1177/1090198105277329
Yi PH, Ganta A, Hussein KI et al (2013) Readability of arthroscopy-related patient education materials from the American Academy of Orthopaedic Surgeons and Arthroscopy Association of North America Web sites. Arthrosc J Arthrosc Relat Surg Off Publ Arthrosc Assoc N Am Int Arthrosc Assoc 29(6):1108–1112. https://doi.org/10.1016/j.arthro.2013.03.003
Parsa A, Nazal M, Molenaars RJ et al (2020) Evaluation of hip preservation–related patient education materials from leading orthopaedic academic centers in the United States and description of a novel video assessment tool. JAAOS Glob Res Rev. 4(6):e20.00064. https://doi.org/10.5435/JAAOSGlobal-D-20-00064
Best Children’s Hospitals for Orthopedics. https://health.usnews.com/best-hospitals/pediatric-rankings/orthopedics. Accessed 30 Nov 2021
Marcoulides KM, Raykov T (2019) Evaluation of variance inflation factors in regression models using latent variable modeling methods. Educ Psychol Meas 79(5):874–882. https://doi.org/10.1177/0013164418817803
National Assessment of Adult Literacy (NAAL). https://nces.ed.gov/naal/index.asp. Accessed 29 Dec 2021
Kirrsch IS, Jungeblut A, Jenkins L et al (1993) Adult literacy in America. US Department of Education, National Center for Education Statistics, Washington DC, p 201
Literacy I of M (US) C on H, Nielsen-Bohlman L, Panzer AM et al (2004) What is health literacy? National Academies Press (US). https://www.ncbi.nlm.nih.gov/books/NBK216035/24. Accessed 28 Dec 2021
CDC (2009) Simply put. https://www.cdc.gov/healthliteracy/pdf/simply_put.pdf. Accessed 28 Dec 2021
Taibanguay N et al (2019) Effect of patient education on medication adherence of patients with rheumatoid arthritis: a randomized controlled trial. Patient Preference Adherence 13:119–129. https://doi.org/10.2147/PPA.S192008
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Author information
Authors and Affiliations
Contributions
CM: Made substantial contributions to the conception and design of the work, analysis and interpretation of the data. Drafted the work and revised it critically for important content. Approved the version to be published. Agrees to be accountable for all aspects of the work in ensuring that the questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CD: Made substantial contributions to the conception and design of the work, analysis and interpretation of the data. Drafted the work and revised it critically for important content. Approved the version to be published. Agrees to be accountable for all aspects of the work in ensuring that the questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. GA: Made substantial contributions to the conception and design of the work, acquisition, analysis and interpretation of the data. Drafted the work and revised it critically for important content. Approved the version to be published. Agrees to be accountable for all aspects of the work in ensuring that the questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. SS: Made substantial contributions to the conception and design of the work, analysis and interpretation of the data. Drafted the work and revised it critically for important content. Approved the version to be published. Agrees to be accountable for all aspects of the work in ensuring that the questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. DK: Made substantial contributions to the conception and design of the work, analysis and interpretation of the data. Drafted the work and revised it critically for important content. Approved the version to be published. Agrees to be accountable for all aspects of the work in ensuring that the questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. GG: Made substantial contributions to the conception and design of the work, analysis and interpretation of the data. Drafted the work and revised it critically for important content. Approved the version to be published. Agrees to be accountable for all aspects of the work in ensuring that the questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. PY: Made substantial contributions to the conception and design of the work, analysis and interpretation of the data. Drafted the work and revised it critically for important content. Approved the version to be published. Agrees to be accountable for all aspects of the work in ensuring that the questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Corresponding author
Ethics declarations
Conflict of interest
The author(s) declare no potential conflicts of interest with respect to the research, ethics approval, authorship, and/or publication of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Michel, C., Dijanic, C., Abdelmalek, G. et al. Readability assessment of patient educational materials for pediatric spinal deformity from top academic orthopedic institutions. Spine Deform 10, 1315–1321 (2022). https://doi.org/10.1007/s43390-022-00545-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s43390-022-00545-1