Background

Patient education materials are an important supplement to verbal communication with eye-care providers [1]. Informational handouts given at the end of a visit and patient-oriented webpages are popular forms of patient education material but often convey complex information at an advanced reading level [2]. High complexity and low readability make these materials difficult to comprehend for patients with a low level of health literacy, generally defined as the ability to read, understand, and act on health information [3]. Low health literacy is prevalent; the 2003 National Assessment of Adult Literacy found that 36 % of American adults have only basic or below basic levels of health literacy [4]. Furthermore, low health literacy is associated with billions of dollars in additional healthcare costs and poor health outcomes [5, 6].

Ophthalmology is no exception to the effect of low health literacy on health. For glaucoma patients, low health literacy is associated with worse vision-related quality of life [7], poor medication adherence [8], worsened visual field loss [9], and decreased understanding about glaucoma compared to patients with adequate health literacy levels [2, 9, 10]. Therefore, it is essential that ophthalmic patient education materials are written at accessible reading levels, both in print and online. The United States Department of Health and Human Services (USDHHS) recommends that patient education materials should be written at a sixth- to seventh-grade reading level (equivalent to years 7-8 in the United Kingdom (UK)) to make them accessible for patients with low health literacy.

The present study presents (1) a systematic review of the literature on readability of ophthalmic patient education materials and (2) evaluation and improvement of the readability and suitability of patient education materials at our own institution. Our academic ophthalmology practice previously developed patient education materials about glaucoma, which we sought to improve by applying a set of recommendations for writing easy-to-understand health material. We then utilized standardized scoring tools and patient interviews to evaluate how well our revisions improved our patient education materials.

Methods

A systematic search for relevant studies on readability of ophthalmic patient education materials was conducted on the PubMed/MEDLINE database. The non-date-restricted search of English-language articles included the following key words: “readability ophthalmology,” “ophthalmology patient education,” and “ophthal* patient education materials.” To expand the search, references of included articles were examined for additional studies.

In addition to reviewing the literature, we also examined the readability and suitability of our own institution’s patient education materials about glaucoma—both before and after revising them using published guidelines. First, we collected evidence-based guidelines for writing easy-to-read patient education materials by compiling recommendations from three national organizations: “Creating and Using Patient-Friendly Written Materials” by the American Medical Association Foundation [11], “Simply Put: A Guide for Creating Easy-to-Understand Materials” by the Centers for Disease Control and Prevention [12], and “How to Write Easy-to-Read Health Materials” by the National Institutes of Health [13]. We used these guidelines to revise our patient education materials about glaucoma. Our focus was not to change the materials’ educational content, but instead to improve their readability, structure, and presentation for patients with low health literacy.

After all documents were revised, the original and revised handouts were scored in random order by two glaucoma specialists (KWM, JAR). Neither evaluator had a role in making the revisions, and neither had seen the revised documents before the masked scoring process. Both evaluators scored all handouts independently using the Suitability Assessment of Materials (SAM) instrument.

The SAM instrument is a widely used rating tool that systematically assesses the suitability of health information materials for a given patient population [14]. The SAM scoring tool has been validated and used to evaluate patient education materials for a number of diseases, including congestive heart failure and chronic kidney disease [15, 16]. SAM rates materials on factors that affect readability and comprehension, which makes it an ideal scoring tool when evaluating the suitability of patient education materials for patients who are low in health literacy. The SAM criteria are each given 0, 1, or 2 points based on adequacy of the handout to address each criterion, with 0 indicating a “not suitable” rating, 1 indicating an “adequate” rating, and 2 indicating a “superior” rating. The total SAM score for each handout is calculated as the sum of earned points divided by the number of possible points. The final SAM score reported as a percentage, with 70-100 % indicating superior material, 40-69 % adequate material, and 0-39 % unsuitable material.

In addition to SAM score, the word count and readability were assessed for each of the original and revised handouts. Readability scores were calculated as Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease Score (FRES), which have been widely used in assessing patient education materials from various fields [17, 18], including ophthalmology [19]. FKGL indicates the academic grade level required to understand written material, determined by a formula that considers the number of words per sentence and the number of syllables per word [14]. For example, a FKGL score of 6.4 indicates that material is written at a sixth- to seventh-grade reading level (equivalent to years 7-8 in the UK). The FRES uses the same variables with an output between 0 and 100, with a higher score indicating that the material is more easily understandable.

Pearson correlation was calculated to determine inter-observer correlation between SAM scores assigned by the two evaluators. Paired t test was used for analysis of SAM scores, word count, FKGL, and FRES between original and revised patient education materials.

Finally, one-on-one interviews were conducted to receive input from glaucoma patients about the revised handouts. Our interview protocol was approved by the Duke University Institutional Review Board and adhered to the Declaration of Helsinki. Patients were recruited from our glaucoma service to review and to evaluate handouts on various topics with a study team member. Written informed consent was obtained from all participants. English-speaking adult patients age 18 years or older were eligible to participate if they had a diagnosis of glaucoma or glaucoma suspect, including ocular hypertension. Potential participants were excluded if they were unable to read or had vision worse than 20/70 in their better-seeing eye. All participants reviewed the “Top 10 for Glaucoma Patients” handout and selected a second or third topic of their choice from the handouts listed in Table 4. Subjects rated the documents with a validated eleven-question survey about the overall design quality of the handout, called the Consumer Information Rating Form (CIRF) [20], and they answered open-ended interview questions about the handouts (Table 7). Health literacy level was measured using the Rapid Estimate of Adult Literacy in Medicine – Short Form (REALM-SF) [21], and demographic information was obtained. Interview recordings were transcribed, and general themes were derived by iteratively coding responses and developing and applying an analytical framework, consistent with the framework method [22]. All three investigators coded themes independently and compared results.

Results

Overview

We first report the results of our literature review, which includes data from our study on evaluating and improving patient education materials at our institution. The results of our study are then presented in detail.

Systematic review of the literature

Our search yielded 456 results between the three keyword searches. Duplicates were removed, and only studies evaluating patient education material in the field of ophthalmology were considered. In total, 12 studies were identified in our literature search, and addition of our study results makes for a total of 13 studies included in our review (Table 1).

Table 1 Characteristics of the 13 included studies
Full size table

Among the 13 studies, eight evaluated patient education material online and five in print, including one that evaluated ocular medication inserts [23]. Between all included studies, 950 articles were evaluated in total, although this figure almost certainly includes duplicates, as the same webpages were likely included in multiple studies. The included studies evaluated material such as educational brochures from ophthalmologic organizations [2, 24], information leaflets from English ophthalmology departments [25], educational material developed at an academic eye center, and webpages available to patients on the Internet [19, 26–32].

Various measures of readability and quality were used to evaluate educational material in the included studies. The most commonly reported metric was the Flesch-Kincaid Grade Level (FKGL), which ranged from 6.4 (the revised handouts in our study) to 12.9 (glaucoma medication inserts) [23]. Notably, the second-lowest reading grade level, 8.3, also comes from revised material (American Academy of Ophthalmology brochures revised in 2008) [2]. Among all reported mean FKGL scores, the median mean across all studies is 11, representing an eleventh-grade reading level (equivalent to year 12 in the UK).

Ebrahimzadeh, et al. [24] first evaluated the readability of ophthalmic patient education materials in 1997 and found that only 32 % of brochures published by the American Academy of Ophthalmology (AAO) were written at or below an eighth-grade reading level. Muir and Lee [2] later demonstrated that 38 AAO brochures, revised in 2008, improved significantly in readability since 1997, but many still fell short of the recommended sixth- to seventh-grade reading level; the average FKGL was 8.3 (i.e., eighth grade), with a range of 5.1 to 11.4. Muir and Lee also evaluated eleven patient education materials from other non-profit organizations, none of which was written at the recommended reading level (mean 9.7, range 8.4-12.0) [2]. Of all studies in this review, Khurana, et al. [23] reveal that medication inserts fare worst in readability; nonglaucoma medication inserts averaged an FKGL of 11.1, and glaucoma medication inserts scored 12.9, a readability at the university education level.

Although many patients use the Internet to learn about eye diseases [1], readability of online ophthalmic patient information does not fare better than print. Among the largest web-based studies, Edmunds, et al. [19] reviewed ten webpages for 16 different eye conditions in 2013. Of the 160 total websites from commercial and non-profit organizations, the mean FKGL was 11.3, with a range of 8.5 to 15.1. Not a single webpage adhered to the USDHHS guideline of a sixth- to seventh-grade reading level. In 2015, Huang, et al. [29] expanded upon this work by evaluating the readability of 339 online patient education materials from seven ophthalmologic association websites. Not a single document was written at or below the recommended sixth-grade reading level, and the authors conclude with a call for revising online patient education materials. Smaller studies of online material focused on topics like flashes and floaters [26], thyroid-associated ophthalmopathy [27], common pediatric ophthalmology conditions [30], and retinopathy of prematurity [31], and all concluded that the information available scores poorly in readability and requires revision (Table 1).

The Duke experience: Improving suitability and readability of patient education materials

Twelve patient education handouts about glaucoma at our academic eye center were revised according to published guidelines (Table 2). Inter-observer correlation of SAM scores between the two evaluators was significant at a Pearson correlation of 0.73 (p < 0.01, n = 24). The revised handouts represent a significant improvement in scoring criteria and total SAM score compared to the original handouts (Table 3). Specifically, the mean (± standard deviation) SAM score improved from 60 ± 7 % (adequate) for the original versions (n = 12) to 88 ± 4 % (superior) for the revised handouts (n = 12) (p < 0.001). Criteria from all five graded areas improved upon revision (content, literacy demand, layout and type, learning stimulation and motivation, and cultural appropriateness) (Table 3). The SAM score for all 12 glaucoma topics improved upon revision (Table 4).

Table 2 Guidelines used for revising patient education materials
Full size table
Table 3 SAM criteria and SAM scores for original versus revised handouts
Full size table
Table 4 SAM scores, word count, and readability level by handout topic
Full size table

In addition to suitability score, readability level also improved after revision of the original handouts. The average FKGL improved from 10.0 ± 1.6 to 6.4 ± 1.2 (p < 0.001), the mean FRES increased from 53 ± 8 to 68 ± 6 (p < 0.001), and the average word count decreased from 604 ± 201 to 488 ± 166 (p = 0.006) (Table 4).

The revised patient education materials were further evaluated by glaucoma patients. Following a regularly scheduled office visit, a total of five study subjects agreed to participate in a one-on-one interview to reflect on the quality of the handouts. Subjects ranged from 31 to 75 years of age, with a mean of 58 years. On the REALM-SF, four subjects scored at “ninth grade level or higher” and one scored at the “fourth-to-sixth grade” level. Demographic characteristics are summarized in Table 5.

Table 5 Demographic Characteristics of Interviewed Glaucoma Patients
Full size table

Before the interview, subjects completed Consumer Information Rating Form (CIRF) evaluations for “Top Ten List for Glaucoma” and another topic of their choosing. The CIRF scales range from 1 to 5, with higher scores indicating better quality. Four different educational topics were selected and evaluated, with all five subjects rating “Top Ten List for Glaucoma.” The highest scoring areas of this scale were organization and finding the handout helpful, but almost all CIRF items rated above a four on the five-point scale. (Table 6).

Table 6 Patient evaluation of revised handouts
Full size table

Structured interviews were conducted using open-ended questions outlined in Table 7, and interview transcripts revealed themes of patient preferences. In particular, subjects emphasized using concrete language, providing practical information, and having a simple format. A positive tone and emphasis that the provider is available to help were other characteristics that stood out to interviewed subjects. Although subjects were mixed about keeping the handout for reference or reading it only once, all generally appreciated having the key points highlighted. For some handouts, a picture of the eye or graphic of drop instillation were suggested, but images were not perceived as necessary for all handouts. Lastly, while brevity is essential, our subjects also desired a picture of the road ahead to glean what to expect during the course of their experience with glaucoma. Table 8 pairs themes with supporting quotations.

Table 7 Structured interview questions
Full size table
Table 8 Themes Derived from Patient Interviews
Full size table

Discussion

Ophthalmic patient education materials are written at a difficult readability level, both in print and online. We conducted a systematic review of 13 studies that measured readability or quality of ophthalmic patient education materials, and we found that the median Flesch-Kincaid Grade Level (FKGL) across all studies represents an eleventh-grade reading level, which far exceeds the level that many patients can understand. Various measures of readability and quality were used across the studies in this review, but all papers called for improvements in the material available for ophthalmic patients. Despite a universal call for improvement, no included study assessed methods for how to improve readability of patient education material. Drawing from other literature, we revised our patient handouts at our academic eye center using guidelines for writing easy-to-understand material (Table 2), and these changes significantly improved their readability and suitability for a low-health-literacy population. Additionally, we solicited feedback from glaucoma patients, and individual interview sessions reflected published guidelines, including providing practical information, being concrete, and highlighting the key points. This process has demonstrated that patient feedback is a valuable tool to ensure proper implementation of these recommendations.

Previous literature has examined effective recommendations for improving patient education materials. These studies outline steps for writing [33–35] and revising [36–39] health education material, and the recommendations reflect the collated list in Table 2, with an emphasis on maintaining an accessible readability level. Putting one of these recommendations into practice, Sheppard, et al. [36] improved readability level of patient education websites in orthopedics by shortening sentences to no more than 15 words, as recommended by NIH guidelines. This simple intervention improved the readability of eight articles by an average of 1.41 grade levels. Recommendations from various sources can be used to improve readability and suitability levels of patient education materials in ophthalmology and other specialties.

Despite a history of poor readability scores, ophthalmic patient education materials may be improving for patients with low health literacy. The AAO recently released updated versions of its patient education brochures in 2014, which are written at an eighth-grade level or lower and feature improved font and format for patients with low vision and patients with a low health literacy level [40].

Although these improvements should be lauded, many important patient education topics are not commercially available as brochures. Critical information in ophthalmology, such as pre-operative instructions and post-operative expectations, is not readily available for purchase, and specific content will vary depending on the individual ophthalmic practice or surgeon. As such, many ophthalmology clinics must develop at least some patient education material on their own. The guidelines presented in Table 2 may be a helpful reference for other groups to revise the suitability and readability of their ophthalmic patient education material with attention to low health literacy. In fact, even patients with a high level of health literacy prefer and more easily understand simplified language in written healthcare materials [10, 41, 42]. Lastly, with the overwhelming volume of medical information available to patients, clinicians and institutions share a responsibility in the “arc of health literacy” for population health to provide critical take-home messages that patients can easily understand [43, 44].

Our study has several limitations. First, we restricted our literature search to studies on ophthalmic patient education materials in the English language, which excludes a body of work on readability published in other fields and limits generalizability to non-English-language material. Second, readability is just one component of measuring suitability of patient education material, and we suggest that future studies include comprehensive suitability scores, such as SAM [14], to include factors such as layout, content, and learning stimulation. Third, in our study, we evaluated patient education material about glaucoma from a single academic ophthalmology practice, which may not be generalizable to other organizations. Additionally, feedback from glaucoma patients came from only five subjects, and just one had health literacy below the ninth-grade level. Finally, we did not measure patient knowledge of glaucoma or monitor health behavior after reading the handouts, as these metrics are outside the scope of this study.

Conclusions

Our systematic review of research on ophthalmic patient education materials showed that materials are consistently written at a readability level that is poorly suited for patients with low health literacy. Fortunately, patient educational materials can be improved. By revising our institution’s educational handouts using guidelines on writing easy-to-understand material (Table 2), we significantly improved the documents’ suitability for patients with low health literacy; the average reading level decreased from the tenth-grade level to the sixth-grade level after revision. Additionally, feedback from glaucoma patients demonstrated positive evaluation of the handouts, and open-ended patient interviews provided further insight for areas of improvement. A similar systematic approach of applying the guidelines we collated in Table 2 may improve the suitability, readability, and patient evaluation of other ophthalmic educational materials.