FormalPara Key Points for Decision Makers

• Patient-reported outcome measures (PROMs) are health questionnaires asking patients to report on their symptoms and quality of life

• Normally patients are involved in the rigorous process to develop PROMs to ensure the questionnaires make sense and are relevant to them. However, this study demonstrates that people with low literacy skills and learning disabilities are generally left out of the development of PROMs so they may not be able to use these measures

• Increased involvement of these groups in research and specific efforts to ensure PROMs are accessible for them could prevent unequal access to further uses of PROMs, for example, in healthcare quality improvement

1 Background

Patient-reported outcome measures (PROMs) are quickly becoming an important part of healthcare quality improvement processes internationally [1]. Health professionals and authorities can use these questionnaires to understand the burden of disease from the patients’ perspective and gather information to improve the quality of care they receive. Although PROMs are mainly used for acute conditions, expanding their use into long-term conditions is imminent [2].

Multiple stages involving patients and subject experts are required to develop PROMs so that they will produce meaningful data. Typically, developers first consult with the concerned groups about what to include in the PROM. Several forms of validation and reliability testing follow before the PROM is finalized. The development of a PROM does not stop at its conception; the process of development is ongoing, encompassing further validation and reliability testing of PROMs through use with different patient groups. The initial and ongoing development and validation are meant to ensure the items and tool are meaningful to clinicians and patients and that it is easy to administer [3]. Since patients are consulted, PROM development appears patient centered. But not all groups of patients may have an opportunity to offer their views on PROMs.

People with low literacy skills or learning disabilities may be excluded from PROM development and ultimately from the uses of PROMs, countering national and international policies. The causes of learning disability and low literacy vary greatly, and people with low literacy and those with learning disabilities are not usually grouped together. Nevertheless, the functional difficulties (e.g., reading ability and comprehension) they face can be similar and both groups may find help in processing written material and managing health interventions useful [4]. Together, these groups form a large proportion of people in the UK, where one in five adults have low literacy skills [5] and there are approximately 1.5 million people with learning disabilities [6]. Literacy difficulties are associated with general exclusion from trials and research [7]. If people with low literacy skills or learning disabilities are left out of the development of PROMs specifically, they may struggle to complete them since patient involvement is crucial to ensure appropriate wording, format, and other needs are met. When PROMs are then used in further healthcare initiatives, they may face exclusion again. This exclusion would counter the UK Government’s policy for the NHS to achieve full inclusion [8]. It also violates the principles outlined in the United Nations Convention on the Rights of Persons with Disabilities [9].

To prevent people with low literacy skills and/or learning disabilities from possible exclusion from PROMs initiatives, knowledge of the extent and nature of exclusion in the development of PROMs is required. There is little previous research on this issue in the PROMs literature. Others have, however, acknowledged the potential for the exclusion of people with low literacy skills and learning disabilities from PROMs initiatives. Devlin and Appleby [10] write that gathering PROMs data from people with “cognitive deficits” (p. 12) is difficult. Mencap, a leading charity in the UK for people with learning disabilities, say their clients are at risk of being left out of efforts to gather health information from patients [11]. But neither describes the current extent of exclusion from the development of PROMs in the first place.

Chronic obstructive pulmonary disease (COPD) is a good exemplar case to investigate potential exclusion. This long-term condition causes significant burden to over 3 million people in the UK [12]. It is also more prevalent in lower socio-economic groups where lower levels of literacy are also more common. This study examines the most commonly used PROMs for COPD to investigate whether there is systematic exclusion of people with learning disabilities and/or low literacy from PROM use and, if so, aims to suggest how PROMs can be developed more inclusively.

2 Method

Our methods had two phases.

In Phase I, we identified the existing PROMs that are considered suitable for use with people with COPD using systematic reviews by Fitzpatrick et al. [13] and Davies et al. [14]. The most promising PROMs they identified included the respiratory condition-specific Clinical Respiratory Questionnaire (CRQ) and St George’s Respiratory Questionnaire (SGRQ), and the generic EQ-5D and SF-36. We went through Fitzpatrick et al. [13] and Davies et al.’s [14] reference lists to select only the publications involving the above-listed PROMs. The inclusion and exclusion criteria are summarized in Table 1. Given the quality of their review, these were the basis for our study for articles published prior to or in 2008.

Table 1 Inclusion and exclusion criteria for the papers identified in Phases I and II
Full size table

In Phase II, we updated the search to identify papers describing PROMs developed from 2009 to 2012. To identify these papers, we used a pragmatic approach based on the search strategy provided to us by one of the authors of Davies et al. [14], searching in MEDLINE. We were not in a position to recommend which of the newly identified PROMs were suitable for population use, as was done in Fitzpatrick et al. [13] and Davies et al. [14], so we included all PROMs that were used with COPD patients, were in English, and were not too narrowly focused (for example, focusing only on one or two specific symptoms such as breathlessness). We also included two PROMs that were identified, but had only just been developed, at the end of Davies et al.’s [14] systematic review in 2009: the COPD Assessment Test (CAT) and the EXAcerbations of Chronic Obstructive Pulmonary Disease Tool (EXACT). The inclusion and exclusion criteria for this phase are also summarized in Table 1.

The abstracts from Phase I and Phase II were independently reviewed by two researchers for inclusion and exclusion. If there was a discrepancy, the abstract went to a third reviewer to decide. We retrieved the full text of articles that met the inclusion criteria and extracted the relevant data from each paper including:

  1. 1.

    The aim of the study

  2. 2.

    The PROM(s) used

  3. 3.

    The participant recruitment and data collection processes (if stated)

  4. 4.

    The inclusion and exclusion criteria for research participants (if stated)

  5. 5.

    Any mention of people with low literacy skills or learning disabilities

We distinguished between ‘explicit’ and ‘implicit’ modes of exclusion. Studies were classed as having ‘explicit exclusion’ if they excluded people with low literacy skills or learning disabilities ‘explicitly’ or immediately as a result of the study’s eligibility criteria. While we did not consider a specific category of disability severity, the terms considered to imply the potential exclusion of groups with low literacy skills or learning disabilities included those who were ‘unable to read,’ or ‘unable to complete or comprehend the questionnaire,’ or those with ‘cognitive deficits,’ ‘cognitive impairments,’ or ‘cognitive limitations.’ Studies that used literacy-dependent methods for recruitment or data collection (e.g., unassisted self-completion of the PROM) were classed as leading to ‘implicit exclusion.’

A second researcher in the team checked the data extraction for 20 % of the articles. Finally anything else in the study set-up that could impact the inclusion of these groups was recorded.

3 Results

We reviewed 108 abstracts and included 53 papers in Phase I. We identified six new PROMs in Phase II and reviewed 67 abstracts, eventually including 17 papers. The processes for Phase I and Phase II are summarized in Figs. 1 and 2, respectively. In addition to the CRQ, SGRQ, EQ-5D, SF-36, CAT, and EXACT tools, the new PROMs identified and included were:

Fig. 1
figure 1

Phase I review flow chart

Full size image
Fig. 2
figure 2

Phase II review flow chart

Full size image
  1. 1.

    The McGill COPD Quality of Life Questionnaire

  2. 2.

    The Functional Assessment of Chronic Illness Therapy (FACIT)-Dyspnea Scale

  3. 3.

    The Disability Related to COPD Tool (DIRECT)

  4. 4.

    The Capacity of Daily Living during the Morning Questionnaire

  5. 5.

    The Visual Simplified Respiratory Questionnaire (VSRQ)

  6. 6.

    The Living with COPD Questionnaire

We extracted data from 70 papers in total. None of the studies mentioned specific efforts to include people with low literacy skills or learning disabilities. Three studies excluded these groups explicitly, or based on eligibility criteria alone. For example Aaron et al. [15] excluded those with cognitive impairments. Thirty-two studies had implicit exclusion only, relating to non-inclusive recruitment and questionnaire administration processes. For instance Schofield and Mishra [16] recruited study participants through postal packages. People with low literacy skills or learning disabilities may not be able to read or comprehend such packages excluding them from the study at the outset; other examples included recruitment through newspaper advertisements [17] and the requirement for self-administration [18], which was the most common reason for classifying studies as having implicit exclusion. Sixteen studies had both explicit and implicit exclusion. Finally, 19 studies were unclear about recruitment or administration processes so we could not verify whether they excluded people with low literacy skills or learning disabilities (Table 2).

Table 2 Data extraction results
Full size table

We also recorded other notable issues in the studies concerning people with low literacy skills and learning disabilities. Some of the studies’ protocols may have given consideration for these groups. For example, Aiken et al. [19] used telephone interviews to administer the PROM, although they did not accept proxy responses. Williams et al. [20] provided an opportunity for participants to clarify the self-administered questionnaire at the end. Some studies also described the sample composition, which could give an indication of the prevalence of low literacy. In Sprenkle et al. [21], the majority of participants had minimum high school education, while 26 % of Wyrwich et al.’s [22] sample had a grade school education only. However, the latter study did use telephone interviews. Ringbaek et al. [23] found patients needed a lot of assistance to complete PROMs, particularly those with less education, and especially while attempting the SGRQ. Yount et al. [24] were explicit about their sample’s high education level and used a touch screen to administer the FACIT-Dyspnea Scale. Technology was also used to send automatic alerts through an e-diary to support patients to complete the Capacity of Daily Living during the Morning Questionnaire [25].

4 Discussion

We have found that people with low literacy skills or learning disabilities are excluded from the development of PROMs for COPD in two distinct ways: explicitly through the study eligibility criteria and, of potentially greater significance, implicitly through recruitment or administration processes. These results indicate that these groups may struggle to complete PROMs. Healthcare initiatives like quality improvement for COPD that depend on PROMs may exclude people with low literacy skills or learning disabilities as a result.

Implicit exclusion is where researchers and readers may not realize exclusion takes place; we found implicit exclusion was most common largely due to the majority of studies’ requirement for self-administration of the PROM. Almost all PROMs are only validated to be self-administered but their readability and administration may be more or less inclusive. For example, technology support or easy read formatting can enhance accessibility. While some studies did use technology [24, 25], none of the papers we reviewed included discussion of the groups they potentially excluded in the recruitment processes they described. This could partly reflect the word limitations imposed on authors, but at least to some extent indicates a lack of critical awareness of the potential for exclusion. Where the authors did highlight heterogeneity in their sample demographic [24], we could not find specific mention of approaches to include people who may have low literacy skills or learning disabilities. There was also a lack of description of efforts to accommodate specific support to complete PROMs, for example through assisted completion. While other authors have found substantial disagreements between self- and proxy reporting especially to collect subjective information [26–28], the valid provision of assistance in completing PROMs has proved useful in population survey research, for example in Kaye [29].

Identifying and distinguishing forms of exclusion in research to develop PROMs is a prerequisite for future research aiming to make PROMs practice more inclusive. We consider implicit exclusion to be more concerning than explicit exclusion because it implies a lack of awareness of the potential for inequality in resulting healthcare and research practices. Studies with explicitly stated criteria excluding people with low literacy skills or learning disabilities set a clear barrier preventing them from participating. This suggests awareness on the part of the researchers and gives clear notification to readers that the findings apply only to the included population groups, provided this caveat is not overlooked when discussing generalizability [30]. Implicit exclusion practices have a higher likelihood than explicit exclusion to result in barriers to inclusive participation without any realization. The more recent increased focus on equality and diversity issues in research may improve future development practices compared with those used to develop the PROMs we identified. Inclusion might be given a higher priority and researchers given more support to overcome inevitable time, skill, and funding constraints that can limit feasibility to include people with low literacy skills or learning disabilities in their work. But if we do not realize there are barriers, we will not recognize the need for efforts to be more inclusive. In a current policy environment with increasing interest in PROMs [10], the potential result is unequal access to quality improvement processes.

Specific efforts can support the inclusion of people with low literacy skills or learning disabilities in research to develop PROMs. Research funders can confront issues of inequality by attaching equality criteria as a condition for funding the development of a PROM (similar to the requirement from the National Institutes of Health in the USA [31] to justify not making efforts to include groups normally left out of research). Responsibility among researchers to think through the implications of their methods and practices for people’s ability to participate in their study may also be helpful. This may involve consulting service users in the commissioning, design, and conduct of research to determine best practice options for PROM development including when (timeliness), where (setting), and how (visual assistance) they can be administered, and any required assistance [32]. Other literature outlines issues and provides guidance to conduct research with people with learning disabilities [33], and tools to aid communication also exist [34]. Guidance on increasing the accessibility of information including the need for ample white space and large font sizes [35] are also easily applicable to typical research documents like information leaflets, consent forms, and PROM design. Our findings also suggested some promising results to build on. The e-diary with automatic alerts used to administer the Capacity of Daily Living during the Morning Questionnaire [25] and the touch screen used to administer the FACIT-Dyspnea Scale [24] may be promising as technology-based methods to support PROMs administration for people with low literacy skills or learning disabilities. These are similar to previous efforts to administer PROMs to people with low literacy skills in a cancer setting using an audiovisual touch screen [36].

Our work is a first step in highlighting the presence of exclusive practices in the development of PROMs and it has limitations. The findings have to be interpreted in the light of word limits imposed on article length. We did not infer sample demographics, recruitment, or administration practices where they were not explicitly stated in the studies and authors’ descriptions of these may have in turn been limited by journal word counts. The data extraction was partly subjective as we sometimes assumed terms used by the authors to describe people who are not eligible for their study (like excluding those with ‘cognitive impairments’) would exclude someone with low literacy skills or learning disabilities. However, the extraction was checked by a second researcher to ensure consistency. Finally, we recognize that although people with low literacy skills and learning disabilities should not be ‘lumped together’ as one group, both groups face similar practical and functional challenges with reading and understanding written material. Other groups may also face similar exclusion from PROM development and validation processes, and substantial difficulties with PROMs, for example people with dementia or people with acquired brain injuries resulting in cognitive impairments.

5 Conclusion

While the use of PROMs is growing as a major part of healthcare quality improvement internationally, we have demonstrated that, at least in relation to COPD, people with low literacy skills or learning disabilities might be left out because they are not included in the research to develop PROMs. Most exclusion is implicit, suggesting the potential for lack of awareness of exclusionary practices. To prevent this form of exclusion in the future, efforts on the part of funders and researchers and more research into validating PROMs to be used with support through technology-based accessible completion or assisted completion are important. Ultimately PROMs are a promising opportunity to involve patients in quality improvement for COPD healthcare but the exclusive nature of PROM development may further alienate people with low literacy skills and learning disabilities from healthcare practices.