Background

Patient-reported outcomes (PROs) allow clinicians and researchers to better understand patients’ perceptions of their health, goals, treatment, and healthcare experiences across different domains, such as physical, mental, and social well-being [1, 2]. Patient-reported outcome measures (PROMs) are tools used to measure PROs in various disease contexts. Ideally PROMs are rigorously tested and validated. While many PROMs were developed for use in clinical trials [3], the use of PROMs in routine clinical care has become more widespread and has been shown to improve symptom control, supportive care measures, patient satisfaction, and health outcomes [4, 5]. In addition, there is increasing demand from healthcare payors, regulators, and administrators to incorporate PROs into routine clinical care for quality improvement and reimbursement purposes [6, 7].

Several logistical challenges regarding the implementation of PRO data collection into routine clinical practice have been noted, such as selection of appropriate and relevant PROMs, adequate staffing and data resources, patient compliance, and non-integration with electronic health records [6, 8,9,10]. However, an additional concern is ensuring equitable PRO implementation, such that PROs capture diverse and underrepresented patient populations [6, 11].

Given the current landscape of biomedical research, where less than 2% of 10,000 clinical trials included sufficient numbers of minority participants in one 2015 report [12], for example, it is unlikely that PRO implementation research would be immune to these issues of health equity and healthcare disparities in clinical research. PRO implementation in routine clinical care has been documented to have low adoption nationwide [13], which may further impede efforts to capture diverse populations. Additionally, reporting of certain factors such as race and ethnicity in clinical trials has not been widespread [14].

Routine PRO data collection is therefore likely predominant in White, higher socioeconomic, and English-speaking populations [15]. This is further supported by a systematic review of PRO implementation in routine care for patients with breast cancer, in which only 2 of 34 identified studies targeted minority patient populations [16] and a study of PRO implementation in integrative medicine, in which the majority of patients were White and had commercial insurance [17]. In addition, clinical PRO implementation studies have identified disparate response rates for many diverse and underrepresented populations across clinical specialties [18,19,20,21].

The lack of participation of certain patient populations in PRO data collection is concerning in light of a growing body of evidence demonstrating disparities in PROs among diverse and underrepresented patient populations. For example, PROs were significantly associated with race, education, and neighborhood poverty in a study of hip and knee arthroplasty [22], and lower income families experienced higher symptom burden and worse quality of life in a pediatric oncology study [23]. As the demand for PRO implementation in routine clinical care grows, it will therefore be critical to collect PROs from diverse and underrepresented patient populations to ensure representative sampling and reduce health disparities [24].

However, there is presently a limited understanding of the landscape of PRO implementation in these populations. This study therefore aims to review the literature to determine what is presently known regarding implementation of PROs in routine clinical care for diverse and underrepresented populations. The primary aim of this review is to characterize PRO implementation in routine clinical care, in terms of patient populations studied and clinical specialties that have evaluated PRO implementation. The secondary aims of this review are to analyze implementation outcomes and population-specific concerns, needs, and preferences regarding PRO collection. We believe that the review will advance our understanding of existing PRO programs and identify important areas of future research.

Methods

Given the lack of knowledge regarding available evidence and breadth of coverage of PRO implementation in diverse and underrepresented populations across clinical specialties, a scoping review methodology was adopted for this study. Per guidance by Munn et al. [25], a scoping review enables determination of the studies available and overall focus of a research area, aligning with this study’s aims to identify the landscape of available studies, populations and clinical specialties represented, and implementation outcomes evaluated to date. The review was conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews [26].

Data source and search

Studies were identified by searching the following electronic databases: MEDLINE (Ovid), Embase (Elsevier), Web of Science Core Collection (Clarivate Analytics), Cumulative Index to Nursing and Allied Health Literature (CINAHL: EBSCO), and PsycINFO (EBSCO). Searches were conducted between September 23 and 27, 2021. No date restriction was applied. The terms used to search each database were informed by previously published systematic reviews investigating PRO implementation in routine clinical care [16, 27, 28], as opposed to PROs used as evaluative assessments only for interventions in clinical trials, and relevant MeSH terms related to health disparities or diverse and underrepresented populations known to be affected by healthcare disparities [29]. The search was designed and conducted by a medical librarian with expertise in systematic and scoping reviews (PAB). Search strategy for the included databases is provided in the Additional file 1: Table S1.

Study selection and eligibility

Studies were compiled into the Covidence (Veritas Health Innovation) citation manager. First, abstracts and titles were screened by three independent reviewers (CJH, RG, RD) using inclusion and exclusion criteria. Full texts were then reviewed by three independent reviewers (CJH, RG, RD) to determine whether studies met inclusion criteria. Disagreements at each phase were resolved through discussion among the three reviewers.

Inclusion criteria were studies that (1) assessed PRO implementation in routine clinical care, (2) had a specific and explicit aim of studying PRO implementation in diverse or underrepresented populations, such as racial or ethnic minorities, sexual and gender minorities, elderly or geriatric populations ≥ 65 years of age, and populations with diverse literacy, educational, socioeconomic or geographic (rural or urban) status, (3) used a validated PROM, and (4) English language. Studies were limited to those conducted in the United States because diverse and underrepresented populations are in part defined by the specific historical, economic, and cultural contexts of their country of origin, as detailed in a previously published review [30]. In addition, certain sociodemographic factors such as insurance status and results concerning routine clinical care and PRO collection would be more generalizable to the unique healthcare system within the United States. Studies that focused on the development, creation, and/or validation of PROMs; use of PROMs as an outcome measure for some other primary intervention; and case reports, study protocols, editorials, dissertations, conference abstracts, and these were excluded. Citations within review articles from the search were manually reviewed to identify additional primary studies that met inclusion criteria. Results of the search as well as inclusion and exclusion of studies are reported according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses for Scoping Reviews (PRISMA-ScR) flow diagram (Fig. 1) [31].

Fig. 1
figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Scoping Reviews (PRISMA-ScR) diagram detailing inclusion and exclusion of studies included in the analysis

Full size image

Data analysis

Data from relevant studies were extracted into an a priori defined form by three reviewers (CJH, RG, RD), including study aims, sample size and demographics, geographic location of study settings, type of study setting, clinical specialty, validated PROM used, study limitations, if technology was used and type, and implementation outcomes. Implementation outcomes were categorized according to previously defined conceptual distinctions for implementation research: acceptability, adoption, appropriateness, costs, feasibility, fidelity, penetration, and sustainability [32]. These categorizations have been further refined for evaluation of technological interventions in healthcare [33] and have been used in previously published reviews of PRO implementation [28]. The quality and level of evidence (1–7) of studies were determined based on previously developed criteria by Melnyk et al., where level 1 corresponds to the highest quality evidence (systematic reviews, meta-analyses of randomized controlled trials, etc.) [34, 35].

Results

Study selection

The electronic database search yielded 8,687 records, and 1 record was identified through a review article known to study authors [16]. After duplicates were removed, the titles and abstracts of 4,452 records were each screened in duplicate according to pre-defined inclusion criteria (agreement > 95%). After screening these records, a total of 76 full-text articles were reviewed, of which 55 were excluded. Manual review of bibliographies from review articles identified in the search yielded an additional 7 studies that met inclusion criteria. A total of 28 studies were included in the analysis (Fig. 1).

Quality assessment and study characteristics

The aims, quality, and general characteristics of studies are included in Table 1. Most studies were observational cohort studies with level 4 evidence (n = 21, 75%), followed by level 2 randomized controlled studies (n = 5, 17.9%), and level 6 qualitative studies (n = 2, 7.1%). Sample sizes within studies ranged from 10 [36] to 6454 [37]. Primary care was the most common specialty represented (n = 10, 35.7%) [11, 36, 38,39,40,41,42,43,44,45], followed by oncology (n = 4, 21.4%) [46,47,48,49,50,51], of which two studies were specific to breast oncology [46, 47] and two studies were specific to urologic/radiation oncology [49, 50]. Remaining specialties included rheumatology [52,53,54,55] (n = 4, 14.3%), psychology/psychiatry [56,57,58] (n = 3, 10.7%), and one study each (3.6%) for neurology [37], geriatrics [59], trauma [60], home health care [61], and orthopedic hand surgery [62].

Table 1 Characteristics of the studies (n = 28) included in the review
Full size table

The demographics and characteristics of study populations are detailed in Table 2. Most included a study population that was majority female (n = 19, 67.9%) compared to majority male (n = 3, 10.7%). Some studies reported multiple samples with variable gender majorities (n = 4, 14.3%) or did not report gender (n = 2, 7.1%). Most studies (n = 15, 53.6%) included majority non-White populations and reported educational attainment and/or literacy (n = 19, 67.9%). In contrast, most studies did not report income or employment status (n = 17, 60.7%) or insurance status (n = 19, 67.9%). For studies that reported participants’ age, the age range among studies was 26.7 to 75.9 years. The majority of studies (n = 23, 82.1%) used electronic PRO (ePRO) data collection.

Table 2 Study population demographics and characteristics
Full size table

Implementation outcomes

Implementation outcomes are detailed in Table 3. Most studies evaluated feasibility (n = 27, 96.4%), followed by acceptability (n = 19, 67.9%), adoption (n = 18, 64.3%), and appropriateness (n = 10, 35.7%). Four studies assessed fidelity (14.3%) and three studies assessed costs (10.7%). One study each evaluated for penetration (3.6%) and sustainability (3.6%). Specific details and examples of each implementation outcome are detailed in Additional file 2: Table S2. Of those assessing feasibility, the majority assessed PRO completion rates over time (n = 17, 60.7%). Fewer assessed time needed to complete PRO assessments (n = 7, 25%) or impact on clinic workflows, such as the need for staff assistance in PRO completion or work burden (n = 7, 25%). Of those assessing adoption, few reported patients’ intention to try the studied PRO collection method again or in another setting (n = 3, 10.7%) and one reported clinicians’ intention to adopt new PRO technology in practice [53]. Of the three studies that assessed costs, none assessed cost-effectiveness. With regard to the one study assessing penetration and sustainability, the number of providers interfacing with the PRO system increased over time and the program was able to be maintained over a 5-year study period [40].

Table 3 Implementation outcomes of studies classified according to Proctor et al. [32] taxonomy
Full size table

Overall concerns, needs, and preferences of populations studied

Overall concerns, needs, and preferences were abstracted from studies and detailed in Table 4. Among the concerns were disparities in PRO completion among racial and ethnic minorities [11, 38, 50, 62], Spanish-speaking patients [11], populations with low income/employment status and low educational or health literacy [62], and elderly or geriatric populations [11, 38, 62]. Needs among populations included additional help in completing surveys among patients with low income/education [55] and a suggestion for a tutorial video for ePRO use among elderly patients [59]. Preferences among populations included a higher likelihood of Black patients selecting automated telephone over web-based surveys [49, 50], conflicting results of Spanish-speaking patients preferring face-to-face interviews vs. electronic data collection [43,44,45], symptom report display using bar graphs with “emojis” for low health literacy, majority Black patients [41], and preference for using a finger over stylus for tablet-based PRO collection in elderly patients [61].

Table 4 Overall concerns, needs, and preferences of populations identified in the review
Full size table

Discussion

This review synthesizes studies to date that have explicitly aimed to evaluate PRO implementation in routine clinical care for diverse and underrepresented patient populations in the United States across all clinical specialties, thereby taking an important step in furthering our understanding in this area. Given growing demands for routine clinical PRO collection, this review specifically evaluated study quality, demographics, implementation outcomes, and patient perspectives in order to inform existing and emerging PRO programs as well as to identify areas requiring further research.

This review demonstrated that relatively few high-quality studies such as randomized-controlled trials have been conducted. In addition, studies are not representative of all clinical specialties and skew predominately toward primary care settings. In particular, there was a paucity of studies conducted in surgical, obstetric, and pediatric settings. While this may be partly explained by pre-existing disparities in access to and quality of care in these settings [63,64,65,66], it will be important to study PRO implementation across the entirety of the healthcare spectrum with particular attention to those presently unrepresented in research to date.

In recent years, several health information technology interventions have been developed to better facilitate electronic PRO (ePRO) implementation, such as web-based platforms, tablets, and mobile applications. While concerns have been raised, given that there are disparities in smartphone and tablet computer ownership as well as internet access [67], our study highlights that ePRO collection is widely acceptable and feasible among diverse and underrepresented patient populations. While some studies suggested that face-to-face PRO collection may be preferred by certain populations (e.g., primarily Spanish-speaking individuals [43, 44]), another study conducted in a similar population demonstrated preference for computerized data collection [45]. Moreover, these studies were published over 20 years ago and may not be representative of populations today that have had more time to adjust to new technologies. This is supported by recent research showing that reliance on smartphones for internet access has become increasingly more common among Americans with lower socioeconomic status as well as racial and ethnic minority populations [67]. However, relying on ePROs alone may not be sufficient to maintain equitable PRO collection, as evidenced by one report demonstrating profound racial and ethnic disparities when transitioning from tablet-based to web-based PRO collection [68]. Our study highlights several key findings regarding the unique needs and concerns of populations in various clinical settings using ePROs, such as inclusion of bar graphs with “emojis” for symptom reports [41] or touchscreens with visual and audio components, for example [48]. This not only emphasizes the need for further research in these populations, but also suggests that programs would likely benefit from specifically tailoring ePROs to the populations they serve.

Notably, the least studied implementation outcomes in studies to date were penetration and sustainability. While it is still helpful to understand elements of PRO implementation such as acceptability and feasibility in the short-term, it is evident that less is known regarding longer-term and institution-wide outcomes of these interventions. This is problematic, given that PRO implementation programs can require additional staffing and data resources [8, 10], which may be prohibitive for low-resourced settings where many diverse and underrepresented populations receive care, especially outside of short-term, intensive study settings. As evidenced by the 5-year study period in the one study in this review with sustainability and penetration outcomes [40], another potential barrier to studying these outcomes is the time needed for longer, prospective studies. Bolstering partnerships between higher-resourced academic centers with existing PRO programs and lower-resourced settings may therefore provide one solution.

Moreover, cost was another infrequently assessed outcome. Consequently, cost-effectiveness data of PRO implementation was largely missing. Alongside staffing and data resource concerns, the cost of PRO implementation may preclude implementation in resource-constrained settings where many diverse and underrepresented populations receive care. While one study reported on the ability to maintain long-term costs [40], the financial requirements and impacts of PRO implementation for diverse and underrepresented populations remains largely to be determined. Nonetheless, the reduction of healthcare disparities itself has been estimated to substantially reduce healthcare spending [69, 70]. With the aim of reducing disparities through these implementation programs, cost data will also be helpful in further characterizing the cost-effectiveness and healthcare value of routine PRO collection.

While the studies within this analysis elucidated important findings within underrepresented patient populations, it is important to note that more work is needed to elucidate which specific determinants of health within populations drive certain outcomes as well as the way in which identities and social determinants intersect [71]. Most studies, for example, reported racial and ethnic demographics of their study sample, however much fewer reported income, employment, and insurance status. This is problematic, given that racial or ethnic minority outcomes may be confounded by other unreported social factors, such as low income or unemployment. In addition, results and data analysis were often stratified by race or ethnic minority status alone, without delving deeper into the specific social determinants of health (i.e. transportation, perceived and actual racism, work environments). As such, future work evaluating implementation within these populations should more deeply investigate how social determinants of health drive disparate acceptability, adoption, or feasibility outcomes, for example.

Our study had some limitations. First, studies were limited to the United States in order to increase generalizability of results, given that diverse and underrepresented populations are defined in part by the history, demographics, culture, and economy of their country of origin. Although international studies such as those investigating routine PRO collection in rural Australia [72], use of robots for routine PRO collection among elderly adults in the Netherlands [73], and ePRO collection among a diverse, urban clinic population living with HIV in Canada [74] may have relevant and generalizable findings for clinical environments in the United States, we deemed these beyond the scope of this review. However, the narrower scope of this review allowed for greater specificity of findings for certain populations, such as Black Americans and non-English-speaking populations, within the unique context of PRO programs in the United States. Second, we only included studies that had a specific and explicit aim of studying PRO implementation in a diverse and/or underrepresented population. There is a possibility that studies investigating clinical PRO implementation more broadly had potentially relevant incidental findings for these populations, however this review importantly highlights studies that fill critical gaps in the literature by intentionally aiming to study these populations. In addition, this review studied diverse populations known to be affected by healthcare disparities in the United States. Third, our study focused on the implementation of already developed and validated PROMs. Addressing disparities in PRO data collection will also require equitable PROM development, testing, and validation within diverse and underrepresented populations before implementation occurs [15].

Conclusions

As existing healthcare systems expand and new systems develop PRO data collection programs, it will be imperative to ensure that PRO data collection is not only representative of all patients but also equitable in its implementation in routine clinical care [15]. While this study highlights several important considerations for PRO implementation in diverse and underrepresented populations, it simultaneously calls attention to the paucity of research in this area to date. Future studies of PRO implementation will be needed across the healthcare spectrum in order to address existing disparities and promote health equity alike.