Background

Every year, more than 4 million children are born in the US [1]. Shortly after delivery, parents face the important task of selecting a pediatrician for their new baby. Receiving recommended care from a pediatric provider can impact child health, and is particularly important in the first two years of life [2–8]. Variation in care quality exists between health plans, communities, hospitals, outpatient practices, and providers [9–11], but patients may not be aware that this variation exists [12].

One way to help patients become aware of differences in physician performance is through the public reporting of quality measures. If patients use information about performance to choose practices with higher quality care or if public reporting stimulates providers to engage in quality improvement activities, public reporting of quality information should result in better patient outcomes [13]. However, the impact on patients’ choices has been limited to date [14], perhaps in part because of a lack of awareness of publicly reported quality data [15]. Even when patients are aware of websites reporting health care quality data, the information may not effectively reach patients because the websites are difficult to navigate and the data are difficult to interpret [16–19]. This failure to reach patients may be especially pronounced in vulnerable populations, such as those with lower health literacy [18, 20].

Massachusetts Health Quality Partners (MHQP) is a non-profit organization that has been working to improve care quality in Massachusetts since 1995. The organization works with stakeholders, including patients, physicians, insurance companies, and academic partners to collect and report quality-of-care data in the Commonwealth. Since 2005, MHQP has included a quality reporting tool known as “Quality Insights” on its website [21]. These reports show adult and pediatric practices’ performance on clinical quality and patient experience measures (Table 1). These reports have been recognized for excellence in methodology and accessibility, and are, to our knowledge, the only source of publicly reported, ambulatory pediatric practice quality data in Massachusetts. The website is currently only available in English.

Table 1 Categories of pediatric clinical quality and patient experience reported on the MHQP website
Full size table

Patient navigators assist patients in overcoming barriers to achieving care goals. Traditional patient navigators have successfully helped patients navigate complex care environments and achieve better health outcomes [22–24], with those at risk for low health literacy and numeracy benefiting the most [25–27]. These traditional navigator functions may assist patients attempting to access and interpret publicly reported information about quality of care, tailoring the assistance to the patient’s needs. Pregnant women are generally healthy and have ample time to explore options for pediatric care along with multiple contacts with the healthcare system, making this condition ideal for a navigator intervention to support use of publicly reported quality data.

We plan to implement a randomized controlled trial (RCT) design to test the efficacy of a patient navigator intervention. The goal of the intervention is to educate women about the availability and meaning of quality measures in pediatrics and to review information about the quality of local pediatric practices, enabling them to make a more informed decision about where to obtain care for their child.

Methods/Design

Study objectives

The primary study outcomes are a) the mean clinical quality score and b) the mean patient experience score of the pediatric practices selected. The study goals are: to determine the efficacy of a patient navigator intervention to assist low-income pregnant women in the use of publicly available information about quality of care when choosing a pediatrician; to evaluate the relative importance of factors influencing women’s choice of pediatric practices; to evaluate the effect of the intervention on patient engagement in management of their own and their child’s health care; and to assess variation in efficacy of the intervention for sub-groups based on parity, age, and race/ethnicity. This study is approved by the Baystate Medical Center Institutional Review Board.

Study design

This study is a two-arm RCT of patient navigators. Participants are randomized to either the intervention arm (navigator plus informational pamphlet) or the control arm (informational pamphlet only).

Study site and participants

The Obstetric and Gynecology Clinics at Baystate Medical Center, located in Springfield, Massachusetts, perform approximately 1,500 to 1,800 deliveries annually. The clinics are staffed by eleven obstetricians, six nurse practitioners, and twenty-one house staff from the Baystate Medical Center (BMC)/Tufts University School of Medicine (TUSM) residency training program in obstetrics and gynecology. The majority (89%) of the women attending the clinic are insured through Medicaid, the national public insurance available to women with no employer-based insurance and/or a low-income level [28]. Children of women who qualify for Medicaid often also qualify for this public insurance. A survey undertaken by the American Academy of Pediatric in the year 2000 showed nearly 95% of pediatric practices in Massachusetts accepted all patients with public health insurance [29], but practices may elect to limit the number of patients they take with any type of insurance. Of these, 57% are Hispanic (predominantly from Puerto Rico), 17% are African-American, 23% are non-Hispanic white, and 3% are of other race/ethnicity [28]. Among Hispanic women attending the clinic, nearly 80% prefer speaking English [28]. English-speaking women ages 16 to 50 years between 20 and 34 weeks of pregnancy will be eligible for enrollment in the study.

Recruitment and informed consent

Women will be given a one-page study fact sheet by office staff when they first present to the Wesson Women’s Clinic. When women check in for their prenatal visit during weeks 20 to 34 of gestation, they will be reminded of the study by one of the two trained patient navigators and will be given the one-page fact sheet again as a reminder of study goals. If a woman expresses interest in enrolling, informed consent will then be obtained. Baseline data will be collected and the patient will be randomly assigned to the navigator intervention or the pamphlet control. Randomization will be stratified based on parity because women who have already selected a pediatric practice for their older children may be less likely to change practices based on quality performance alone. If randomized to the intervention arm, the first intervention session will ensue. If in the control arm, the patient will be given an informational pamphlet (described below). If the study process is interrupted, it will be continued at subsequent prenatal visits or by phone when feasible. A flow diagram for the study is shown in Figure 1.

Figure 1
figure 1

IDEAS (Inform, Describe, Educate, Assist, support) for a healthy baby sessions flow diagram PAM, Patient Activation Measure; P-PAM, Parental Patient Activation Measure; NVS: Newest Vital Sign.

Full size image

IDEAS intervention

Patients randomized to the patient navigator intervention will participate in the IDEAS for a Healthy Baby’ sessions. The conceptual model for the intervention is shown in Figure 2. The intervention will include:

Figure 2
figure 2

IDEAS for a healthy baby sessions conceptual model.

Full size image

  • Information about the purpose of quality measures.

  • Description of how quality of care varies among practices.

  • Education about the existence of publicly reported quality data.

  • Assistance in navigating and understanding quality-of-care data on the MHQP website.

  • Support for informed decision making in health care.

A list of eligible patients is generated each morning. Patients are then approached by one of the patient navigators in the waiting room of the prenatal clinic and offered an opportunity to participate in the study. If interested, informed consent is obtained and baseline data collected. This may take place in the waiting room, an examination room, or an educational office, when available. Patients randomized to the intervention arm then participate in a brief in-person educational session led by the patient navigator, whose experience includes teaching small undergraduate courses. The navigator’s training for the study included developing an understanding of the methods used to develop the quality measures, use of the MHQP website and obtaining informed consent. Both the primary and secondary navigators participated in extensive rehearsal of the intervention with the principal investigator (PI) feedback prior to piloting then enrolling patients. The information sessions include the rationale for quality measurement, how quality varies among practices, and that information about variation is publicly available. The information session is followed by in-person guided use of the MHQP website with the navigator. Each of two guided website sessions last approximately 20 minutes and take place at two prenatal visits. The first session takes place between 20 and 34 weeks gestation and the second 2 to 6 weeks after the first session. The navigator guides the participant in use of the MHQP website on a laptop computer. There is a standard explanation of the meaning of the quality performance measures and patientexperience survey data. The patient navigator provides the same demonstration to all participants about how the website can be used to assess a practice’s performance. Following this standard intervention, patients are shown performance data for practices they would like to view within a 25-mile radius of their home. If the participant does not have any practices she wishes to view, a standard set of three area practices with high and low scores are shown as examples. Practices shown are recorded. A copy of the study information pamphlet is given after the first guided website session and the participant is asked if they have questions about the pamphlet. During the second intervention session, the patient navigator asks if the participant had an opportunity to look at the website themselves. The navigator then invites them to look at local pediatric practice data again and will respond to any questions about performance data for these practices. Participants may complete a worksheet during the intervention in which they fill out a table of scores on performance measures for the practices they review. Patient navigators do not recommend one practice over another but do discuss the practices’ quality scores and what the scores mean. Objectives of the IDEAS for a healthy baby intervention are outlined in Table 2, along with methods to achieve objectives.

Table 2 Objectives and associated navigator actions
Full size table

Information pamphlet control

The pamphlet will provide information about clinical quality performance measures, patient satisfaction measures, and information about the MHQP website, including the purpose of the site as well as the web address (URL) where the quality-of-care data relevant to pediatric care are located. Participants randomized to the control group will receive the information pamphlet following collection of baseline measures on the day of enrollment.

Outcome assessment

All enrolled patients will be interviewed after delivery, prior to hospital discharge for outcome assessment. This assessment will include the name of the pediatric practice where the newborn has a scheduled visit (required by Baystate prior to infant discharge) and follow-up measures outlined.

Measures

Primary outcome

The MHQP website uses a star system for rating practices on each of nine measures of pediatric clinical quality with a maximum of four stars. Similarly, the site uses a star system for rating practices on each of seven patient experience measures. A mean quality score and a mean patient experience score will be created for each practice within a 25-mile radius of the hospital that has data reported on the MHQP website for at least three relevant measures. The primary outcomes will be a) the mean MHQP clinical quality and b) mean patient experience scores of the pediatric practices selected by the participants in the navigator intervention compared to the control. All measures are summarized in Table 3.

Table 3 Summary of outcome and other measures
Full size table

Secondary outcomes

Patient activation measures The Patient Activation Measure (PAM) [30] and Parental Patient Activation Measure (P-PAM) [31] are valid, reliable, uni-dimensional, probabilistic Guttman-like scales that reflect a developmental model of activation. The measures can be used to assess changes in individuals’ levels of activation for self-management of health care [30, 31]. Prior studies have shown individual and parental activation can differ for the individual, with parental activation being higher than individual [31]. Interventions with decision aids and to improve access to health care have each improved activation measure scores [32, 33].

Importance of factors in selecting a pediatric practice As part of the study survey, participants will be asked to rate the importance of several factors in selecting a pediatric practice. Factors will be rated on a five-point Likert scale and will include recommendations of friends or family, advice from a physician, an existing relationship with a provider, insurance, and proximity to home.

Other measures

Demographics Demographic data collected will include age, race/ethnicity, country of birth, parity, household income, marital status, number of adults in the household, number of children in the household, insurance status, primary language spoken at home, and name of current pediatric practice if the participant has children.

Use of publicly reported data As part of the study survey, we will determine participants’ familiarity with health care quality measures, variation in quality of care, awareness of publicly reported health care quality data, and their personal use of publicly reported health care quality data outside of the study.

Health literacy and numeracy The Newest Vital Sign (NVS) is a screening tool that assesses general literacy and numeracy skills as applied to health information. Individuals examine information on a nutrition label and answer six questions about how they would interpret and act on that information. The NVS takes approximately 3 minutes to complete.

Challenges encountered using website data to guide provider choice At the post-delivery interview, women will be asked what difficulties, if any, they encountered acting upon the information they accessed either with the navigator or on their own using the information pamphlet. Anticipated barriers include limitations imposed by the participant’s insurance carrier and transportation difficulties.

Analysis

Intervention efficacy

Descriptive statistics will be used to characterize the study population and evaluate the distributions of outcomes, predictors and covariates. We will use t-tests and Wilcoxon rank-sum tests to compare mean quality and patient experience scores of the pediatric practices selected between women randomized to the navigator intervention and the information pamphlet control. This analysis will be both overall and within strata defined by parity (0, 1+). Our primary analysis will be intent-to-treat, based upon randomization, whether or not the intervention sessions occurred. Patients who choose practices that do not appear on the MHQP website will not be included in the primary outcome analysis but will be included in secondary outcome analyses. Secondary analysis will adjust for compliance with the intervention. We will also evaluate analysis of variance models for the mean quality and patient experience scores, adjusting for parity and any baseline patient characteristics that were not balanced between the intervention groups. An interaction effect for intervention with parity will be included to test for difference in the intervention efficacy between women selecting a pediatrician for the first time versus those with children. Confounding by covariates will be assessed. In addition we will explore the impact of clustering of women within neighborhoods defined by residential zip code, since location of practice relative to home may have a large impact on choice of practice. Final models will provide an estimate of the patient navigator effect on the quality and patient experience ratings of selected pediatric practice, both overall and by parity, adjusting for important maternal characteristics. If practice scores on the website should change during the course of the study, the scores present during the prenatal period will be used in the analysis.

Intervention efficacy among subgroups

We will use t-tests and Wilcoxon rank-sum tests to compare mean quality and patient experience scores of pediatric practices between intervention and control arms among subgroups of women defined by age, race/ethnicity, literacy, and other baseline characteristics, both overall and stratified by parity. We will then use analysis of variance models to evaluate quality and experience scores as functions of intervention status and patient baseline characteristics. Effect modification will be evaluated by means of interaction of the intervention with selected patient baseline characteristics. Results of these models will give insight into characteristics of those mothers for whom the patient navigator intervention proved more valuable.

Importance of factors in choosing a pediatrician

We will evaluate the distribution of responses to the five-point scale (‘Did not matter at all’ to ‘Mattered a lot’) for each factor in choosing a pediatrician at baseline and follow up. We will compute a within-subject change in importance for each factor, using t-tests and sign tests to determine whether there is a change from baseline and follow up. Initial analyses will be stratified by parity.

Impact of intervention on patient and parent activation to self-manage health care

We will evaluate differences in patient and parental activation change between control and intervention by means of t-tests or Wilcoxon rank-sum tests. Analysis of variance models will be used to evaluate change in activation, adjusting for baseline activation levels as well as other patient. Initial models will include an interaction effect for intervention with parity. Results from these models will provide information on the impact of the intervention on changing patient and parental health attitudes, whether or not there was an impact on the quality scores of practices selected.

Power/sample size

Power was evaluated to detect a difference between intervention groups in mean quality and patient experience ratings of pediatric practices, within strata defined by parity (0, 1+). In an 18-month period, we anticipate 2,250 pregnant women will be seen at the Wesson Women’s OB and Midwifery Clinic. Based upon prior experience with research recruitment in these clinics, we expect to be able to approach about two thirds, or approximately 1,500 women. Of these we anticipate 1,200 will meet age and language eligibility requirements and that 730 will agree to participate, with 650 completing baseline and post-delivery data collection.

Expecting approximately 40% nulliparous women [28], we will need to have at least 130 women in each study arm within strata defined by parity. Quality data were identified on the MHQP website for 18 practices within a 25-mile radius of Springfield, with a mean (SD) quality score of 3.1 (0.5) on nine pediatric measures. Mean (SD) patient experience scores were similar at 3.0 (0.6). Using a significance level of 0.01, we will have power greater than 80% to detect a mean difference of 0.2 units in quality scores between control and intervention, within each stratum of parity. Power is greater than 80% to detect a 0.25 difference in mean patient experience ratings. The magnitude of the differences (0.2, 0.25) would indicate a meaningful difference in mean quality or patient experience rating of practices selected, yet this difference is large enough that we expect it could be detected.

Discussion

In this paper, we describe our rationale and methods for an RCT to test the efficacy of a patient navigator intervention for increasing the impact of publicly reported quality of care data on the choice of a pediatric care provider. The potential impact of this study is great. With more than 73 million children under the age of 18 residing in the US [34], the quality of ambulatory pediatric care can have an enormous impact on population health.

The impact of publicly reported quality data on patients’ choice has been disappointing, due in part to challenges in communicating the information to patients in a meaningful way [14]. Our study takes a novel approach to making the science of public reporting available and understandable to a population known to be at greatest risk for limited capacity to use these data. Two elements of this study design enhance the potential for success: use of patient navigators and engagement of participants while they are receiving prenatal care. Although patient navigators have been used to assist patients in obtaining preventive care and treatment for complex medical conditions [22–24], we are unaware of studies in which they have been used to enhance informed decision-making about choice of providers for a low income population. Navigators have demonstrated the greatest effectiveness among populations with lower socioeconomic status and low health literacy [25–27]; thus, targeting a population with these characteristics may offer the greatest potential for effectiveness of the intervention. Pregnancy is also a time when women may be more engaged in their health and healthcare than at other times [35]. This motivation, coupled with sufficient lead time between knowing they need to select a pediatrician and selecting one, makes this an opportune time to test navigator efficacy.

If successful, this model has important ramifications. Team approaches to care are increasingly used in the ambulatory setting [36]. Although it is unlikely that a single practice can sponsor a dedicated patient navigator, it seems reasonable that case managers and ancillary staff can be trained to provide information about care quality to patients as part of a global effort to engage patients in health care decision-making. Should this efficacy trial favor use of navigators, this may have important implications for design and scope of responsibilities for care team members.

This study has limitations. First, women who already have children may be less likely to change pediatricians, even if quality scores for the current pediatrician are low. We have attempted to account for this by stratifying randomization to multiparous and nulliparous women and by asking mothers to rate the level of importance of a range of factors that may impact choice of pediatrician. Second, because the website is only available in English, we are not enrolling patients who are not comfortable with the English language. Few organizations have the resources to make websites available in multiple languages, but this continues to limit access to these data. Third, there is debate about how meaningful these types of quality measures are, with questions about the quality of the data used to develop them and the paucity of data linking them to improved patient outcomes. However, the MHQP website has been recognized for the soundness of the MHQP methods of compiling the performance scores, and there is growing use of these measures for assessment of care quality. Fourth, although we do not include a cost analysis for this intervention, it is possible that it may be prohibitively expensive to undertake outside of the study setting. If the intervention is successful, we plan to explore mechanisms of delivering the intervention in an affordable manner, such as using existing staff in obstetricians’ offices. Additionally, while the study was designed with power to detect differences within strata defined by parity, we may be underpowered to detect differences in secondary outcomes, or within subgroups of patients defined by literacy or ethnicity. It may be possible that as websites improve and become more usable for patients with limited health literacy, the need for a navigator may become less important.

In this paper, we have given a brief description of the rationale for a patient navigator intervention to increase access and understanding of publicly reported pediatric quality data for low-income pregnant women. This study will help determine whether removing barriers to use of quality-of-care data impacts women’s choice of pediatrician. If navigators are found to be effective in this study, patient navigator effectiveness in settings with other vulnerable populations may be warranted.

Trial status

Piloting of study materials, intervention, and navigator training are complete and we are actively enrolling patients.