Advertisement

BMC Health Services Research

, 18:941 | Cite as

Successful implementation of a combined learning collaborative and mentoring intervention to improve neonatal quality of care in rural Rwanda

  • Jennifer Werdenberg
  • Francois Biziyaremye
  • Merab Nyishime
  • Evrard Nahimana
  • Christine Mutaganzwa
  • David Tugizimana
  • Anatole Manzi
  • Shalini Navale
  • Lisa R. Hirschhorn
  • Hema Magge
Open Access
Research article
  • 100 Downloads
Part of the following topical collections:
  1. Health systems and services in low and middle income settings

Abstract

Background

Globally, neonatal mortality remains high despite interventions known to reduce neonatal deaths. The All Babies Count (ABC) initiative was a comprehensive health systems strengthening intervention designed by Partners In Health in collaboration with the Rwanda Ministry of Health to improve neonatal care in rural public facilities. ABC included provision of training, essential equipment, and a quality improvement (QI) initiative which combined clinical and QI mentorship within a learning collaborative. We describe ABC implementation outcomes, including development of a QI change package.

Methods

ABC was implemented over 18 months from 2013 to 2015 in two Rwandan districts of Kirehe and Southern Kayonza, serving approximately 500,000 people with 24 nurse-led health centers and 2 district hospitals. A process evaluation of ABC implementation and its impact on healthcare worker (HCW) attitudes and QI practice was done using program documents, standardized surveys and focus groups with facility QI team members attending ABC Learning Sessions. The Change Package was developed using mixed methods to identify projects with significant change according to quantitative indicators and qualitative feedback obtained during focus group discussions. Outcome measures included ABC implementation process measures, HCW-reported impact on attitudes and practice of QI, and resulting change package developed for antenatal care, delivery management and postnatal care.

Results

ABC was implemented across all 26 facilities with an average of 0.76 mentorship visits/facility/month and 118 tested QI change ideas. HCWs reported a reduction in barriers to quality care delivery related to training (p = 0.018); increased QI capacity (knowledge 37 to 89%, p <  0.001); confidence (47 to 89%, p <  0.001), QI leadership (59 to 91%, p <  0.001); and peer-to-peer learning (37 to 66%, p = 0.024). The final change package included 46 change ideas. Themes associated with higher impact changes included provision of mentorship and facility readiness support through equipment provision.

Conclusions

ABC provides a feasible model of an integrated approach to QI in rural Rwanda. This model resulted in increases in HCW and facility capacity to design and implement effective QI projects and facilitated peer-to-peer learning. ABC and the change package are being scaled to accelerate improvement in neonatal outcomes.

Keywords

Quality improvement Health care system Children Quality culture Developing countries 

Abbreviations

ABC

All Babies Count

ANC

Antenatal Care

HCW

Healthcare Worker

HMIS

Health Management Information System

MOH

Ministry of Health

PNC

Postnatal Care

QI

Quality Improvement

Background

Despite declines in under-five mortality in Rwanda and globally, neonatal mortality remains high [1]. Contributors to neonatal mortality include poor coverage and quality delivery of evidence-based interventions known to improve maternal and newborn outcomes [2]. To date, approaches to improve delivery of these interventions have included improving individual provider skills through mentoring and skills-training; health systems strengthening to ensure availability of essential equipment; and quality improvement (QI) interventions, such as learning collaboratives, to drive system change [3, 4].

A learning collaborative brings together QI teams from healthcare facilities to seek improvement in a focused topic area through QI methods and peer learning [5]. Collaboratives often create change packages, collections of high impact change ideas tested during a collaborative, which are used to accelerate quality improvement by spreading potential solutions to common challenges [6, 7, 8, 9, 10, 11]. The change package includes change concepts, grouping of similar change ideas into a broader conceptual categories [12].

Building on previous success of mentoring and learning collaboratives [11, 13], the Rwanda Ministry of Health (MOH) in partnership with Partners In Health (PIH), a US-based non-governmental organization, designed and implemented the All Babies Count Initiative (ABC) in 2013. ABC was an 18-month district-wide QI learning collaborative and mentorship effort designed to reduce neonatal mortality by improving health system performance and individual provider behavior to prevent newborn death.

ABC was implemented in two rural Rwandan districts which have been supported by the MOH-PIH partnership since 2005, with additional work starting in 2009 through a health systems strengthening project funded through the Doris Duke Charitable Foundation Africa Health Initiative [14]. The two districts serve a catchment population of approximately half a million and included 24 nurse-led health centers and two district hospitals at the start of ABC [15].

This paper reviews the implementation process and implementation outcomes of the ABC initiative including feasibility and fidelity, acceptability, self-reported changes in health care worker (HCW) attitudes and practice of QI, QI project implementation and the resulting change package.

Methods

The ABC intervention

ABC had three primary components. First, based on routine facility data on infrastructure and supplies, baseline gaps in essential neonatal care medical equipment were identified, and essential equipment was provided to close critical needs [16]. Second, utilizing PIH and MOH staff, ABC provided targeted clinical skills training on national neonatal care protocols which included essential newborn care, neonatal resuscitation (using Helping Babies Breathe); and advanced neonatal care for hospital staff. Third, these components were supported by a district-wide 18-month Learning Collaborative organized by ABC mentors with integrated onsite clinical and QI mentorship to improve HCW skills and support system-level QI projects in neonatal and maternal care. Additional program description details can be found elsewhere [17].

The learning collaboratives included four learning sessions during the active collaborative and a final “Harvest Session”. Between the learning sessions, facilities were visited monthly by an ABC mentor who provided individual clinical and team QI mentorship to support improvement in clinical care delivery and in QI project implementation to address identified gaps in the core processes of maternal/newborn health [antenatal care (ANC), delivery management, and postnatal care (PNC)]. ABC mentors were experienced nurses in neonatal care delivery with additional training PIH Pediatric Program Director in QI methods, learning collaborative facilitation, data collection and analysis and coaching. To facilitate sustainability, mentoring visits were conducted jointly with existing, district-based MOH mentors whenever possible.

QI teams were formed at each facility, composed of 3–4 strategic team members, usually including a maternity, antenatal, or neonatal charge nurse, the community health officer, and the facility data manager. QI teams were supported to review their performance in eight core indicators and develop QI projects based on the results. QI projects were designed to test specific interventions (change ideas), using standard “Plan, Do, Study, Act” (PDSA) cycles based on the Model for Improvement [18].

Core learning collaborative indicators were chosen through a combination of literature review, expert consultation and review of data available in the Rwandan National Health Management Information System (HMIS) (Table 1).
Table 1

ABC Learning Collaborative Core Indicators

Care Domain

ABC Learning Collaborative Indicators

Data Source

Antenatal care

Percent of women completing four standard antenatal care visits (ANC)

HMISa

Delivery

Number of babies with birth asphyxia

HMIS

Percent of preterm births in which women received antenatal steroids

non-HMIS

Percent of prolonged rupture of membranes in which women received antibiotics

non-HMIS

Percent of births at which a skilled birth attendant was available

HMIS

Time to cesarean section (hospital only)

non-HMIS

Postnatal care

Percent of infants receiving immediate skin-to-skin

HMIS

Percent of infants checked for danger signs within 24 h of delivery

HMIS

a HMIS Health Management Information System

Core indicators included those available from routinely reported data (HMIS) and ones which required review of facility registers (non-HMIS). Non-HMIS indicators were extracted from registers by the mentors and facility QI teams. HMIS indicators were extracted from existing reports with data validation done with facility QI teams by comparing HMIS reports with facility registers. Indicators were collected retrospectively at baseline (minimum 3 months prior to QI project start for non-HMIS indicators and 6 months for HMIS indicators) and monthly during the 18-month implementation. Data were entered into excel databases for review with facility teams, programmatic monitoring and QI activity tracking. A change package for the three care domains of ABC was created at the end of the collaborative based on quantitative and qualitative data collected from the QI projects and participants.

Data sources

Collaborative implementation

Implementation data were collected using routine program monitoring tools. These data included: learning session attendance, duration, and frequency; mentor visit content and frequency; and QI team characteristics.

Healthcare worker attitudes and practice of quality improvement

Participant surveys were completed before the first learning session and after the Harvest Session with shorter surveys before other learning sessions. The surveys were adapted from previously used surveys in other quality improvement programs. They included answers on a 5 item Likert scale and open-ended questions used for immediate programmatic management. The scaled questions were included in this analysis. Questions included: confidence in QI methods, leadership, teamwork, data use, peer-to-peer learning, motivation, and work environment using a Likert scale as well as presence of barriers to care such as knowledge, time, and resources.

Change package

Change ideas generated and tested by facility teams were captured by mentors using a QI tracking tool. Quantitative pre-post mean analysis of change in the targeted measure was used to identify successful change ideas for inclusion in the change package. During the Harvest Session, each team was asked to rank all tested change ideas using a priority matrix which included: potential impact on health outcomes; volume of patients affected; time to impact; feasibility; and level of other support required. Results of the quantitative analysis and qualitative ranking were used to generate district-wide lists of higher and lower ranked change ideas in each care domain to guide focus group discussions.

During the Harvest Session, one focus group discussion was conducted on each care domain - ANC, PNC and delivery care - with ABC participants in each district for a total of 6 focus group discussions. Discussions were used to capture feedback on the prioritized change ideas and lessons learned to inform future ABC program design and scale. Focus groups were conducted in Kinyarwanda and French using a standardized guide by a focus group facilitator and two note-takers. Follow-up structured debriefs of the facilitator and note takers were conducted by a qualitative expert to extract key themes of the discussions.

Analysis

Collaborative implementation

We assessed the fidelity and completeness of the ABC initiative implementation comparing key activities including mentor visit frequency, site participation and QI activities with the program design. Qualitative data on facilitators and challenges to ABC success were also collected through the focus groups described above.

HCW attitudes and practice of quality improvement

Learning session surveys were entered into a database using EpiInfo version 7.1.5. Individual surveys with < 20% of questions completed were classified as incomplete and excluded. Likert scale questions responses were converted into dichotomous variables (4 or 5 versus < 3). Descriptive statistics were used with significance testing using chi-squared or two-tailed t-tests to measure difference between the baseline (pre-learning session) and endpoint (Harvest Session) for dichotomous and continuous variables respectively. Missing responses to individual questions were excluded and results with a p <  0.05 were considered significant. Quantitative analyses were conducted in Stata v14 (College Station, TX: StataCorp LP).

Change package

Quantitative success of QI projects was defined as significant change (p <  0.05) from the mean facility baseline versus QI project endpoint for the targeted core indicator using two-tailed t-tests.

Qualitative success for QI projects was based on QI team priority matrix rankings of change ideas as noted above and on the thematic coding of the focus group discussions. Both deductive and inductive approaches were used to determine underlying themes. To reduce reporting bias, the codes extracted from the interviews were validated by an expert in ABC implementation (ABC mentor) and by an expert in qualitative analysis.

Qualitative and quantitative data were subsequently integrated to determine which QI projects and change ideas warranted inclusion in the change package using rules summarized in Table 2.
Table 2

Rules for Determining Change Package Inclusion

Rule 1: All QI projects (which may contain multiple change ideas) demonstrating significant, positive pre/post mean change are included in the change package.

RULE 2: All QI projects that do not have data available for pre/post mean analysis (small ‘n’ or rare occurrence) but were identified through the priority matrix and subsequently verified by focus groups as important change ideas are included in the change package.

RULE 3: QI projects which were identified by the priority matrix and focus group as high impact, but quantitative data is available and does not demonstrate impact are excluded from the change package.

RULE 4: If there are QI projects (ANC, postnatal, delivery) that contain multiple change ideas, implemented simultaneously and leading to improvement in a core indicator, then qualitative data was used to determine the relative merits of the different change ideas introduced. If a given change idea was identified by focus group participants as low impact then this idea is excluded from the change package.

Results

During the collaboratives, learning sessions were conducted every 3–5 months with an average of 40 participants per session and 98% of all health facilities represented at each session. Each health facility received, on average, 0.76 mentorship visits per month (range 0.6–0.9), slightly less than the planned 1 visit per month. Clinical trainings were implemented on average every 12.5 months with > 92% of facilities achieving the goal of retaining ≥2 trained nurses at the end of ABC (Table 3).
Table 3

ABC Learning Collaborative vs Typical Learning Collaborative Implementation Comparison [7, 10, 17]

Characteristic

ABC Learning collaborative

Common Learning Collaborative Design Components

Focus

Neonatal Mortality

Single clinical subject

# and type of Indicators addressed

8 indicators focused on drivers of neonatal mortality

1–9 indicators focused clinical subject

Duration of intervention

18 months

12–24 months

# people on QI team

3–4 people

3–7 people

Composition of QI team

Community health worker, nurse, data manager

+/−leadership

Multi-professional team

Learning Session Characteristics

2 days

1–3 days

4 learnings sessions

+ Harvest Session

> 2 sessions

Every 3–5 months

every 4–6 months

Average % health facilities represented at all LS

98%

N/A

Average total # LS attendees

40.7

30-40people

Inclusion of Clinical Training or Skills Building

every 12.5 months

not standard

% HC with at least 2 nurses trained in newborn training package

> 92% throughout collaborative

not standard

Frequency of assessment for essential equipment

every 3 months

not standard

Mentor Characteristics

nurses with QI training

QI experts

Average mentor visits per facility per month

0.76 visits/month

Mentoring should occur between Learning Sessions

Monthly mentor visit content

QI project and QI skills mentorship

QI project and QI skills mentorship.

Clinical skills mentorship and observation checklists for [1] Labor and delivery [2] neonatal care and/or [3] postpartum care

not standard

routine equipment/commodities assessment and support

not standard

HCW attitudes and practice of quality improvement

Self-reported QI capacity increased significantly by the end of the collaborative. This included increase in self-rated QI knowledge (37% vs 89%, p <  0.001), confidence (47% vs 89%, p <  0.001) and leadership (59% vs 91%, p <  0.001) (Table 4). QI team members also reported an increase in being asked for advice to improve neonatal care at the health facility (64 to 93%, p <  0.001).
Table 4

Results from Participant Survey Pre and Post Learning Collaborativea, b

 

Pre-ABC LC

Post-ABC LC

P-value

Number participants

71

67

 

Percent rating QI Knowledge and Confidence 4 or 5 scale 1 (little/none) to 5 (very/extremely)

 QI Knowledge

24/65 (37%)

58/65 (89%)

<  0.001

 Confidence in helping to improve quality at your clinic or department/district

30/64 (47%)

58/65 (89%)

< 0.001

 Confidence in leading QI at your clinic

34/58 (59%)

61/67 (91%)

< 0.001

Percent rating Leadership as very Interested in QI (scale 1 (not at all) to 3 (very interested)).

 Facility management interest in measuring and improving quality for neonatal patients

43/68 (63%)

63/66 (95%)

< 0.001

 Facility management interest in hearing ideas from you and other staff for QI?

38/67 (57%)

43/64 (67%)

0.2

Participant motivation at work (percent always or often) (scale never, sometimes, often always)

 My work is rewarding

17/63 (29%)

18/63 (27%)

0.84

 My work is stressful

4/65 (6%)

3/63 (5%)

1.0

 I feel emotionally drained by my work

53/63 (84%)

43/60 (72%)

0.09

 I feel isolated in my work

4/66 (6%)

0/64 (0%)

1.0

 I can help my clients

62/65 (97%)

65/67 (98%)

0.62

 I have confidence in my skills

63/64 (98%)

61/63 (97%)

0.62

 I am motivated to perform well in my job

42/67 (63%)

35/63 (55%)

0.35

Working Environment (percent very good or Excellent) (scale poor, fair, good, very good, excellent)

 Maternity/neonatal services at my clinic/hospital are excellent

31/59 (53%)

23/64 (36%)

0.06

 I have enough basic clinical equipment and supplies to provide good care and services to patients

41/62 (66%)

51/61 (84%)

0.03

 I have adequate support and information to help make clinical decisions

53/67 (79%)

41/57 (72%)

0.35

 There is enough space for me to be able to provide care for the patients and ensure privacy

45/66 (68%)

44/61 (72%)

0.42

 The clinic has a very good environment (friendliness, teamwork, respect, lack of chaos)

44/63 (70%)

47/63 (75%)

0.79

Peer to Peer learning

N (% yes)

N (% yes)

 

 Since the last LS, have you heard any ideas or asked for help from the other participating HC/hospitals?

17/47 (36%)

40/60 (66%)

0.024

Team work

 Have you been involved in activities that look at the quality of neonatal care and work to improve problems at your site?

36/64 (56%)

55/63 (87%)

< 0.001

 Do staff members work together to improve quality of care for neonatal patients?

59/63 (93%)

61/64 (95%)

0.72

 Have you been asked for your input how to improve neonatal care?

43/67 (64%)

59/63 (93%)

< 0.001

Data Utilization

 Have you seen or used any routine reports for measuring the quality of care?

32/63 (50%)

37/62 (59%)

0.32

Identified barriers to delivery of quality care at their site (Yes/No)

n = 71

N = 67

 

 High number of patients

29 (40%)

34 (50%)

0.24

 Number of staff

30 (42%)

27 (40%)

0.32

 Social or economic problems in patients’ lives (getting to clinic, food, etc)

30 (42%)

21 (31%)

0.18

 Training of staff

28 (39%)

14 (20%)

0.018

 Communication between providers and patients

15 (21%)

9 (13%)

0.23

 The staff does not work together as a team

14 (19%)

5 (7%)

0.03

 Complexity of care (severity of illness, complicated pregnancy, etc)

8 (11%)

5 (7%)

0.56

 Clinic flow

7 (9%)

6 (8%)

0.86

a “Pre” data obtained at the start of the first learning session. “Post” data obtained at the start of the “Harvest Session”

b Missing data for a specific question were excluded

Other significant improvements included QI team member engagement in activities to measure the quality of neonatal care (56 to 87%, p <  0.001) and peer-to-peer learning (36 to 66%, p = 0.024). Surveys showed no significant change in motivation at work or use of routine reports (50 to 59%, p = 0.32).

Reported leadership interest in measuring and improving quality of care increased during ABC (63 to 95%, p <  0.001), although leadership interest in hearing health care worker input on QI remained unchanged (57 to 67% p = 0.2).

In addition to self-reported increases in peer collaboration, 61% of change concepts from the change package were implemented successfully by more than one facility. Focus group discussions revealed that the project also fostered intra-facility collaborations: “through [QI teams] implementing their QI projects, it made more services within the health facilities develop their own projects.” This led to the spread of QI beyond ABC core indicators to lab services, pharmacy, vaccination, family planning and vitamin K administration.

Perception of availability of adequate equipment to provide care and services increased (66 to 84%, p = 0.03). However, although perception of training as a barrier improved (39 to 20% p = 0.018) the top four reported barriers to quality care delivery remained unchanged, including: high patient volume, inadequate staffing, socioeconomic challenges of patients, and staff training.

ABC change package

Facilities initiated a total of 52 QI projects spanning all 3 care domains, testing 118 change ideas. Of the ideas tested, 63% were in ANC, 27% were in delivery management, and 10% were in PNC (Table 5). Six of the eight core indicators were addressed. No projects explicitly address asphyxia (which was covered during clinical trainings) and due to changes in documentation, QI projects promoting placing newborns skin-to-skin with mothers immediately after delivery which were implemented early in ABC were not captured by mentors in the QI tracking templates. Forty-six (38.9%) change ideas were determined to be high impact through quantitative or qualitative criteria and were summarized into 17 change concepts for the change package (Table 6).
Table 5

Total QI Projects, Change Ideas and Care Domain Associated High Impact Themes

 

Antenatal Carea

Delivery Care

Postnatal Care

N (%) QI Projectsb

23 (44%)

17 (33%)

12 (23%)

N (%) Change Ideasc

74 (63%)

32 (27%)

12 (10%)

Change package associate themes

 Integration of services

 Teamwork & communication

 Mentorship & Clinical Teaching

 Data utilization

 Importance of leadership

 

 Essential equipment

 

a Check mark indicates theme coded focus group debrief

b QI Project: Goal set for improvement in targeted care area

cChange Idea: Specific intervention planned to achieve QI Project goal

Table 6

All Babies Count Change Package

Quality Indicator Targeted

Care Gap

Change Concept

Related Qualitative theme from focus groups

Facilities successfully implementing

QI PROJECT: IMPROVING ANTENATAL CARE SERVICES (# Facilities attempting n = 23)

 4 Antenatal care visits

Low 4 ANC visit completion because women miss 1st ANC appointment

Test women for pregnancy in all departments and transfer for ANC enrollment or same day ANC care if pregnant.

Integration of services

Teamwork & Communic-ation

7

Low community awareness of importance of ANC visits

Increase community awareness of ANC importance by educating women in waiting rooms of the health facility

Teamwork & Communic-ation

2

Engage health center leadership to provide ANC care and demonstrate importance of ANC to the community.

Importance of Leadership

1

Engage community leadership to help CHWs emphasize the importance of antenatal care in monthly community meetings

Importance of Leadership

1

Health Centers too far for women to reach

Decentralize ANC services to Health Posts on a regular basis

 

4

No mechanism to follow up women who miss appointments

Make (a) a filing system of medical records or (b) a register modification to facilitate identification of women who miss appointments for outreach by CHWs

Teamwork & Communica-tion

3

No mechanism to remind women of up-coming appointments

Have CHWs remind women who have upcoming appointments.

Teamwork & Communica-tion

1

Women cannot attend ANC clinic on the day offered

Offer ANC care at the health center more frequently (ranges from 2 times per week to daily) and at times that coordinate with community activities (such as market day)

 

4

Women don’t come for ANC because partner is not available.

See women for 1st ANC and send her with invitation for her partner to attend following visit

Mentorship &Training

2

QI PROJECT: IMPROVING DELIVERY CARE SERVICES (# Facilities attempting n = 17)

 Time to C-section (District Hospitals only)

Poor communication and coordination between maternity and neonatal services

Set aside a regularly scheduled time for collaboration between neonatal and maternity services

Teamwork & Communication

2

 Appropriate administration of steroids & antibiotics for preterm labor management

Low case identification of women in active preterm labor

Define scope of problem at health center by comparing preterm infants recorded versus women in preterm labor identified

Data utilization

5

Refresher trainings for staff on calculation of gestational age and management of PPROM in order to improve recognition of labor complications & management

Mentorship & Training

4

Medical Staff forget that steroids and/or antibiotics may be indicated in preterm birth

Modify existing maternity registers to prompt appropriate management

Integration of services

Teamwork & Communication

2

Antibiotics not given because not available

Have nurses proactively checking on supply of steroids available.

Essential equipment

1

Facility-based delivery

Women not deliver at the facility because inadequate anticipatory planning

Assist mothers with anticipatory planning of items to have prepared to bring for delivery at their 3rd ANC appointment

Teamwork & Communication

1

QI PROJECT: IMPROVING POSTNATAL CARE SERVICES (# Facilities attempting n = 12)

 Checking infants for Danger Signs within 24 hours

Low maternal knowledge & nurse vigilance in checking for neonatal danger signs

Make checking for danger signs part of the maternity register and assign the filling of the register as a daily nursing responsibly to prompt staff to educate the mother and check the newborn for Danger Signs

Integration of services

5

Short hospital stays (often same day discharge) preclude staff from checking danger signs

When women come to get child’s BCG vaccination makes sure they are also screened for neonatal danger signs

Integration of services

1

Successful change concepts included improving access to and convenience of ANC services, improving community engagement and awareness of ANC service importance, and leadership engagement. Interventions targeting delivery included improving coordination of care to reduce time to caesarian section, preterm labor management included refresher trainings, collaborations with pharmacy to ensure stock of necessary medications and gestational age calculation training and verification. QI projects focused on PNC included danger sign recognition within 24 h of delivery and return for postnatal check-up after discharge. Successful PNC change concepts included patient register modification to act as a clinical reminder for HCW to check mother-baby dyad and integration of postnatal check-ups into routine vaccination services. Participants identified areas which could not be addressed by the ABC scope such as “retaining patients for 72 hours [for PNC care] was difficult and unsuccessful because of high volume of patients. They don’t have enough rooms, staff, or beds.”

Table 5 describes six general themes of high impact projects identified by focus group participants. One theme was the importance of the integration of services. For example, an intervention to improve ANC visit attendance was reported as working well because it “integrate[d] the tracking of women who tested positive for pregnancy when they came in to other services.” Similarly, PNC service attendance was improved by taking a “strategy to attract more mothers [to post-natal services by getting] them when they came for vaccination clinics.” Projects were noted to be more sustainable when there was integration, “... between health system, local leaders and community health workers.”

Teamwork and communication were also identified as important themes for success across all care domains. “[Teamwork] helps make a project be successful. Working together and learning from each other are what make projects successful.” Leadership support emerged as another key element to high impact project implementation with debriefs mentioning that “Local leadership collaboration allowed for QI projects to be successful.” Integrated clinical and QI mentorship were also identified as important facilitators of high impact QI projects. “The mentors would come to health facilities while they worked and correct them on the job. They would see where they needed more tools or supplies…this would keep the participants active and working on their QI projects.”

Clinical training where the “staff gained skills through (ABC) to implement QI projects” on key practices between or during learning sessions was also reported as valuable. Availability of essential equipment was also noted to be key to the success, as “equipment and materials were a problem at in the beginning of the QI projects but then they received the materials from ABC”. Finally, participants reported the importance of data utilization to QI efforts “[the ability to do] data monitoring in order to track their progress in implementation” was key to the execution of high impact projects.

Discussion

The ABC initiative successfully integrated clinical and QI mentorship with a learning collaborative and targeted training and equipment support designed to improve newborn care in district health care facilities in rural Rwanda. ABC was implemented with strong fidelity and acceptability to participants, resulting in wide-spread QI activities at the health center and district hospital levels. The initiative also increased district-wide QI capacity and peer-to-peer learning. The program success was accomplished by engaging individuals and teams through mentorship and learning sessions to improve individuals’ and system’s ability to create, implement and spread QI projects across the district.

Our finding that a multi-faceted intervention centered around a learning collaborative was an effective model in building QI capacity and practice improvement was similar to Franco’s findings from a review of learning collaboratives in resource limited settings [5]. Quality delivery of evidence-based interventions is dependent on healthcare worker skills and availability of supplies, especially in resource limited settings [19, 20, 21]. However, studies have found that addressing facility readiness in isolation does not lead to improved delivery of high quality care. ABC’s explicit attention to essential equipment at the start of the project and baseline training in essential care practices to address potential barriers of knowledge and skills was identified as contributing to the successful implementation and acceptability of ABC in addition to the ongoing mentoring and QI. These findings emphasize the importance of comprehensive interventions, such as ABC, which have the ability to address facility readiness as well as continuous learning through on-going training and effective supportive supervision or mentoring [22, 23, 24, 25, 26].

The success of ABC in building QI capacity was evident in the reported increase in QI confidence and the range and scope of change ideas implemented. Additionally, work to increase data use for performance measurement, a key step of PDSA cycles, was emphasized in some of the change ideas. For example, maternity and neonatal registers were compared to identify cases of inadequate preterm labor screening, and stock checks for essential medications were integrated into routine activities.

Change packages generated through collaboratives are an important product for spreading change [27]. However, there is little consistency or description of how change packages are created [6] with previous collaboratives in LMIC settings describing a range of methods. Project Fives Alive! in Ghana took a strictly quantitative approach using run chart rules [28]. Others, such as projects supported by the USAID ASSIST [12, 29, 30] describe a more mixed-methods approach to change package creation. We chose a mixed-methods approach to incorporate front line HCWs and identify projects which may not be seen through quantitative analysis, but which were feasible, high impact and applicable to the setting. This addition of qualitative data was particularly relevant when events were rare so statistical significance was unlikely to be reached, yet had potentially critical impact on patient outcomes (i.e. antibiotics for prolonged rupture of membranes), or the number of data points too few to use run charts or statistical process control charts, other methods commonly used in QI [31].

Our study had a number of limitations. Changes in capacity and peer-to-peer learning were solely based on HCW self-report. We also could not analyze change at the individual level since some providers’ attending of the learning sessions changes over time due to staff turnover. We also did not have any counterfactual evidence, so could not prove that changes in QI capacity and activities were due to the ABC intervention nor which components of ABC were most important in driving those changes. However, we do know that no other independent neonatal-focused or other QI initiatives were newly active in the districts during ABC.

The quality of the data and documentation likely improved over the course of ABC implementation, potentially contributing to some of the measured improvement associated with change ideas. We also did not have sufficient data subgroups at the change idea level for the application of run chart rules. Therefore, we used pre-post means of project level measures and included qualitative assessment of change ideas. Impact on quality and cost-effectiveness are not included and will be published separately.

Conclusion

Incorporating clinical mentorship and facility readiness support in the targeted areas into district-wide learning collaboratives was a feasible and effective strategy to support the development of a culture and capacity for QI in rural districts in Rwanda. Including evaluation of implementation contributed to the ability to proceed with scale-up of the intervention including timely application of the change package in Rwanda. Final analysis of ABC impact on care outcomes and sustainability one year post-ABC is underway.

Notes

Acknowledgements

We would like to thank the dedicated mentors without whom this work would not have been possible and the health care providers whose dedication to improving the care and lives of the women and children in their districts is inspirational. We would also like to thank the Institute for Healthcare Improvement and the Ghana Project Fives Alive staff for their advice and input during the design phase.

Funding

This work was supported by a grant from the Doris Duke Charitable Foundation Africa Health Initiative and Partners In Health/Inshuti Mu Buzima. The funder (Doris Duke Charitable Foundation Africa Health Initiative) had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Availability of data and materials

The data that support the findings of this study are available from Partners in Health but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of Partners in Health.

Authors’ contributions

JW, LRH, SN, and FB interpreted and analyzed quantitative and qualitative data associated with this study. JW, FB, HM, LRH, EN, CM, MA, SN, MN, and DT participated in key design, implementation and data collection associated with ABC. JW, FB, LRH and HM were major contributors to writing of the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate

The study was approved by the Institutional Review Board of Brigham and Women’s Hospital Boston, MA and the Rwanda National Ethics Committee Kigali, Rwanda. All participants completed written informed consent.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.
    Victora CG, Requejo JH, Barros AJ, Berman P, Bhutta Z, Boerma T, et al. Countdown to 2015: a decade of tracking progress for maternal, newborn, and child survival. Lancet. 2016;387(10032):2049–59.CrossRefGoogle Scholar
  2. 2.
    Edmond KM, Quigley MA, Zandoh C, Danso S, Hurt C, Agyei SO, et al. Aetiology of stillbirths and neonatal deaths in rural Ghana: implications for health programming in developing countries. Paediatr Perinat Epidemiol. 2008;22(5):430–7.CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Das J, Gertler PJ. Variations in practice quality in five low-income countries: a conceptual overview. Health Aff (Millwood). 2007;26(3):w296–309.CrossRefGoogle Scholar
  4. 4.
    Board on Global Health, Institute of Medicine, National Academies of Sciences, Engineering, and Medicine. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Buckley GJ, Pittluck RE, editors. Washington, D.C.: National Academies Press; 2015. Available from: http://www.nap.edu/catalog/21736. [cited 2016 Jul 1]
  5. 5.
    Franco LM, Marquez L. Effectiveness of collaborative improvement: evidence from 27 applications in 12 less-developed and middle-income countries. BMJ Qual Saf. 2011;20(8):658–65.CrossRefPubMedGoogle Scholar
  6. 6.
    Sorensen AV, Bernard SL. Accelerating what works: using qualitative research methods in developing a change package for a learning collaborative. Jt Comm J Qual Patient Saf. 2012;38(2):89–95.CrossRefPubMedGoogle Scholar
  7. 7.
    Øvretveit J, Bate P, Cleary P, Cretin S, Gustafson D, McInnes K, et al. Quality collaboratives: lessons from research. Qual Saf Health Care. 2002;11(4):345–51.CrossRefPubMedGoogle Scholar
  8. 8.
    Plsek PE. Collaborating across organizational boundaries to improve the quality of care. Am J Infect Control. 1997;25(2):85–95.CrossRefPubMedGoogle Scholar
  9. 9.
    Kilo CM. Improving care through collaboration. Pediatrics. 1999;103(Supplement E1):384–93.PubMedGoogle Scholar
  10. 10.
    Wilson T, Berwick DM, Cleary PD. What do collaborative improvement projects do? Experience from seven countries. Jt Comm J Qual Patient Saf. 2003;29(2):85–93.Google Scholar
  11. 11.
    Singh K, Speizer I, Handa S, Boadu RO, Atinbire S, Barker PM, et al. Impact evaluation of a quality improvement intervention on maternal and child health outcomes in northern Ghana: early assessment of a national scale-up project. Int J Qual Health Care. 2013;25(5):477–87.CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    MaNHEP Change Package for Improving Community Support Processes for Maternal and Newborn Health | USAID ASSIST Project. [cited 2016 Jun 26]. Available from: https://www.usaidassist.org/resources/manhep-change-package-improving-community-support-processes-maternal-and-newborn-health
  13. 13.
    Magge H, Anatole M, Cyamatare FR, Mezzacappa C, Nkikabahizi F, Niyonzima S, et al. Mentoring and quality improvement strengthen integrated management of childhood illness implementation in rural Rwanda. Arch Dis Child. 2015;100(6):565–70.CrossRefPubMedGoogle Scholar
  14. 14.
    Drobac PC, Basinga P, Condo J, Farmer PE, Finnegan KE, Hamon JK, et al. Comprehensive and integrated district health systems strengthening: the Rwanda population health implementation and training (PHIT) partnership. BMC Health Serv Res. 2013;13(Suppl 2):S5.CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    National Institute of Statistics of Rwanda (NISR) [Rwanda], Ministry of Health (MOH) [Rwanda], and ICF International. Rwanda demographic and health survey 2010. Calverton: NISR, MOH, and ICF International; 2012.Google Scholar
  16. 16.
    Iyer HS, Kamanzi E, Mugunga JC, Finnegan K, Uwingabiye A, Shyaka E, et al. Improving district facility readiness: a 12-month evaluation of a data-driven health systems strengthening intervention in rural Rwanda. Glob Health Action. 2015;1:8 Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4490804/. [cited 2017 May 8].Google Scholar
  17. 17.
    Magge H, Chilengi R, Jackson E, Wagenaar B, Kante A. Tackling the hard problems: implementation experience and lessons learned in newborn health from the African health initiative. BMC Health Serv Res. in press.Google Scholar
  18. 18.
    Institute for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement, IHI Innovation Series white paper. Boston: Institute for Healthcare Improvement; 2003. Available on http://www.ihi.org/resources/Pages/IHIWhitePapers/TheBreakthroughSeriesIHIsCollaborativeModelforAchievingBreakthroughImprovement.aspx.
  19. 19.
    Marquez L. Helping healthcare providers perform according to standards. Oper Res Issue Pap. 2001;2(3):1–34.Google Scholar
  20. 20.
    Nolan T, Angos P, Cunha AJ, Muhe L, Qazi S, Simoes EA, et al. Quality of hospital care for seriously ill children in less-developed countries. Lancet. 2001;357(9250):106–10.CrossRefPubMedGoogle Scholar
  21. 21.
    Das J, Holla A, Das V, Mohanan M, Tabak D, Chan B. In urban and rural India, a standardized patient study showed low levels of provider training and huge quality gaps. Health Aff (Millwood). 2012;31(12):2774–84.CrossRefGoogle Scholar
  22. 22.
    Hermida J, Robalino ME. Increasing compliance with maternal and child care quality standards in Ecuador. Int J Qual Health Care. 2002;14(suppl 1):25–034.CrossRefPubMedGoogle Scholar
  23. 23.
    Kelley E, Geslin C, Djibrina S, Boucar M. Improving performance with clinical standards: the impact of feedback on compliance with the integrated Management of Childhood Illness algorithm in Niger, West Africa. Int J Health Plann Manag. 2001;16(3):195–205.CrossRefGoogle Scholar
  24. 24.
    Rowe AK, Hamel MJ, Flanders WD, Doutizanga R, Ndoyo J, Deming MS. Predictors of correct treatment of children with fever seen at outpatient health facilities in the Central African Republic. Am J Epidemiol. 2000;151(10):1029–35.CrossRefPubMedGoogle Scholar
  25. 25.
    van Lonkhuijzen L, Dijkman A, van Roosmalen J, Zeeman G, Scherpbier A. A systematic review of the effectiveness of training in emergency obstetric care in low-resource environments: systematic review of training in obstetric emergencies. BJOG Int J Obstet Gynaecol. 2010;117(7):777–87.CrossRefGoogle Scholar
  26. 26.
    Delaney MM, Maji P, Kalita T, Kara N, Rana D, Kumar K, et al. Improving adherence to essential birth practices using the WHO safe childbirth checklist with peer coaching: experience from 60 public health facilities in Uttar Pradesh. India Glob Health Sci Pract. 2017;5(2):217–31.CrossRefGoogle Scholar
  27. 27.
    Nembhard IM. Learning and improving in quality improvement Collaboratives: which collaborative features do participants value Most? Health Serv Res. 2009;44(2 Pt 1):359–78.CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20(1):46–51.CrossRefPubMedGoogle Scholar
  29. 29.
    Kivwanga M, Chome I. Change package for improving the quality of antenatal care services and skilled deliveries in Kwale, Kenya. Technical report. Published by the USAID ASSIST project. Bethesda: University Research Co., LLC (URC); 2014.Google Scholar
  30. 30.
    Change Package for Quality Improvement in Orphans and Vulnerable Children Programmes in Kenya | USAID ASSIST Project. [cited 2016 Jun 26]. Available from: https://www.usaidassist.org/resources/change-package-quality-improvement-orphans-and-vulnerable-children-programmes-kenya
  31. 31.
    Provost LP, Murray S. Wiley: The Health Care Data Guide: Learning from Data for Improvement. Hoboken: Wiley. Available from: http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470902582.html. [cited 2017 Oct 31]

Copyright information

© The Author(s). 2018

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors and Affiliations

  • Jennifer Werdenberg
    • 1
    • 2
    • 3
  • Francois Biziyaremye
    • 1
  • Merab Nyishime
    • 1
    • 4
  • Evrard Nahimana
    • 1
  • Christine Mutaganzwa
    • 1
  • David Tugizimana
    • 1
  • Anatole Manzi
    • 1
    • 5
  • Shalini Navale
    • 6
  • Lisa R. Hirschhorn
    • 7
  • Hema Magge
    • 1
    • 2
    • 8
    • 9
  1. 1.Partners In Health/Inshuti Mu BuzimaRwinkwavuRwanda
  2. 2.Boston Children’s HospitalBostonUSA
  3. 3.Harvard TH Chan School of Public HealthBostonUSA
  4. 4.University of Global Health EquityBostonUSA
  5. 5.University of Rwanda School of Medicine and Health SciencesKigaliRwanda
  6. 6.Widener University Center for Human and Sexuality Studies, One University PlaceChesterUSA
  7. 7.Northwestern University Feinberg School of MedicineChicagoUSA
  8. 8.Brigham and Women’s HospitalBostonUSA
  9. 9.Institute for Healthcare ImprovementCambridgeUSA

Personalised recommendations