Keywords

Background

From 2007 to 2014, the two USAID-funded projects worked with the Ministry of Health (MOH) of Guatemala to strengthen maternal, neonatal, and child health (MNCH) and nutritional care. Both projects relied heavily on the use of quality improvement collaboratives to improve health-care services and SBCC. A quality improvement collaborative consists of a large number of improvement teams from multiple health-care facilities working together and meeting often to promote mutual learning and data exchange in a specific area of care to trigger even more rapid health-care improvement (USAID Health Care Improvement Project 2008). Teams in the MNCH collaboratives in Guatemala conducted PDSA cycles to test process changes to improve compliance with health care and SBCC standards.

While the collaboratives produced recognizable results in facility-based clinical care during the projects’ initial stages, project leaders, stakeholders, and participants realized that most deliveries and maternal deaths occurred in the communities – outside the clinical care setting. They, therefore, decided that the way forward was to focus on improving MNCH-related knowledge and skills within communities by improving: pregnant women’s recognition of danger signs; individual, family, and community preparation of emergency plans; traditional midwives’ participation in referrals; and culturally appropriate, patient-centered delivery services. Project leaders, subsequently, looked to develop community collaboratives that focused on using QI to improve SBCC interventions.

To do this, the projects had to take into account the organization of the health-care system , the number and type of health personnel working at the primary level (see Box 3.1), and the nature of the SBCC strategies that were being implemented at the time. Among the many channels in which SBCC interventions are carried out, interpersonal communication and counseling play a particularly critical role. Project leaders and participants realized that the data yielded by current SBCC documentation left them unable to measure the quality of counseling and, more importantly, its effects on pregnant women’s knowledge and practices. For example, while clinical teams had added checklists in their registers as reminders or ways to measure quality of care indicators, such as development of a birth/emergency plan, provision of postpartum family planning counseling, and delivery of communication materials to clients, these forms did not provide the evidence needed to ascertain quality or measure results. In addition, although this type of counseling was also conducted in home visits, no records were kept of counseling provided during home visits, and results were never measured or analyzed by auxiliary nurses and community health workers.

Box 3.1 Snapshot: The Health Care System in Guatemala

From 1996 to 2014, primary health-care services in Guatemala were provided in health posts directly managed by the MOH, a few health facilities were supported by municipalities, and community centers were managed by NGOs. The NGOs operated in the Extension of Coverage Program, which was part of the government’s primary health-care program, under contracts with the MOH. In the health posts and the community centers, a single auxiliary nurse provided health care, and occasionally, a professional nurse or a last-year medical student provided health care in health posts. In the community, mostly in areas covered by the Extension of Coverage Program, auxiliary nurses, rural health technicians, and health educators were in charge of health promotion and SBCC activities. In some places, volunteers or health promoters and mother counselors supported by different NGOs or projects also participated.

Designing the Improvement Effort

After a first community-based collaborative proved critical in effecting changes to maternal health knowledge and practices in a demonstration area, the project introduced, in 2012, a second community-based collaborative in 166 health posts and 429 community centers. The initiative focused on improving processes of care and nutritional results during the first 1000 days of life (from pregnancy to the child’s second birthday). They were able to introduce the collaborative at a large scale because leaders promoted the community effort as one that would build upon earlier successes with clinical and community collaboratives.

All improvement starts with an aim. The primary objective of the second large-scale community-based collaborative was to improve the effectiveness of interpersonal communication and counseling and other SBCC strategies in order to increase the health knowledge of women and their families and achieve change in 19 health and nutrition-related behaviors. These behaviors were the core of the SBCC strategy and the focus of a strategic counseling tool known as the “Wheel of Practices for Better Living”Footnote 1 that was introduced by the project to promote maternal health and prevent chronic malnutrition (Fig. 3.1).

Fig. 3.1
figure 1

The wheel of practices for better living

Full size image

To generate interest in expanding the QI collaborative’s focus on SBCC for MNCH , the project initially used a local advocacy approach. Project leaders worked with social workers, rural health technicians, and health educators from the MOH Health Promotion and Education Department, the mandate of which was to implement SBCC interventions. Project leaders showed MOH staff how information and data on health promotion activities and behavioral outcomes was critical to improving their ability to communicate with and interact with clients in the community. Further, they sought to demonstrate how the QI methods being used to improve clinical health care could be used to improve community SBCC. To garner buy-in from the central-level MOH, the project invited the authorities to the health districts’ quarterly learning sessions, where health-care improvement data on selected process indicators for maternal and neonatal care were presented and discussed by primary health-care worker teams. Conducted as celebratory events with activities to recognize and highlight accomplishments, the learning sessions proved effective in instilling enthusiasm for QI work. In addition, the sessions provided an opportunity for the MOH Health Promotion and Education Department officials to learn about QI methods that could be used to support SBCC.

After MOH officials observed and participated in sessions and talked with participants and project staff, they began to envision how applying QI methodology could support health promotion. Support for using improvement methods to strengthen SBCC messaging and campaigns grew among officials of the department as they became convinced of the effectiveness of this approach. Their advocacy for this work was critical to the ability of the project to work with health area technical teams on indicators, sampling, data collection instruments, analysis, coaching, and improvement interventions.

Formation of Collaborative Teams

The establishment of the 2012 community collaborative’s quality improvement teams was a participatory process. At first, the project attempted to form teams in each primary health-care facility. However, frequently there was only one auxiliary nurse in a facility, especially in health posts, which made collaborative learning difficult or impossible. Therefore, Municipal Health District authorities and project staff decided instead to create “supervision areas” that joined auxiliary nurses from contiguous health posts within each health district and all personnel in each nongovernmental organization (NGO) jurisdiction. These supervision areas typically had a population of 10,000 inhabitants who were expected to be covered by the health facilities in the area.

The project also envisioned the formation of two QI teams in each supervision area: one devoted to the improvement of clinical health care, led by the auxiliary nurses, and the other devoted to SBCC initiatives, led by the rural health technicians or the health educators. However, district authorities and health providers found this was not feasible for several reasons. First, in these settings, the auxiliary nurse was responsible not only for clinical care but for health promotion as well, and thus participated in SBCC initiatives. In addition, there were too few staff members in these rural areas, and many had very limited education. Learning to apply QI methods also requires a certain amount of training in data collection, recording, analysis, and planning in order to be implemented successfully; on top of this, there was a high number of coaching visits and learning sessions needed. As a result, project leaders strongly recommended assembling only one QI team in each supervision area, but having different actors lead an effort, depending on whether it was a clinical or community initiative.

In the end, the health districts defined 104 supervision areas,Footnote 2 38 of which were MOH jurisdictions, composed of 4–6 health posts each, and 66 of which were NGO jurisdictions, composed of 4–5 community centers. Supervision areas, rather than the municipal health center, became the locus of activity. Each one had a QI team with up to five members of different types of health personnel – auxiliary nurses, health educators, rural health technicians, and community facilitators – depending on the types of personnel available in the different health-care delivery models and at the various facilities. These teams examined both clinical quality of care in records and maternal knowledge and practices indicative of the quality of behavior change interventions.

The project also organized 30 higher level health district QI teams comprised of a physician coordinator, a professional nurse, a rural health technician, and the NGO teams in charge of monitoring the collaborative effort (Hurtado et al. 2011). In 2012, the project signed agreements with 19 NGOs and trained NGO staff, generally professional nurses, in facilitation techniques and tools for community-based collaboratives under the Extension of Coverage health-care delivery model. In turn, NGO staff often conducted training of their community QI teams.

Within a single QI team in each supervision area, the auxiliary nurses conducted the collection and analysis of the health-care data from clinical records, while the rural health technicians, health educators, and/or community facilitators collected and consolidated data from community-based interviews. All staff were considered members of the QI team and participated in both clinical and community QI efforts, especially in analyzing results and planning changes that would lead to improvement.

Understanding SBCC Processes

Assessing the quality of SBCC processes such as interpersonal communication and counseling cannot be performed entirely through record review, as is typically done for assessing clinical health care. It is uncommon to find records of counseling, and even less so with enough detail to determine quality. As mentioned, checklists on counseling and for registering graphic materials provided to clients had already been added to clinical records (prenatal, postnatal, neonatal, and child) and were being reviewed in the clinical QI component, but data were insufficient to ascertain quality and effectiveness to improve SBCC activities.

The project then considered using other techniques such as direct observation and/or individual exit interviews to determine quality of counseling. Although the teams agreed that direct observation would be the gold standard for QI monitoring, they had to be realistic about the requirements to gathering these data, as it would require considerable training and standardization, time, effort, and resources. Instead, they decided that the community-based behavior change component would focus on changes in knowledge and practices of pregnant women and/or mothers in the community. These changes were theorized to occur as outcomes of improved interpersonal communication and counseling primarily through home visits by community health workers (see “Outputs/Outcomes” in Fig. 3.2).

Fig. 3.2
figure 2

SBCC inputs, processes, and outcomes

Full size image

It is fair to say that with the approach taken (measuring outcome knowledge and practices), the project relied on the validity of indicators to reflect the quality and effectiveness of SBCC. Also, it relied on the validity of the indicators in improving the health and nutrition of the people who are in the coverage area of health facilities. However, these assumptions are often made in QI work; specific appropriate studies would have needed to be conducted to firmly establish the link between QI collaborative efforts and health outcomes in the population.

Based on the available evidence and the health and nutrition situation in Guatemala, the MOH and the SBCC QI collaboratives developed indicators for selected key essential behaviors or practices for families with pregnant women and/or children under 2 years of age. Most of these were topics already included in the MOH counseling protocols and SBCC strategies, but they were turned into standards and indicators with specific descriptions. Examples of standards and the indicators developed are presented in Table 3.1.

Table 3.1 Standards and indicators in the community-based collaboratives
Full size table

Measuring indicators entailed that each quarter a sample of 19 pregnant women or mothers of children 0–2 years of age from each supervision area were selected to interviewFootnote 3 using Lot Quality Assurance Sampling (LQAS). The project trained the teams in the sampling method, provided the tools, and supervised teams in the selection of households and respondents and the LQAS interview process. After measuring knowledge and practice indicators, the QI team consolidated and analyzed results and identified those indicators that scored lower. Then, they discussed the reasons for low performance and planned changes in SBCC activities to improve results that could be implemented during home visits, health talks, and even mass media campaigns.

To plan these changes, the community-based collaboratives initially used tools from the behavior change fieldFootnote 4 and adapted them for use in QI. An important tool was the BEHAVE model developed by the Academy for Educational Development (Table 3.2). The BEHAVE model focused health workers on specific actions to convey danger signs to pregnant women.

Table 3.2 BEHAVE format for planning improvement changes in the community-based collaboratives
Full size table

Later on, the community collaborative used the standard QI “planning matrix” being used in the clinical health-care collaborative to plan changes and improve results. It contained six columns: (1) gap found, (2) proposed change, (3) activities to be conducted, (4) where, (5) when, and (6) person responsible for leading the effort.

Example of Carrying Out the SBCC Improvement Effort in Chichicastenango, Quiché

The QI team in one supervision area of Quiché applied QI methods to improve the indicators related to pregnant women having an emergency plan and recognizing danger signs. The team noted that they were not meeting their targets regarding pregnant women recognizing danger signs and having an emergency plan to respond to them. Therefore, they decided to follow a major QI principle: Always involve in the improvement effort those who directly perform the process, in whole or in part. This meant including traditional birth attendants (TBAs) in the intervention and inviting them to be part of the QI team. In Guatemala , TBAs are generally illiterate older women, experienced and well respected in the community, who care for women from pregnancy to the postpartum period. In rural indigenous communities, they can attend as many as 80% of deliveries.

The intervention required challenging a perception that TBAs would be reluctant or unwilling to talk about emergency birth plans with their patients lest the family think they were incapable of attending women in their homes. Whether or not this perception was valid, the teams knew they had to find a way to get the TBAs to assume ownership of the intervention.

MOH guidelines required only that TBAs be taught about danger signs and that they refer to the hospital pregnant women who displayed those symptoms. However, trainings had not covered having TBAs review the specific components of an emergency plan, such as identifying the referral hospital, preparing for transportation, designating a person to help at home, and how to put the plan into action. Women were filling out the plans in the prenatal clinic, but TBAs were not encouraged to discuss the plans with the women.

The QI team, in partnership with representatives of the TBAs (e.g., better known TBAs, those with more patients, and/or those with closer ties to health services who represent the other TBAs), developed a new approach to emergency plans. The new procedures required that each TBA visit the referral hospital to develop an understanding of the components of the plan including:

  • Knowing the cost of transportation and vehicles available in the community

  • Designating someone to help at home, if they accompanied the woman to the hospital

  • Communicating with the community health commission to activate the community emergency plan

In addition, TBAs were given emergency planning cards to provide their patients in case their patients had not received such a card in the health post or community center.

The QI team tested the interventions during two quarters by training TBAs, organizing guided visits to the appropriate hospital, and meeting with the health commission to discuss the community emergency plan.

After this intervention began to yield successful results (see Box 3.2), the health district continued to actively work with the TBAs to address other obstacles. One challenge was the low number of women receiving prenatal and neonatal services. In particular, very few pregnant women were initiating prenatal care in the first trimester. This lack of care during the first 12 weeks delays the initiation of micronutrient supplementation, nutritional counseling, and other interventions and puts women at risk of worse health and pregnancy outcomes. Again, TBAs were involved since they usually have contact with pregnant women before the official health services do.

Box 3.2

During 2013 our NGO target of pregnant women detected before 12 weeks of pregnancy was 25, but it was not met. In 2014, our target was set at 30, something which worried us very much. In our analysis meetings of both clinical and community results, we observed that the referral of pregnant women with complications had increased after actively involving the TBAs. Therefore, we proposed a strategy of involving the TBAs in wider referral of women and children to health services and created ‘the TBA’s folder.’

–NGO nurse in Quiché

In addition to referring pregnant women with danger signs to health services, TBAs became involved in referring all pregnant women at the onset of pregnancy, all postpartum women, newborns, and all healthy and sick children under 2 years of age to health services. In addition, they were also asked to refer nonpregnant women of reproductive age to health services for family planning or for treatment of an illness. For this purpose, each TBA in the jurisdiction received a folder with her identification details, including name, address, and a list of the communities she attended. The folder contained five different colored envelopes, each containing cards with pictures: (1) red for postpartum mothers and newborns, (2) yellow for pregnant women, (3) green for nonpregnant reproductive-aged women, (4) blue for healthy children under 2 years; and (5) gray for a sick child or woman, or a pregnant woman showing a danger sign.

TBAs gave their clients an appropriately colored card to give to the facility health provider. The health provider registered the referral, signed and stamped the card, and gave it back to the TBA who had sent it. At the end of the month, the TBAs got together at the health facility and counted the number of cards of each color that had been signed and stamped by the facility personnel. The TBAs explained to the nurse the cases referred, received feedback, and were acknowledged and congratulated for their work (see Box 3.3). Those with the largest number of referrals at the end of the quarter also received a certificate of accomplishment from the health facility.

Box 3.3

These folders are a useful and productive tool for us. Before I could not demonstrate my work, but with the folders I can show the work that I do… I am very happy with the folders.

–A TBA in the supervision area

The change was implemented from April to June 2014. By June 2014, in the same supervision area, 55 pregnant women had been identified in the first 12 weeks of pregnancy (above the target of 30). Similarly, the health facilities observed a rise in the number of postpartum mothers and newborns returning for care, family planning clients, and children attending growth-monitoring checkups due to TBA referrals.

The card and colored envelope referral system also permitted a certain amount of discretion. A TBA said that she gave the yellow cards to pregnant women who did not want to attend prenatal care because they did not want to make their condition known. These women liked the fact that they just had to present their card and were attended by the nurse. The same was true, she explained, for women wanting family planning services; they could just give the green card to the health provider and get the service. After observing these initial successes, two other health districts in the health area decided to implement the change.

Documentation and Job Aids

In QI projects in Guatemala, especially in community-based collaboratives, documentation of results and change interventions has presented a formidable challenge. The formal educational level of the staff is generally low: auxiliary nurses have 2 years of secondary education, while community facilitators generally have completed only primary education. To show quantitative results, the project experimented with visual tools where health providers colored columns with results according to an established color scheme. These were pasted on the wall for analysis and presented at learning sessions.

Documenting the qualitative results on changes implemented presented an even greater challenge. The project tried various means, including giving oral presentations, recording and transcribing these presentations, presenting panels, providing photographs of improvement changes, and photocopies of each facility’s notebook (containing notes that were taken on meetings held, visits received, comments, decisions taken, and recommendations from coaching visits). Project staff produced copies for team members and staff whenever possible, whether through photo or carbon copies and/or notes describing interventions and lessons learned.

Measurement and Results

To measure indicators and identify those that were not reaching the quality targets, the collaborative used LQAS, a relatively low-cost and simple sampling method that could be used locally in the health posts’ and NGOs’ supervision areas. Sampling was carried out more easily by the NGOs than by health posts, since the NGOs were required to update their population census annually as part of their contracts with the MOH.

Most indicators showed increases after successive QI cycles of measuring, planning, trying out changes, and adopting those that resulted in improvement. For example, the percentage of mothers who could correctly state the ideal birth interval (3–5 years) rose from 57% in June 2013 to above 80% (LQAS QI target) in September 2014Footnote 5 (Fig. 3.3). The percentage of clinical recordsFootnote 6 showing that postpartum women received a home visit within 48 hours of birth rose from 31% to 77% during the same time period (Fig. 3.4). Improvement plans and activities that community QI teams carried out to achieve these outcomes included developing key messages and talks on the ideal inter-pregnancy interval for use at every clinic visit and during household visits, as well as conducting postpartum exams at home and bringing clinical records to register findings.

Fig. 3.3
figure 3

Percentage of mothers who correctly mentioned optimal birth spacing period (3–5 years), Community SBCC collaborative, Guatemala (June 2013 to September 2014)

Full size image
Fig. 3.4
figure 4

Percentage of mothers who received a postpartum home visit within 48 hours after delivery, community SBCC project, Guatemala (June 2013 to September 2014)

Full size image

Support for Improvement

Coaching Visits

Coaching visits to supervision areas were conducted each quarter for both the clinical and the community-based collaboratives. Because transportation restrictions such as lack of vehicles or gasoline and lack of per diem affected district level staff’s ability to make regular coaching visits, the NGOs’ professional nurses often accompanied the project staff to make visits. They followed coaching guidelines jointly developed by the MOH and project staff.

Before a visit, project coaches reviewed data on the supervision area, the type of facility, the composition of the QI team, as well as previous measurements and visits. Data were available in the supervision areas’ project information system which also included open source Google maps with all supervision areas. Coaches coordinated with the health district and NGO, since they were required to be accompanied by MOH staff.

The QI team, which included staff from four to five community centers or four to six health posts, met at one facility to verify and exchange results and share change plans and experiences. Among the activities in which teams participated, guided by coaches, were to:

  • Explore – through open-ended questions – the team’s knowledge and use of QI methods.

  • Review baseline or current measurement of indicators and the forms used, identify low-performing indicators, and prioritize one or two for improvement.

  • Reinforce PDSA cycles.

  • Use the improvement-planning matrix to register and monitor improvement activities.

  • Identify gaps by comparing the improvement-planning matrix with implementation documentation or by having the team describe the methodology used.

  • Take notes of knowledge of the methodology and application and the aspects that needed to be improved, leaving a record of the visit in the facility’s notebook.

  • Use the planning matrix to note where and when changes had been implemented.

As the QI intervention progressed, the coaches recognized that teams often required more intensive coaching in how to apply LQAS to draw the community samples. Coaches supported team members by accompanying them on visits to selected households to observe how they identified the household and selected the respondent(s), as well as the interview process and recording of responses.

Due to limited resources, the USAID-funded project provided copies of the questionnaires and planning matrices to all supervision areas. Coaches found that they often had to supply the QI teams with tools, such as instructions on how to select a sample or fill out questionnaires, and information, education, and communication materials for SBCC activities.

During coaching visits, teams verified data quality under the guidance of coaches. The coaches guided team members in checking records to ensure that the numbers correctly added up. Following visits, coaches summarized results in the project’s quarterly report disseminated to USAID and the MOH central and health area level technical teams.

Institutionalization

Since 2014, there have been profound changes in Guatemala. The health-care system has faced budgetary and staff challenges that have produced a climate in which it is hard for institutional changes to take root and for spread and scale-up to take place. Changes within the top ranks of the MOH and at the central and health area levels have made it difficult to sustain positive change.

The government’s termination of the NGO health delivery model dramatically curtailed the collaborative’s growth plan which had relied heavily on continuing to involve community organizations’ representatives in measurement, analysis of results, and subsequent planning of improvement.

Even in the face of these challenges, QI methods and tools have become part of MOH guidelines and are being used by other cooperating agencies and projects. QI methodology has taken root with some health providers who participated in QI efforts and remained in their posts. They continue to use them to measure and improve processes and results, even when, in practice, the central-level MOH does not actively promote and support their use any longer.

Reflection

Collaborative participants emerged with lessons learned about how to apply QI interventions to the SBCC cycle and how the process differs from applying the methods to clinical care. For clinical interventions, the project typically defined standards and indicators of prenatal, postnatal, and neonatal childcare; measured them in a sample of each type of clinical record; and undertook planned actions to improve the levels of compliance with these standards and indicators.

Because SBCC documentation was not sufficient to measure progress, teams had to define indicators based on maternal knowledge and practices. These indicators were best measured through sampling the population through lot quality assurance sampling.

Due to the large number of primary health-care facilities and dispersed areas of coverage and community-level personnel, the collaborative had to be prescriptive in the organization of supervision areas, timing of measurement, the amount of time for the action period, and scheduling learning sessions. Participants found that PDSA improvement cycles that could be done rapidly in clinical interventions were only able to be performed “relatively rapidly” in community settings. Community-based teams needed at least one quarter between measurements to effectively implement changes planned.

Through trial and error, project participants came away with significant lessons learned about the formation of community QI teams. They found that organizers of QI efforts would be well served to first consider the structure of a country’s health-care system and the number and type of health providers available. In addition, in this primary health-care setting, clinical and community-based collaboratives had to work together to optimize involvement of the limited number of personnel and achieve results.

Project participants also found that it was sometimes critical to challenge existing perceptions when implementing a QI intervention. The QI intervention with TBAs described in this case study required that the team trust that TBAs would be willing and able to assume ownership of the strategy.