Significance

The systemic nature of factors affecting maternal and child health merits action by multiple sectors partnering with families. Decision makers are increasingly seeking to ensure active participation of all stakeholder groups in developing and implementing community strategies. The current study found the Wilder Collaboration Factors Inventory effective for helping coaches work with community action teams to improve their collaboration over time. The instrument was easy for participants to complete and yielded findings about specific collaboration factors that teams were able to use. Community leaders may find North Carolina’s experiences useful for comparison as they build and track their collaborative capacity.

Introduction

Preterm birth and low birth weight are associated with long-term negative health conditions and high related spending for children (Behrman and Butler 2007; CDC 2018). Preterm birth and low birth weight rates have recently increased (Martin et al. 2018). There are also persistent disparities in these outcomes. For instance, the preterm birth rate is 50% higher for African American women than for white women (CDC 2018). Reasons for these increases are likely to have systemic origins, and hence require broad-based solutions (Black et al. 2017).

Funders increasingly require collaboration toward sustainable improvement (Gillam et al. 2016). One such example has been Improving Community Outcomes for Maternal and Child Health (ICO4MCH), funded by the North Carolina General Assembly (Morgan et al. 2020). ICO4MCH began with 2-year funding cycles, the first being June 1, 2016–May 31, 2018, from the state Division of Public Health to five local health departments with a combined service area of 14 counties. These funds were competitively awarded for implementing evidence-based strategies to lower infant mortality rates, improve birth outcomes, and improve the overall health of children ages birth to five.

The developers of ICO4MCH used a collective impact framework (Kania and Kramer 2011), implementation science (Fixsen et al. 2009, 2010), and a health equity lens. The funding required creation of a community action team in each site including health department staff, consumers, and other partners. These teams support implementation of evidence-based strategies and health equity to promote systems change, with a secondary goal of creating sustainable collaboration.

The intention of the collective impact framework is to overcome the usual “isolated impact” of actors with common goals who may not work together effectively, if at all. Instead, collective impact is envisioned as improving social outcomes at scale through intersectoral cooperation (Kania and Kramer 2011). In addition, the collective impact approach entails mutually reinforcing action by partners across sectors, as well as continuous learning. In the context of ICOMCH, this included collaboration within and beyond public health agencies. ICO4MCH also drew on implementation science, including an emphasis on ongoing feedback and coaching to participants in community innovation (Fixsen et al. 2005, 2009).

A central tool for providing ICO4MCH teams with feedback about their collaboration was the Wilder Collaboration Factors Inventory (Inventory). Technical Assistance (TA) coaches based in the University of North Carolina and State ICO4MCH leadership chose the Inventory as an easily interpretable instrument for tracking changes in perceived community action team collaboration over time. The Inventory draws on extensive reviews of prior research to assess factors previously found to be associated with success in collaboration, defined as a mutually beneficial relationship among organizations for common goals (Mattessich and Johnson 2016).

The purpose of the current paper is to describe lessons from the ICO4MCH program’s use of the Wilder Collaboration Factors Inventory that may apply to other communities implementing collective impact initiatives.

Methods

Procedure

TA coaches asked members of each community action team to complete the Wilder Collaboration Factors Inventory three times between 2016 and 2018: shortly after forming, and 12 and 24 months thereafter. Team members were given time to complete paper versions of the Inventory during meetings, as well as sent electronic links to the Inventory for completion at another time.

This project was conducted in accordance with prevailing ethical principles and reviewed by the University of North Carolina Institutional Review Board.

Participants

Community action team members represent stakeholders invested in reducing adverse birth outcomes and improving child health. As specified by the authorizing legislation, these include individuals from women’s health and pediatric providers, local health department staff, and the supplemental nutrition program for women, infants, and children (WIC), as well as consumers living in the service areas who use public health and human services, and other community leaders. At least 25% of their members are consumers and represent local demographics.

Assessment and Measures

The Wilder Collaboration Factors Inventory version used was that available in 2016, which entailed 40 items relating to 20 factors (Table 1). Those factors included participation by a range of stakeholders; clear roles and policies; sufficient resources for the participant time and administrative support collaboration requires; trust and commitment; and open and frequent communication (Ales et al. 2011; Gillam et al. 2016; Perrault et al. 2011). The study team made minor changes to Inventory item wording to enhance ease of reading without losing meaning. For instance, “collaborative” was sometimes replaced with “team” or omitted as an unnecessary qualifier, and “accomplish” was replaced with “do.” After this study was completed, a new version of the Inventory was released, with four more items reflecting two additional factors (evaluation and continuous learning and engaged stakeholders) (Wilder Foundation 2019).

Table 1 Wilder Collaborative Factors Inventory, 2016 Version
Full size table

For each Inventory item, response options ranged from 1 (Strongly Disagree) to 5 (Strongly Agree). The score for each factor was calculated as the mean of all community action team members’ responses about the item(s) representing that factor. According to the Inventory developers, a score of 4.0 or above for a factor is a strength that does not require additional attention, factors with scores ranging from 3.0 to 3.9 are borderline and should be discussed by the group, and factors scoring 2.9 or below are areas of concern that should be addressed (Mattessich and Johnson 2016).

Each time after completing the Inventory, the TA coaches reflected with members of each community action team about how to interpret their scores, and how to build on strengths and identify opportunities for improvement. This entailed linking the team’s particularly high or low scores in the Inventory to the conditions for collective impact. For instance, coaches related team responses to Inventory items about concrete, attainable goals and objectives to the collective impact emphasis on a common agenda. These discussions included incorporating action steps into plans for the following year.

In addition, community action team conveners were asked open-ended questions each year: “What is working well in this group?” and “What needs improvement in this group?” Responses to these questions were content analyzed for insights about Inventory scores and related strategies.

Results

Each of the five health departments represented a diverse area of North Carolina, with varying community characteristics, including the largest county in the state as well as smaller rural, urban, and mixed regions (US Census Bureau 2018) (Table 1). All teams sought to improve access to reproductive life planning, including long-acting reversible contraception. To reduce infant mortality, two teams implemented a tobacco reduction intervention; the other three supported breastfeeding. To improve children’s health, two teams addressed secondhand smoke exposure; two implemented parenting programs, and another implemented both a parenting program and nurse home visiting (Morgan et al. 2020).

TA coaches found the Inventory feasible to use, taking an average of 10 min for community action team members to complete. In the first year, 51 members across the five teams completed the Inventory, for an average of 10 respondents (ranging from 5 to 20) per site. In year 2, 60 team members completed the Inventory, for an average of 12 (ranging from 5 to 19) per site. In year 3, 78 team members completed the Inventory, for an average of 16 (ranging from 8 to 23) per site (Table 2).

Table 2 Characteristics of Sites in Improving Community Outcomes for Maternal and Child Health Initiative (ICO4MCH)
Full size table

For each year we calculated the average scores for each factor across all sites (Table 3 and Fig. 1). In year 1, the only factor with an average score above 4.0 (‘strength’) was members seeing their collaboration as in their self-interest. Most factors had an average score between 3.0 and 3.9 (‘borderline’). The three factors with the lowest average scores were appropriate cross section of members, multiple layers of participation, and development of clear roles and policy guidelines. No factors averaged 2.9 or below (‘concern’).

Table 3 Wilder Collaboration Factors Inventory, Average Scores across all sites, Year 1–Year 3
Full size table
Fig. 1
figure 1

Wilder Collaboration Factors Inventory Average Scores across all sites each year

Full size image

Inventory results were not used to choose interventions. Instead, Inventory results informed improvements in team functioning needed to achieve collective impact. The number of members in leadership roles was designated by the funder, and so did not change. However, the increasing numbers of people completing the Inventory over time suggest team growth.

In year 1, community action team convener responses to open-ended questions indicated perceptions of leaders and team members as passionate about their work. We interpreted these comments as related to the Inventory factor of members sharing a stake in processes and outcomes, which had a borderline average score in year 1 before becoming a strength in years 2 and 3.

However, members also reported that key stakeholders were missing from the teams, including providers as well as community members using services; we saw these comments as related to the Inventory factor about having an appropriate cross section of members, which tended to have borderline scores all 3 years. Further, members noted that their roles and specific team goals were unclear. These comments aligned with borderline average team scores all 3 years for the Inventory factor relating to development of clear roles and policy guidelines. Average scores for the related factor concerning concrete, attainable goals and objectives increased from borderline in years 1 and 2 to a strength in year 3.

[T]his project involves disparate health outcomes (e.g. tobacco cessation, family planning) and different members are involved in different parts of project so [it] is challenging to have [a] unified goal for collaborative; helpful to review/clarify what is needed from each partner, timeline of where we are going and of project goals (Year 1 single county urban site))

TA coaches helped members identify how to build on strengths and address areas for improvement. For instance, teams developed communication strategies including newsletters and using social media to increase visibility of ICO4MCH. Challenges in identifying clear roles and clarifying goals led teams to reevaluate their common agendas.

In year 2, members seeing collaboration as in their self-interest remained a strength across the five teams, and nine other factors also emerged as strengths, including mutual respect, understanding, and trust; flexibility; open and frequent communication; shared vision, and skilled leadership. In addition to the three lowest scoring factors from year 1, sufficient funds, staff, materials, and time also emerged as low scoring in year 2.

In year 3, 10 factors were again identified as strengths. Two factors shifted categories: favorable political and social climate moved from a strength to borderline; while concrete, attainable goals and objectives moved from borderline to a strength.

Community action team convener responses to open-ended questions in years 2 and 3 were consistent with year 1 responses. Common responses about what was working well included the energy and motivation of diverse participating members; improving coordination and communication; and good leadership. In addition, team conveners often noted location and meeting times, which is not in the Inventory.

The level of communication and how the leaders keep everyone in the group informed of our mission and the things that are being done from other organizations and programs. (Year 3 multicounty urban/rural site))

Common responses in years 2 and 3 about needed improvements included clear timelines and actions; identification of additional community members and agencies needed, including managers; and sustainable models of engagement.

Tangible goals and actions for group members to participate in both within and beyond the meeting setting. Additional consumers needed at the table. Co-leadership model is also in progress and would be ideal for sustainability purposes. (Year 2 multicounty rural site)).

In years 2 and 3 the TA coaches led the community action team members through additional strategic planning processes to address the lower scoring factors. The teams began implementing action steps over the year in an effort to address those factors. These included more use of social media to highlight team activities and targeted invitations to potential team members.

Overall, despite challenges identified by community action team members, by year 2, half of the factors across all sites were identified as strengths, which were largely sustained in year 3.

Discussion

The experience reported here demonstrated the utility of the Wilder Collaboration Factors Inventory for improving maternal and child health community action teamwork over time. One of the lessons is the importance of editing the Inventory for ease of reading without losing the original meaning. It is also important to ensure that all members have access to the Inventory, evidenced in ICO4MCH through high and increasing survey participation rates over time. Study limitations include the absence of team members’ demographic information, which the Inventory does not address, and not having the number of team members in any given year.

Improving participant assessments of collaboration between years 1 and 2 suggest that tailored technical assistance using Inventory scores supported teamwork. In year 1, initial responses indicated shared perception that collaborating was in participants’ self-interest. By year 2, participants had slightly higher average assessments of such relational bases for successful collaboration as mutual respect and flexibility; evidence of improving collaboration, such as frequent and open communication (Gillam et al. 2016); and skilled leadership. In year 3 members continued to show enthusiasm for working together, valuing their comfort sharing information and mutual awareness achieved through meetings. ICO4MCH patterns fit those reported by other partnerships using the Inventory, such as initial exploratory participation, in search of increasing collective impact; early clarifications of participant roles and responsibilities paying off in later collaborative dynamics; and positive reinforcement over time between informal communication and trust (Ales et al. 2011; Gillam et al. 2016; Perrault et al. 2011).

Although four factors continued to have borderline average scores through year three, three of those four—appropriate cross section of members, multiple layers of participation, and development of clear roles and policies—did increase over time within that range. This progress is similar to that reported in prior research using the Inventory, such as initial identification of missing stakeholders and clarification of governance (Ales et al. 2011).

The one factor with an average score virtually unchanging from years one to three was sufficient funds, staff, materials, and time. Despite each of the partnerships receiving state funding to support the community action team and implement strategies, the continuing challenges relating to sustainability in ICO4MCH also mirror other community partnerships’ accounts of very substantial time and other resource requirements (Ales et al. 2011; Gillam et al. 2016). The collective impact framework encourages collaborative groups to have a backbone organization responsible for organizational and administrative support (Kania and Kramer 2011).

Finally, specifically aligning teams’ Wilder Collaboration Factors Inventory scores with collective impact goals can facilitate more effective collaboration. For instance, coaches noted how high Inventory scores in open and frequent communication were related to the continuous communication needed for collective impact (Kania and Kramer 2011). Similarly, coaches facilitated discussion of how the Inventory factor about awareness of mutual interest fit the collective impact framework’s emphasis on complementary activities. Results from this North Carolina initiative suggest that incorporating Inventory results into strategic action planning may help communities increase their collective impact in maternal and child health.