Key messages: summary

Key findings

This paper addresses CHW programme performance and how it is assessed from a systems perspective

• A variety of existing frameworks are useful when assessing the performance of CHW programmes

• CHW programme performance assessments need to consider measurement of individual CHW performance and community-level outcomes, as well as look at health systems and other contextual factors that influence CHW programme performance

Key implications

• CHW programme assessments ideally need to be based on data derived from a mix of reliable sources and obtained using a mix of methods

• Investment in CHW programme performance assessments is instrumental for continually innovating, upgrading, and improving CHW programmes at scale

Background

This is the tenth paper in our series, “Community health workers at the dawn of a new era”. Previous papers have provided an overview of national community health worker (CHW) programmes at present and the challenges they face (Paper 1 [1]), approaches to building partnerships, collaborations, and governance (Papers 2 [2] and 3 [3]), financing (Paper 4 [4]), roles and tasks (Paper 5 [5]), recruitment and training (Paper 6 [6]), supervision (Paper 7 [7]), motivation and reimbursement (Paper 8 [8]), and relationships with communities and with health systems (Paper 9 [9]). This series has been produced in recognition that CHW programmes are in a historical moment where the importance of their contribution to health systems and to population health improvement is becoming more widely recognized [10]. Rather than being an underfunded afterthought, CHWs need to be understood as an integral part—if not the foundation—of health systems, particularly (but not exclusively) in low- and middle-income countries (LMICs) [11, 12].

This historical moment has been produced by several powerful and concurrent circumstances. First, the evidence of the effectiveness of trained and supported CHWs in providing quality health services and improving population health continues to grow [13]. Countries that are making the strongest gains in improving the health of their populations have strong CHW programmes, such as Bangladesh, Brazil, Ethiopia, Nepal, and Rwanda [14]. Second, the Declaration of Alma-Ata [15], which was the first major global health document to assert the importance of CHWs in 1978, was reaffirmed on its 40th anniversary in the 2018 Declaration of Astana [16]. On this anniversary, the World Health Organization (WHO) for the first time created a guideline for optimizing the contribution of CHW programmes to health systems [10, 17]. The following year, the Director-General of WHO and the World Health Assembly strongly affirmed the importance of CHWs for achieving universal health coverage and strengthening primary healthcare (PHC) [18, 19]. Finally, the COVID-19 pandemic has clearly demonstrated the need for health workers at the community level who can prevent, detect, and respond [20, 21]. The pandemic has led to calls throughout the world for CHWs to receive appropriate respect and status (including pay), not to mention the personal protective equipment that is essential for them to carry out their duties safely [22]. The pandemic has also led to broad recognition of the importance of CHWs in high-income countries [23, 24].

Improved investment and integration of CHW programmes in the health sector in LMICs require evidence on performance. This again requires monitoring and evaluation. The performance of CHW programmes is influenced by support (or the lack thereof) from two sides—the health sector and the community. It is also influenced by the broader social, economic, and political context [25,26,27]. Performance can be measured at the individual CHW level as well as at the programme and population level [25, 27, 28]. As discussed in Paper 1 of this series, CHWs are embedded in the community health system, which is a subsystem of the overall health system [29]. This makes the assessment of overall CHW programme performance a complex undertaking because of the number of (formal and informal) actors involved and the many relationships and processes at play in improving health at the community level. It is therefore difficult to specifically attribute CHWs’ impact on health outcomes. Nevertheless, monitoring and evaluation of CHW programmes is of utmost importance to be able to continuously improve CHW programming and build more evidence on what works, why, and in which context [30].

Key message 1

While the assessment of CHW programme performance is a complex undertaking, it is of utmost importance to be able to continuously improve CHW programming

Methods

In this discussion paper, we address CHW programme performance and how it is assessed from a systems perspective, acknowledging that health systems are complex social arrangements [31]. We start by defining CHW and CHW programme performance, after which we reflect upon various frameworks that have been developed for considering the performance of CHW programmes. Next, we provide an overview of approaches to assessing CHW programme performance, and finally, we reflect upon identified gaps and needs for the future with regard to assessment of CHW programme performance.

This paper builds on previous work of our writing team, most notably the widely used 2014 CHW Reference Guide, which targeted CHW programme managers and policymakers, supporting them in developing and strengthening CHW programmes at scale [12]. It is also informed by a recently released compendium of case studies of 29 national CHW programmes [14]. In addition, the WHO guideline on health policy and system support to optimize CHW programmes (2018) is discussed in this paper, as this recent guidance not only aims to assist national governments and national and international partners in improving the design and implementation of CHW programmes, but also aims to improve their performance and evaluation [17]. We also discuss the CHW Assessment and Improvement Matrix (AIM) toolkit [32, 33]. Lastly, we include illustrative examples of studies on CHW programme performance and community health information systems (CHIS) published in the past 5 years (i.e., since publication of the CHW Reference Guide). For this purpose, we conducted a search on PubMed in August 2020, with a broad focus on CHWs, performance, information systems, and LMICs. We used a wide definition of CHWs, which includes a spectrum extending from volunteers with brief informal training to paid, professionalized CHWs, as outlined in Paper 1 of this series [1]. The 551 search results were screened on their titles and abstracts, and examples of partial or full CHW programme assessments, conducted with a variety of methods, were taken up in this paper to illustrate the discussion.

Results

Defining CHW and CHW programme performance

Well-performing health workers function in ways that are responsive, fair, and efficient to achieve the best health outcomes possible for clients, given the available resources and circumstances [34]. We use this definition of health worker performance for CHWs as well and stress that for CHWs to be responsive, fair, and efficient in achieving the best health outcomes in their communities, the available support from and relationships with actors in the health sector and the community are instrumental.

CHW performance at the individual CHW level is the sum of different interrelated attributes, such as knowledge, guideline adherence, competencies, motivation, job satisfaction, agency, attitudes, and self-esteem [25]. These attributes can be measured in various ways—some more difficult than others—and can thereby give an indication of CHW programme performance. However, CHW programme performance is not only the sum of performance-related individual attributes of CHWs; it is also defined by community-level outcomes, such as health service use and community empowerment, and, ultimately, health outcomes (impact) [25, 27, 28]. We will explore this in the next section.

CHW programme performance frameworks

Over the past decade, several frameworks on CHW programme performance have been published. While these frameworks differ in terminology used and sometimes in focus, all address determinants of CHW individual and programme performance. We draw on three of these in this paper: Kok et al.’s framework of intervention design factors influencing CHW performance [25], Naimoli et al.’s CHW logic model [27], and Agarwal et al.’s conceptual framework for measuring CHW performance within PHC systems [28].

Agarwal et al. (2019) [28] recently developed a CHW performance measurement framework using the common input-process-output-outcome logic model approach, while recognizing that community health systems are nonlinear and complex. Determinants of CHW programme performance are presented as inputs and programmatic processes. The inputs include policies (e.g., CHW selection procedures and CHW roles and tasks), governance and stakeholders, logistics (e.g., transport and commodities), funding, and information management systems. Other widely used frameworks label these as health systems factors [25, 27]. The programmatic processes entail supportive systems (e.g., supervision, performance appraisal, and data use), CHW development (e.g. recruitment, training, and incentives), and support from community-based groups [28].

Naimoli et al. (2014) [27] label these as programme-level activities, where they distinguish technical support, social support, and incentives from the perspective of both health sector actors and community actors. In Kok et al.’s (2015) [25] conceptual framework on factors influencing CHW performance, these are labelled as intervention design factors; in Ballard and Montgomery’s (2016) systematic review, they are called intervention components [35]. These determinants of CHW programme performance are important, and we will come back to them later in this paper. We now focus on how performance can be defined and measured.

As indicated above, most frameworks make a distinction between individual CHW performance and community-level outcomes as indicators of CHW programme performance. Between the frameworks, the categorization of the attributes of individual CHW performance differs, but the attributes themselves are in line with each other. Following the categorization (indirect and direct) used in the work of Naimoli et al. (2014) [27], Table 1 contains the attributes of individual CHW performance from the three frameworks combined [25, 27, 28]. Some of these attributes slightly overlap. What is important, though, is how the attributes are operationalized, and this will vary by country and programme context. In the next section, we discuss how they could be measured.

Table 1 Indirect and direct attributes of individual CHW performance [25, 27, 28]
Full size table

The three frameworks cover community-level outcomes as indicators of CHW programme performance. Naimoli et al. (2014) [27], unlike the others, distinguish CHW-attributable changes among individual clients, in the community, and in health systems. For example, CHWs could influence changes in health system structures, processes, and behaviours. In Table 2, we list community-level outcomes reported across the three frameworks. They include outcomes that can relate to individual community members and communities more broadly. Some of them overlap, and again, their usefulness depends on how they are operationalized in the context of a specific CHW programme.

Table 2 Community-level outcomes as measurements of CHW programme performance [25, 27, 28]
Full size table

The performance of individual CHWs and the community-level outcomes will (ideally) lead to improved health outcomes at the population level (impact): equitable reductions in mortality and morbidity, improved well-being, reduced disease incidence, or changes in the incidence of health-related conditions (such as teenage pregnancy) (Fig. 1). While changes in population-level health indicators are readily measurable, attributing any observed changes to the impact of CHW programmes is a challenging methodological task and is often difficult in the absence of evaluations that have a control area where the CHW programme was not present. In addition, other influences that are producing measurable improvements in population health are also present, such as improved socioeconomic conditions and facility-based healthcare.

Fig. 1
figure 1

Impact at the population level of CHW programme performance

Full size image

Key message 2

CHW programme performance assessments need to consider measurement of individual CHW performance, community-level outcomes, and impact at the population level

Approaches to assessing CHW programme performance

Depending on the programme and evaluation focus, a combination of the above-presented attributes of individual CHW performance and community-level outcomes can be included for assessing CHW programme performance. Different methods are suitable for this, depending upon the attribute or outcome and associated indicators.

The use of meeting minutes, training and accreditation logs, and reports from CHWs and supervisors

CHW programmes often have mechanisms for reporting which are different from data captured in the routine health information system. Minutes of meetings between CHWs and supervisors, or of village health or other committees involving CHWs, can provide insight into, for example, CHW satisfaction or productivity and their influencing factors. Training and accreditation logs, if available, can provide information about (gained) knowledge of CHWs. Supervisor reports and standardized checklists can include observations on guideline adherence and quality of care provided by CHWs [28, 39]. Supervisor reports can provide useful quantitative data if there is regular supervision of all CHWs, because infrequent supervision or supervision of only a part of the CHW workforce may distort indicator measurement [40].

There are various examples of studies in the recent academic literature in which these types of data are used to assess CHW performance, sometimes combined with data from other sources, such as surveys or qualitative interviews. These studies mostly focus on specific attributes of individual CHW performance in the context of vertical programmes. For example, in Nepal, Thapa et al. (2020) [41] assessed the knowledge of female community health volunteers (FCHVs) before and after a (cascade) training on postpartum family planning. They also reviewed FCHV registers, interviewed clients, and conducted focus group discussions with FCHVs. They concluded that the intervention resulted in increased knowledge and improved community-based counselling by FCHVs. However, long-term effects were not measured.

In Nigeria, Abimbola et al. (2016) [42] analysed the minutes of 129 community health committees (CHCs) across four states. These minutes provided a wealth of information about committee processes, deliberations, actions, decisions, and relations. CHCs had five modes of functioning in promoting community engagement in PHC. They provided forums (called village squares) for community stakeholders to interact with and support the health system, they reached out within their communities (community connectors), they lobbied governments for support (government botherers), they induced and augmented government support (back-up government), and took control of healthcare in their community (general overseers). While performing these functions, contextual factors, in particular power imbalances, limited their influence on improving health service delivery. It was therefore concluded that CHCs tended to promote collective action for self-support (of the committee) more than collective action for demanding accountability.

The use of information from the routine health information system

Key CHW programme activities and outputs need to be routinely documented and analysed, beginning at the most peripheral tier of the PHC system. The health information system provides a basis for this. However, the inclusion of community health services into the routine health information system varies by country. Depending on the available system, activities and outputs of CHWs can be specifically identified or not. For example, information systems may have data on referral from CHWs to health facilities. Other indicators, such as antenatal care (ANC) utilization, can be disaggregated based on whether it was offered at a community or facility level. Calculations to measure performance can then be made by comparing ANC use to the use based on the number of expected pregnancies in the CHW catchment area. If disaggregation between facility- and community-level data is not possible, it is more difficult to assess CHW programme performance.

The recent academic literature provides examples of studies that used data from routine health information systems to assess CHW performance. In Mozambique, Jose et al. (2020) [43] evaluated the role of CHWs in promoting facility-based tuberculosis (TB) screening and household contact tracing using routine national TB programme data in five intervention and seven control districts. They concluded that the involvement of CHWs led to an increase in TB notifications over the period of 1 year. In Sierra Leone, CHW reports were discussed in community health data review meetings where community leaders, members of ward development and health management committees, CHW peer supervisors, and representatives of the peripheral health unit assessed CHW performance and developed action plans for improvement [44].

Roberton et al. (2016) [40] examined the feasibility of collecting integrated community case management (iCCM) routine indicators as included in the iCCM Indicator Guide in 10 countries. They concluded that the majority of data needed are being collected through existing monitoring systems. However, they are mostly available at the health facility level and do not get aggregated at district or national level [40].

The use of information from existing periodic national surveys

National demographic and health surveys (DHSs) include indicators that provide valuable information on CHW programme performance. For example, DHS 8 woman’s and man’s questionnaires contain indicators such as the percentage of women visited by a CHW in the last 12 months and the percentage of men who discussed family planning with a CHW [45]. The United Nations Children’s Fund (UNICEF) multiple indicator cluster surveys also contain questions on CHWs. For example, the questionnaire for individual women covers whether CHWs offered antenatal, delivery, and postnatal care [46]. These data can be assessed over longer periods of times (e.g., every 5 years) to get a sense of how CHW programme performance is evolving.

Periodic internal or external CHW programme assessments

Countries can readily carry out their own periodic assessments of their national CHW programmes. One possible approach to this is national evaluation platforms (NEPs) that have been developed in some countries. NEPs build national capacity to systematically identify, compile, and rigorously analyse data from diverse sources to evaluate the effectiveness of maternal, newborn, and child health and nutrition programmes [47]. Efforts such as these could be adapted to assess national CHW programme performance. There are examples of countries that, often with funding from external donors, contract firms to carry out independent assessments of national CHW programmes as is commonly done with other development projects. For example, the health extension programme in Ethiopia was evaluated in 2018–2019 [48]. The national assessment consisted of a comprehensive mixed-methods study, a systematic review, and a participatory process to synthesize evidence and develop recommendations. The assessment revealed the strengths and weaknesses of the programme. Amongst others, it found gaps in health extension worker competencies as a result of suboptimal pre-service training. In 2011, a mixed-methods national assessment of CHW services in Afghanistan was conducted. It was found that training, supervision, and supplies were adequately organized, mainly by implementing nongovernmental organizations, but overload and motivation of CHW were issues of concern [49]. Other examples of national CHW programme assessments are provided in Table 4 (Appendix).

The use of primary studies and research tools

Primary studies often focus on a limited set of attributes or outcomes related to CHW programme performance. In addition, they often look at vertical CHW programmes. They are seldom applied to national and scaled CHW programmes. As mentioned in Paper 1 of this series, evidence from small-scale studies can still be relevant for large-scale programmes.

Primary studies can have various methodologies. Randomized controlled trials (RCTs) often assess the effect of a change in supportive systems, such as certain modalities of CHW supervision or incentives, on attributes of individual CHW performance, such as guideline adherence or productivity [50, 51], or community-level outcomes such as service use [52]. While RCTs are needed for robust evidence, the RCT design is relatively unhelpful in providing full assessments of complex programmes at scale. Comprehensive programme case studies are a useful addition, as well as implementation research, using surveys over time, scales, qualitative methods, and systems-thinking tools [17].

When using surveys to measure attributes or outcomes on CHW (programme) performance, researchers can consider numerous indicators, according to the CHW programme under study and its context. Agarwal et al. (2019) [28] provide example indicators in their work based on a review of peer-reviewed articles, reports and global data collection tools, and consultations with global stakeholders, community health implementers, advocates, measurement experts, and ministry of health representatives across seven countries. One possible indicator of productivity and the use of services is the household visitation rate. Mwanza et al. (2017) [53] reported on CHW supervisors using the lot quality assurance sampling (LQAS) methodology to evaluate CHW performance in terms of household visitation rate in Zambia. LQAS proved to be a useful tool to assess CHW performance and could be used for performance appraisal as well. Smittenaar et al. (2020) conducted large-scale surveys including recently delivered women, their husbands, mothers-in-law, and accredited social health activists (ASHAs) in Uttar Pradesh, India. Household visitation timing and rate, ASHAs’ knowledge and beliefs, and the content and target of their advice were found to influence maternal and newborn health-promoting and health-seeking behaviour [54].

With regard to scales on attributes of individual CHW performance, one can adjust existing scales developed for other purposes unrelated to CHWs to the context of CHW programmes. The engagement survey for measuring the well-being of workers is one example [55]. The engagement survey asks employees to rate items on: their belief in their work and organization; their belief in their ability to succeed; their relations with colleagues and their supervisor; their opportunities for professional advancement; the support and recognition they receive; and the influence they have in decision-making. Another valuable toolkit that can be used to measure attributes of individual CHW performance is CUBES, which focuses on contextual and perceptual drivers (including beliefs, such as self-efficacy) of behaviour [56].

Recently, a validated scale for measuring motivation specifically for CHWs was published, consisting of 12 items related to job satisfaction, organizational commitment, community commitment, and work conscientiousness [57]. Qualitative methods, along with quantitative scales or as a stand-alone method, can provide insight into CHW performance as well.

For example, Kane et al. (2016) [58] presented findings from six country case studies from Africa and Asia regarding CHW empowerment. Their analysis was based on Lee and Koh’s analytical framework of empowerment that identifies four dimensions of empowerment: meaningfulness, competence, self-determination, and impact. The study found CHWs had feelings of empowerment as well as disempowerment. The authors recommend that more attention should be given to the experience of CHWs when aiming for improving CHW programme performance. We fully support this statement, as it is obvious that addressing issues of concern to CHWs themselves is an essential strategy for improving overall CHW programme performance. Vareilles et al. (2015) [59] conducted a realist evaluation on motivation of community health volunteers of the Red Cross in Uganda. They describe how a supportive environment positively influenced three key drivers of volunteer motivation: feelings of autonomy, competence, and connectedness.

For measurement of community-level outcomes, community scorecards can be used, where communities score on performance indicators [60]. With this method, outcomes such as satisfaction with CHW services, and communities’ trust in and perceptions of credibility of CHWs can be assessed. In addition, a variety of qualitative methodologies can be used.

Key message 3

Approaches to assessing CHW programme performance can include monitoring based on available data in the health system, and primary studies or national programme assessments that use a variety of methods

CHW programme assessments: a focus on programmatic determinants of performance

Programme inputs, as mentioned before, entail policies, governance and stakeholders, logistics, funding, and information management systems. Programmatic processes entail supportive systems, CHW development, and support from community-based groups. Many studies focus on these inputs and processes—either one or all of them—because they influence CHW programme performance. Therefore, regular assessment of these programmatic determinants of performance provides information on whether a CHW programme is designed and being implemented in such a way as to yield good performance.

The WHO guideline

Based on available global evidence—including a systematic review of published literature reviews [61], 15 systematic reviews of relevant primary studies, and a stakeholder perception survey—the 2018 WHO guideline on health policy and system support to optimize CHW programmes [10, 17] provides recommendations on 15 programmatic determinants of performance. These are: selection of CHWs, duration of their pre-service training, competency domains for the curriculum for pre-service training, modalities of pre-service training, competency-based certification, supportive supervision, remuneration, contracting agreements, career ladder, target population size, data collection and use, types of CHWs, community engagement, and mobilization of community resources. Depending on the certainty of evidence about positive effects on CHW programme performance, WHO provides conditional or strong recommendations about the design of these programme components. While most recommendations are conditional, a strong recommendation is made for establishing robust community engagement strategies. WHO also strongly recommends remunerating practising CHWs for their work with a financial package commensurate with the job demands, complexity, number of hours, training, and roles that they undertake; and providing paid CHWs with a contracting agreement specifying role and responsibilities, working conditions, remuneration, and workers’ rights. Although evidence regarding the benefits of remuneration was of low certainly, these recommendations were made based on human and labour rights [17].

The CHW AIM toolkit

While the WHO guideline provides recommendations and considerations for optimization of the design, implementation, and performance of CHW programmes, it does not provide specific tools or checklists to assess the extent to which a current programme design can yield optimal performance. The often used CHW Assessment and Improvement Matrix (AIM) toolkit, developed in 2011 and updated in 2018, serves this goal [32, 33]. The updated toolkit contains a programme functionality matrix with 10 programme components that influence performance of CHW programmes: role and recruitment, training, accreditation, equipment and supplies, supervision, incentives, community involvement, opportunity for advancement, data, and linkages to the national health systems (Table 3). It is clear that these 10 programme components cover the 15 components included in the WHO guideline, as listed in the previous paragraph.

Table 3 Programmatic components of the updated CHW AIM tool [63]
Full size table

Each of the 10 components is subdivided into four levels of functionality: 1 = nonfunctional, 2 = partially functional, 3 = functional, and 4 = highly functional. For each of the programme components, each functionality level is described by situations commonly seen in CHW programmes, so that assessors can match the status of the CHW programme components against these descriptions to guide their assessment. For example, with regard to the component supervision, it goes from CHWs not or very occasionally being visited by supervisors or other health staff (score 1) to monthly, supportive supervision by dedicated and trained supervisors (score 4). The CHW AIM toolkit has been used by governmental, nongovernmental, and other stakeholders in the assessment, improvement, and planning of CHW programmes. Recently, it was used for the evaluation of the CORE Group Polio Project over a 20-year period in five countries. The successful programme, which made use of community mobilizers (CHWs), met the basic level of functioning (level 3) for six of the 10 components of the AIM tool [62].

Other ways to assess programmatic determinants of performance

Assessment of logistics and human resources can be informed by data on commodity availability from the logistics information system and by data on, for example, training and certification of CHWs from the human resources information system. Recently, a validated scale on perceived CHW supervision was published, containing six items that cover perceptions of CHWs on how they are supervised [64].

Key message 4

The 2018 WHO guideline [17] (2018) and the AIM toolkit [32, 33] provide guidance to assess a list of main programme components that influence CHW programme performance

Contextual factors influencing CHW programme performance

As stated in the Background of this paper, CHW programme performance is also influenced by the broader context. While it is not possible to change the context, at least in the short term, it is important to understand how certain contextual factors influence CHW programme components, attributes of individual CHW performance, and community-level outcomes. This can contribute to adjusting the programme to better fit in or respond to the context, resulting in better programme performance.

Kok et al. (2015) [65] conducted a literature review on contextual factors that influence CHW performance. The factors were categorized into the community context, economic context, environment, health system policy, and health system practice. The community context includes sociocultural factors (such as gender norms and values as well as disease-related stigma), safety and security, and knowledge of the target group. The environment includes geography and climate. The health system policy includes the existence of a CHW policy, human resource policy, legislation related to CHWs, and political commitment. Finally, a set of health system factors plays a role in influencing performance (health service functionality, human resource provisions, level of decision-making, costs of health services, and the governance and coordination structure). These factors should be taken into account when assessing CHW programme performance. As a recent example, Marston et al. (2020) [66] used qualitative research to provide insights into how family planning services offered by FCHVs in Nepal could be more tailored to the community context. For example, the study showed the importance of FCHVs upholding secrecy and privacy in their interactions with women to increase health-seeking behaviour. Bergström et al. [67] developed the Context Assessment for Community Health (COACH) tool, which is based on the PARIHS [Promoting Action on Research Implementation in Health Services] framework [68]. The COACH tool can be used to map and measure aspects of the local healthcare context that can influence the implementation of evidence-based interventions, including CHW programmes, in LMICs [67].

Key message 5

It is important to understand how contextual factors influence CHW programme performance, so that programme adjustments, aiming at tailoring the programme to the context, can be made to optimize performance. Contextual factors of importance are best identified through qualitative research

Gaps and needs for the future

This paper has focused on CHW programme performance and how this could be assessed. Performance assessments will differ by programme, as programmes have different features, in particular with regard to the types of CHWs, the services they provide, and their support mechanisms. As such, standardization of assessments of CHW programme performance is only possible to a certain extent. Given the fact that large-scale CHW programmes are complex entities and part of health systems, their assessment ideally needs to be based on data derived from a mix of reliable sources.

While it is useful and efficient to use existing data sources, such as routine health information systems, these sources have limitations. We already pointed out that in many settings, community-based data cannot be disaggregated from facility-based data. Routine health information systems often fail to capture data on services provided solely at a community level [30], and human resource information systems similarly do not always capture data on CHWs, especially when they are volunteers.

In addition, there are indications that data are not always reliable. Data collectors, including CHWs, sometimes see data collection and reporting as a “tick-the-box exercise”, because it concerns mere reporting up into the administrative ether rather than reporting data that are used at different levels to improve services delivery [69]. Feedback loops are often lacking. Regeru et al. (2020) [70] found that in Kenya and Malawi, there were large discrepancies between data reported by CHWs and those reported by their supervisors. Barriers to the reporting of high-quality data were (1) unavailability of standard data collection and reporting tools, (2) limited training of CHWs on data reporting, (3) parallel reporting requirements of vertical programmes, and (4) overload of CHWs tasks leading even to fabrication of data. In the study in Sierra Leone mentioned earlier, it was also found that CHW reported morbidity and mortality data were of low quality and incomplete [44].

CHISs need strengthening. Data quality assessments can stimulate improvements, as shown in the Democratic Republic of the Congo, Malawi, Mozambique, Niger, and Nigeria in relation to iCCM [71]. Digital data collection also provides opportunities for improvement of routine data collection, quality, and use [72, 73].

Maher and Cometto (2016) [74] have pointed out that research on CHWs should be focused not only on effectiveness (what works) but also on contextual factors and enablers (how, for whom, under what circumstances). George et al. (2018) [75] argue—and we underline this—that efforts to strengthen CHW programmes must “guard against the hubris of relying on a single approach or hierarchy of evidence”. Instead, understanding the implementation realities of CHW programmes will require “humility in understanding communities as social systems, the complexity of the interventions they engage with and the heterogeneity of evidence needs that address the implementation challenges faced”. Most research on CHW programmes has had a narrow focus on a specific disease or intervention, and research on large-scale integrated CHW programmes remains limited. The WHO guideline calls for new health policy and systems methodologies such as implementation research, systems thinking tools, process monitoring, and rapid synthesis of available research [17]. Agarwal et al. (2019) [76] have carried out a literature and consultative review to identify evidence gaps about community health systems and to propose a research agenda to strengthen CHW programmes. Priority research questions they identified included queries on (1) the most effective and efficient supervision and monitoring systems to improve CHW performance, (2) the added value of the use of digital technologies in supervision and monitoring, (3) policies, financing, and governance structures required to support and sustain CHW programmes, (4) CHW models that improve cost-effectiveness and quality of care, and (5) which combinations of training, incentives, and career growth opportunities increase CHW motivation and retention.

Looking at the attributes of individual CHW performance, community-level outcomes and impact at the population level (Tables 1–2, Fig. 1), it is obvious that the academic literature addresses many of these, but others are relatively neglected. More attention to community-level outcomes, such as empowerment and trust, is needed when we aim to fully understand how performance is shaped in different contexts. We also need to recognize that certain CHWs working for a programme may face different factors that influence their performance than other CHWs in the same programme and therefore need different types of support; there is no one-size-fits-all approach to improving CHW performance. As there is evidence that CHW programmes reduce inequities in access to services and promote inclusion of marginalized or vulnerable groups [77], there is also a need to disaggregate metrics to be able to assess for equity. Lastly, as referred to above, it is important to improve methods in assessing cost-effectiveness of CHW programmes [78].

Key message 6

CHW programme assessments ideally need to be based on data derived from a mix of reliable sources, obtained using a mix of methods

Conclusion

Assessments of CHW programme performance can have various approaches and foci according to the programme and its context. Given the fact that CHW programmes are complex entities and part of health systems, their assessment ideally needs to be based on data derived from a mix of reliable sources. Assessments should be focused not only on effectiveness (what works) but also on contextual factors and enablers (how, for whom, under what circumstances). Serious attention to assessments is instrumental for continually innovating, upgrading, and improving CHW programmes at scale. Programmes will need steadily increasing funding to be able to respond to the gaps and needs identified. Now is the time for a new frontier of implementation research for strengthening CHW programming.