Background

The global burden of mental, neurological and substance use (MNS) disorders is high, resulting from chronic disability combined with premature mortality [1, 2]. Untreated MNS disorders also impact negatively on global health priorities [3], and may be associated with human rights abuses [4]. The majority of people with MNS disorders in LMICs are unable to access effective mental health care, with the treatment gap higher than 90 % in some low-income countries [5].

Health system constraints are potent threats to the scale-up of evidence-based mental health care for people affected by MNS disorders in LMICs [6]. Policy-makers and planners play a critical role in the successful strengthening of mental health systems, but despite historical attempts at capacity-building [7], may not be appropriately equipped for the task. The limited evidence and evaluation of capacity-building interventions for policy-makers in LMICs has been recognized in physical health care [8] and is likely to be even more significant in mental health care. In a mixed-methods report of implementing a national mental health policy in South Africa, barriers included the relatively low priority given to mental health care by planners, provincial bureaucracy around service coordination, insufficient staff for policy-making and service planning, and disinclination by some local authorities to lead mental health policy implementation [9].

In a qualitative study involving national and regional stakeholders in Ghana, South Africa, Uganda and Zambia, low perceived legitimacy of the problem of scaling up mental health services and inadequate government support were identified as factors perpetuating the low priority accorded to mental health care [10]. A qualitative survey of leaders and specialists in international mental health specifically identified the need for a broader and more holistic view of mental health care, adopting a more public health-level perspective among mental health policy-makers [11]. In particular, the lack of training and experience of clinicians to fulfil leadership roles was emphasized as a barrier to high quality mental health policy-making.

There is international consensus on the need for mental health system strengthening and for a specific focus on building the capacity of key stakeholders, including policy-makers and planners, and service users [12]. ‘Systems thinking’ casts health services as a type of complex adaptive system, characterized by large numbers of individuals occupying a range of roles, acting in the context of and adapting to constant internal and external changes [13]. It advocates a ‘systems perspective’ which unifies health care with education, research and policy-making, through “collaboration across disciplines, sectors and organizations; ongoing, iterative learning; and transformational leadership” [14]. This approach in part represents a reaction to the former emphasis on targeted, often disease-specific health investments, which have not demonstrated benefits for the wider health system [15]. However, the specific means by which the capacity of mental health policy-makers and planners should be built, to facilitate such systems-wide scaling up, are less well-documented. A recent systematic review of the involvement of mental health service users and their caregivers in mental health system strengthening, identified a lack of high quality research [16].

Given the widespread support for building the capacity of policy-makers and planners in LMICs to achieve mental health system strengthening, this systematic review aimed to identify the best evidence and experience for specific capacity-building intervention models. A secondary aim was to identify methods of evaluating models of capacity-building for mental health policy-makers and planners in LMICs.

Methods

This systematic review formed part of the ‘Emerging mental health systems in LMICs’ (Emerald) program, which focuses on the health system inputs, processes and outputs required for mental health service scale-up in LMICs [17]. A systematic investigation of existing evidence-based strategies was an important precursor of the Emerald project, since one of its aims is to build the capacity of policy-makers and planners to strengthen the mental health system. This work was registered on the PROSPERO international prospective register of systematic reviews (Registration Number: CRD42016032798).

Eligibility criteria

In view of the anticipated scarcity and heterogeneity of relevant literature, the criteria used for selection of studies into this review were intentionally broad and inclusive. The review set out to include any type of study design, review or report on the evidence, experience or evaluation of capacity-building of policy-makers, service planners or managers in mental health system strengthening in LMICs. Studies were included whenever the recipients of system-focused capacity-building were either currently involved in policy-making, planning or co-ordination of services, or when recipients had the potential to take on such a role, regardless of their professional background. Mental health systems were interpreted to include services addressing the priority MNS disorders outlined in the World Health Organization’s mental health Gap Action Program [18]. Papers written in the following languages were included: English, Spanish, Portuguese, French or German. Studies were excluded where system level interventions to support implementation and expansion of mental health care were implemented by external agencies and did not report specifically on the building of local capacity. A common reason for exclusion was that studies described mental health training of clinical staff without discussing training needs of mental health policy-makers, or commented on the need for capacity-building without presenting any interventions to build policy-makers’ capacity. Studies that reported solely from high-income countries were excluded.

Search strategy

The following databases were searched on 12 November 2013: MEDLINE (from 1946 until December 2013), Embase (1974 to December 2013), PsycINFO (1806 to December 2013), Web of Knowledge, Web of Science, CINAHL, LILACS, ScieELO, Google Scholar and Cochrane and on 27 November 2013, Scopus (all from the start date of the database to November 2013). The search was updated on 25 November 2015 to look for any papers published in the intervening period of two years. Details of the search strategy used are given in Additional file 1. In addition to database searches, the reference lists of included papers were hand-searched for relevant studies and experts in the field were contacted to identify any further studies. Grey literature including reports and web-based resources were identified via the Google Scholar search and experts in the field.

Study selection and data extraction

Figure 1 shows the flow of papers from identification to selection and data extraction. The titles and abstracts of all 2950 papers identified in the search were assessed by two independent reviewers. This number comprises 2040 papers identified in the first database search up to November 2013, a further 882 papers identified in the second database search up to November 2015 and 28 papers identified by contacting experts or key informants, hand-searching references, and searching the internet. In the first search, the 1704 papers excluded at the stage of abstract review were either not eligible for inclusion due to their subject matter (1581), because they were duplicates (66), based in high income countries (51) or due to language (6). In the second search, 837 papers were excluded at the stage of abstract review because of ineligible subject matter (827), based in high income countries (8) or due to language (2). Papers were considered to pertain to LMICs if the work described took place in a year in which the country was classified by the World Bank [19] as a low or middle-income country.

Fig. 1
figure 1

PRISMA flow diagram for selection of peer-reviewed articles

Full size image

This process yielded a total of 409 papers which were reviewed as full text articles. For the screening of full text papers identified by the database search, all articles were reviewed independently by at least two reviewers (RK, SE-L, JA or MT). Additional papers identified via experts and informants and hand-searching references and the internet were screened independently by SE-L, JA, MT or RK. Where there was disagreement between independent reviewers of full text articles regarding inclusion, the paper was reviewed by two senior independent reviewers (CH and MS), who made a final decision. In several papers, the language used to describe participant groups and interventions was vague, creating ambiguities about whether the work constituted clinical training alone or clinical training combined with capacity-building. In such cases, papers were reviewed again and where necessary, input from a third independent reviewer was obtained. MS reviewed 7 full texts where there was disagreement, leading to inclusion of 4 studies. CH reviewed 5 full texts where there was disagreement and included 3 studies. This was sufficient in all cases and contacting the authors of such papers was not required.

Of the 409 full text papers screened, 395 were excluded due to not addressing the focus of this review: capacity-building interventions for mental health policy-makers and planners.

Data from included papers were extracted by independent reviewers (RK, MT and JA) and inputted into a data extraction form (Additional file 2). The data extracted included: authorship (noting whether or not this included authors from LMICs), type of publication, countries involved, study design, capacity-building participants, sample size, nature of capacity-building and any guiding framework, type of evaluation, type of evaluation data and a summary of findings.

Quality evaluation

Given the mixed methods employed by included studies, their quality was evaluated using two methods: one to assess quantitative aspects (Additional file 3) and one to assess qualitative aspects (Additional file 4). Included papers were assessed by MT and RK using the Effective Public Health Practice Project (EPHPP) Quality Assessment Tool for Quantitative Studies [20], which rates studies from 1 (‘strong’) to 3 (‘weak’) for selection bias, study design, confounders, blinding, data collection method, withdrawals and dropouts. These scores are used to obtain a global rating of strong (no ‘weak’ ratings), moderate (one ‘weak’ rating) or weak (two or more ‘weak’ ratings). Included papers were also evaluated by MT or RK against 12 criteria for high quality qualitative research, derived from the literature and previously employed to distinguish papers of low, medium and high quality [21, 22]. The specific criteria are included in a footnote to Additional file 4 and pertain to quality of reporting, use of strategies to increase reliability, validity and rigor. Studies were assessed as of low quality if they met seven criteria or fewer, medium quality if they met between seven and nine criteria and high quality if they met ten or more criteria.

Results

Following the process of systematic searching, 14 papers fulfilled the inclusion criteria and were included in this review. The data extracted from included papers were not amenable to statistical synthesis through meta-analysis, being heterogeneous and largely non-numerical. The study findings were therefore reviewed in a narrative manner instead.

Included papers: capacity-building interventions for mental health policy-makers and planners

The 14 papers included in the review described nine capacity-building interventions. One intervention aimed to equip policy-makers and planners with “knowledge, skills and attitudes required to make the paradigm shift and the translation into the effective provision of health care” for people with substance use disorders [23]. Another implemented a range of interventions to scale up a harm reduction program for people who inject opioids [24]. Three interventions of the nine were orientated towards developing skills in planning and co-ordination of mental health care [25–30] and a fourth did so with a specific focus on de-institutionalization of children’s services [31]. Two programs focused specifically on enhancing mental health leadership [32–35] and a further intervention aimed to build regional capacity to prevent and control mental illness and other non-communicable diseases (NCDs) across 24 countries [36].

Each capacity-building intervention combined short training courses or workshops with a system of continued mentorship or dialogue. In most cases the interventions were not specific to policy-makers and planners but to any person involved in mental health service co-ordination, planning or development, including mental health professionals. The nine capacity-building interventions described by the papers included in this review are summarized in Table 1.

Table 1 Summary of capacity-building interventions
Full size table

Methods of evaluating capacity-building interventions for mental health policy-makers and planners

The strength of evidence to support any particular capacity-building approach for policy-makers and planners was low. One study used a conceptual framework for evaluation, which included the collection of primary evaluation data [23]. In that case, external and internal evaluation processes were employed to collect complementary data on the success of the program. The external evaluator collected data from documentary analysis of subsequent funding applications, project manager reports, telephone and face-to-face interviews with UK partners and Russian participants. This was combined with an ‘insider’ evaluation influenced by ‘utilization-focused’ approaches [37]. The authors applied a systematic framework used in previous substance misuse research [38] because it was considered to account for the complex nature of cross-cultural education and training of health care staff. The mental health leadership and advocacy (mhLAP) intervention embraced a systematic approach to evaluation, with consideration of numerical process indicators, such as numbers of policy-makers and planners participating in the national stakeholder council, output indicators, such as numbers of advocacy activities by type and outcome indicators, such as preparation of a country-specific situational analysis report [35].

For the other capacity-building interventions, a descriptive account of how the mental health system had been strengthened was given, but with no clear methodology to guide the evaluation. In none of the reports did the evaluation methodology attempt to isolate the impact of the capacity-building intervention from other contextual factors that might also have led to the reported outcomes. For example, when multi-faceted interventions were implemented which targeted both clinical service delivery and capacity-building of policy-makers and planners, efforts were not made to independently assess the impact of capacity-building activities, in isolation.

Impact of capacity-building interventions for mental health policy-makers and planners

The 14 papers included in this narrative review presented evidence of success, including process indicators (e.g. number of people trained within the program), output indicators (e.g. participation of trained people in mental health system strengthening activities) and outcome activities (e.g. indications of increased commitment to mental health care scale-up, through preparation of implementation plans, committing more resources and efforts to support reduction of stigma, discrimination and abuse). Details of the evaluations are presented in Additional file 5. Information about outcome indicators was generally not collected but when presented, was not robustly evaluated. For example, the outcome indicator “evidence of heightened awareness about the salience of improving mental health services and/or reduction in stigmatization” following mhLAP varied from “efforts to improve” one mental health centre in Liberia and “rehabilitation of homeless mentally ill citizens in some states” in Nigeria to “plans to review the mental health legislation are on-going” in Gambia [35]. More general ‘lessons learned’ were more frequently reported, which focused on the importance of building sustainable, high quality relationships and the need for continued mentoring and joint activities to support mental health systems strengthening.

Quality assessment

Quality assessment of quantitative aspects using the EPHPP tool found all included studies to be ‘weak,’ with between 3 and 6 ‘weak’ ratings across 8 domains. Assessment of qualitative aspects found 12 included papers to be of low quality (scoring less than 7 out of 12), and two more recent papers to be of medium quality (scoring between 7 and 9 out of 12: [30, 31]).

Excluded papers

Many papers were excluded because they referred to service level capacity-building, for example, building capacity in health workers to deliver mental health care. In several excluded papers, policy-making and planning activities to strengthen mental health systems were described, for example, raising awareness and advocating for change, establishing a Ministry of Health mental health co-ordination desk or implementing guidelines for reliable drug supplies, but no specific interventions to build the capacity of policy-makers and planners to implement these activities were described [39–43]. Similarly, four of the 28 papers identified outside the database search described innovative mental health development work in Latin America and the Caribbean [44–47]. Despite detailed documentation of the work’s historical context, political shifts and clinical detail, the lack of description of capacity-building of policy-makers and planners prohibited their inclusion.

In some cases, an outside agency carried out the system strengthening activity [40, 41] and, in other cases, a ‘sustained policy dialogue’ was undertaken [39, 43]. These studies were excluded because the policy dialogues were not described or evaluated. Similarly, WHO developed training materials (including PowerPoint presentations, workshop facilitation guidelines and case materials) to build capacity for mental health policy and service development for LMICs, linked to the 13 modules of its Mental Health Policy and Service Guidance Package [7]. These resources provide practical, step-by-step guidance for developing national mental health policies, information systems and strengthening human resources and can be adapted for training purposes. However, since no formal evaluation of their impact on policy-makers and planners was published, these interventions were not included in this review.

Discussion

This systematic review was prompted by the need for evidence on the most effective approaches to building capacity of mental health policy-makers and planners in LMICs. Health policy and systems research methods as they pertain to LMICs have been a neglected field [48]. Challenges identified include under-funding, fear that health policy and systems research does not yield generalizable findings, poor quality and a lack of alignment between research foci and policy-makers’ needs. The negative impact of a weak evidence base in health systems research on clinical outcomes, such as those specified in the Millennium Development Goals, is not a new observation [49].

Our study demonstrates that a restricted number of capacity-building interventions for policy-makers and planners in LMICs to strengthen mental health systems have been developed, implemented, evaluated and described in published literature. However, the quality of evidence evaluating the effectiveness of the different capacity-building approaches trialled is generally low and little attempt has been made to isolate the impact of the intervention, distinct from other contextual factors. Nevertheless, several features of effective capacity-building (Table 2) and evaluation of such interventions (Table 3) are suggested by this review.

Table 2 Features of effective capacity-building suggested by this review
Full size table
Table 3 Recommendations for evaluation of capacity-building interventions suggested by this review
Full size table

Management and leadership skills for general health systems strengthening, targeting decision-makers in low-income countries, are one area which has begun to be explored, with evidence of effective programs in post-conflict Liberia [50]. The importance of tailoring leadership capacity-building interventions to the stakeholder context was identified by a qualitative study involving health care leaders from Ethiopia, Ghana, Liberia and Rwanda [51]. The authors criticized the largely Western-centric literature on leadership capacity-building for neglecting the historical, political, and socio-cultural aspects of leadership in different countries, with health care leadership in sub-Saharan Africa still less well-studied. It is, perhaps, unsurprising, that capacity-building for mental health leadership in LMICs has been even less thoroughly researched.

This review identified two courses building mental health leadership capacity for participants from nine African [35] and 18 Asian countries [32–34]. Process and output indicators supported the favorability and perceived relevance of both leadership courses and reported that delegates went on to engage in mental health strengthening activities in their countries of origin.

Disease-specific health investment can distort national priorities and allocation of staff [15]. As a result, the need for renewed focus on wider health systems strengthening has been increasingly supported [14]. In each capacity-building intervention identified by this review, specific training for policy-makers and planners was embedded within a larger program of activities to support mental health system strengthening. A range of inter-related achievements were reported by each multi-faceted intervention, including staff attitude change [23, 31], increased funding [24, 25] policy change [26–29], education and qualifications for stakeholders [30], pursuit of doctoral and other research by participants [32–34] and creation of national [35] and international stakeholder councils or consortia [36]. These findings support a ‘systems thinking’ approach, whereby unified interventions collaboratively target health care, education, research and policy-making, acknowledging the dynamic nature of health services as complex adaptive systems [13]. The need for longer-term mentoring and partnership, often between low and high income countries [23, 25, 30, 32–35], underpinning shorter-term interventions, built on high quality relationships, emerged as being particularly important for success. Many of the partnerships reported developed over years, usually for time periods dictated by the duration of external funding. While the optimal duration of partnership could not be inferred by this review, the importance of long-term partnership working was evident across the range of published studies.

However, despite yielding demonstrable benefits, the complex systems-wide approach to capacity-building reported by each study rendered isolated evaluation of specific components difficult, especially given uncertainty about which ‘active ingredients’ of the intervention contribute to outcomes. The International Mental Health Leadership Programme (iMHLAP) model prioritized leadership skills, but also provided training in a broad range of topics relevant to mental health system strengthening [32, 33, 35]. Training courses developed for provincial and district-level planners in Kenya and Tanzania [26–29] were not well-described, but focused on practical skills in mental health service planning, budgeting and co-ordination. The difficulty associated with presenting evidence for an individual capacity-building intervention for mental health policy-makers and planners epitomizes the neglect of health policy and systems research methods in LMICs [48].

No study presented evaluation findings on the impact of specific capacity-building initiatives upon the mental health system strengthening competencies of participants. Furthermore, the optimal duration of training workshops for this target group remains unclear, having ranged from several days in Kenya and Tanzania to two weeks in West Africa and four weeks in Asia; a balance is required between adequate content and duration and feasible roll-out to busy professionals. Neither was evidence presented for the impact of course accessibility and staff turnover in course leadership positions on the outcomes of such interventions. As has been previously argued, the weakness of health policy and systems research methods in LMICs may compromise the efficacy and longer-term impact of capacity-building programs [49].

The evaluation of capacity-building interventions embedded in larger programatic interventions is inevitably complex. The indicators identified by this review provide potential starting points for more in-depth evaluation. The use of a systematic evaluation framework, as used in the capacity-building intervention targeting substance misuse in Russia, may be less susceptible to bias than more descriptive accounts of program outcomes [23]. Qualitative exploration, incorporating semi-structured interviews with all individuals responsible for policy-making and planning of mental health services, including those who did not participate in specific interventions, may yield insights into the mechanisms by which capacity-building activities work [10].

Training evaluation offers an important process indicator that may help to isolate the immediate impact of capacity-building interventions for mental health policy-makers and planners. Formal case study methodology may also facilitate a systematic, in-depth but contextualised evaluation of impact [3–52]. A recent study of capacity-building activities to apply evidence from general health research to policy-making in four LMICs proposed a framework for measurement and evaluation [8]. Advocating a more systematic approach to evaluation, the authors propose that capacity-building interventions should be assessed in terms of the extent that they increase access to, interaction with and receptivity to research evidence, with respect to the three levels of the individual, the organization and the institution. For example, increased access to research papers works at the individual level, whereas evidence-based policies work at the institutional level. A similar framework could provide a more systematic approach to evaluating interventions specifically focused on capacity-building of mental health policy-makers and planners.

A potential limitation of our review could be the tendency for reports on capacity-building for this target group of policy-makers and planners to be located within the grey literature, so that we may have missed relevant studies.

Conclusions

Despite international consensus regarding the need for in-depth, system-wide capacity-building interventions for mental health policy-makers and service planners in LMICs, there is a paucity of published research studies and service evaluations in this field. This systematic review identified just fourteen studies describing nine different approaches, focused largely on systems-wide interventions to enhance mental health services, or on leadership courses. All studies shared a broad focus on partnership and high quality relationships, in keeping with the shift in focus from disease-specific investment to wider health system strengthening to build mental health capacity in LMICs. The preliminary evaluations presented here support a positive impact on process and output indicators. However, systematic research and evaluation methods were absent from all studies, compromising the potential to draw conclusions regarding outcomes on mental health system strengthening. This review demonstrates the need for high-quality capacity-building interventions for mental health policy-makers and planners, assessed using rigorous mixed methods to capture both quantitative and qualitative indices, as part of longitudinal evaluation of multi-faceted partnership approaches to mental health system strengthening in LMICs.