Background

Rationale

There is wide recognition in principle about the importance of involving service users and their caregivers in health system policy and planning processes, service monitoring and health research. There is also some evidence from high-income countries that their involvement can directly lead to improved mental health system strengthening [1–5]. Participatory policy-making, planning and service monitoring is of particular importance in the field of mental health, where a large majority of people with mental disorders or psychosocial disabilities do not receive any effective treatment or care, and may at times receive treatment against their will [6]. To avoid fragmented and tokenistic inclusion of service users and caregivers in planning, or their marginalisation, and to facilitate meaningful and effective contributions, it is necessary for professionals working within the formal health system to share responsibility with representative organisations. Mental health service user networks have been emerging and are on the increase globally [7, 8]. However, to date there have been very few reports of efforts to involve service users and caregivers in mental health service strengthening in low- and middle-income countries (LMICs). Yet this information is vital in planning capacity-building programmes for service users and caregivers to effectively participate in improving mental health programmes.

This systematic review was undertaken as part of the ‘Emerging mental health systems in LMICs’ (Emerald) programme [9], which aims to improve mental health outcomes by generating evidence and capacity to strengthen mental health system performance in six African and Asian LMICs (Ethiopia, India, Nepal, Nigeria, South Africa and Uganda). One of the objectives of the Emerald programme is to empower and equip service users and caregivers to participate more fully and effectively in the mental health system, for example at the level of (i) mental health policy-making and planning processes, (ii) service development and monitoring, and (iii) mental health research. This review served to provide an evidence base to inform the Emerald programme’s capacity-building activities for service users and caregivers.

Objectives

The objectives for this systematic review were: (i) to systematically synthesise the current evidence and experience base for models of involvement of mental health service users/caregivers in mental health policy-making, mental health service development, quality monitoring and evaluation of services, and mental health research in LMICs; and (ii) to identify models of best practice for evaluation of capacity-building of service users and caregivers to facilitate greater involvement in these mental health system strengthening activities.

Methods

Eligibility criteria

The PRISMA checklist is included in Additional file 1. A protocol for the review is available upon request from the authors. Given the scarcity of evidence from LMICs with respect to service user involvement in mental health care more generally, and since initial pilot literature searches carried out by the authors did not identify many relevant papers, a limited number of pertinent resources was anticipated. A broad eligibility framework was therefore adopted, and both peer-reviewed journal papers and grey literature (such as books, unpublished reports, website resources, or training materials) were included in the review.

Whilst the primary focus of the review was capacity-building of service user and caregivers in mental health system strengthening, and the evaluation of capacity-building, a secondary focus was a broader examination of service user and caregiver involvement per se. We therefore included any kind of study design, which reviewed or reported on evaluation or experience of service user (i.e. service users with any kind of mental health problem, including those with intellectual disabilities, dementia, or child and adolescent mental health problems), family or caregiver (though not community) involvement in LMICs, and which were relevant to mental health system strengthening. This entailed direct involvement of service users and caregivers in: (i) development of policies or strategies; (ii) planning or development of services; (iii) training of health workers in mental health care; (iv) service monitoring, evaluation or quality control; or (v) mental health research. Given the dearth of evidence of service user and caregiver involvement in mental health systems, the search was expanded to also include involvement in service delivery and/or support groups, even though their applicability to mental health system strengthening was considered to be limited, unless the support groups contained an advocacy, empowerment or mobilisation component. No studies were excluded based on the type of service, for example whether they were delivered within the formal health system or through alternative avenues.

Although an explicit evaluation of service user/caregiver involvement was required within the original criteria for inclusion, a post hoc decision was made to also include studies that described service user/caregiver involvement in mental health system strengthening according to the categories outlined above, even when there was no clear evaluation strategy. Papers written in English, Spanish, Portuguese, French or German were included. Studies that reported on data solely from high-income countries were excluded.

Search strategy, study selection and data extraction

To identify relevant peer-reviewed literature, we searched the following databases: MEDLINE [1946 to December 2013], Embase [1974 to December 2013], PsycINFO [1806 to December 2013], Web of Knowledge, Web of Science, Scopus, CINAHL, LILACS, SciELO, Google Scholar and Cochrane [all from the start date of the database to December 2013]. The search strategy is detailed in Table 1.

Table 1 Search strategy used in the database search
Full size table

Further relevant literature was identified by: (i) hand-searching reference lists (for example in relevant reviews or journal papers); (ii) a snowballing process of contacting experts active in the area; and (iii) by searching the internet (through use of the Google search engine). Two researchers reached consensus on whether resources that were identified via the grey literature process were eligible for inclusion into the review. Any non-peer-reviewed grey literature found during this search fed into the ‘Discussion’ section of this paper.

For literature that was identified through the database search, each of the titles and abstracts of papers that the search generated were assessed by two independent reviewers. Any paper that was considered to be relevant by either of the two reviewers was included for full-text review. Full-text papers were then screened by two independent reviewers. Where there was disagreement between the independent reviewers of the full-text articles, two senior reviewers made the final decision as to whether the paper fulfilled the inclusion criteria.

A data extraction form (see Tables 2, 3, 4 and 5 for the data obtained from the completed forms) was designed prior to running the review, and the data were extracted by two independent reviewers [one reviewer was used for papers that were written in languages other than English]. The data taken from selected papers was not amenable to statistical synthesis through meta-analysis and was, instead, synthesised into a narrative review, as data were heterogeneous between studies and much of the data were qualitative.

Table 2 Overview of quantitative data studies included in the review
Full size table
Table 3 Overview of studies with quantitative and qualitative data that were included in the review
Full size table
Table 4 Overview of qualitative data studies included in the review
Full size table
Table 5 Overview of descriptive non-data-based studies included in the review
Full size table

Quality assessment of papers

To assess the quality of papers that were included in the review, for studies containing quantitative data, the ‘Quality assessment tool for quantitative studies’ by the ‘Effective Public Health Practice Project’ (EPHPP) [10, 11] was used (see also http://www.ephpp.ca/tools.html) (see footnote of Table 2 for the eight categories that the methodology includes). A global quality assessment rating of ‘strong’, ‘moderate’ or ‘weak’ was then assigned based on the responses within each of the eight categories. For studies containing qualitative data, twelve review criteria were used, based on those suggested in the literature on qualitative research, as described in Harden et al. [12] (see footnote of Table 4 for the twelve review criteria). Where studies included both quantitative and qualitative data, an assessment was made for both types of data, using the methodologies described above (see Table 3). No quality assessment was made for non-data-based descriptive studies (see Table 5), or for (non-peer-reviewed) grey literature.

Ethics statement

No ethical review was required as there was no research on human participants.

Results

Study selection and characteristics

Figure 1 displays a flow diagram showing the process that was employed in the selection of peer-reviewed articles. In total, 20 papers were included in the narrative review.

Fig. 1
figure 1

Flow diagram for selection of peer-reviewed articles (format taken from [42])

Full size image

The characteristics of the peer-reviewed studies that were eligible for inclusion are shown in Tables 2, 3, 4 and 5. Of the 20 studies, only four included an explicit evaluation of service user/caregiver involvement and thus met our original criteria for inclusion. A further six studies included an evaluation component (though no evaluation of service user or caregiver involvement), six were descriptive data-based studies, and four studies were non-data-based descriptive pieces. Only one (cluster)-randomised controlled trial (RCT) was identified. Across studies, twelve countries were included that were upper-middle income at the time of publication (seven studies from South America, two from Asia, one from Africa, one from Europe, and one from Europe/Asia [Russia]), five lower middle-income countries (three studies from Asia, one from Africa, and one from Europe), and eight countries that were low-income at the time of publication (five from Africa and three from Asia) (according to the World Bank, see http://data.worldbank.org/); a further study included data from 63 different countries or regions [13]. The majority of studies (n = 16) included at least one author from a LMIC; four did not [13–16]. Of the 20 papers, eleven were published after 2010, and nine studies were published between 2000 and 2010. Fifteen papers were published in English, three in Spanish [17–19] and two in Portuguese [20, 21].

In accordance with the type of data collected in studies, eight studies included quantitative outcomes and eleven studies included qualitative outcomes/evaluation methods (of which three studies included both types of outcomes/evaluation methods) (see Tables 2, 3, 4 and 5). Of these studies, only four (two quantitative [22, 23] and two qualitative [24, 25]) included outcomes that clearly assessed how service user or caregiver involvement impacted on service users and/or caregivers themselves (for example, in terms of number of hospitalisations, mental health status or coping capacities of service users, caregiver burden). Five studies (two quantitative [26, 27], three qualitative [18, 28, 29]) contained study outcomes that were intended to be improved through service user participation (outcomes included, for instance, needle turnover rates, factors around the sustainability of mental health organisations, development of a scale, establishment of a mental health policy). For six of the studies, the primary outcomes related to satisfaction data or data on perceptions or attitudes (one quantitative [30] and four qualitative studies [15, 20, 27, 31], plus one with both types of data [14]). A further two studies [13, 32] included outcomes at the country level (such as access to medications, number of mental health staff, policies). Four studies did not include any outcomes, as they were non-data-based and descriptive (see Table 5).

In the non-peer-reviewed grey literature, in addition to various websites by service user groups or organisations working with service users or caregivers, five relevant reports were identified, as well as one self-advocacy toolkit, and one newsletter. From these seven sources, information was included from eight low-income countries (Indonesia, Kenya, Lao PDR, Malawi, Nepal, Rwanda, Tanzania, Uganda), three lower middle-income countries (Ghana, India, Sri Lanka), and one upper middle-income country (South Africa), at the time of publication. Two reports were produced after 2010, four between 2005 and 2010, and one before 2000.

Quality assessment of papers

The majority of included papers (n = 8) were qualitative or non-data-based studies (n = 4). Five studies presented quantitative data only, with a further three studies including both quantitative and qualitative data. The quality assessment of papers is shown in Tables 2, 3 and 4 for individual studies. Overall, the quality of papers that were included in the review was weak. For studies that included quantitative data, the quality was categorised as weak for all but one paper, for which the quality was categorised to be moderate (the cluster-RCT, which involved peer-facilitated women’s groups, and therefore was one of the papers that was included in the review under the secondary criteria). For papers that included qualitative data, no papers fulfilled all twelve quality assessment criteria, though two papers fulfilled eleven of the criteria. The average number of criteria that were fulfilled was 7.3 (ranging between 4 and 11).

Findings of peer-reviewed studies

Overall, the studies that were identified in this review provided a weak evidence base for service user and caregiver involvement in mental health system strengthening in LMICs. Many were of low quality (see above) and just four studies included an explicit evaluative element of service user/caregiver involvement. Most of the literature reported about service user and caregiver involvement at the service-level rather than the systems-level, and commonly involved service users as research participants for the evaluation of services rather than their direct participation in the development of policies or services, the training of health workers in mental health care, or within mental health research. There were also few reports that evaluated service user involvement. The level of service user involvement reported in the literature varied considerably across countries. However, there were a few favourable studies that provided some indication as to research areas that could be pursued further, in particular about the development of policies and strategies, including advocacy work, and to a lesser extent the development of services, service monitoring and evaluation.

Development of policies or strategies

The main evidence from the peer-reviewed literature in regards to the development of policies or strategies reported on the usefulness and feasibility of consultation processes with service user involvement, and showed that these processes may lead to an improvement in mental health services and/or outcomes. A study that used data from the World Health Organization Assessment Instrument for Mental Health Systems (WHO-AIMS) from 63 countries, for example, showed that participation of family-based organisations in the development of mental health legislation increased availability of psychotropic medication in countries by 17 % [13]. Another paper provided a descriptive account of a consultation process in Colombia with representatives of patients, families, medical students and mental health workers to derive a declaration of mental health patients’ duties and rights [18]. One challenge that has been reported is that people in power may not be willing to give up control, which may result in the exclusion of service users [16]. To address this imbalance in power relations, grassroots public action has been suggested. Hayward & Cutler [16], for instance, described the coalitions between grassroots organisations and people with mental health problems in Romania to develop policies to promote community-based services.

A qualitative study by Ndyanabangi et al. [32] that involved interviews and focus groups with service users, policy makers and health providers in Uganda highlighted the importance, usefulness and feasibility of capacity-building activities for service users and caregivers in advocacy skills. The study established that service users were rarely informed of their rights, so hardly ever made complaints, but that the recent increase in service user organisations had led to an enhanced knowledge base for service users in their rights overall. Similarly, a study in Argentina and Chile found that with structured support, people with intellectual disability (ID) were able to exercise their rights to a level comparable to peers without ID [26]. This may be relevant to policy development in that if service users are not aware or do not have information about their rights, their contribution to policy development is likely to be limited or altogether absent.

Planning or development of services

There was very little evidence in the peer-reviewed literature on how best to involve service users and caregivers in the planning or development of services. The best research identified was from a case study on the involvement of community members in the development and delivery of psychosocial interventions for women with depression in South Africa. Through a participatory implementation framework, the service users and service providers were actively involved in the development of the interventions (through in-depth focus groups and consultations), such as a peer facilitated group intervention based on the principles of interpersonal therapy for people with depression [24].

Service monitoring and evaluation

The review identified several peer-reviewed papers that involved service users and/or caregivers in the evaluation of mental health services, for example using satisfaction data [14, 20, 31, 33], though few of these included either an evaluation of service user involvement or service user involvement in the monitoring of services themselves. That is, service users or caregivers did not participate in the process of assessing satisfaction levels (monitoring) or devising appropriate responses (service development). However, a useful methodological framework for the evaluation of mental health services was presented in a commentary on the evaluation of mental health services in Brazil, where the participation of service users in the service appraisal was ‘gradual’. That is, researchers were exclusively involved at the start of the project (due to funding and time constraints), but then other stakeholders participated later in the analysis of results, and the final workshops and dissemination [21]. The framework proposed a synthesis of perspectives and recommended five dimensions that should be considered in the participatory evaluation process.

Mental health research

The review produced no evidence on how best to involve service users or caregivers within mental health research in LMICs.

Training health workers in mental health care

The review produced no evidence or reports on service user or caregiver involvement in the training of health workers in mental health care in LMICs.

Service delivery and support groups

The review showed that there is some evidence for the benefits of service user or caregiver involvement in service delivery and/or support groups. This has included involvement of peer educators in needle exchange programmes for alcohol and drug abuse in China [27]; the employment of service users’ family members as case managers for people with schizophrenia in Iran when compared to psychology graduates [22]; service user and carer self-help groups in several African countries [28], such as Ghana [25]; as well as women’s groups led by peer facilitators to reduce moderate maternal depression in India [23].

Discussion

Summary of evidence

Overall, this systematic review showed that although there were signs of service user and caregiver involvement in mental health system strengthening in numerous (about 26) countries, there was a lack of high-quality research and a weak evidence base for the work that was being conducted across countries (up to the end of 2013). Most of the literature reported service user and caregiver involvement at the service-level (for example, in regards to the delivery of services such as self-help and support groups) rather than the systems-level (such as at the policy, planning, monitoring or evaluation level), and commonly involved service users as research participants in the evaluation of services (for example, surveying their satisfaction with services) rather than in the direct development of policies or services, the training of health workers in mental health care, or within mental health research. Indeed, no evidence at all was found for the latter two issues, and therefore underlined the substantial gap in the literature in these areas. There were also few reports that evaluated service user involvement. Furthermore, outcomes were often vague in terms of their link to service user involvement; many of the outcomes measured related to perceptions, attitudes or satisfaction data with no clear and direct relation to the impact of service user involvement on mental health system strengthening, particularly in relation to (i) the effects of service user or caregiver involvement on service users or caregivers themselves, or (ii) the impact of their involvement on services.

One reason for these findings may be that most of the service user organisations are still few and fragmented, and that it is difficult to document some of their best practices. However, there were some encouraging studies on the development of policies and strategies, including advocacy work, and to a lesser extent the development of services, service monitoring and evaluation, with some indication to areas that could be pursued in the future.

The review showed overall that service user and caregiver involvement in mental health system strengthening is possible, and there are tentative signs that their direct involvement may lead to improvements in mental health services and outcomes. Generally, research on service user or caregiver involvement in mental health system strengthening seems to be on the rise, as most research has been published in the last ten years. On the other hand, whilst there seem to be good intentions in some countries, these are often not translated into practice. For example, in Argentina, even though there has been an increase in service user involvement over the last twenty years, this has not yet been translated into substantial changes in services or an improvement in service users’ inclusion and quality of life [17]. It was also clear from the literature review that service user and caregiver involvement varied significantly across countries. For instance, whilst in Uganda, quite a large presence of service user advocacy groups is described in the grey literature [8, 34–36], other countries, such as Malaysia, have reported a lack of evolved service user perspectives [15]. Since there are a few countries that have greater awareness in this area than others, it will therefore be important for models of best practice to be shared across countries and regions.

Table 6 provides an overview of recommendations for future studies on service user or caregiver involvement in mental health system strengthening. One important recommendation is for further research to incorporate rigorous evaluative elements of service user and caregiver involvement. This could include high-quality studies such as RCTs, as well as participatory approaches. Case studies that capture the complexity of the impact of service user and caregiver involvement may also yield more revealing insights. Without an evaluation of interventions, the allocation of scarce resources becomes difficult, given the potential for harm (for example, due to stigma), so it is important not to assume that every intervention is beneficial. Future research needs to clarify which interventions are valuable, and also acceptable within different socio-cultural contexts, and which are less so.

Table 6 Recommendations for future studies on service user and caregiver involvement in mental health system strengthening
Full size table

Grey literature

A search of the grey literature identified several reports of encouraging projects and groups that incorporated participation by service users, their families or caregivers across several countries, particularly in relation to advocacy and empowerment work. A growing number of organisations (in particular non-governmental organisations (NGOs)) across various low- and middle-income countries advocate for service users’ rights and needs, with the recognition that not all service users and families may be interested in roles of activism. These include, for example, Action for Mental Illness (ACMI) in India (see http://mhinnovation.net/organisations/action-mental-illness-india-acmi), who campaign for service user involvement at the social level (for example, running empowerment and advocacy programs for family carers and service users); political level (such as lobbying and negotiating); the legal (state) level; and media level (including civil society and publications). Others include the World Network of Users and Survivors (WNUSP) Working Group (see http://www.wnusp.net/), which was established in 2013 and is a global forum to promote the rights and interests of service users; the Pan-African Network of People with Psychosocial Disabilities (PANUSP) [8] (also see http://www.panusp.org/), which was established in Uganda in 2005 and now extends to nine African countries; and the NGO BasicNeeds, which was founded in 1999 and now has programmes (including empowerment and self-help groups) in at least eight countries across Africa and Asia [37]. A useful toolkit has been developed by BasicNeeds together with the NGO CBM [34], for service users and caregivers who are planning to lead advocacy initiatives. The toolkit was pre-tested by six self-help groups in Uganda (one of the countries that is at the forefront of service user involvement in LMICs), and was peer-reviewed by a wide range of self-advocates and development workers. The toolkit contends that district policy-makers and programme implementers may respond better to advocacy issues that are raised by people affected themselves.

Mental health activists have stressed the importance of taking a holistic approach, in which not only service users, but also caregivers, the surrounding community and decision-makers are engaged, and in which the socio-cultural context is considered. It is also important to take into account the national picture of service user involvement when developing interventions. For example, interventions must consider whether service user groups already exist whose expertise and experiences can be utilised for future involvement, or whether interventions need to focus heavily on empowerment or the establishment of new service user groups. The importance of considering the traditions and values of the socio-cultural context in the implementation of intervention programmes or capacity-building activities (for example, in terms of gender, level of education, or rural–urban cultures) has also been highlighted by advocacy organisations such as ACMI (see above).

In regards to service planning, development and evaluation, a report in the grey literature by TPO (Transcultural Psychosocial Organisation) Uganda [35] described a successful model to scale up mental health services and trauma support in war-affected communities. The process involved service users in the evaluation process (through focus groups and key informant interviews), and patient support groups. The report recommends that services should be monitored and/or evaluated once a year by a wide range of stakeholders, including service users. The use of peer educators for needle exchange programmes for alcohol and drug abuse in Indonesia has also been described in the grey literature [38].

Limitations

There were several limitations to this review: 1) in regards to the comprehensiveness of studies included, seven studies were excluded because the full-text papers could not be accessed, and one study had to be excluded based on its language of publication (Korean). This may have resulted in a bias in the types of papers that were included in the review (although due to the low number of excluded studies based on these criteria, this bias is unlikely to be substantial). 2) The quality of studies that were included in the review was weak overall, which will have had an impact on the strength of the evidence. This highlights further the need for high-quality research in the area. 3) Systematic reviews are commonly subject to a publication or reporting bias [39–41], whereby only positive outcomes are reported, and unpublished reports are missed. The inclusion of grey literature in the review may have reduced some of this bias, although it is likely that not all eligible resources were identified, given the unsystematic nature of grey literature searches. 4) The review was not listed on an international prospective register of systematic reviews such as PROSPERO (see http://www.crd.york.ac.uk/PROSPERO/), though the original protocol was agreed by all of the co-investigators and is available upon request.

Conclusions

This systematic review showed that the evidence for how best to involve service users and caregivers in mental health system strengthening in LMICs is not easily accessible in the literature, as well as evidence for the evaluation of user involvement (i.e. how useful or effective user involvement is). Furthermore, despite a few emerging studies, there is still a paucity of high-quality research, especially in regards to service user involvement in the development of policies and strategies, the planning and development of services, the training of health workers in mental health care, and within mental health research. It will therefore be important to develop, test and evaluate models of best practice in the future through rigorous and systematic research. One way in which to do this may be to empower service user organisations to deliver those services that they are best at, which in turn may facilitate the documentation of their best practices.