Background

The 2006 United Nations Convention on the Rights of Persons with Disabilities (UNCRPD) states that the achievement of equal rights, empowerment and social inclusion of people with disabilities requires comprehensive rehabilitation services encompassing educational, social, economic and medical interventions [1]. In particular, article 26 of the UNCRPD calls for rehabilitation services and programs to promote more participation of persons with disabilities in their community and in all aspects of broader society. Community Based Rehabilitation (CBR) is a strategy that promotes the “rehabilitation, equalization of opportunities, poverty reduction and social inclusion of all people with disabilities” [2, 3].

CBR programs were introduced in the 1970s as a participatory strategy to use effective, locally-developed technologies and interventions to prevent disability and transfer knowledge and skills about disability and rehabilitation to people with disabilities, their families and the community at large involved in the program. CBR was conceived as a combined effort of a diverse group of actors, including families, communities, disabled people’s organizations, health and social services provided by governmental and non-governmental actors, and, at the center, people with disabilities themselves.

Advocates of CBR identify several advantages of this approach over other alternatives. First, CBR is comprehensive; experts have demonstrated that all rehabilitation needs can be addressed through CBR interventions [4,5,6]. Second, authors have argued that CBR programs are more cost-effective than hospital or rehabilitation center-based interventions [7]. Third, taking a strong rights-based approach, CBR aims to specifically improve the wellbeing of systematically marginalized people with disabilities [8]. Finally, CBR is oriented toward participation and empowerment of people with disabilities [9, 10].

Despite the stated strengths, CBR programs continue to face several critiques linked to field realities. The first is that, while nominally based in values of participation and empowerment of people with disabilities, CBR often reproduce the same top-down service delivery approach of other methods [11]. Second, it has been argued that CBR are operated and funded by international aid and humanitarian organizations, raising significant questions about the sustainability of programs when donor priorities change [11]. Finally, at implementation level, many programs have scarce resources and lack strong support from the community. Absence of community involvement leads to poor monitoring despite some recent progress with the elaboration in partnership with CBR stakeholders of a monitoring manual [12,13,14]. It also results in limited ownership, empowerment and program relevance and sustainability [8, 15].

Despite the proposed benefits of CBR, extensive empirical literature that provides evaluation of the impact of CBR programs in diverse contexts is lacking. Most existing studies do not evaluate the overall WHO CBR matrix but overwhelmingly focus on its health component [15, 16].

Moreover, studies often focus on one condition or type of disability and do not evaluate the impact of CBR programs across disabilities. Notwithstanding the focus of the CBR matrix and the program design on the participation of people with disabilities in communities, few studies examine the contribution of CBR towards fostering empowerment and increasing social inclusion of people with disabilities and their families or change in community attitudes and behavior towards people with disabilities [8, 13, 17]. The general lack of overall evaluation studies is also explained by the absence of standardized outcomes and of a discrete intervention [18].

Finally, sustainability of CBR is overall neglected [19]. This research gap is in part the consequence of the prioritization of implementation over evaluation in CBR by development organizations, funders, and policymakers. Most existing research focuses on accessibility, reach of the program, identification of needs and specific rehabilitation and service delivery outcomes [20]. Studies that do exist lack consistent methodologies, making comparison across programs challenging and unreliable [15, 21,22,23]. For instance, existing CBR program evaluation studies have non-experimental design with limited size samples [15].

The present impact evaluation study carried out between July 2012 and December 2015 contributes towards filling the gap between theoretical concepts of CBR and the actual completion of a CBR program implemented in 13 provinces of north and eastern Afghanistan by measuring the impact of CBR activities on the circumstances and well-being of participants with disabilities. The study aims at advancing knowledge on CBR program effectiveness by investigating the effect of the Swedish Committee for Afghanistan CBR program on mobility, activities of daily living, communication, social participation, emotional well-being and access to employment of persons with disabilities.

Methods

Study aim, design and setting

Aim and design

The present study investigates the impact of the Swedish Committee for Afghanistan (SCA) CBR using a quasi-experimental design on major outcomes of interest after a maximum of 36 months in the study: How do persons with disabilities who received the CBR program’s services fare in terms of challenges in mobility, activities of daily living, communication, social participation, emotional well-being and employment compared to control with disabilities?

We used a quasi-experiment because we could not randomly select and assign treatment and control to the two arms of the experiment. The program is offered to all persons with disabilities living in its catchment area and it would have been both unpractical and unethical to withdraw the program to some potential participants. We used propensity score matching (PSM) associated with difference in difference (DD) analysis to minimize critical differences between the treatment and control groups and yield unbiased robust estimates of program effects using a non-parametric regression [24]. PSM was used to balance baseline characteristics between treatment and control groups in order to construct an analytical sample in which treatment and control groups are exchangeable [25]. DD analyses adds to PSM by further disentangling the program effect from differences between treatment and controls and from unrelated time-based trends in the outcome. DD allows a longitudinal analysis at two times points (at baseline and again after a maximum of 36 months after baseline) of a difference in outcomes (i.e. mobility, activities of daily living, communication, social participation, emotional well-being and employment) considering a difference in participation in the CBR program [26].

Settings

The CBR is implemented in 13 provinces of northern and eastern Afghanistan (See Map 1) and provides services to an estimated 2301 persons with disabilities in home-based activities and 1443 children in home-based education in 2018. The program covered 48 districts with over 775 staff, 863 (413 female) community volunteers and 151 (60 composed of female) CBR committees. The program is managed from four regional project offices based in Ghazni city (Ghazni province), Jalalabad (Nangharar province), Mazar-e-Sharif (Balkh province) and Taloqan (Takhar province) (See Fig. 1).

Fig. 1
figure 1

Map of the CBR program interventions areas. JPO: Jalalabad Provincial Office, GPO: Ghazni Provincial Office, MPO: Mazar-I-Sharif Provincial Office, TPO: Taloqan Provincial Office

Full size image

Participant characteristics

Selection of program participants

For the present study, we interviewed all new 1680 CBR participants included in the program between July 2012 and December 2013. Besides living in one of the 169 villages or urban areas (mahals) of the catchment area of the program, the other inclusion criteria were the ones defined by the CBR program at its start in 2004. At the creation of the program, inclusion was defined under the following criteria: (i) having a disability defined by the WHO guidelines for grassroot disability programs to account for contextual factors as having difficulties (e.g. seeing, hearing, speaking, learning or moving around, behavioral difficulty) that make it challenging to conduct all the activities that other members of the family or the community do, resulting in community members often considering people with such difficulties as being different or inferior to others and treating them unfairly [27]. Persons were screened for disability using a locally developed, tested and validated questionnaire based on these WHO guidelines; (ii) living in the areas or mahal close to the other SCA activities such as orthopedic workshops and physiotherapy centers/clinics; (iii) mahals had to be the place of residence of the newly recruited CBR workers before the program could expand progressively to nearby villages until it covered the entire district; iv) the absence in the mahal of a similar intervention by any other organization, and v) the intervention had to be welcomed by the overall village community and particularly the village council or shura; vi) the willingness of person with disabilities and the family members to participate in the CBR programs; and vii) the readiness of a family member to be trained by —and implement— the activities set up by the CBR worker in order for such activities to be ongoing daily, while the CBR worker would check progress made on a weekly or sometimes bi-weekly basis. If a village on the province was not part of the program for whatever reason, it was automatically included in the list of villages for random selection of controls. On average, one CBR worker was serving 100 participants with disabilities per year. Each expansion was decided in agreement with the CBR program management and the new targeted areas were surveyed for identification of persons with disabilities using the same WHO instrument [27].

Selection of controls

Controls were randomly selected during the same period with a two-level selection process: At first level, a random group of villages and urban areas of the same provinces but outside of the catchment area of the CBR program and at the second level a random group of households within each village or urban area (See Appendix for more details about the control group selection process).

Study tools and data collection

Disability screening

All heads of households were interviewed with a locally validated disability-screening tool composed of 34 items for adults (DSQ-34) and 35 for children (DSQ-35) based on the same disability definition as participants to identify all members of the household with disabilities [28].

Questionnaire development and testing

All study participants were interviewed by the CBR workers with a locally developed and validated questionnaire that inquired about demographic characteristics, socioeconomic status, access to rehabilitation, health and social services, individual functioning, social participation, and additional needs. The questionnaire examined the effectiveness of the CBR program in improving the agency of persons with disabilities to determine their daily lives, participate in different aspects of community life, escape stigma and prejudice, and access various CBR services from the five domains of the CBR matrix (health, education, livelihood, social inclusion and empowerment) [2]. Disability experts in Afghanistan were asked to review the content of the initial English version of the tool for completeness, content validity, and appropriateness of the questions to the Afghan cultural context. The English version of the tool was then translated into Dari and Pashto by a disability expert from the Ministry of Public Health in Kabul. Several different translators worked independently to back-translate the survey into English and compare results to reconcile discrepancies. A first version of the questionnaire developed by the authors was initially tested end of 2011 with a group of 20 CBR participants in Jalalabad, Nangarhar, Afghanistan. Each respondent was interviewed separately by a researcher for consistency check of responses provided. Additionally, the Dari and Pashto versions of the final questionnaire were tested through a series of 30 interviews in Kabul in 2012 with persons with disabilities of different age groups, gender and ethnicity to verify that response process followed, understanding and interpretation of complex or technical terms, such as access to healthcare, available CBR services, participation in family and community activities, and measures of additional needs as well as satisfaction with life were consistent across different socioeconomic background and with the initial concepts conceived in English by the researchers. Respondents were asked the questions as defined by researchers followed by a series of probe questions aiming at capturing their understanding of the questions in light of their own life experience [29].

Data collection

All study participants were interviewed with the same tool three times respectively between July 2012 and December 2013, between July 2013 and December 2014 and finally between July 2014 and December 2015.

Variables

Outcome variables

Six main outcomes of interest were included in the questionnaire and assessed through a range of questions: mobility, activities of daily living, communication, participation in social and community life, emotional well-being and employment. Indexes for each domain were created by generating a sum index score from the component items in the questionnaire. Because outcomes were sometimes different between age groups (for example, questions pertaining to ability to bathe oneself were not asked of infants too young to do so on their own), sum index scores were based on total points possible for each age group. Each summary index score was then divided by total points possible according to age and converted to a proportional value between 0 and 1. Difference scores between rounds 3 and 1 were then calculated. Thus, a result of .15 indicates a 15% increase in points possible within a given domain. Only employment did not result in an index.

Mobility index

The mobility index was composed of five activities with response choices limited to a Likert scale composed of three choices (I can always, I can with help, I cannot at all): Can you sit (asked to respondents above 1 year old); Can you stand (above 1 year old); Can you move inside the home (above 1 year old); Can you move outside the home (above 2 year old); Can you walk at least ten steps (above 2 years old).

Activity of daily living index

The Activity of daily living index was composed of four activities with the same three response choices: Are you able to eat on your own (asked above 4 years old); Are you able to bathe (above 8 years old); Are you able to use the latrine (above 3 years old); Can you dress and undress (above 4 years old).

Communication index

The communication index focused on the four following functions with the same three response choices: Can you speak (above 2 years old); Can you understand simple instructions (above 2 years old); Can you express needs (above 2 years old); Do you feel confident learning new things (above 4 years old).

Social participation index

The social participation index comprised a first item below with three response choices (I can without difficulty, I can with some difficulties, no, I cannot at all) and four following items with three different choices (I can always, I can sometimes, no never): Can you make friends outside the family; Are you consulted in family decisions (above 15 years old); Can you join in community activities and ceremonies; Do you feel respected in the community (above 5 years old); Do you feel respected in your family.

Emotional well-being index

The emotional well-being index was composed of five items with three response choices (never, sometimes, always): Do you feel sad (above 5 years old); Do you feel angry (above 5 years old); Do you feel worried or distressed (above 5 years old); Do you have nightmares or bad sleep (above 5 years old); Do you have headaches, stomach-aches or nausea (above 5 years old).

Employment

We asked respondents between 18 and 60 years old if they had a paid job, either in cash or in goods, what was the employment status (7 categories), if it was full time or part time and since when did they work.

Exposure variables

We did not measure the specific effect of each service delivered by the CBR program. Each participant received services tailored to their needs. The program delivered a set of services including physiotherapy, group training, loans, home based education, center-based education, inclusion in school, home based training, community advocacy and disability awareness. In other words, the present study does not measure the impact of the discrete interventions offered by the program but rather the overall impact of the program as a whole and the combination of interventions on the participants. We cannot assess whether it was the physiotherapy or the home-based education that contributed most to the change in outcomes over time, but rather a combination of interventions as deemed necessary and useful for each participant.

Covariates

Covariates we kept for PSM calculation included gender (male, female), age (continuous), cause of disability (birth, accident, disease, conflict related, other cause), disability type (physical/locomotor, sensory, intellectual, mental illness & epilepsy, multiple disabilities), ethnicity (Pashtun, Tajik, minority ethnicity), assets owned at baseline in tertile (poorest, 20–80%, highest), region (eastern region, northern region, south eastern region, north eastern region) age of onset of the disability (continuous), household education level (illiterate, some education), working status at baseline (not working, working) and household income at baseline (continuous). Moreover, people were also matched according to the baseline levels of the impact variables to ensure that CBR participants and controls started off at similar levels. We also included community-level variables such as distance to a road (continuous, in kilometres) and its usability for motorized vehicles (usable or not), availability of electricity (available or not), availability or not and distance (continuous, in minutes) to a school and a health center, presence or not of different types of social and/or political groups in the village (international NGO, religious group, political party, village shura, education or health shura, district development assembly, CBR committee), and exposure to different types of disasters or negative events (natural disaster, attack or other type of crisis such as landslide, drought or inundation) in the last 3 years.

Statistical methods

We used a quasi-experimental approach that mixed propensity score matching with difference in difference (PSM-DD) to measure the effect of the CBR program. We analyzed the three waves of data collected from treatment and control from surrounding communities, from the onset of the program at baseline until end-line. Yet, some CBR participants left the program before because they did not need the services anymore, some migrated and a few died. Some controls left the study because they did not want to participate anymore, migrated or died. As a result, we only have two interviews for them. The PSM framework used baseline data to find the best possible control match to the persons that received the CBR program. It is based on potential outcomes: The comparison group includes people with disabilities who would have been eligible to receive the CBR program services but who live outside of the program’s catchment areas. The assumption is that the decision on which communities are part of the program is based on observable characteristics. We use a vast set of control variables that are expected to influence both exposure of the program and outcome of interest. We controlled in the analysis for personal (e.g. gender, age, ethnicity, age at disability onset, cause and type of disability, residence, marital status, education level, employment status, assets index, individual and household income) and village (distance to road, electricity, distance to school and the healthcare facility, presence of multiple organizations, negative event outcome in the previous 3 years) characteristics that might have an effect on the impact of the program and were found to also be predictive of exposure to be in the treatment group. We calculated the likelihood of assignment into treatment in the CBR using a probit model based on the same variables used for matching in the PSM. All variables with a few exceptions (household income, presence of a domestic NGO, religious group, village, education or health shura) were found to be significant predictors at p < .05 level of assignment into the CBR treatment program when compared to reference categories.

Because all those who are eligible within the catchment area were included in the program we initially made the reasonable assumption that participant and control groups have similar characteristics overall [30, 31]. Yet, persons with disabilities in the catchment area could decide not to participate. Therefore, we tested for common support. The balancing tests show that propensity score matching using the nearest neighbor matching (with a 1:1 ratio, and a 0.10 calliper) removed most of the bias between the CBR participant and control groups. We reproduced the matching process using the Gaussian Kernel-based matching algorithm which is characterize by the ability to minimize total distance between matches with its weighting function without significant loss of observations. The Gaussian Kernel estimator was conducted with six separate specifications as sensitivity analysis of findings. These included using the .6 default bandwidth, a medium (.4) and large (.9) bandwidth, as well as 2, 5, and 10% sample trimming, each with the default bandwidth. We finally tested propensity score weighting including a village cluster effect and found consistent results (See Table 1 for the Gaussian Kernel estimator with default bandwidth and the propensity score weighting results).

Table 1 Average treatment effect on the treated (ATT) on all outcomes of interest using other matching methods
Full size table

We combined PSM with the difference in difference (DD) approach when different points in time (Y0, Y1) were captured to account for all unobservable differences that are stable over time therefore eliminating the risk of selection bias even if some unobservable characteristics that lead to the decision on whether to receive the program could not be captured with the variables (X) [24]. We identified the effect of the CBR program by comparing the change in outcomes of interest \( \mathrm{E}\left[{\mathrm{Y}}_{\mathrm{t}+1}^1-{\mathrm{Y}}_{\mathrm{t}}^0|\mathrm{D}=1\right] \) of the CBR participants between the period (t) and (t + 1) to the counterfactual \( \mathrm{E}\left[{\mathrm{Y}}_{\mathrm{t}+1}^0-{\mathrm{Y}}_{\mathrm{t}}^0|\mathrm{D}=1\right] \) they would have experienced in the absence of the program. This counterfactual is approximated by the change in outcomes of interest \( \mathrm{E}\left[{\mathrm{Y}}_{\mathrm{t}+1}^0-{\mathrm{Y}}_{\mathrm{t}}^0|\mathrm{D}=0\right] \) observed in the control group considering the common trend assumption:

$$ \mathrm{E}\left[{\mathrm{Y}}_{\mathrm{t}+1}^0-{\mathrm{Y}}_{\mathrm{t}}^0|\mathrm{D}=1\right]=\mathrm{E}\left[{\mathrm{Y}}_{\mathrm{t}+1}^0-{\mathrm{Y}}_{\mathrm{t}}^0|\mathrm{D}=0\right] $$

We estimated these two counterfactuals by matching treatment and control with key characteristics. To overcome the possible selection bias and the absence of independence between effect and treatment, we assumed conditional independence, i.e. that we observed all the baseline variables (X) that led a person to receive the program. We assumed the existence of common support which implies that only CBR participants that have a probability of being treated also found in any of the controls were included in the analysis. Similarly, controls with an extremely low probability of being treated were not included either. This method has the advantage of not requiring any assumption on whether the program has homogeneous or heterogeneous effects on the model errors and by being non-parametric it can be combined with other methods in order to yield more precise impact measures [32]. We used a logistic regression of the likelihood to receive the program based on baseline variables (X) for the propensity score estimation. After the propensity score, we estimated the average treatment effect on the treated (ATT):

$$ {\mathrm{ATT}}^{\mathrm{D}\mathrm{D}-\mathrm{PSM}}=\frac{1}{{\mathrm{N}}_{{\mathrm{D}}_1}}\sum \limits_{\mathrm{i}\in {\mathrm{D}}_1\cap S}\left[\left({\mathrm{Y}}_{\mathrm{i},\mathrm{t}+1}^1-{\mathrm{Y}}_{\mathrm{i},\mathrm{t}}^0\right)-\sum \limits_{\mathrm{j}\in {\mathrm{D}}_0\cap S}{\mathrm{W}}_{\mathrm{i}\mathrm{j}}\left({\mathrm{Y}}_{\mathrm{j},\mathrm{t}+1}^0-{\mathrm{Y}}_{\mathrm{j},\mathrm{t}}^0\right)\right] $$

Where D1 (D0) represents the treatment (control) group, wij the nearest neighbor matching weights, and S the area of common covariate support. PSM makes the standard DD assumption more plausible by forming statistical twin pairs before performing the DD estimator.

PSM-DD allowed for measuring the relative difference in change in outcomes over time between CBR participants and controls, and counteract the fact that not all variables that led to the definition of a catchment area could be considered, and thus addresses the bias generated by this limitation.

We estimated the PSM using only baseline variables to ensure that people were comparable before any interventions took place. We calculated Cohen’s d effect size estimates of the effect of the CBR program. We used STATA 16 for all analyses.

We conducted sensitivity analysis to assess the robustness of our model results in which we modified the socioeconomic and community-level characteristics of covariates. We used household monthly income (continuous) instead of the welfare index based on assets owned at baseline. We used a different age limit for working at baseline (15 years old instead of 18). We used the 13 provinces instead of the four regional offices. We also tested whether our results were sensitive to an alternative specification of the outcome variables. We calculated the model for program effect separately on the different items of the five outcomes: mobility (sit, stand, move inside/outside the home, walk ten steps alone), activities of daily living (can eat, bath, use the restroom and dress without help), communication (speak, understand instructions, express needs, learn new things), social participation (making friends, consulted in family decisions, participate in community activities/ceremonies, feel respected in the community and in the family), emotional wellbeing (being sad, being angry, being worried, having nightmare as dummy variables). We also measured the effect on employment considering working or not as an adult with two different cuts-off at 15 and 18 years old. We found similar significant effects.

Results

Participant flow

We interviewed 1875 new CBR participants and 1305 controls at baseline between July 1st 2012 and December 31st 2013. At endline, 1349 CBR participants and 927 controls were interviewed between July 1st 2014 and December 31st 2015 (See Fig. 2). Attrition rate was 28,0 and 28.8% between round 1 and 3 respectively for participants and controls. Reasons for attrition were as follows: death or migration outside of the catchment area of the study as well as no need for further services for participants. Refusal to participate in the study was very minimal among participants (n = 14, 0.01%) and higher among controls (refusal at baseline was n = 173, 15.3%). Yet, there were few significant differences between respondents and non-respondents with respect to the measured characteristics and we therefore assumed that unobserved data were missing at random [33] (See Table 2).

Fig. 2
figure 2

Study participants selection, intervention and follow up process

Full size image
Table 2 Baseline characteristics of treatment and control comparing those with complete exposure and those lost to follow up
Full size table

Baseline data

Table 3 presents the difference in various characteristics between CBR participants and controls at baseline. Examining the balance before and after PSM (Fig. 3 and Table 2) illustrates that the unmatched CBR and control groups differ substantially in terms of important confounding factors: CBR participants were younger, more often with mobility/locomotor limitations, disabled at birth than controls. They came from families with more material assets and slightly higher average income and were more likely to work than controls. They live in villages more connected to a road, more likely to have electricity, a school or an healthcare facility. Once PSM is calculated, Fig. 3 shows that covariates are well balanced and matching significantly reduces the mean standardised bias for each covariate below the standard threshold of 5% [34].

Table 3 Characteristics at baseline of study participants
Full size table
Fig. 3
figure 3

a Balance results of the propensity-score matching of the sample for the various CBR outcomes. b Histogram of propensity-score matching distribution with common support for control and treatment

Full size image

Bivariate analysis of effect of intervention exposure on outcomes

At baseline, CBR participants showed on average higher limitations for all outcomes of interest (See Table 4). A significant higher proportion of CBR participants could not carry-out any of the four mobility activities or had some difficulties compared to controls. Similarly, CBR participants had significantly more difficulties to accomplish any of the activities of daily living. Results for communication comparisons were consistent with limitations of basic activities of daily living and mobility: overall CBR participants had higher rates of complete or partial limitation than controls except for learning new things, where controls have slightly higher limitations. Study results show that CBR participants faced higher barriers to social participation. Interestingly, perception of lack of respect by family was observed to be very low and less frequent in both groups than perception of lack of respect by the community. Yet, participation in family decisions was high as approximately 9% of both CBR participants and controls above age 15 were never consulted. Finally, we found that CBR participants and controls demonstrated very similar and relatively low levels of severe mental distress and anxiety; feeling sometimes worried or angry was even significantly higher among controls.

Table 4 Effect of the exposure to the CBR program on outcomes of interest
Full size table

At end-line, all bivariate analysis showed that the difference between CBR participants and controls remained statistically significant on all variables of interest. The proportion of CBR participants who had severe activity limitations or functioning difficulties in terms of mobility, activity of daily living, communication and social participation or a low level of emotional wellbeing at baseline was smaller at endline. The proportion of CBR participants with severe activity limitations or functioning difficulties or low level of emotional wellbeing at end-line were similar or lower than controls.

Measuring the effect of the CBR program using propensity score matching with difference in difference analysis

We use PSM-DD to assess the CBR program impact on the various outcomes of interest. Findings reported in Fig. 4 and Table 5 show a positive impact of the CBR program on all outcomes of interest. Table 5 assesses the ATT including Cohen’s d effect size. The first graph in Fig. 4 presents the change over time in the mobility index for both CBR participants and controls. While the controls saw their mobility worsen slightly by 1.4%, CBR participants’ mobility index improved by 13%. The 14.4 percentage point difference between both groups is statistically significant and the effect size (ES) is of − 0.36 (95% Confidence interval − 0.44– − 0.29), regarded in the literature to be between small and medium size [35, 36]. The second graph shows that both CBR participants and controls improved their capacity to carry out activities of daily living. However, CBR program participants improved by 8.4% more than controls with an ES of − 0.26 (95%CI -0.35– -0.18). Similarly, as shown in the third graph, CBR participants improved their communication abilities by 9.1% more than controls (ES = -0.38, 95%CI -0.46– -0.30). The fourth and fifth graphs show a difference in improvement in social participation skills and emotional wellbeing between CBR participants and controls of 17.8% (ES = -0.45, 95%CI -0.53– -.036) and 10.2% (ES = -0.48, 95%CI -0.58 – − 0.38) respectively. Controls even saw their mobility, social participation skills and emotional wellbeing decrease during the 3 years period of intervention. The last graph in Fig. 4 shows that the CBR program promoted access to employment for participants —by 7.5% on average during the three-year period— while controls underwent a reduction in employment rate during the same period of 4.4% but with a relatively small effect size (ES = -0.21, 95%CI -0.33– − 0.10), in fact the smaller of all program effect.

Fig. 4
figure 4

Impact of the CBR program on outcomes of interest using PSM-DID

Full size image
Table 5 Average treatment effect on the treated (ATT) on all outcomes of interest
Full size table

Discussion

Our study shows that the CBR program had a significant positive impact on several outcomes of interest promoted by the WHO, namely individual mobility, activities of daily living, communication skills, emotional wellbeing, social participation and employment. Our findings suggest that CBR programs can improve livelihoods and wellbeing of persons with disabilities in LMICs [8, 13, 20, 37]. Yet, existing studies present multiple limitations —small sample size, inadequate sample methodology, mostly observational or qualitative approaches, recall bias in the only existing quasi experiment and lack of accounting for confounding factors in regression analysis [8, 38]. Our study follows and interviews a large group of CBR participants and a random group of controls multiple times over a period of 3 years.

Our study is unique in that: (i) it combines a measure of outcomes associated with one of the five components of the CBR matrix —health, education, livelihood, social and empowerment; (ii) it addresses the issue of CBR program sustainability by looking at data from three rounds of interviewing the same CBR cohort of participants and controls with disabilities, showing the lasting effect of the rehabilitation interventions; and (iii) it is one of so far only nine identified studies looking at CBR effectiveness carried out in a low income country; the only one in a conflict context. This paucity of existing evidence makes studies like the present one essential to legitimize the promotion of CBR while shedding light on the conditions of its success. This is of utmost importance in LMICs as CBR has been, especially in crisis and emergency contexts, the only set of services available to persons with disabilities, a particularly disadvantaged group in such contexts [39, 40].

Interpretation

The CBR program has a differential impact on the various outcomes indicating that improvement is not uniform but depends on the effectiveness of the program in improving the condition of persons with disabilities within a given domain of the CBR matrix rather than the capacity for change within each domain. The highest positive impact is observed on emotional wellbeing. After 3 years on average in the program, the estimated effect size is − 0.48 The gap between treated and controls is particularly salient because the emotional wellbeing of controls without any dedicated support decreased by 34% during the period. While we observed such an increase in anxiety and distress among controls during the three years of observation, CBR participants showed a continuous improvement. Anxiety and distress are a major ill in conflict affected contexts such as Afghanistan. Literature traditionally attributed mental distress to the violence itself [41,42,43,44,45] including sexual violence and intimate partner violence [46,47,48,49]. Increasingly, scholars have also shown that daily stressors play an important role in common and more severe mental disorders, particularly among vulnerable groups. Conflict exacerbates economic deprivation adding an important source of mental distress, particularly for those at risk of being economically and socially excluded [50,51,52]. Literature has shown that persons with disabilities are also particularly at risk of mental distress in conflict settings because of poverty, unemployment but also stigma associated with disability [53, 54].

Another important effect of the program was observed on social participation which encompasses important dimensions of social life. The social participation index reveals a person’s sense of self-worth and placement within their family and community, and also indicates to what extent disability isolates a person from family and community daily life. Effectively changing community attitudes resulting in reduced marginalization and discrimination of persons with disabilities is a major milestone difficult to achieve because of deeply entrenched negative beliefs [54, 55]. Early CBR literature showed that it was easier to restore individuals’ functions than to change contextual factors such as attitudes towards disability [56]. Yet, it is a significative achievement, one that embodies the philosophical underpinning of the CBR principles as defined by the World Health Organization: a bottom-up multipronged set of interventions relying on ownership and empowerment of persons with disabilities [2]. If this outcome reflects genuine social participation and reduction in prejudice, such achievement also contributes to the principle of “full inclusion and participation in all aspects of life” promoted by the article 26 of the UNCRPD. The fact that controls saw their social involvement in their community and family slightly reduced by 5.1% suggests that cultural norms and beliefs foster social isolation that only an external intervention promoting awareness and advocacy can progressively and partially remedy [8, 53, 57].

The significant effect on activities of daily living, communication and mobility demonstrate that a CBR program implemented with limited resources, family support in remote areas prone to violence and disruption associated with conflict, can be effective in providing rehabilitation services through a combination of low cost home-based services such as physiotherapy and access to free orthopedic workshops [58]. Literature has shown that caregivers’ involvement is key in effectively improving participants with disabilities’ functioning outcomes [59]. However, by improving the autonomy of persons with disabilities through rehabilitation processes involving family caregivers, the CBR program reduces the burden that otherwise rest exclusively on them in most LMICs in absence of rehabilitation services [60]: The limited amount of improvement made over the same period of time among controls with disabilities tends to suggest the restricted access to such services in the mainstream healthcare system of Afghanistan.

The program also had a significant marginal effect of 12% on employment of adult participants between 18 and 60 years of age compared to control with disabilities of the same age group. This is a major finding considering there are few interventions and even fewer studies showing the impact of CBR programs tackling the livelihood component of the CBR matrix and supporting employment in LMICs [8]. Such gap in intervention makes escaping poverty highly elusive for persons with disabilities. Overcoming barriers to employment of persons with disabilities is a considerable economic challenge overall. An already 16 years old study estimated at the time that the global gross domestic product loss due to disability to be between $1.71 trillion to $2.23 trillion annually [16, 61]. Furthermore, lack of livelihoods intervention undermines resilience and self-esteem of persons with disabilities, particularly men. In Afghanistan, persons with disabilities have been shown to face lower employment and higher multidimensional poverty compared to non-disabled people particularly for young men [62]. Yet, men are traditionally expected to be breadwinners and look after the needs of the family [63, 64]: For persons with disabilities, the CBR intervention helps secure a job, an essential step towards building a sense of wellbeing resulting from meeting their social obligations and obtaining recognition within the community [65].

The fact that the program was effective in tackling emotional wellbeing, social participation and access to jobs shows that it had an effect on prejudice and discrimination that persons with disabilities face in their community, often the most complex and arduous barrier to overcome [20]. In particular, there is abundant literature exploring the impact of stigma on emotional wellbeing [66, 67]. One of the central goals of CBR programs in general is to challenge stigma of rural communities towards disability in order to promote people with disabilities’ community participation and inclusion [68]. Rehabilitation remains incomplete unless it addresses stigma and prejudice towards people with disabilities who frequently cannot envision the future and have no prospect of social inclusion. It is well established that community acceptance is associated with higher emotional wellbeing and self-esteem but we lack evidence of effective stigma-reduction interventions particularly in low income countries [69]. Similarly to the Swedish Committee for Afghanistan CBR program, another program in Afghanistan promoted by HealthNet TPO also relied on including people with mental disorders and the community in advocacy [70].

Limitations

Our study presents some limitations. First, there was a risk of self-selection in the CBR program. Elderly people and people with associated disabilities were significantly less likely to participate in the program [58]. There was a risk of selection on non-observable variables as well that we tried to minimize by including a large set of control variables in the propensity score calculation and by using difference in difference as an additional methodology to estimate an unbiased impact. We ran different PSM-DD matching algorithm and found similar impact of the CBR program. We also conducted robustness checks by changing the set of baseline variables (X) for the propensity score estimation and our results hold. Second, there is a risk of information bias as data collection was carried out by CBR workers under the supervision of the investigators and research officers because of security issues. This might have introduced a social desirability bias among CBR workers willing to show a good image of their program. Thorough training, careful and ongoing supervision in the field and after data collection, consistency checks as well as random re-interviews of participants has made very unlikely that this bias could change our findings. The items in the emotional index did not have a time limit. We asked respondents to assess if currently they felt sad, angry, worried, or had nightmare without indicating in the last seven days for instance. There has been some contamination of the program as few activities benefited controls. In particular, persons with disabilities from villages outside the catchment area would seek assistive devices (orthotics, orthopedics, crutches, wheelchairs) in urban workshops –as well as health rehabilitation services such as physiotherapy in health clinics– run by Swedish Committee for Afghanistan. We found that less than 2% received any kind of service, mostly assistive devices,except for physiotherapy received by 10% of controls (data not shown). Contamination of the program to the benefit of controls means that our findings are probably slightly underestimated. Finally, the selection of catchment areas was not random. However, baseline information shows that CBR participants started worse off than their counterparts, reducing the chance that observed impacts were due to initial differences favoring CBR participants.

Conclusions

Our results have important implications regarding the capacity of CBR to complement existing government initiatives in addressing the issues related to rehabilitation, autonomy social and economic inclusion of persons with disabilities in a conflict or disaster context and beyond in other LMICs. Better mobility activities of daily living and communication skills are indicators of a CBR program that offers effective rehabilitation services. Most importantly, improvement in emotional wellbeing, social participation and even access to jobs show that the CBR program was also effective in advocating for the rights of persons with disabilities and promoting awareness of disability in the community to fight stigma [20].

The lack of standardization in the CBR program we examined did not allow for identification of specific processes that were most successful in promoting social inclusion. Future research should explore such processes and mechanisms that lead to effective CBR interventions towards persons with disabilities. Process evaluation based on mixed methods to dwell deeper into the process of change using implementation research would help better understand how the program work to be able to standardize the intervention and translate it in a different context.