Descriptive statistics
Table 1 provides descriptive statistics by wave for the 10,925 individuals in the full sample. The average respondent is in moderate to poor mental health. Approximately 75% of individuals live in households receiving CSG and about 87% live in a household that is eligible to receive the CSG. This is consistent with previous studies [8, 10, 12, 16, 18] and is evidence of potential selection into the CSG, as in about 50% of the non-recipient households there is at least one eligible child. Every third person in the sample is male. The average respondent is about 39 years old and lives with six other individuals together in a household in a moderately safe neighbourhood. About 13% of individuals in the sample are younger than 19 years of age. One in nine households has experienced the death of a household member in the past 2 years. 38% of the individuals report to have at least one individual in their household who receives the old age pension, 12% the disability grant, 5% the foster care grant and 1% the care dependency grant.
Table 1 Summary statistics 2008–2014 Splitting the samples by waves shows the increase in the coverage of the CSG policy nationwide, with increasing numbers of individuals living in CSG receiving and/or in a CSG eligible household over time. Mental health improves slightly from an average of 19.0 points in 2008 to an average of 20.4 points in 2014. Table A2 discussed in section A1 of the Additional file 1 compares individuals living in recipient and non-recipient households. We observe slightly higher CES-D scores for individuals living in receiving households (19.89 vs 19.37), similarities in other grant receipt and disparities in household composition with more household members, more female household members and lower mean age in receiving households. Statistical difference in means of control variables between individuals living in CSG-receiving households and non-recipient households suggest the potential absence of random selection of CSG-receipt.
Pooled versus fixed effects
Table 2 reports the results of the pooled and FE estimation. We find a positive and significant effect on mental health for individuals living in a CSG receiving household across the models. In the pooled model without covariates in column (1), individuals living in a CSG receiving household have on average a 0.325 point higher CES-D score than individuals living in non-recipient households.
Table 2 Effect of CSG on mental health: Pooled OLS and Fixed Effect estimation Controlling for all covariates in the pooled model in column (2) the coefficient increases to 0.405. This implies a 2% increase in CES-D score compared to the mean of CES-D. When controlling for unobserved heterogeneity in the models using FE without and with covariates in columns (3) and (4), results show that the size of the coefficient increases, and it remains statistically significant. This indicates a downward bias in the pooled OLS estimate due to correlation of the main explanatory variable with unobservable idiosyncratic factors. The coefficient size reported in column (4) is 0.536, which indicates a 2.7% increase in CES-D from the mean of CES-D.
In both the pooled and FE models, being younger than 19 years of age is associated with better mental health. In the FE model age has a positive non-linear association with mental health. Males have on average a significantly better mental health than females in the pooled model. In the pooled model, age has a non-linear, and positive above 30 years, association with mental health. In the pooled model, mental health is better in households of larger size. Economic decision making and negative events are both negatively associated with mental health.
Notably, attrition is significantly and negatively associated with transfer receipt in the pooled model in column (2). However, when using FE estimation this association loses statistical significance which tells us that FE address the correlation of attrition with unobservable factors determining mental health.
We perform a Hausman test to determine if FE or Random Effects models should be used. The Hausman test rejects the null hypothesis p < 0.001 in favour of the FE estimation.
2SLS and 2SLS FE estimation
The estimates of the first stage regression of the different instrumental variable models are presented in Table 3, for the whole population in columns (1) to (4) including pooled OLS without and with covariates and FE estimations without and with covariates, and FE estimation for male and female separately in columns (5) and (6). The regression of the household level CSG receipt on the instrumental variable in the top row of the table shows positive significant effects throughout all specifications.
Table 3 Effect of eligibility on CSG receipt: First Stage 2SLS, Fixed effect 2SLS estimation The magnitude of the coefficient of the instrumental variable varies over the specifications with highest magnitude for the 2SLS estimation without covariates (0.833). Adding covariates to the OLS estimation in model (2) reduces the coefficient to 0.723, indicating correlation of the instrumental variable with other factors. When using FE with 2SLS estimation without covariates, the magnitude is further reduced to 0.712, which indicates that FE account for unobservable constant factors. Adding covariates in model (4) to the FE estimation further reduces the magnitude of the coefficient to 0.644. The estimated coefficient gives the compliance rate for the group of transfer recipients with the instrumental variable. The compliance rate shown in model (1) is 83%, which implies that 83% of the recipients comply in their treatment status conditional on the instrumental variable.
Table 4 presents the findings of the second stage of the pooled and FE instrumental variable estimations for the same six models. The coefficient associated with the CSG is positive and significant in all models, except for the one estimated on the male sub-sample where it is not statistically significant (p = 0.392).
Table 4 Effect of CSG on mental health: Second Stage 2SLS, Fixed Effect 2SLS In column (1) the effect of living in a receiving CSG household on mental health outcome, estimated using pooled 2SLS, increases mental health by about half a unit on the CES-D scale (0.614) compared to individuals living in non-receiving household. The magnitude of the effect is higher (0.749) in model (2) when controlling in the pooled 2SLS estimation for confounding factors in the determining mental health. The FE 2SLS estimation in column (3) shows a marginally larger magnitude compared to the 2SLS with a coefficient of size 0.621 which is a 15% standard deviation increase in mental health. The marginally larger size of the coefficient in column (3) compared to column (1) shows that the 2SLS estimation without covariates in (1) is downwardly biased due to unobserved heterogeneous factors.
Adding control variables in the FE 2SLS estimation in column (4) shows a transfer effect of size 0.822 on individuals’ mental health which is a 4.1% improvement in the mean value of mental health. The 2SLS coefficient of transfer receipt without FE shows low variation with and without conditioning on covariates whereas in the 2SLS estimation with FE it shows significant difference conditioning on covariates and without covariates. An explanation is that covariates are correlated with unobservable factors adding bias to the estimation, which we address by using individual FE model. The CSG effect is not significant for the male sub-sample in column (5) and the strongest cash transfer effect amongst all models is observed in the female sub-sample in column (6) with an improvement on the CES-D scale of one unit or an increase in mental health of 5 %.
The FE 2SLS model with covariates in column (4) is twice the size of the pooled model coefficient in column (2) of Table 2, indicating a potential downwardly bias of the pooled model-coefficient due to unobserved heterogeneity in mental health. The bias remains when controlling for unobserved heterogeneity in health but not instrumenting for the treatment receipt as the coefficient size of the FE in column (4) of Table 2 is still 0.3 points lower than the one estimated with FE 2SLS in column (4) of Table 4.
When controlling for determinants of mental health, the following coefficients estimated with the pooled 2SLS in column (2) are associated with mental health. We find positive significant effects for being under 19 years of age, being male and the size of the household. Significant negative associations for age (though with increasing positive non-linear age effects), economic decision making in a household, death of a household member and attrition. The attrition dummy is only significant in the pooled model. This indicates for the FE models that individuals who leave the survey for reasons of death or non-response are not systematically biasing the estimation.
We find in the FE model in column (4) a positive significant association of age with mental health. Both FE models in columns (4) and (5) show a positive effect of being under 19 years of age on mental health. The FE model for males in column (5) shows positive effects of negative events on male mental health. In contrast the FE model for females in column (6) identifies negative associations of negative events with mental health.
We test the statistical strength of the instrumental variable in all presented models. The results from these tests emphasise and support the statistical strength of our instrumental variable in all 2SLS models. In all first-stage regressions is the endogenous regressor neither weakly nor under-identified by the instrumental variable (F-statistics > 10) [41]. For brevity, we present in the following only results from the preferred FE 2SLS model in brackets, however the findings apply to all 2SLS models. Since only one instrumental variable is applied, the F-test, the Angrist-Pischke multivariate F test, and Kleibergen-Paap test are similar [4, 23]. The tests reject the null hypothesis at the 99% level (F = 633.76 Prob >F < 0.001). Since we can reject the null hypothesis, the instrument is strong and adequate [41]. Both the Stock-Wright and the Anderson-Rubin Wald test specifications tests reject the null hypothesis (Stock-Wright: p = 0.00212; Anderson-Rubin Wald test p = 0.0159), indicating that endogenous regressors are relevant. The critical value of the Stock-Yogo ID which is 16.38 and indicates a weak instrument threshold is far below the F-statistics (10% = 16.38 < 633.75) indicating a strong instrument [45]. The endogeneity test (Durbin-Wu-Hausman test or short Hausman form) is not rejected (t = 1.162, p = 0.2810). This tells us that self-selection is not based on mental health characteristics.
Robustness checks
Our robustness checks show that the results are robust to sample selection and attrition, that mental health effects remain strong when actual cash transfer recipients are excluded from the sample estimation, which supports the argument of cash-transfer sharing on the household level, and that no placebo-effects of cash transfer receipt occur, supporting the claim that no anticipation effects occur. We find further strong support for the causality of cash transfer effects on mental health and the exclusion assumption (e.g. validity) of the instrumental variable to hold as living with a child in the CSG age-eligibility brackets in non-poor households (income above Rand800 and not applicable to apply for the CSG programme) has no statistically significant effect on mental health. An elaborate discussion of the findings from the robustness analysis is presented in section A5 of the Additional file 1.