Evaluation Strategies

Abstract

The aim of the quantitative analysis of social programmes is to assess the causal impact of the programme on a certain outcome variable. However, this causal impact is never observed in reality. Hence, the fundamental problem of every programme evaluation is that one would like to compare the labour market outcome of a programme participant with the labour market outcome of the same participant, if he or she had not participated in the programme. The latter, however, is never observable. The fundamental problem is solved if the participants (treatment group) of a programme and the non-participants (control group) do not differ systematically in their characteristics and behaviour before the programme starts, i.e. in the absence of a sample selection bias. Such a situation is typically built up by a social experiment. In principle, the social experiment is viewed as the best solution of the evaluation problem, however experiments are very expensive (see Heckman and Hotz, 1989) and e.g. in Germany social experiments are not allowed because of ethical reasons. Moreover, Heckman and Smith (1995) show that social experiments also face several selection problems. Since in reality a sample selection bias is present for most social programmes either due to self-selection or due to programme-selection processes a naive comparison of participants and nonparticipants would not yield the causal impact of the programme. Hence, as an alternative to the experimental method a non-experimental (econometric) procedure is applied to address the sample selection bias. In principle, the econometric procedures assemble the control group ex post out of the pool of non-participants and aim to estimate a hypothetical value of the outcome variable for the treatment group on the basis of that control group.