# Programme Evaluation and Treatment Choice — An Overview

Chapter
Part of the Lecture Notes in Economics and Mathematical Systems book series (LNE, volume 524)

## Abstract

Policy and programme evaluation in a wide sense is concerned with measuring how far a policy or a programme has achieved its intended aims. A policy is hereafter defined as a bundle of R different programmes. This includes the case of evaluating a single programme (R = 2, participation versus non-participation) and evaluating multiple programmes (R > 2). One example of policies consisting of multiple programmes are active labour market policies, which often comprise various public employment programmes, on-the-job training, retraining, classroom training, job search assistance, wage subsidies etc. Another example are rehabilitation policies for the re-integration of people with long-term illnesses, which may consist of different forms of vocational workplace training, vocational schooling, medical rehabilitation and social and psychological programmes. In the following, often the neutral term treatment will be used synonymously for programme, since the methods presented here are not restricted to the evaluation of social policies but apply similarly to, for example, the evaluation of the effectiveness of medical drugs or of different schooling choices, or of the effects of participation in the military. Since participation in a policy is often voluntary, or since full compliance in a ’mandatory’ policy might not always be enforceable, the set of different treatments usually includes a ’no-programme’ or ’non-participation’ option. As it is assumed that all individuals are untreated before participation in the policy, i.e. that they had not participated previously in the programmes,1 this ’non-participation’ treatment is often special in the sense that it is the treatment most similar to the situation before participation in the policy. To illustrate this asymmetry, the treatment set will be indexed by r ∊ {0,..,R − 1}, i.e. consisting of a ’non-participation’ treatment (r = 0) and R − 1 active treatments. In the case of the evaluation of a single programme the treatment set consists of r = 0 (non-participation) and r = 1 (participation).

## Keywords

Propensity Score Instrumental Variable Propensity Score Match Potential Outcome Average Treatment Effect
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## Notes

1. 1.
For the evaluation of sequential programmes see Lechner and Miquel (2002).Google Scholar
2. 2.
For an introduction to causal reasoning see Holland (1986) and, particularly, Pearl (2000).Google Scholar
3. 5.
Even if a proper experiment is conducted, it might still occur by chance that the treatment groups differ substantially in their characteristics particularly if the sample sizes are small. Although the differences in sample means provide unbiased estimates of average treatment effects, adjusting for the differences in the covariates, as discussed in Section 2.1.5, can reduce the variance of the estimates (Rubin 1974).Google Scholar
4. 7.
The control-for-confounding-variables evaluation strategy is widely applied in the evaluation of active labour market programmes, see for instance Heckman, Ichimura, and Todd (1997) and Dehejia and Wahba (1999) for the USA, Lechner (1999) for Eastern Germany, Gerfin and Lechner (2002) for Switzerland, Brodaty, Crepon, and Fougere (2001) for France, Larsson (2000) for Sweden and Jalan and Ravallion (2002) for Argentina.Google Scholar
5. 15.
Further applications of the regression-discontinuity approach include the effects of unemployment benefits on recidivism rates of prisoners (Berk and Raufna 1983), the effects of classroom size on students’ test scores (Angrist and Lavy 1999), or parents’ willingness to pay for higher quality schooling for their children (Black 1999), among others.Google Scholar
6. 17.
Estimation of the common support is discussed in Heckman, Ichimura, and Todd (1998), Lechner (2002b) and in Chapter 4.Google Scholar
7. 24.
However, in specific situations, for instance if the outcome variable is bounded, nonparametric regression on X might work better than is widely thought, see Frölich (2001a).Google Scholar
8. 30.
See OECD (1998), Colpitis (1999), de Koning (1999), DOL (1999), Black, Smith, Berger, and Noel (1999) and Eberts and O’Leary (1999).Google Scholar
9. 31.
This is not unusual with active labour market programmes, see e.g. Bloom, Orr, Bell, Cave, Doolittle, Lin, and Bos (1997), Fay (1996), Gerfin and Lechner (2002), Lechner (2000) or Puhani (1999).Google Scholar
10. 33.
General method of moments estimator (Hansen 1982).Google Scholar
11. 35.
Other widespread nonparametric regression techniques include splines, wavelets and series regression, see Eubank (1988), Hardie (1991) or Pagan and Ullah (1999).Google Scholar