An Introduction to the Econometrics of Program Evaluation

  • Giovanni Cerulli
Part of the Advanced Studies in Theoretical and Applied Econometrics book series (ASTA, volume 49)


It is common practice for policymakers to perform ex post evaluation of the impact of economic and social programs via evidence-based statistical analysis. This effort is mainly devoted to measure the “causal effects” of an intervention on the part of an external authority (generally, a local or national government) on a set of subjects (people, companies, etc.) targeted by the program. Evidence-based evaluation is progressively becoming an integral part of many policies worldwide. The main motivation resides in the fact that, when a public authority chooses to support private entities by costly interventions, a responsibility towards taxpayers is assumed. This commitment, constitutionally recognized in several countries, draws upon the principle that, since many alternative uses of the same amount of money are generally possible, any misuse of it is seen as waste, especially under severe budget constraints.


Ordinary Little Square Program Evaluation Binary Treatment Average Treatment Effect Conditional Independence Assumption 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Abadie, A., Angrist, J., & Imbens, G. (2002). Instrumental variables estimates of the effect of subsidized training on the quantiles of trainee earnings. Econometrica, 70, 91–117.CrossRefGoogle Scholar
  2. Althaus, C., Bridgman, P., & Davis, G. (2007). The Australian policy handbook. Sydney: Allen & Unwin.Google Scholar
  3. Angrist, J. D. (1991). Instrumental variables estimation of average treatment effects in econometrics and epidemiology (NBER Technical Working Papers No. 0115).Google Scholar
  4. Angrist, J. D., & Imbens, G. W. (1995). Two-stage least squares estimation of average causal effects in models with variable treatment intensity. Journal of the American Statistical Association, 90, 431–442.CrossRefGoogle Scholar
  5. Angrist, J. D., & Pischke, J. S. (2008). Mostly harmless econometrics: An empiricist’s companion. Princeton, NJ: Princeton University Press.Google Scholar
  6. Blundell, R., & Costa Dias, M. (2002). Alternative approaches to evaluation in empirical microeconomics. Portuguese Economic Journal, 1, 91–115.CrossRefGoogle Scholar
  7. Cameron, A. C., & Trivedi, P. K. (2005). Microeconometrics: Methods and applications. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  8. Cattaneo, M. D. (2010). Efficient semiparametric estimation of multi-valued treatment effects under ignorability. Journal of Econometrics, 155, 138–154.CrossRefGoogle Scholar
  9. Cerulli, G. (2010). Modelling and measuring the effect of public subsidies on business R&D: Critical review of the economic literature. Economic Record, 86, 421–449.CrossRefGoogle Scholar
  10. Cerulli, G. (2014a). Identification and estimation of treatment effects in the presence of neighbourhood interactions (Working Paper Cnr–Ceris, N° 04/2014).Google Scholar
  11. Cerulli, G. (2014b). CTREATREG: Stata module for estimating dose–response models under exogenous and endogenous treatment (Working Paper Cnr–Ceris, N° 05/2014).Google Scholar
  12. Chernozhukov, V., & Hansen, C. (2005). An IV model of quantile treatment effects. Econometrica, 73, 245–261.CrossRefGoogle Scholar
  13. Cobb-Clark, D. A., & Crossley, T. (2003). Econometrics for evaluations: An introduction to recent developments. Economic Record, 79, 491–511.CrossRefGoogle Scholar
  14. Cooley, T. F., & LeRoy, S. F. (1985). Atheoretical macroeconometrics: A critique. Journal of Monetary Economics, 16, 283–308.CrossRefGoogle Scholar
  15. Frölich, M. (2004). Programme evaluation with multiple treatments. Journal of Economic Surveys, 18, 181–224.CrossRefGoogle Scholar
  16. Frölich, M., & Melly, B. (2010). Estimation of quantile treatment effects with Stata. Stata Journal, 10(3), 423–457.Google Scholar
  17. Frölich, M., & Melly, B. (2013). Unconditional quantile treatment effects under endogeneity. Journal of Business & Economic Statistics, 31(3), 346–357.CrossRefGoogle Scholar
  18. Guo, S., & Fraser, M.-W. (2010). Propensity score analysis. Statistical methods and applications. Thousand Oaks, CA: SAGE.Google Scholar
  19. Heckman, J. J. (2000). Causal parameters and policy analysis in economics: A twentieth century retrospective. Quarterly Journal of Economics, 115, 45–97.CrossRefGoogle Scholar
  20. Heckman, J. J. (2001). Micro data, heterogeneity, and the evaluation of public policy: Nobel lecture. Journal of Political Economy, 109, 673–748.CrossRefGoogle Scholar
  21. Heckman, J., Ichimura, H., Smith, J., & Todd, P. (1998). Characterizing selection bias using experimental data. Econometrica, 66, 1017–1098.CrossRefGoogle Scholar
  22. Heckman, J. J., Lalonde, R., & Smith, J. (2000). The economics and econometrics of active labor markets programs. In A. Ashenfelter & D. Card (Eds.), Handbook of labor economics (Vol. 3). New York: Elsevier.Google Scholar
  23. Hirano, K., & Imbens, G. (2004). The propensity score with continuous treatments. In A. Gelman & X. L. Meng (Eds.), Applied Bayesian modeling and causal inference from incomplete-data perspectives (pp. 73–84). New York: Wiley.Google Scholar
  24. Holland, P. (1986). Statistics and causal inference (with discussion). Journal of the American Statistical Association, 81, 945–970.CrossRefGoogle Scholar
  25. Hoover, K. D. (2001). Causality in macroeconomics. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  26. Hudgens, M. G., & Halloran, M. E. (2008). Toward causal inference with interference. Journal of the American Statistical Association, 103(482), 832–842.CrossRefGoogle Scholar
  27. Husted, J. A., Cook, R. J., Farewell, V. T., & Gladman, D. D. (2000). Methods for assessing responsiveness: A critical review and recommendations. Journal of Clinical Epidemiology, 53, 459–468.CrossRefGoogle Scholar
  28. Imai, K., & Van Dyk, D. (2004). Causal inference with general treatment regimes: Generalizing the propensity score. Journal of the American Statistical Association, 99, 854–866.CrossRefGoogle Scholar
  29. Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87, 706–710.CrossRefGoogle Scholar
  30. Imbens, G. W. (2004). Nonparametric estimation of average treatment effects under exogeneity: A review. The Review of Economics and Statistics, 86(1), 4–29.CrossRefGoogle Scholar
  31. Imbens, G. W., & Wooldridge, J. M. (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature, 47, 5–86.CrossRefGoogle Scholar
  32. Koopmans, T. C. (1947). Measurement without theory. The Review of Economic Statistics, 29(3), 161–172.CrossRefGoogle Scholar
  33. Koopmans, T. C. (1949). Reply to Rutledge Vining. The Review of Economic Statistics, 31, 86–91.CrossRefGoogle Scholar
  34. Lee, M. J. (2005). Micro-econometrics for policy, program and treatment effects. Oxford: Oxford University Press.CrossRefGoogle Scholar
  35. Lucas, R. E. (1976). Econometric policy evaluation: A critique. In K. Brunner & A. H. Meltzer (Eds.), The Phillips curve and labor markets (pp. 19–46). Amsterdam: North-Holland.Google Scholar
  36. Lucas, R. E. (1980). Methods and problems in business cycle theory. Journal of Money, Credit and Banking, 12, 696–715.CrossRefGoogle Scholar
  37. Lucas, R. E., & Sargent, T. J. (1981). After Keynesian macroeconomics. In R. E. Lucas & T. J. Sargent (Eds.), Rational expectations and econometric practice (pp. 295–319). Minneapolis, MN: University of Minnesota Press.Google Scholar
  38. Manski, C. F. (1993). Identification of endogenous social effects: The reflection problem. The Review of Economic Studies, 60(3), 531–542.CrossRefGoogle Scholar
  39. Manski, C. F. (2003). Partial identification of probability distributions. New York: Springer.Google Scholar
  40. Manski, C. F. (2013). Identification of treatment response with social interactions. The Econometrics Journal, 16(1), S1–S23.CrossRefGoogle Scholar
  41. Manski, C. F., Sandefur, G. D., McLanahan, S., & Powers, D. (1992). Alternative estimates of the effect of family structure during adolescence on high school graduation. Journal of the American Statistical Association, 87, 417.CrossRefGoogle Scholar
  42. Millimet, D., Smith, J., & Vytlacil, E. (Eds.). (2008). Advances in econometrics, Vol 21. Modelling and evaluating treatment effects in econometrics. Amsterdam: JAI Press, Elsevier.Google Scholar
  43. Moran, M., Rein, M., & Goodin, R. E. (Eds.). (2008). The Oxford handbook of public policy. Oxford: Oxford University Press.Google Scholar
  44. Morgan, S. L., & Winship, C. (2007). Counterfactuals and causal inference: Methods and principles for social research. New York: Cambridge University Press.CrossRefGoogle Scholar
  45. Neyman, J. (1923). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Translated in Statistical Science, 5(1990), 465–480.Google Scholar
  46. Pearl, J. (2000). Causality: Models, reasoning and inference. Cambridge: Cambridge University Press.Google Scholar
  47. Pearl, J. (2009). Causal inference in statistics: An overview. Statistics Surveys, 3, 96–146.CrossRefGoogle Scholar
  48. Potì, B., & Cerulli, G. (2011). Evaluation of firm R&D and innovation support: New indicators and the ex-ante prediction of ex-post additionality. Research Evaluation, 20(1), 19–29.CrossRefGoogle Scholar
  49. Rosenbaum, P. R. (2007). Interference between units in randomized experiments. Journal of the American Statistical Association, 102(477), 191–200.CrossRefGoogle Scholar
  50. Rosenbaum, P., & Rubin, D. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.CrossRefGoogle Scholar
  51. Rothman, K. J., Greenland, S., & Lash, T. L. (2008). Modern epidemiology. Philadelphia: Lippincott Williams & Wilkins.Google Scholar
  52. Rubin, D. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688–701.CrossRefGoogle Scholar
  53. Rubin, D. B. (1977). Assignment to treatment group on the basis of a covariate. Journal of Educational Statistics, 2, 1–26.Google Scholar
  54. Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Annals of Statistics, 6(1), 34–58.CrossRefGoogle Scholar
  55. Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.Google Scholar
  56. Sims, C. A. (1980). Macroeconomics and reality. Econometrica, 48, 1–48.CrossRefGoogle Scholar
  57. Sims, C. A. (1996). Macroeconomics and methodology. Journal of Economic Perspectives, 10, 105–120.CrossRefGoogle Scholar
  58. Sobel, M. E. (2006). What do randomized studies of housing mobility demonstrate? Causal inference in the face of interference. Journal of the American Statistical Association, 101(476), 1398–1407.CrossRefGoogle Scholar
  59. Tchetgen-Tchetgen, E. J., & VanderWeele, T. J. (2010). On causal inference in the presence of interference. Statistical Methods in Medical Research, 21(1), 55–75.CrossRefGoogle Scholar
  60. Trochim, W., & Donnelly, J. P. (2007). The research methods knowledge base. Mason, OH: Thomson.Google Scholar
  61. Vining, R. (1949a). Koopmans on the choice of variables to be studied and the methods of measurement. The Review of Economics and Statistics, 31, 77–86.CrossRefGoogle Scholar
  62. Vining, R. (1949b). Rejoiner. The Review of Economics and Statistics, 31, 91–96.CrossRefGoogle Scholar
  63. Wooldridge, J. M. (2001). Unobserved heterogeneity and estimation of average partial effects. Michigan State University, Department of Economics, mimeo.Google Scholar
  64. Wooldridge, J. M. (2002). Econometric analysis of cross section and panel data. Cambridge: MIT Press.Google Scholar
  65. Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data. Cambridge, MA: MIT Press. Chapter 21.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Giovanni Cerulli
    • 1
  1. 1.Research Institute on Sustainable Economic GrowthCNR-IRCrES National Research Council of ItalyRomaItaly

Personalised recommendations