Skip to main content

Targeting labour market programmes — results from a randomized experiment

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Summary

We evaluate a randomized experiment of a statistical support system developed to assist caseworkers in Swiss employment offices in choosing appropriate active labour market programmes for their unemployed clients. This statistical support system predicted the labour market outcome for each programme and thereby suggested an ‘optimal’ labour market programme for each unemployed person. The support system was piloted in several employment offices. In those pilot offices, half of the caseworkers used the system and the other half acted as control group. The allocation of the caseworkers to treatment and control group was random. The experiment was designed such that caseworkers retained full discretion about the choice of active labour market programmes, and the evaluation results showed that caseworkers largely did not follow the statistical support system. This indicates that stronger incentives are needed for caseworkers to comply with statistical profiling and targeting systems.

References

  1. Black, D., J. Smith, M. Berger, and B. Noel (2003), “Is the Threat of Reemployment Services More Effective than the Services Themselves? Evidence from Random Assignment in the UI System”, American Economic Review 93, pp. 1313–1327.

    Article  Google Scholar 

  2. Black, D., J. Smith, M. Plesca, and S. Plourde (2002), “Profiling UI claimants to Allocate Employment Services: Evidence and Recommendations for the States”, Final report to the US Department of Labour.

  3. Behncke, S., M. Frölich, and M. Lechner (2007), „Pilotprojekt Statistisch assistierte Programmselektion (SAPS)“, Swiss State Secretariat for Economic Affairs (seco), http://www.seco.admin.ch/dokumentation/publikation/00004/00005/index.html?lang=de.

  4. Bell, S., and L. Orr (2002), “Screening (and Creaming?) Applicants to Job Training Programs: The AFDC Homemaker-Home Health Aide Demonstrations”, Labour Economics 9, pp. 279–301.

    Article  Google Scholar 

  5. Berger, M., D. Black, and J. Smith (2001), “Evaluating Profiling as a Means of Allocating Government Services”, in: Econometric Evaluation of Labour Market Policies, M. Lechner and F. Pfeiffer (eds), Physica/Springer, Heidelberg, pp. 59–84.

    Chapter  Google Scholar 

  6. Brownell, K. and T. Wadden (1991), “The Heterogeneity of Obesity: Fitting Treatments to Individuals”, Behavior Therapy, 22, pp. 153–177.

    Article  Google Scholar 

  7. Bryson, A., and D. Kasparova (2003), “Profiling Benefit Claimants in Britain: A Feasibility Study”, Department for Work and Pensions, Research Report no. 196.

  8. Colpitts, T. (2002), “Targeting Reemployment Services in Canada: The Service and Outcome Measurement System (SOMS) Experience”, in: Eberts, R. W., D. J. O’Leary, S. Wandner (eds), Targeting Employment Services, Kalamazoo, Michigan: W.E. Upjohn Institute, pp. 283–301.

    Chapter  Google Scholar 

  9. Dehejia, RH. (2005), “Program Evaluation as a Decision Problem”, Journal of Econometrics 125, pp. 141–173.

    Article  Google Scholar 

  10. De Koning, J. (1999), “The Chance-Meter: Measuring the Individual Chance of Long-Term Unemployment”, Paper presented at the Conference ‘Current Developments in the Evaluation of Employment Policies’, Barcelona, 19–20 July 1999, Division of Labour and Education, Rotterdam.

    Google Scholar 

  11. Dickinson, K., P. Decker, and S. Kreutzer (2002), “Evaluation of WPRS sytems”, in: Eberts, R. W., D. J. O’Leary, S. Wandner (eds), Targeting Employment Services, Kalamazoo, Michigan: W. E. Upjohn Institute, pp. 61–81.

    Chapter  Google Scholar 

  12. Eberts, R. (2002), “Design, Implementation and Evaluation of the Work First Profiling Pilot Project”, Report for the US Department of Labor. http://wdr.doleta.gov/research/keyword.cfm?fuseaction=dsp_resultDetails&pub_id=2287&bas_option=Keywords&start=1&usrt=4&stype=basic&sv=1&criteria=work%20first.

  13. Eberts, R., and C. O’Leary (2002), “A Frontline Decision Support System for Georgia Career Centres”, W. E. Upjohn Institute Staff Working Paper no. 02-84, Kalamazoo, Michigan.

  14. Eberts, R., C. O’Leary and S. Wandner (2002), Targeting Employment Services, Kalamazoo, Michigan: W. E. Upjohn Institute.

    Book  Google Scholar 

  15. Egger, M., and C. Lenz (2006), “Wirkungsevaluation der öffentlichen Arbeitsvermittlung – Evaluationsbericht”, SECO Publikation Arbeitsmarktpolitik No 18 (10.2006), Swiss State Secretariat for Economic Affairs (seco), http://www.seco.admin.ch/dokumentation/publikation/00004/00005/index.html?lang=de. French version: http://www.seco.admin.ch/dokumentation/publikation/00004/00005/index.html?lang=fr.

  16. Fraser, N. (2000), “How Strong is the Case for Targeting Active Labour Market Policies”, International Journal of Manpower 20, pp. 151–164.

    Article  Google Scholar 

  17. Fredriksson, P., and P. Johansson (2003), “Program Evaluation and Random Program Starts”, IFAU Discussion Paper 2003:1.

  18. Frölich (2006), “Statistical Treatment Choice: An Application to Active Labour Market Programmes”, IZA Discussion Paper 2187.

  19. Frölich (2008), “Statistical Treatment Choice: An Application to Active Labour Market Programmes”, forthcoming in the Journal of the American Statistical Association.

  20. Frölich, M., M. Lechner, S. Behncke, S. Hammer, N. Schmidt, S. Menegale, A. Lehmann, and R. Iten (2007), “Einfluss der Rav auf die Wiedereingliederung von Stellensuchenden”, SECO Publikation Arbeitsmarktpolitik no. 20 (1.2007), Swiss State Secretariat for Economic Affairs (seco), http://www.seco.admin.ch/dokumentation/publikation/00004/00005/index.html?lang=de.

  21. Frölich, M., M. Lechner, and H. Steiger (2003), “Statistically Assisted Programme Selection – International Experiences and Potential Benefits for Switzerland”, Swiss Journal of Economics and Statistics 139, pp. 311–331.

    Google Scholar 

  22. Horrace, W., and P. Schmidt (2000), “Multiple Comparisons with the Best, with Economic Applications”, Journal of Applied Econometrics 15, pp. 1–26.

    Article  Google Scholar 

  23. Keum, J. (2001), “Review of the Effectiveness of Public Employment Services”, in: Park, F., Y. Park, G. Betcherman, A. Dar (eds), Labor Market Reforms in Korea: Policy Options for the Future, Korea Labor Institute.

  24. Kreuter, M., and V. Strecher (1996), “Do Tailored Behavior Change Messages Enhance the Effectiveness of Health Risk Appraisals?: Results from a Randomized Trial”, Health Education Research 11, pp. 97–105.

    Article  Google Scholar 

  25. Lalive, R., J. van Ours, and J. Zweimüller (2005), “The Effect Of Benefit Sanctions On The Duration Of Unemployment”, Journal of the European Economic Association 3, pp. 1386–1417.

    Article  Google Scholar 

  26. Lechner, M., and J. Smith (2007), “What is the Value Added by Caseworkers?”, Labour Economics 14, pp. 135–151.

    Article  Google Scholar 

  27. Manski, C. (2000), “Identification Problems and Decisions under Ambiguity: Empirical Analysis of Treatment Response and Normative Analysis of Treatment Choice”, Journal of Econometrics 95, pp. 415–442.

    Article  Google Scholar 

  28. Manski, C. (2004), “Statistical Treatment Rules for Heterogeneous Populations”, Econometrica 72, pp. 1221–1246.

    Article  Google Scholar 

  29. Manski, C. (2006), “Search Profiling With Partial Knowledge of Deterrence”, The Economic Journal 116, pp. F385–F401

    Article  Google Scholar 

  30. Manski, C. (2007), “Minimax-Regret Treatment Choice with Missing Outcome Data ”, Journal of Econometrics 139, pp. 105–115.

    Article  Google Scholar 

  31. Moffitt, R (2006), “Welfare Work Requirements with Paternalistic Government Preferences”, The Economic Journal 116, pp. F441–F458.

    Article  Google Scholar 

  32. Moisala, J., I. Suoniemi, and R. Uusitalo (2006), “Työttömien tilastollinen profilointi”, http://www.mol.fi/mol/fi/99_pdf/fi/06_tyoministerio/06_julkaisut/06_tutkimus/tpt316.pdf.

  33. Murphy, S. (2003), “Optimal Dynamic Treatment Regimes”, Journal of Royal Statistical Society, Series B 65, pp. 331–366.

    Article  Google Scholar 

  34. Murphy, S., K. Lynch, D. Oslin, J. McKay, and T. TenHave (2007), “Developing Adaptive Treatment Strategies in Substance Abuse Research”, Drug and Alcohol Dependence, Supplement 2, 88S, pp. S24–S30.

    Article  Google Scholar 

  35. OECD (1998), “Early Identification of Jobseekers at Risk of Long-Term Unemployment: The Role of Profiling”, OECD Proceedings, Paris.

  36. O’Leary, D., P. Decker, and S. Wandner (2005), “Cost Effectiveness of Targeted Reemployment Bonuses”, Journal of Human Resources 40, pp. 270–279.

    Article  Google Scholar 

  37. Plesca, M., and J. Smith (2005), “Rules versus Discretion in Social Programs: Empirical Evidence on Profiling in Employment and Training Programs”, unpublished, University of Michigan.

  38. Persico, N., and P. Todd (2005), “Passenger Profiling, Imperfect Screening, and Airport Security”, American Economic Review, Papers and Proceedings 95, pp. 127–131.

    Article  Google Scholar 

  39. Persico, N., and P. Todd (2006), “Generalising the Hit Rates Test for Racial Bias in Law Enforcement, With an Application to Vehicle Searches in Wichita”, The Economic Journal 116, pp. F351–F367.

    Article  Google Scholar 

  40. Rudolph, H., and M. Müntnich (2001), “Profiling zur Vermeidung von Langzeitarbeitslosigkeit – erste Ergebnisse aus einem Modellprojekt”, MittAB, pp. 530–553.

  41. Rush, A. (2005), “Algorithm-Guided Treatment in Depression: TMAP and STAR*D”, in M. Bauer, A. Berghofer and M. Adli (eds), Therapieresistente Depressionen – Aktueller Wissensstand und Leitlinien für die Behandlung in Klinik und Praxis, Springer, Heidelberg.

    Google Scholar 

  42. Smith, J. (2002), “Comments on Chapter 10”, in: Eberts, R.W., D.J. O’Leary S. Wandner (eds), Targeting Employment Services, Kalamazoo, Michigan: W.E. Upjohn Institute, 2002, pp. 303–308.

    Google Scholar 

  43. Staghøj, J., M. Svarer, and M. Rosholm (2007), “A Statistical Programme Assignment Model”, mimeo.

  44. Stephan, G., S. Rässler, and T. Schewe (2006), “Das TrEffeR-Projekt der Bundesagentur für Arbeit. Die Wirkung von Maßnahmen aktiver Arbeitsmarktpolitik”, Zeitschrift für Arbeitsmarktforschung, 39, pp. 447–465.

    Google Scholar 

  45. U.S. Department of Labor, Employment and Training Administration (1999), “Evaluation of Worker Profiling and Reemployment Services Policy Workgroup: Final Report and Recommendations”, Washington D.C.

  46. Velicer, W, and J. Prochaska (1999), “An Expert System Intervention for Smoking Cessation”, Patient Education and Counselling 36, pp. 119–129.

    Article  Google Scholar 

  47. Wandner, S. (2002), “Targeting Employment Services under the Workforce Investment Act”, in: Eberts, R.W., D.J. O’Leary, S. Wandner (eds.), Targeting Employment Services, Kalamazoo, Michigan: W.E. Upjohn Institute, 2002, 1–25.

    Google Scholar 

  48. Wunsch, C. (2007), “Optimal Use of Labour Market Policies”, Discussion Paper 2007–26, Department of Economics, University of St.Gallen.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Stefanie Behncke.

Additional information

We are very grateful to Heidi Steiger for her support in the early stages of this project. We also thank Chris O’Leary and Thomas Ragni for helpful comments. We are grateful to the Swiss government (seco) for providing the administrative database as well as substantial financial support for this project. All remaining errors are our own.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Behncke, S., Frölich, M. & Lechner, M. Targeting labour market programmes — results from a randomized experiment. Swiss J Economics Statistics 145, 221–268 (2009). https://doi.org/10.1007/BF03399281

Download citation

JEL-Classification

  • J68

Keywords

  • Profiling
  • active labour market programmes
  • ALMP
  • statistical treatment rules
  • unemployment
  • public employment services