Skip to main content
SpringerLink
Log in

Do dropouts with longer training exposure benefit from training programs? Korean evidence employing methods for continuous treatments

  • Published:
Empirical Economics Aims and scope Submit manuscript

Abstract

Failure of participants to complete training programs is pervasive in existing active labor market programs, both in developed and developing countries. From a policy perspective, it is of interest to know if dropouts benefit from the time they spend in training since these programs require considerable resources. We shed light on this issue by estimating the average employment effects of different lengths of exposure by dropouts in a Korean job training program, and contrasting it to the ones by program completers. To do this, we employ methods to estimate effects from continuous treatments using the generalized propensity score, under the assumption that selection into different lengths of exposure is based on a rich set of observed covariates. We find that dropouts with longer exposures exhibit higher employment probabilities one year after receiving training, but only after surpassing a threshold of exposure of about 12–15 weeks. In contrast, program completers exhibit higher returns from their time of exposure to the program than dropouts, but these tend to decline for longer program durations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. We note, however, that their Table I reports the fraction of individuals assigned to the treatment group that received the intended services. By this definition, individuals that never enroll are counted as dropouts. Our definition will only consider individuals that do enroll but do not complete the program.

  2. The programs they consider include six programs in Latin American countries, one in Germany, and one in the United States.

  3. Some of these studies explicitly state that they have included dropouts in their analysis, while others simply do not mention whether dropouts are included in the analysis or not.

  4. De Crombrugghe et al. (2010b) also focus on dropouts, but they analyze dropout behavior (e.g., who drops out) and do not estimate the effects of the program.

  5. As a simple illustration in the context of the Korean program analyzed here, the objective of the Webmaster (training) Program is to train participants as web developers. In this case, participants with longer training spells would obtain more knowledge and skills regarding web development. Relative to individuals with shorter training spells, those with longer spells accumulate more skills that should lead in principle to a higher employment probability (our outcome variable).

  6. In addition, since we analyze individuals who have enrolled in different types of training programs, our results are average effects for different lengths of exposure to a variety of types of programs.

  7. Our data provide the exact calendar time of the start and termination (completion or dropout) of the job training program. Thus, we use as our treatment variable a measure of participants’ enrollment in the program in days—which we rescale to weeks by dividing the total number of days spent in the program by five.

  8. Overall, about 90 % of the unemployed in South Korea are eligible for unemployment benefits or training.

  9. Nevertheless, a local case worker typically provides information and discusses the options of types of training with the unemployed individual (Lee and Lee 2009). Exogenous variation in this factor likely contributes to the identification of the DRF, as explained in Sect. 2.4.

  10. More specifically, they find that some of the effects of training reported in Lee and Lee (2005) disappear with their sensitivity analysis, while others are corroborated, such as the positive effects for “finance/insurance” and “information/communications” trainings, and the negative effects for “service” and “industrial application” trainings.

  11. One must keep in mind that under the instrumental variables approach a different parameter from the average treatment effect on the treated is identified if the effects are heterogeneous, namely, the average treatment effect for those who are induced to participate in the program as a result of a change in the value of the instrument.

  12. Particularly, their variables related to the local supply of training programs (which they use as instruments) are unavailable to us, while we have the detailed information on dates that allow us to measure the length of enrollment for both completers and dropouts. Choi and Kim (2012) exclude dropouts from their analysis.

  13. For instance, for dropouts, while the maximum duration in the sample is 44.7 weeks, after the removal of individuals with these durations the maximum is only 26 weeks.

  14. Economically Active Population Survey is a survey similar to the Current Population Survey in the United States.

  15. These factors are less critical for completers, since their length of enrollment is determined by the preset duration of the training program they enrolled in.

  16. More specifically, out of the four samples they consider, in one the correlation is highly statistically significant, in another insignificant, and in the other two it is statistically significant at the 10 % level.

  17. They refer to this assumption as weak unconfoundedness since it does not require joint independence of all potential outcomes but instead requires conditional independence to hold for each value of the treatment.

  18. This bandwidth selector has been previously used in economics (e.g., Ichimura and Todd 2007), especially an adaptation of it to the regression discontinuity context (e.g., Lee and Lemieux (2010)).

  19. The distributions considered were the log-normal, inverse Gaussian, and gamma distributions. Within the inverse Gaussian and gamma distributions, we employed link functions corresponding to the identity; inverse powers 1, 1.5, 2; and a log link. To choose a model, we employ the Akaike Information Criteria (AIC) to decide over different distributions, and the deviance measure of McCullagh and Nelder (1989) and the value of the log-likelihood function to decide over link functions. The models estimated, along with the goodness of fit measures, are presented in the Internet Appendix.

  20. According to the AIC measure, the log-normal model appears to be best among those models considered for dropouts. However, the GPS estimated with this model has trouble satisfying the balancing property, and thus was discarded in favor of models with a Gamma distribution. Among the models with a Gamma distribution, those with a log link and an inverse power 1 link have a very similar deviance measure and value of the log likelihood function, and both satisfy the balancing property. Since all results employing these two links are almost identical for dropouts, we decided to report the results employing the log link. Note that a gamma model with log link and scale parameter equal to one is equivalent to an exponential regression model, commonly used in duration analysis. However, our GLM model does not restrict the scale parameter to one, and thus it is more general.

  21. As discussed below, 2007 was a year of strong economic growth in South Korea, which is related to shorter average lengths of enrollment in training for dropouts, relative to the other years in our data.

  22. If the groups of dropouts and completers are combined and a single GPS is estimated for them, a large amount of units do not fall in the overlap region (over 40 %), indicating that the two groups are largely non-comparable in terms of the overall set of observable characteristics available.

  23. We use a cubic specification of the GPS to make it consistent with the specification of the PPM estimator in (4).

  24. Both of these models are estimated with the common-support restricted sample.

  25. For the PPM estimator the derivative at \(t\) is obtained as the “forward” change of one additional week of training: \(\hat{{\mu }}(t+1)-\hat{{\mu }}(t)\). This is the usual approach when using this estimator (e.g., Bia and Mattei 2008). For the IW estimator, the derivative estimate at \(t\) can be computed as the slope coefficient of the linear term from a local quadratic regression of \(Y\) on \(T\) using the re-weighted kernel defined in Sect. 3, \(\tilde{K}_h (T_i ,X_i ;t)\). When computed this way, we choose the appropriate bandwidth by using the procedure described in Fan and Gijbels (1996).

  26. In a previous version of the paper we conducted a separate analysis for male and female dropouts. While the general patterns reported for the full sample hold, males generally show higher effects than females. The estimated average derivative for the entire range of training durations (1–99) for males was three times as high as that for females; and the average derivative for the range 50–99 for males was about 50 % percent higher than that for females. Additionally, as it is typically found, the precision of the estimates for females was lower: none of their estimated average derivatives were statistically significant, while the estimates for training durations 1–99 and 50–99 for males were statistically significant. These results can be found in the Internet Appendix.

  27. Also for this reason, computing the derivatives for the IW estimator using the method described in footnote 27 results in estimates that are too wiggly. Thus, for completers, we compute the derivatives as the “forward” change of one additional week of training (as we do for the PPM estimator).

  28. As it was the case in the results for dropouts, in the sample of completers the IW and PPM estimates are within each other’s 95 % confidence bands.

  29. In those other studies, though, the estimated effects do not recover toward the higher end of training durations.

  30. One possible way to formally gage the relative importance of these two potential explanations would be to compare dropouts and completers that have the same training duration. Unfortunately, doing this is difficult in our data since the degree of overlap in training durations between the two groups is relatively small (see also footnote 24).

  31. One referee illustrated this notion as follows: early dropouts plausibly exit due to finding out a mismatch between their needs and the training program, while late dropouts plausibly exit due to the arrival of a job offer.

  32. We also experimented with three subsamples of percentages of completed planned duration (\(<\)33 %, between 33 and 66 %, and more than 66 %) and found similar results, albeit much less precise.

  33. The ensuing heuristic explanation is based under the premise that there is positive state dependence, but the same arguments hold if there is negative state dependence (with the opposite sign of the corresponding selection effects).

  34. Fitzenberger et al. (2010) find a negative and statistically significant correlation between the random effects of each of the employment and training equations they estimate. This suggests that “...those individuals who have a higher unobserved propensity to enter a program and to stay in a program tend to have a lower unobserved propensity to be employed.”

  35. We note, however, that the precision of the results for both subgroups of dropouts is reduced given their smaller sample size. Due to this loss in precision, despite the clear differences in DRF between the two subgroups, their differences are largely not statistically different from each other (see Fig. 1A.14 in the Internet Appendix).

References

  • Aedo C, Nuñez S (2004) The impact of training policies in latin America and the Caribbean: the case of programa Joven. Research network working paper R-483, Inter-American Development Bank

  • Attanasio O, Kugler A, Meghir C (2011) Subsidizing vocational training for disadvantaged youth in Colombia: evidence from a randomized trial. Am Econ J Appl Econ 3:188–220

    Article  Google Scholar 

  • Betcherman G, Olivas K, Dar A (2004) Impacts of active labor market programs: new evidence from evaluations with particular attention to developing and transition countries. Social protection discussion paper series 0402, World Bank

  • Bia M, Mattei A (2008) A STATA package for the estimation of the dose-response function through adjustment for the generalized propensity score. Stata J 8:354–373

    Google Scholar 

  • Bia M, Mattei A, Flores C, Flores-Lagunes A (2013) A STATA package for the application of semiparametric estimators of dose-response functions. CEPS/INSTEAD working paper no 2013–07, Luxembourg

  • Busso M, DiNardo J, McCrary J (2008) Finite sample properties of semiparametric estimators of average treatment effects. University of Michigan, Mimeo

    Google Scholar 

  • Campos-Vazquez R, Chiapa C (2011) Estudio sobre la temporalidad en el programa [oportunidades] y nuevos esquemas de apoyo y corresponsabilidad para las familias. Mimeo, El Colegio de Mexico

    Google Scholar 

  • Calderón-Madrid A (2006) Revisiting the employability effects of training programs for the unemployed in developing countries. RES working paper R-522, Inter-American Development Bank

  • Card D, Ibarraran P, Regalia F, Rosas D, Soares Y (2011) The labor market impacts of youth training in the dominican republic: evidence from a randomized evaluation. J Labor Econ 29:267–300

    Article  Google Scholar 

  • Card D, Kluve J, Weber A (2010) Active labor market policy evaluations: a meta-analysis. Econ J 120: F452–F477

    Google Scholar 

  • Choi H, Kim J (2012) Effects of public job training programmes on the employment outcome of displaced workers: results of a matching analysis, a fixed effects model and an instrumental variable approach using Korean data. Pac Econ Rev 17:559–581

    Article  Google Scholar 

  • Dammert A, Galdo J (2013) Program quality and treatment completion for youth training programs. Econ Lett 119:243–246

    Article  Google Scholar 

  • De Crombrugghe D, Espinoza H, Heijke H (2010a) Job training programmes with low completion rates: the case of projoven-peru. Working paper ROA-RM-2010/4, Research Centre for Education and the Labor Market, Maastricht University

  • De Crombrugghe D, Espinoza H, Heijke H (2010b) Determinants of dropout behavior in a job training programme for disadvantaged youth. Working paper ROA-RM-2010/8, Research Centre for Education and the Labor Market, Maastricht University

  • Dehejia R, Wahba S (2002) Propensity score-matching methods for nonexperimental causal studies. Rev Econ Stat 84:151–161

    Article  Google Scholar 

  • Fan J, Gijbels I (1996) Local polynomial modeling and its applications. Chapman and Hall, London

    Google Scholar 

  • Fitzenberger B, Osikominu A, Paul M (2010) The heterogeneous effects of training incidence and duration on labor market transitions. IZA Discussion Paper No. 5269

  • Flores C (2005) Estimation of dose-response functions and optimal doses with a continuous treatment. Dissertation, University of California at Berkeley

  • Flores C, Mitnik O (2013) Comparing treatments across labor markets: an assessment of nonexperimental multiple-treatment strategies. Rev Econ Stat 95:1691–1707

    Article  Google Scholar 

  • Flores C, Flores-Lagunes A, Gonzalez A, Neumann T (2012) Estimating the effects of length of exposure to instruction in a training program: the case of job corps. Rev Econ Stat 94:153–171

    Article  Google Scholar 

  • Flores-Lagunes A, Gonzalez A, Neumann T (2010) Learning but not earning? The value of job corps training for hispanic youth. Econ Inq 48:651–667

    Google Scholar 

  • Flores-Lagunes A, Light A (2010) Interpreting degree effects in the returns to education. J Hum Resour 45:439–467

    Google Scholar 

  • Galdo J, Chong A (2012) Does the quality of public-sponsored training programs matter? Evidence from bidding process data. Labour Econ 19:970–986

    Article  Google Scholar 

  • Gerfin M, Lechner M (2002) A microeconometric evaluation of the active labour market policy in Switzerland. Econ J 112:854–893

    Article  Google Scholar 

  • Heckman J, Hohmann N, Smith J, Khoo M (2000) Substitution and dropout bias in social experiments: a study of an influential social experiment. Q J Econ 115:651–694

    Article  Google Scholar 

  • Heckman J, Hotz J (1989) Choosing among alternative nonexperimental methods for estimating the impact of social programs: the case of manpower training. J Am Stat Assoc 84:862–874

    Article  Google Scholar 

  • Heckman J, LaLonde R, Smith J (1999) The economics and econometrics of active labor market programs. In: Ashenfelter O, Card D (eds) Handbook of labor economics, 3A. Elsevier Science North-Holland, Amsterdam, pp 1865–2097

    Chapter  Google Scholar 

  • Heckman J, Navarro S (2007) Dynamic discrete choice and dynamic treatment effects. J Econom 136: 341–396

    Google Scholar 

  • Heckman J, Smith J, Taber C (1998) Accounting for dropouts in evaluations of social programs. Rev Econ Stat LXXX:1–14

    Article  Google Scholar 

  • Hirano K, Imbens G (2004) The propensity score with continuous treatments. In: Gelman Andrew, Meng Xiao-Li (eds) Applied bayesian modeling and causal inference from incomplete-data perspectives. Wiley, West Sussex, pp 73–84

    Google Scholar 

  • Hotz J, Imbens G, Klerman J (2006) Evaluating the differential effects of alternative welfare-to-work training components: a reanalysis of the California gain program. J Labor Econ 24:521–565

    Article  Google Scholar 

  • Ibarrarán P, Rosas Shady D (2009) Evaluating the impact of job training programs in latin America: evidence from IDB funded operations. J Dev Effect 1(2):195–216

    Article  Google Scholar 

  • Ibarrarán P, Villa J (2010) Labor insertion assessment of conditional cash transfer programs: a dose-response estimate for Mexico’s oportunidades. Inter-American Development Bank

  • Ichimura H, Todd P (2007) Implementing nonparametric and semiparametric estimators. In: Heckman J, Leamer E (eds) Handbook of econometrics, 6B. Elsevier Science North-Holland, Amsterdam, pp 5369–5468

    Google Scholar 

  • Imbens G (2000) The role of the propensity score in estimating dose-response functions. Biometrika 87: 706–710

    Google Scholar 

  • Imbens G (2004) Nonparametric estimation of average treatment effects under exogeneity: a review. Rev Econ Stat 86:4–29

    Article  Google Scholar 

  • Kluve J, Lehmann H, Schmidt C (1999) Active labor market policies in Poland: human capital enhancement, stigmatization, or benefit churning? J Comp Econ 27:61–89

    Article  Google Scholar 

  • Kluve J, Schneider H, Uhlendorff A, Zhao Z (2012) Evaluating continuous training programs using the generalized propensity score. J R Statl Soc A Stat 175:587–617

    Article  Google Scholar 

  • Lechner M (2002) Program heterogeneity and propensity score matching: an application to the evaluation of active labor market policies. Rev Econ Stat 84:205–220

    Article  Google Scholar 

  • Lechner M, Wunsch C (2011) Sensitivity of matching-based program evaluations to the availability of control variables. IZA Discussion Paper No. 5553

  • Lee M, Lee S (2003) Analyzing effects of job-trainings suffering dropouts with an optimal multiple matching, office of research. Singapore Management University, Mimeo

    Google Scholar 

  • Lee M, Lee S (2005) Analysis of job-training effects on Korean women. J Appl Econ 20:549–562

    Article  Google Scholar 

  • Lee M, Lee S (2009) Sensitivity analysis of job-training effects on reemployment for Korean women. Empir Econ 36:81–107

    Article  Google Scholar 

  • Lee D, Lemieux T (2010) Regression discontinuity designs in economics. J Econ Lit 48:281–355

    Article  Google Scholar 

  • Lubyova M, van Ours J (1999) Effects of active labor market programs on the transition rate from unemployment into regular jobs in the Slovak Republic. J Comp Econ 27:90–112

    Google Scholar 

  • McCullagh P, Nelder J (1989) Generalized linear models, 2nd edn. Chapman and Hall/CRC, London

  • Mitnik O (2008) Intergenerational transmission of welfare dependency: the effects of length of exposure. Working Paper, University of Miami

  • Na Y, Jeong W, Lee S, Lee J (2007) The study to improve the vocational training system in South Korea. Korea Research Institute for Vocational Education and Training

  • Newey W (1994) Kernel estimation of partial means and a general variance estimator. Econ Theor 10: 233–253

    Google Scholar 

  • Rodriguez-Planas N, Jacob B (2010) Evaluating active labor market programs in Romania. Empir Econ 38:65–84

    Article  Google Scholar 

  • Rosenbaum P, Rubin D (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70:41–55

    Article  Google Scholar 

  • Smith J, Todd P (2005) Does matching overcome lalonde’s critique of nonexperimental estimators? J Econom 125:305–353

    Article  Google Scholar 

  • Waller M (2008) Further training for the unemployed: what can we learn about dropouts from administrative data? FDZ Methoden Report No. 4/2008, Institute for Employment Research, Nuremberg

  • Yoo G, Kang C (2010) The impacts of vocational training on earnings in Korea: evidence from the economically active population survey. KDI J Econ Policy 32:29–53

    Google Scholar 

Download references

Acknowledgments

We thank two anonymous referees and Bernd Fitzenberger (the Editor) for comments that greatly improved the paper. We also thank Carlos A. Flores and seminar participants at CEPS/INSTEAD Research Institute and at Korea Institute of Public Finance for their comments. Useful comments were also provided at the 2010 meeting of the European Society for Population Economics, the 2010 Midwest Economic Association meeting, and the 2010 Korea Economics Association meeting.

Author information

Authors and Affiliations

  1. Korea Information Society Development Institute, 36 Jang gun maeul 3 gil, Gwacheon-si, Gyeonggi-do , 427-710, Korea

    Chung Choe

  2. Department of Economics, State University of New York at Binghamton, P.O. Box 6000, Binghamton, NY , 13902-6000, USA

    Alfonso Flores-Lagunes

  3. IZA, Bonn, Germany

    Alfonso Flores-Lagunes

  4. Korea Research Institute for Vocational Education and Training, 15-1 Chongdam 2-dong, Kangnam-Gu, Seoul, 135-949, Korea

    Sang-Jun Lee

Authors
  1. Chung Choe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alfonso Flores-Lagunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sang-Jun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alfonso Flores-Lagunes.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1 (PDF 1994 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Choe, C., Flores-Lagunes, A. & Lee, SJ. Do dropouts with longer training exposure benefit from training programs? Korean evidence employing methods for continuous treatments. Empir Econ 48, 849–881 (2015). https://doi.org/10.1007/s00181-014-0805-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00181-014-0805-y

Keywords

JEL Classification

Search

Navigation