The Use of Administrative Data to Evaluate the Impact of Active Labor Market Policies: The Case of the Italian Liste di Mobilità

Abstract

Administrative data are a valuable source of information in several ways for the design of an impact evaluation. Conditional on providing the information required for the evaluation, they feature major advantages: they cover the whole (sub)population of reference for their administrative purposes, and not just a sample of individuals; very often they extend over long time periods; they are not susceptible to the typical measurement errors of survey data; and, finally, they are free of charge. The case of the Italian Liste di Mobilità, an Active Labor Market Policy (ALPM) to handle collective redundancies, illustrates how much empirical labor economics can benefit from having access to this kind of data. The accessibility of administrative archives for scientific purposes is a critical issue, as the history of this scheme demonstrates.

1 Introduction

High-quality data are essential to design an effective evaluation of the effects of a public intervention. Leaving aside the notion of quality relevant in all scientific fields—available information should provide valid and reliable measurements for the concepts they are intended to measure—this chapter takes “high quality” to have two different but equally relevant meanings. First, the data should provide information on the outcomes relevant for the evaluation of the intervention, i.e., the individual status and behaviors the intervention might have an impact on, whether intentionally or as a side effect. Second, the data should allow identification of a comparison group, made up of individuals not exposed to the intervention under evaluation and equivalent to the group of individuals exposed to the intervention, in all respects relevant to the outcomes considered in the evaluation.

The first condition—which may seem obvious—often requires a great deal of effort by the analyst to recover information on the outcomes the intervention might have an impact on.

The second condition is more technical. To identify the average causal effect of an intervention on the pool of individuals exposed to it, one needs to properly approximate what those individuals would have experienced in the so-called counterfactual world, i.e., one in which the intervention does not happen. This counterfactual experience being by definition unobservable, one has to resort to the factual experience of a pool of individuals not exposed to the intervention, the comparison group. To be a credible approximation of the counterfactual, the comparison group must be made up of individuals as similar as possible to those exposed to the intervention.

By far the most convincing way of obtaining two groups that are equivalent in all respects—one exposed to the intervention, the other providing the approximate counterfactual—is to select them at random in the same way as experiments run in laboratories to test the effectiveness of a new drug. In an observational study—i.e., where the analyst has no control over who is going to receive the exposure—an effective impact evaluation design requires the analyst to understand the selection process as well as possible, i.e., the set of rules and behaviors according to which the exposure state of the individuals is determined. Knowledge of the selection process implies knowledge of the differences in composition of the two groups of individuals.

This raises an important issue. Knowledge that, as a result of the selection process, individuals exposed to the intervention are on average, for example, younger, more educated, or less experienced, specifies the information required to select a proper comparison group. This entails observing age, education, and previous labor market experience for each individual included in the study. With this information at hand, the analyst is in a position to compare the two groups of individuals, controlling for the characteristics with respect to which the two groups are on average different as a result of the selection process.

Administrative data are a valuable source of information in several ways for the design of an impact evaluation. Their obvious main limitation is that they are developed and maintained to meet the specific administrative requirements of their reference institution and are not primarily intended to serve scientific purposes. As a straightforward implication, the (sub)population covered by the archive and the information it provides may be useless for the specific problem in which the analyst is interested. On the other hand, conditional on providing the information required for the evaluation, administrative data feature major advantages: they cover the whole (sub)population of reference for their administrative purposes, and not just a sample of individuals, very often they extend over long time periods, they are not susceptible to the typical measurement errors of survey data, and, finally, they are free of charge.

To illustrate the potential as well as the limitations of administrative data for evaluating public interventions, this chapter presents the case of the Italian Liste di Mobilità (LM), a program to handle collective redundancies which was introduced in the early 1990s and was in operation until recently. All the existing studies on the LM have exploited administrative data, with a clear distinction between first-generation studies, which relied on poor information, and subsequent studies that over the years have had access to much richer data, which gave insight into the effects of the LM unknown to first-generation studies.

This chapter is organized in five sections. Section 2 presents the basic provisions of the LM program. Section 3 describes the evaluation questions on which existing studies focused. Section 4 presents a review of the evaluation designs adopted by researchers and of the results obtained. The emphasis in this section is on the advantages and limitations of the administrative archives from which researchers drew their data. Section 5 concludes. For those unfamiliar with the econometrics of program evaluation, an appendix briefly describes the basic ideas on which the evaluation of LM has so far been developed.

2 Basic Provisions of the Liste di Mobilità Program

The LM program was introduced by Law 233 (1991) and then slightly modified by Law 236 (1993). It was in operation until the end of 2016. Some provisions varied according to industry, worker’s occupation, geographic area, etc. and underwent frequent modifications over time. Here, the main provisions relevant to the area and the period covered in this chapter are outlined. For additional details, see Paggiaro et al. (2009) and references therein. Firms with more than 15 employees—referred to here as “large” firms—could collectively dismiss redundant workers, because of plant closure or restructuring, and automatically enroll them in a special register maintained by a regional authority. Workers dismissed by firms with up to 15 employees—referred to here as “small” firms—could also enroll in the LM on a voluntary basis. Evidence indicates that most eligible workers dismissed by small firms did register in the LM. To be eligible for the LM, a worker must have had been on a permanent contract with the dismissing firm for at least 1 year. Workers in the LM were in principle required to fulfill some obligations with respect to training and job offers. An LM worker who refused an appropriate job offer from the local public labor exchange was dropped from the program. However, enforcement of these rules was largely absent. In practice, a worker’s willingness to accept a job offer was not tested, and a worker enrolled in the LM could refuse any job offer and retain LM status up to the end of the eligibility period. The basic features of the program are summarized in Table 1. It is apparent that there were two separate subprograms, targeted at two non-overlapping populations:

• 1. The first subprogram applied to workers who had been collectively dismissed by large firms. Upon dismissal, they entered the LM by default and received a monetary benefit, which was partly transferred to any firm that later hired them. The same firm also benefited from a rebate on social security contributions (SSCs) for up to 2 years. This subprogram was fundamentally different for workers aged 50 years or over who met the requirements for “long mobility.” For these workers, the active component of the program was largely dominated by the passive component.

• 2. The second subprogram applied to workers either collectively or individually dismissed by small firms. They entered the LM on a voluntary basis and were eligible only for the active component of the program, which could be supplemented with the much less generous standard unemployment benefits where applicable.

Table 1 Basic features of the Liste di Mobilità (LM)

The common feature of the two subprograms was the duration of eligibility as determined by the worker’s age at dismissal. However, the effect of the eligibility duration was very likely to vary across subprograms, as well as across the 40- and 50-year thresholds. Calculations by Paggiaro et al. (2009) show that the best strategy for the employer was to hire a worker from the LM on a temporary 1-year contract and then to switch it to a permanent one. If the worker was not eligible for monetary benefits, this strategy provided a saving over 2 years, worth approximately 23% of the labor cost. The employer saved an additional 13% of the labor cost by hiring a worker eligible for monetary benefits on his or her first day in the LM, irrespective of the duration of eligibility.

The bulk of savings for the hiring firm was represented by the significant rebate on SSCs. In the case of the best hiring strategy, it ranged from 65% to 70% of total savings. In addition, it coincided with total savings—which at 23% of the 2-year labor cost were still substantial—in all circumstances when there was no benefit transfer and specifically at the end of the eligibility period. One implication is that it might not have been all that relevant to a potential employer whether or not an LM worker to be hired was entitled to monetary benefits and whether he or she was above or below the 40-year threshold, because in all cases the employer received the same rebate on SSCs, as long as the worker was eligible.

3 The Evaluation Problem

Most of the studies on the LM so far have focused on the effect of the length of the eligibility period. As an example, looking at the effect of being eligible for 2 years in the LM instead of just 1 year, the issue can be formulated as follows: does allowing workers just above the 40-year threshold to stay in the program for 2 years affect their chance of reemployment—and the quality of the job they eventually find—relative to what would happen to them with 1-year eligibility? With the standard job search model as a background (Mortensen and Pissarides 1999), it is apparent that the LM provides two contrasting incentives: (i) the incentive to firms, which benefit from the rebate on SSCs (and in some cases from the benefit transfer), to provide workers enrolled in the LM with more job offers than they would otherwise receive and (ii) the incentive to workers drawing monetary benefits from the LM program to lengthen their unemployment spell by increasing their reservation wage and to refuse any job offers they receive, at least over a large fraction of their eligibility period.

In comparative terms, both these incentives tend to be higher under the 2-year regime than under the 1-year regime. From a theoretical point of view, indication of the effect is a priori uncertain and depends on which of the two incentives prevails. Furthermore, it is difficult to isolate the effects of these incentives from those of provisions about engagement in temporary employment. Thus, the effect can be identified only empirically.

Research on the LM so far has focused largely on program participants aged under 50 years, focusing on the causal effect of being eligible for 2 years versus 1 year either on the duration of the unemployment spell or on the probability of having working status at selected post-enrollment periods (see Brunello and Miniaci 1997, Paggiaro and Trivellato 2002, Paggiaro et al. 2005). The prevailing evidence can be summarized in two statements: (i) for workers entitled only to the active component of the LM, i.e., dismissed by small firms, the additional year of eligibility has no effect on reemployment probabilities or on the time spent waiting for the first permanent job, and (ii) for workers also entitled to monetary benefits, i.e., dismissed by large firms, the additional year of eligibility has a negative impact: older workers, who draw benefits longer, have significantly lower reemployment probabilities and a significantly longer time spent waiting for a first permanent job than their younger colleagues. This effect tends to be larger for women, but not consistently across all the case studies.

4 How the Availability of Data Drove the Design of the Evaluations

There are three main sources of administrative information relevant to the evaluation of the impact of the LM:

• 1. The archive resulting from the management of the LM itself

• 2. The archive resulting from the operations of the public labor exchange (Centri per l’Impiego (CPI))

• 3. The archive of the Italian social security agency (INPS)

These three sources differ greatly in terms of the information they provide on the labor market history of workers enrolled in the LM. Broadly speaking, the first archive provides information only on transitions to permanent employment after enrollment in the LM; the CPI archive is much more detailed, providing information on each single employment spell, whether temporary or permanent, and both before and after enrollment in the LM; and, finally, the INPS archive adds data on employee wages to the CPI information.

The first-generation studies on the LM made use of the first archive only. More recent studies had access to the CPI archives and in some instances to the INPS archive. All these studies are broadly similar in terms of the basic feature of the evaluation design: one way or another, they exploit the discontinuity along the age dimension in the duration of eligibility. But they are quite different with respect to the variety of outcomes taken into consideration as well as with respect to the robustness of the results obtained. All these differences across studies are driven entirely by the administrative information available to the researchers. The main studies are reviewed below, focusing specifically on the value of each piece of information to the design of the impact evaluation.

4.1 Studies Based on the Liste di Mobilità Archive

The LM was managed by regional employment agencies, which were also responsible for data collection. There was no common format for collecting individual data on the program across the country; therefore, there is no consistent national database available in Italy resulting from the operation of those agencies. To illustrate what first-generation studies did, the work of Paggiaro and Trivellato (2002) is considered here. They use data from the administrative records of the Veneto Regional Employment Agency. The Veneto region is a large, relatively well-developed region of Northeastern Italy. With more than 4.4 million inhabitants, it makes up 7.7% of the Italian population. At the time the authors refer to (the late 1990s), Veneto had an employment rate close to 42%, an unemployment rate around 5.2%, and a per capita gross national product (GNP) some 15–20% higher than the national average. These traits of comparatively low unemployment and high economic activity characterize Veneto as similar to the rest of Northern Italy but far from representative of the much less developed South.

The analysis by Paggiaro and Trivellato (2002) is restricted to the period from January 1995 to March 1999. Each worker is followed from enrollment in the LM up to the occurrence of one of the following events: (i) exit into a permanent job and (ii) exit from the lists at expiration of the eligibility period. Incomplete durations are registered for workers still enrolled in the lists at the end of the observation window. There are clear limitations arising from these data.

The main limitation is that there is no explicit information on spells of temporary work. It is clear that some of these spells did exist because there were cases where the duration of enrollment in the lists was longer than the legal duration. This prevented the authors from discriminating between periods spent in the lists as temporarily employed and time spent unemployed (drawing income support, if so entitled). Those periods are then collapsed into a single spell of enrollment in the lists, waiting for one of the two alternative final events, i.e., transition to a permanent job or expiration of the eligibility period.

In principle, there is also a problem of self-selection into the program, since workers dismissed by small firms who decide not to register are not included in the study. Fortunately, in practice this problem turned out to be irrelevant: evidence provided by officials of the Veneto Regional Employment Agency indicates that, essentially, all workers dismissed by small firms did register in the lists.

The analysis focuses on the first two age groups, while workers aged 50 or over are excluded. The final sample is thus reduced to 36,405 workers. The parameter of interest is the effect of being offered 2 years of eligibility instead of 1 on the rate of transition to permanent employment. The analysis is performed separately for workers entitled and not entitled to income support, and by gender. Since duration of eligibility depends only on age at dismissal, the assignment process fits the sharp regression discontinuity design (RDD). In principle, one could exploit this feature to identify and non-parametrically estimate the causal effect of the additional year of eligibility by comparing workers just above the 40-year threshold with those just below it.

Considering the size of the sample and the relatively low rate of transition to permanent employment, the authors chose a semiparametric strategy exploiting the whole sample. They fit a proportional hazard model to the duration of the spell in the lists, estimating the difference between the two groups of workers—up to 39 and 40–49, respectively—controlling for age (as well as for other observable characteristics of the workers). Figure 1 reports their main results.

Fig. 1

Estimated baseline hazards controlling for age. Source: Paggiaro and Trivellato (2002)

For workers entitled to income support, the additional year of eligibility appears to have a significant positive effect on the time spent searching for permanent employment, varying with the duration of the spell and by gender. For men, the effect is negligible during the first year but has a strong negative peak just at the end of the first year, i.e., when the eligibility period for younger workers expires. Within the second year, the effect is still negative but less pronounced, possibly because the reference group of workers under 40 is somewhat selected, as it consists only of workers who obtained temporary contracts. The pattern of the effect is quite different for women. It is negative from the start of the spell, stays essentially constant throughout the first year, including the last week, and then increases during the second year up to its end. Similar differential treatment effects, but much less pronounced, are also found for workers dismissed by small firms, for whom benefit packages do not include any benefit transfer component.

Summing up, the passive component of LM appears to drive the results. A major open question is that with these data, there is no way of understanding what happens to the overall employment chances—rather than simply permanent employment—of workers enrolled in the LM, both during enrollment and in the years after expiry of the eligibility period.

4.2 Studies Based on the Archives of the Public Labor Exchange and the Social Security Agency

Compared with the LM archive, the key additional piece of information provided by the CPI archive is a much more detailed description of the labor market history of workers enrolled in the LM, before, during, and after the eligibility period. Two major advances are thus possible with respect to the first-generation studies. First, the causal effects of the duration of eligibility are established with respect to the monthly employment rate in each month since enrollment in the lists. This provides a much richer picture of the short- and medium-term effect of the program on the pattern of reemployment of these workers.

Second, the rich set of information on preenrollment labor market history of workers entering the LM allows implementation of a set of specification tests to validate the design of the impact. This strengthens the credibility of the results.

Finally, access to the INPS archive allows addition of information on wages to the analysis. The next part of this chapter presents the analysis developed by Paggiaro et al. (2009). They refer to workers enrolled in the LM in Veneto, 1995–1998. Their analysis is carried out separately (i) for the two LM subprograms, (ii) by gender, and finally (iii) both at the 40-year and at the 50-year thresholds.

As in the first-generation studies, Paggiaro et al. (2009) identify the causal effect by exploiting the discontinuity along the age dimension in the duration of the eligibility. The internal validity of this strategy rests on the assumption that, if individuals just above and just below the threshold were assigned to the same eligibility regime, they would experience the same average outcome. Clearly, any evidence providing support to this claim would be most welcome. The authors also argue that, at least in principle, one could imagine reasons why the assumption might be violated. For instance, one could argue that workers enrolled in the LM and over the threshold are on average different from those below it, if firms and unions bargain on the composition of the pool of workers to be dismissed.

Figure 2 shows the distribution of enrolled workers by age, separately for the two groups of entitlement to monetary benefits and for gender. The main evidence is a large discontinuity at the 50-year threshold for workers with monetary benefits. The McCrary (2008) test turns out to be statistically significant for both men and women. As this is the threshold at which most of the workers are entitled to “long mobility,” it is clear that workers dismissed by large firms have been selected in consideration of the peculiar differential advantages brought to them by the “long mobility” provisions. In principle, this selection opens the door to a violation of the identifying restriction of the RDD.

Fig. 2

Distribution of workers enrolled in the Liste di Mobilità by age, 1995–1998, Veneto. Source: Paggiaro et al. (2009)

To investigate this, the authors follow Lee (2008) and carry out a set of overidentification tests to validate the strategy. These tests are based on comparing individuals just above the threshold and their younger colleagues just below it with respect to their preprogram employment history. As it is hard to think of a causal effect of the LM program on employment status and wages experienced by a worker 3 years before entering the LM, for example, any discontinuity with respect to those variables at the cutoff points should be interpreted as a sign of a differential composition around the cutoff point with respect to characteristics relevant to subsequent employment status and wages. Hence, it should be taken as evidence against the validity of the RDD restriction.

Figure 3 plots the employment rate by age for workers enrolled in the LM 3 years before enrollment. No significant discontinuity appears at the cutoff points. Similar evidence is presented in the paper for the weekly wages of workers, as well as looking at other preenrollment periods. This evidence confirms that workers aged 39 and 40, as well as workers aged 49 and 50, had the same employment histories before entering the program.

Fig. 3

Employment rate 3 years before enrollment in the Liste di Mobilità (95% confidence interval represented by the dashed lines; the vertical dashed lines represent threshold ages). Source: Paggiaro et al. (2009)

Figure 4 shows the age profile of the reemployment probability 3 years after enrollment, when most workers have already dropped out of the LM. The age profile shows no significant discontinuity at the thresholds, except for a major statistically significant drop at 50 years for workers with monetary benefits, i.e., those who can use “long mobility” as a bridge to retirement. The drop is as large as 32.6% for men, whereas it is 27.1% for women.

Fig. 4

Employment rates 3 years after enrollment in the Liste di Mobilità (95% confidence interval represented by the dashed lines; the vertical dashed lines represent threshold ages). Source: Paggiaro et al. (2009)

Having established that adding a second year of eligibility did not affect the chances of getting a job, the issue is whether or not it affected the quality of the jobs these workers in their early 40s eventually found. In particular, did it have any effect on wages once a worker enrolled in the LM secured a new job? Keeping the standard job search model as a background, it might well be the case that entitlement to a longer eligibility period allows a worker to be more selective during the job search, particularly if he or she draws monetary benefits, ultimately obtaining a higher wage.¹

The evidence presented by Paggiaro et al. (2009) shows that there are no significant effects for all the groups taken into consideration. The only slight difference discernable is again at the 50-year threshold for both men and women with monetary benefits. However, this difference is hardly significant and is of the same size as that observed for wages 3 years before enrollment (not reported here; see Figure 3 in Paggiaro et al. 2009). Overall, the evidence is that the additional year of eligibility has no impact on wages 3 years after enrollment in the LM.

Summing up, the main substantive evidence on the LM program is that at 40 years, there is no effect of the 2-year vs. 1-year eligibility regime on reemployment rates 3 years after enrollment. There is also no effect for workers eligible only for the active component or for those eligible for the additional passive component. That is, the active component exactly counteracts the passive one. Moreover, the additional year of eligibility has no effect on wages once a new job is secured.

There is some evidence (not reported here) that women with monetary benefits postponed their reentry to work: after 2 years, the effect of the additional year of eligibility on the employment rate is −15%, which disappears 1 year later.

There is a strong negative effect at 50 for workers with monetary benefits: the effect is as large as −35% and −30% for men and women, respectively. This is the effect of providing them with the option of using LM as a bridge to retirement.

4.3 Disentangling the Roles of the Active and Passive Components

Studies on the effects of the LM reviewed in the preceding sections have been motivated by the fact that, from a theoretical point of view, the sign of the effect of the 2-year regime of eligibility vs. the 1-year regime is a priori uncertain. Indeed, the same uncertainty is faced with respect to the effect of the “passive + transfer” component, since it is apparent that this component provides two contrasting incentives: on the one hand, the monetary benefit to the worker increases his or her reservation wage, lengthening the unemployment spell; on the other hand, the benefit transfer to the hiring firm makes the worker more likely to receive job offers.

Mazzarella et al. (2014) exploit the discontinuity along the firm size dimension to identify the causal effect of the monetary benefit provided to workers dismissed by firms above the 15-employee cutoff. Because of errors in measuring the exact status with respect to eligibility for the monetary benefit, even if the probability of receiving the monetary benefit as a function of firm size is discontinuous at 15 employees, the size of the discontinuity is smaller than 1 (see the case of workers up to age 39 in Fig. 5; there is similar evidence for older workers). This problem is dealt with by resorting to the so-called fuzzy RDD.

Fig. 5

Probability of receipt of the “passive + transfer” component as a function of firm size by age group and gender (point estimates and 95% confidence interval). Source: Mazzarella et al. (2014)

Figure 6 summarizes the results for workers aged 40 to 49. The estimated causal effect of the “passive + transfer” component on the reemployment probability is nearly always negative over months 1 to 36 for each age–gender group, suggesting that the effect of the passive component prevails over the effect of the transfer component. The only exception is the group comprising men younger than 40. The order of magnitude of the estimated effect is around −5% for all groups except young men and is fairly stable over the 36 months.

Fig. 6

Causal effect of the “passive + transfer” component on employment rate at the 15-employee threshold, from 36 months before to 36 months after enrollment in the Liste di Mobilità (LM), by age group and gender (point estimates and 95% confidence intervals). Source: Mazzarella et al. (2014)

5 Conclusions

This chapter concludes by emphasizing a collateral aspect of the story of LM and its evaluation exercises. The LM has been in operation since the early 1990s. First results on its effects started becoming available only at the end of the 1990s (Brunello and Miniaci 1997), when first-generation studies provided some estimates, albeit confined to transitions to permanent employment. Preliminary results on the impact of the LM on overall post-enrollment labor market history became available in 2005 (Paggiaro et al. 2005). Evidence of the dramatic impact of the “long mobility” clause on the labor market participation of over 50 workers became available only 4 years later, along with evidence for the lack of any impact of the duration of eligibility on earnings (Paggiaro et al. 2009). This was some 15 years after the program started its operations.

One might wonder what caused this large delay in the availability of a credible estimate of the impact of the LM. The answer is clear: it is not due to a lack of data. The administrative data required for the evaluation have been there since the very beginning. This means that there were conditions for obtaining a complete picture on the (mal)functioning of the program at least by the mid-1990s. The delay is first of all due to a lack of demand for evaluation on the part of policy-makers. In addition, and perhaps on a related note, the delay has been caused by the lack of an established protocol to access data for scientific purposes. Researchers took the lead in providing evaluations of LM even in the absence of explicit demand, making use of the information they were able to obtain.

There is still a missing piece of evidence on the effect of LM. It is known that at the 40-year threshold, the additional year of eligibility did not cause any improvement in the probability of securing a job nor did it boost wages once a job had been obtained. What has yet to be established is the impact of that additional year of eligibility on the costs of the program: how much did it cost to realize that zero impact on employment and wages? Again, the data to answer this question do exist. However, they were simply not accessible to researchers at the time the papers reviewed here were developed.

A.1 Appendix: A 30-second Overview of the Econometrics of the LM Studies

The institutional rules of the program induce discontinuities in the level and/or type of benefits received by eligible individuals, either along the age dimension or along the firm size dimension. One way or another, all the studies reviewed in this chapter exploit those discontinuities to identify causal effects, following the logic of the regression discontinuity design (RDD) (Imbens and Lemieux 2008, Lee and Lemieux 2010). Here the RDD identification strategy is briefly reiterated, with reference to the 40-year threshold.

The duration of the eligibility period is assigned on the basis of a worker’s age at dismissal only, the running variable in RDD terminology. Comparing workers assigned to the 2-year regime with those assigned to the alternative 1-year regime, the econometric problem is how to disentangle the causal effect of the second year of eligibility from a pure age effect. Let treatment I, denoting eligibility for the second year, be equal to 1 for individuals aged 40–49 and 0 for those aged up to 39.

The outcomes are the economic performances in the post-enrollment months of workers enrolled in the LM, e.g., employment status, duration of unemployment, earnings, and so on. Let Y ₁ and Y ₀ be the potential outcomes a specific worker would experience after being exposed to and being denied the treatment, respectively. Let Y be the outcome observed for a specific individual. It is linked to the potential outcomes and to the treatment status by the following identity:

$$ Y={Y}_0+I\times \left({Y}_1-{Y}_0\right), $$

(A1)

i.e., for each specific individual, either Y ₀ or Y ₁ is observed, depending on eligibility status.

By contrasting the average outcome experienced by the treatment group with the average outcome experienced by the comparison group, the following identity is obtained:

$$\begin{array}{llll} E\left[Y\left|I=1\right.\right]-E\left[Y\left|I=0\right.\right]&=&E\left[{Y}_1-{Y}_0\left|I=1\right.\right]\nonumber\\&&+\left\{E\left[{Y}_0\left|I=1\right.\right]-E\left[{Y}_0\left|I=0\right.\right]\right\} \end{array}$$

(A2)

where E[Y ₁ − Y ₀|I = 1] is the average treatment effect on the treatment group (ATT) and the difference in brackets is the so-called selection bias, i.e., the difference observed between the two groups even in the absence of any difference in the duration of eligibility. It is induced by the differential composition of the two groups with respect to age.

By conditioning on age = 40, the only age around which there are both individuals assigned to the 2-year regime and individuals assigned to the 1-year regime, the selection bias becomes:

$$ E\left[{Y}_0\left|I=1\right.,\mathrm{age}=40\right]-E\left[{Y}_0\left|I=0\right.,\mathrm{age}=40\right]. $$

(A3)

The classic sharp RDD identifying restriction states that this selection bias is zero by claiming that the conditional mean E[Y ₀ |age] is a continuous function of age in a neighborhood of age = 40. The rationale for this restriction is that, since age is by design the only individual characteristic relevant for the assignment of the duration of eligibility, it is the only individual characteristic possibly raising a selection bias problem. Hence, by comparing individuals who are very similar with respect to age, the selection bias shrinks to zero, implying that the quantity:

$$ E\left[Y\left|I=1\right.,\mathrm{age}\approx 40\right]-E\left[Y\left|I=0\right.,\mathrm{age}\approx 40\right] $$

(A4)

identifies a meaningful causal parameter.

The drawback of this design is that if the program impact is heterogeneous across subjects—as is likely in most cases—then the quantity (A4) identifies the ATT only near the threshold, i.e., the RDD is useless to identify the causal effect away from the threshold value. An additional problem is that by the very nature of the causal parameter identified by the RDD—it is a causal parameter at a specific value of the running variable—only units within a certain distance of the threshold value contribute to the estimation of the causal effect. Hence, it is not the overall sample size that matters for the precision of the estimate but the number of units in a suitably defined neighborhood of the threshold value.

To estimate the ATT at the threshold value requires estimating the conditional expectations of the two potential outcomes to the left and to the right of the threshold, respectively. There are several methods available; one of the most popular is the local linear regression as proposed by Hahn et al. (2001).

A too small sample size is precisely the problem that Paggiaro and Trivellato (2002) deal with. Instead of restricting the analysis to individuals close to the threshold value (in their case, workers enrolled in the LM in an age range around 40), they use the whole sample to model the relationship between the outcome and age using a semiparametric model. Then they use the estimated model to evaluate the average outcome just above the threshold and the average outcome just below it.

The fuzzy RDD used by Mazzarella et al. (2014) arises when it is not the exposure to the treatment that jumps suddenly from 0 to 1 as the running variable crosses the threshold. Rather, it is the probability of being exposed that jumps suddenly but with the size of the jump smaller than 1. Mazzarella et al. (2014) observe that this happens essentially because of a measurement error problem on the running variable (see the discussion in Sect. 4.3). In this instance, the difference in average outcome between those just above the threshold (i.e., eligible for the passive component of the LM) and those just below it (i.e., not eligible for the passive component of the LM) understates the average causal effect because there are units below the threshold receiving the passive component as well as units above the threshold not receiving it. To correct for this bias, it is sufficient to rescale the difference in outcome across the threshold by dividing it by the jump at the threshold in the probability of exposure to the treatment. Mazzarella et al. (2014) consider it to be the jump documented in Fig. 6.

Hahn et al. (2001) show that the resulting estimate is an instrumental variable (IV) locally at the threshold value. The eligibility status—as defined by the running variable being above/below the threshold value—acts as an IV for the actual exposure to the treatment.