Causal tree with instrumental variable: an extension of the causal tree framework to irregular assignment mechanisms

Abstract

This paper provides a link between causal inference and machine learning techniques—specifically, Classification and Regression Trees—in observational studies where the receipt of the treatment is not randomized, but the assignment to the treatment can be assumed to be randomized (irregular assignment mechanism). The paper contributes to the growing applied machine learning literature on causal inference, by proposing a modified version of the Causal Tree (CT) algorithm to draw causal inference from an irregular assignment mechanism. The proposed method is developed by merging the CT approach with the instrumental variable framework to causal inference, hence the name Causal Tree with Instrumental Variable (CT-IV). An improved version, named Honest Causal Tree with Instrumental Variable (HCT-IV), able to estimate more reliably the heterogeneous causal effects, is also proposed. As compared to CT, the main strength of CT-IV and HCT-IV is that they can deal more efficiently with the heterogeneity of causal effects, as demonstrated by a series of numerical results obtained on synthetic data. Then, the proposed algorithms are used to evaluate a public policy implemented by the Tuscan Regional Administration (Italy), which aimed at easing the access to credit for small firms. In this context, HCT-IV breaks fresh ground for target-based policies, identifying interesting heterogeneous causal effects.

Appendices

Appendix A: Estimation of the expected mean squared error

Let $ITT_Y(x)$ be the true intention to treat, conditional on a certain set of covariates’ values $X_i=x$:¹²

$$\begin{aligned} ITT_Y(x)={\mathbb {E}}\big [Y_i(Z_i=1)-Y_i(Z_i=0)|X_i=x\big ]. \end{aligned}$$

The (adjusted) Expected Mean Squared Error (henceforth, EMSE¹³) is the expectation over the test sample $\varOmega ^{te}$ and the estimation sample $\varOmega ^{est}$ of the following adjusted Mean Squared Error ($MSE^{adj}$), whose precise expression is given later in equation (60):

$$\begin{aligned} EMSE(\varOmega ^{te}, \varOmega ^{est})= {\mathbb {E}}_{\varOmega ^{te}, \varOmega ^{est}}\big [MSE^{adj}(\varOmega ^{te}, \varOmega ^{est})\big ]. \end{aligned}$$

First, the MSE can be defined as the average over the test sample of the squared error of prediction associated with the conditional estimator obtained on the estimation sample. It is expressed as:

where $\#(\varOmega ^{te})$ is the number of observations in the test sample, $ITT_{Y,i}$, the unit level intention to treat, is

$$\begin{aligned} ITT_{Y,i} = Y_i(Z_i=1) - Y_i(Z_i=0)\,, \end{aligned}$$

and $ITT_{Y,i}^{te}$ denotes its value on an element of the test sample. Following [8], we can adjust the MSE by the empirical mean (on the test sample) of $(ITT^{te}_{Y,i})^2$. Since this term does not depend on the choice of the estimator, subtracting it does not affect the way the criterion ranks different estimators [8]. The adjusted version of the MSE is the following:

(60)

Nevertheless, the unit level intention to treat $ITT_{Y,i}^{te}$ is infeasible, since one cannot observe for the same unit i, and at the same time, the effects under its assignment to the treatment and under its assignment to the control. However, if one puts aside this problem of infeasibility for a moment, one can expand the EMSE, on a partition of a given tree ${\mathbb {T}}$, as follows:¹⁴

Since ${\mathbb {E}}_{i \in \varOmega ^{te}} \big [ \big (ITT_{Y,i}^{te} - ITT_Y(X_i^{te})\big )\big ]$ is zero and the covariance between the two terms $\big (ITT_{Y,i}^{te} - ITT_Y(X_i^{te})\big )$ and is zero,¹⁵ then the term cancels out:¹⁶

leading to the following:

where denotes the conditional variance of given $\varOmega ^{est}$.

Now it is possible to proceed with the estimation of $EMSE^{HCT-IV}$ for the Honest Causal Tree with Instrumental Variable. For $X_i^{te} \in {\mathbb {X}}_j$, the conditional variance in the second term of (54) can be approximated by the within-leaf conditional variance estimated on the training sample divided by the number of observations in the leaf (in the estimation sample):

The expected value can be estimated as:

where the ${\mathcal {P}}^{est}_{{\mathbb {X}}_j}$’s are the leaf shares on the estimation sample. Assuming approximately equal leaf size, we get:

With respect to the first term in (54), $ITT^2_Y(X_i^{te})$ can be now approximated using the square of the estimated in the training sample minus an estimate of the within-leaf variance of , obtained by taking into account the number of observations (in the training sample) in the leaf ${\mathbb {X}}_j$ associated with $X_i^{te}$:¹⁷

Assuming again that the leaves are of equal size, the expected value of $ITT^2_Y(X_i^{te})$ in (54) can be approximated as follows:

Merging the formulas above, we get an estimator of $EMSE^{HCT-IV} (\varOmega ^{te}, \varOmega ^{est})$ for every partition:

The first component of (55) is the conventional causal tree criterion, which rewards the partitions with a stronger heterogeneity in the causal effect, while the second component penalizes those partitions that create variance in the leaf causal estimates. This algorithm tends to balance the causal tree tendency to reward heterogeneity in the causal estimates by penalizing imprecise causal estimates within the leaves.

Fig. 2

CT-IV built on the PDC data (the colors in the figure are reported in the online version of the article)

Fig. 3

CT built on the PDC data (the colors in the figure are reported in the online version of the article)

Moreover, one can estimate the terms $\left( {ITT}_Y^{tr}(X_i^{tr})\right) ^2$ and , respectively, as follows:

where ${\mathbb {X}}_i$ is the leaf to which $X_i^{tr}$ is assigned by the tree ${\mathbb {T}}$, $N_{1, {\mathbb {X}}_i}^{tr}$ is the number of units assigned to treatment within the leaf ${\mathbb {X}}_i$, $N_{0, {\mathbb {X}}_i}^{tr}$ is the number of units assigned to control within the leaf ${\mathbb {X}}_i$, and, for a generic leaf ${\mathbb {X}}_j$ of the tree ${\mathbb {T}}$, $s^2_{1, {\mathbb {X}}_j}$ is the within-leaf variance of $ITT_Y$ for the units assigned to the treatment, and $s^2_{0, {\mathbb {X}}_j}$ is the within-leaf variance of $ITT_Y$ for the units assigned to the control.¹⁸ Concluding, one can estimate the overall $EMSE^{HCT-IV}$ as follows:

In practice, one can use the same sample size for both $\varOmega ^{tr}$ and $\varOmega ^{est}$, so the estimator above becomes:

Appendix B: Case study with causal tree with IV

Figure 2 depicts the CT-IV built using the data from the case study in Sect. 5.

Appendix C: Case study with causal tree and honest causal tree

Figures 3 and 4 depict the CT and the HCT built using the data from the case study in Sect. 5, respectively.

Fig. 4

HCT built on the PDC data (the colors in the figure are reported in the online version of the article)