How much should we trust regression-kink-design estimates?

Abstract

In a regression kink (RK) design with a finite sample, a confounding smooth nonlinear relationship between an assignment variable and an outcome variable around a threshold can be spuriously picked up as a kink and results in a biased estimate. In order to investigate how well RK designs handle such confounding nonlinearity, I firstly implement Monte Carlo (MC) simulations and then study the effect of fiscal equalization grants on local expenditure in Japan using an RK design. Results in both the MC simulations and the empirical application suggest that RK estimation without covariates can be easily biased, and this problem can be mitigated by adding observed covariates to the regressors. On the other hand, a smaller bandwidth or a higher-order polynomial, even a quadratic polynomial, tends to result in imprecise estimates although they may be able to reduce estimation bias. In sum, RK estimation with confounding nonlinearity often suffers from bias or imprecision, and estimates are credible only when relevant covariates are controlled for. I also examine how placebo RK estimation can effectively address these issues.

Appendices

Appendix 1: MC simulation results for OLS estimation

See Table 5.

Table 5 MC Simulation results for OLS estimation

Appendix 2: Description of fiscal variables

1.1 Expenditure need: NEED

This index measures the cost of a “standard” level of local public services for a municipality. It is officially referred to as “Standard Fiscal Need” (Kijun Zaisei Juyo Gaku) and calculated annually by the Ministry of Internal Affairs and Communications. Standard Fiscal Need is calculated as follows:

$$\begin{aligned} {\mathrm{NEED}}_i = \Sigma _{k}({\textit{Unit}}\,{\textit{Cost}}_k \times {\textit{Measurement}}\,{\textit{Unit}}_{{\textit{ik}}} \times {\textit{Adjustment}}\,{\textit{Coefficient}}_{{\textit{ik}}}), \end{aligned}$$

where k expresses kth public service. $Unit\ Cost_k$ is a kind of net standard cost per measurement unit for each service item. $Measurement\ Unit_{ik}$ is in most cases the number or size of the beneficiaries of a particular service. $\textit{Adjustment Coefficient}_{ik}$ is a modification ratio that reflects the socioeconomic diversity of a local body and modifies the unit cost in order to make it fit the local body’s socioeconomic circumstances.

1.2 Revenue capacity: CAP

This index measures the fiscal revenue capacity of a municipality before fiscal equalization. It is officially referred to as “Standard Fiscal Revenue” (Kijun Zaisei Syunyu Gaku) and calculated annually by the Ministry of Internal Affairs and Communications. CAP is calculated as follows:

$$\begin{aligned} {\mathrm{CAP}}_i = {\textit{Standard}}\,{\textit{Tax}}\,{\textit{Revenue}}_{i} \times \dfrac{3}{4} + {\textit{Local}}\, {\textit{Transfer}}\,{\textit{Tax}},{\textit{etc}}._i \end{aligned}$$

where Standard Tax Revenues are estimated based on standard tax rates, standard tax collection rates, and estimated tax bases which are calculated using relevant statistics or past tax revenues. Local Transfer Tax, etc. represents the sum of revenues from the Local Transfer Tax and the Special Grant for Traffic Safety Measures. In brief, CAP captures the potential amount of local general revenues before fiscal equalization, which cannot be manipulated by municipalities in the short run.

There are two main reasons that Standard Tax Revenue is multiplied by 3/4.¹⁹ First, the remaining 1/4 of Standard Tax Revenue is excluded from the fiscal equalization process and left for municipalities so that they can cover some remaining fiscal needs that are not taken into account by the Standard Fiscal Needs calculation. Second, this portion of tax revenue is excluded from the fiscal equalization process so that municipalities have some incentive to increase their local tax revenues by enhancing local economic growth. In other words, if the exact amount of Standard Tax Revenue were taken into account in CAP, LAT-receiving local bodies would have less incentive to enhance local economic growth because the increase in Standard Tax Revenue caused by this economic growth would be completely canceled out by the decrease in the LAT grant.

1.3 Revenue capacity (modified for a predetermined covariate)

As is explained above, CAP itself does not represent “real” pre-equalization revenue capacity as it takes into account some policy objectives of the fiscal equalization scheme such as providing economic incentives to municipalities. We can, however, easily recover real pre-equalization revenue capacity by simply replacing 3/4 for 1 in the above definition of CAP.

When I use pre-equalization revenue capacity as a control variable in Sect. 4, I use this modified version of revenue capacity that reflects the real pre-equalization revenue capacity of municipalities. However, because available statistics are only CAP and Local Transfer Tax, I have to assume that revenue from Special Grant for Traffic Safety Measures is negligible. This assumption should not be a major problem because the amount of the Special Grant for Traffic Safety Measures is in general much smaller than the sum of Standard Tax Revenues and Local Transfer Tax.

I therefore estimate this “real” pre-equalization revenue capacity as follows:

$$\begin{aligned}&{\mathrm{RealCAP}}_i \\&= {\textit{Standard}}\,{\textit{Tax}}\,{\textit{Revenues}}_i \times 1 + {\textit{Local}}\,{\textit{Transfer}}\,{\textit{Tax}},{\textit{etc}}._i \\&=({\mathrm{CAP}}_i - {\textit{Local}}\,{\textit{Transfer}}\,{\textit{Tax}},{\textit{etc}}._i) \times 4/3 + {\textit{Local}}\,{\textit{Transfer}}\,{\textit{Tax}},{\textit{etc}}._i \\&\simeq ({\mathrm{CAP}}_i - {\textit{Local}}\,{\textit{Transfer}}\,{\textit{Tax}}_i) \times 4/3 + {\textit{Local}}\, {\textit{Transfer}}\,{\textit{Tax}}_i. \end{aligned}$$

Appendix 3: Stylized description of Japanese fiscal equalization

In this appendix, I explain the stylized features of the Japanese fiscal equalization scheme and describe how the kink based on the LAT grants is generated in more detail. In order to make this description as concise as possible, throughout this appendix I redefine CAP as follows:

$$\begin{aligned} \mathrm{CAP}_i = Standard\ Tax\ Revenue_i \times \dfrac{3}{4}. \end{aligned}$$

In other words, compared with the actual definition in Appendix “Description of fiscal variables,” CAP is simplified by dropping the second term (Local Transfer Tax and some miscellaneous revenues), which is actually much smaller than the first term (Standard Tax Revenue) in the majority of municipalities.

Then, further assuming that there are no additional revenues other than local tax revenues and LAT grants, the relation between pre-equalization standard revenue (denoted as PreRev) and post-equalization standard revenue (denoted as PostRev) can be expressed as follows:

$$\begin{aligned} PostRev_i = \left\{ \begin{array}{ll} PreRev_i,&{} \quad \text {if}\ \ V_i\le 0\\ PreRev_i + \mathrm{GRANT}_i, &{} \quad \text {if}\ \ V_i > 0.\\ \end{array} \right. \end{aligned}$$

By inserting $PreRev_i = Standard\ Tax\ Revenue_i,V_i = \mathrm{NEED}_i-\mathrm{CAP}_i,\mathrm{GRANT}_i = V_i = \mathrm{NEED}_i - \mathrm{CAP}_i$ (if $V_i>0$), and the above definitions of $\mathrm{CAP}_i$ into these equations, they can be rewritten as

$$\begin{aligned} PostRev_i = \left\{ \begin{array}{ll} \mathrm{CAP}_i + Standard\ Tax\ Revenue_i \times \dfrac{1}{4},&{} \text {if}\ \mathrm{CAP}_i \ge \mathrm{NEED}_i\\ \mathrm{NEED}_i + Standard\ Tax\ Revenue_i \times \dfrac{1}{4},&{} \text {if}\ \mathrm{CAP}_i < \mathrm{NEED}_i. \\ \end{array} \right. \end{aligned}$$

(12)

These two equations represent an essential function of the Japanese fiscal equalization scheme. First, if CAP is larger than NEED, no LAT grant is distributed and post-equalization standard revenue is identical to the sum of CAP and $Standard\,Tax\,Revenue\,\times 1/4$. Second, when CAP is smaller than NEED, the LAT grant ensures that municipalities receive the sum of NEED and $Standard\ Tax\ Revenue \times 1/4$. In both cases, this additional amount, $Standard\ Tax\ Revenue \times 1/4$, exists due to the fact that CAP is calculated by $Standard\ Tax\ Revenue \times 3/4$ and the other 1/4 of $Standard\ Tax\ Revenue$ is excluded from the fiscal equalization formula. Because of this excluded part of $Standard\ Tax\ Revenue$, which is officially referred to as “Reserved Revenues,” a wealthier municipality is always wealthier even after fiscal equalization. Figure 9 presents actual scatter plots and local polynomial smoothing of PreRev, PostRev, and NEED against the assignment variable V. It illustrates how the LAT grant phases in at the cutoff point $V=0$.

Fig. 9

Scatter plots of PreRev, PostRev, and NEED). Notes The same sample that is described in Sect. 4 is used for this scatter plot. The local polynomials are obtained using the lpoly command in Stata 13 with the default setting. Sources: reports on the Municipal Public Finance, Census, and CPI

Notice that in this graph PostRev is well above NEED around $V=0$. This implies that municipalities just after $V>0$ have ample additional fiscal resources in excess of NEED. These additional fiscal resource come from the term $Standard\ Tax\ Revenue \times 1/4$ in the second equation of (12). This fact benefits our empirical analysis because LAT grants, which phase in after $V>0$, can be plausibly considered as “general” and “lump-sum” around the threshold, without any difficulties caused by complicated institutional settings of these grants. I conclude this appendix by examining this issue in greater detail.

In this paper, I implicitly assume that LAT grants are “general” and “lump-sum” and local bodies have full discretion in their decision making on spending and taxation. In other words, I presuppose that an estimated coefficient can be straightforwardly interpreted as the effect of general lump-sum grants on local spending under the full discretion of local municipalities.

But it could be misleading to simply assume that LAT grants are completely general and lump-sum as the so-called Bradford-Oates equivalence theorem, and some previous empirical studies in the literature of local public finance have done. The LAT grant is nominally a general grant that a local body can spend on whatever it wants, but at the same time the LAT grant is the grant that guarantees every single municipality a sufficient amount of revenues to cover centrally determined “standard costs” for local public services, which is referred to as Standard Fiscal Needs and denoted as NEED in this paper. It is sometimes pointed out that the central government takes advantage of LAT grants to control local spending by arbitrarily adjusting NEED. In addition to these possibly “centralized” aspects of LAT grants, the provision of local public services is often strongly regulated by the central government through various centralized legal frameworks.

In sum, although local bodies do not have to strictly follow these centrally determined standards, they quite often cannot control their expenditures on some local public services because the basic legal and provisional frameworks of these local services are centrally determined. Hayashi (2000, (2006) provides critical reviews of empirical studies on flypaper effects in Japan and points out that these previous studies do not consider these institutional settings of the Japanese general grant and naively treat it as a general and lump-sum grant as studies of similar schemes in the USA do.

In fact, this obligatory and centralized feature of local public services is part of the institutional basis of LAT grants: Since the central government forces all local bodies to provide particular levels of local public services, fiscal resources for these services have to be guaranteed by the intergovernmental fiscal transfer which reflects the expected costs of these services. This feature of the LAT grant is officially referred to as a function of “fiscal resource guarantee.”

According to Fig. 9, however, I would argue that LAT grants can be considered to be general and lump-sum around the threshold $V=0$ regardless of the centralized features of these grants and local administration. In other words, around the threshold, PostRev is well above NEED, and therefore, the relatively obligatory local public services that are reflected in the calculation of NEED can be financed even without the LAT grant. It is thus possible to assume that the marginal increase in the LAT grant around the threshold affects local bodies’ expenditure in exactly the same way that standard general and lump-sum grants do.

Appendix 4: Description of data arrangement

Japan is a unitary state which has three tiers of administrative authorities: the central government, 47 prefectures, and 1750 municipalities (as of 2010). Municipalities are classified into four categories: cities (shi), towns (cho), villages (son), and special districts (ku). Cities are generally larger than towns and villages, and in principle the minimum population required to become a city is 50,000. Even if the population of a city becomes less than 50,000, however, it does not have to become a town or village. The 23 special districts are all located in Tokyo Metropolis (prefecture) and have similar duties to other municipalities but follow a different vertical fiscal equalization scheme managed by the prefecture. Cities and towns/villages have similar duties under the LAT grant fiscal equalization scheme, but cities have more responsibilities in some areas.

In this paper, I only use the datasets for cities, but not all cities are included in my analysis. First, I exclude so-called designated cities, which consisted of the 12 largest cities in Japan during the sample period. I drop these cities from the sample because their response to the marginal increase in their LAT grants might be institutionally different from other cities as a result of the fact that some of the duties normally assigned to prefectures are delegated to them and their administrative responsibility is thus larger than that of normal cities. Second, I also remove the cities that experienced amalgamation between 1975 and 1999 because the calculation of the LAT grants for these merged cities was affected by special measures. Because this special measure was in effect for 5 years after amalgamation, municipalities which merged before 1975 were not affected by this measure after 1980. Finally, there are some LAT-receiving municipalities whose need-capacity gap is apparently different from the amount of their LAT grant, possibly due to measurement errors or typos. Therefore, I drop 18 observations in which |Need-capacity gap − LAT grant per capita| is larger than 10,000 yen.

Appendix 5: RK estimation validity check

1.1 Smooth density of the assignment variable

Following a density test applied to an RD design in McCrary (2008), Card et al. (2015) presents a density test applied to an RK design using collapsed data with equal-sized bins based on an assignment variable. Two required variables in this collapsed data set are the number of observations in each bin and the midpoint values of the assignment variable in each bin. I use bins with width 2 and bandwidth $[-50,+50]$, which is a benchmark bandwidth for local regressions in this paper. Table 6 shows that in each sample an RK estimate is statistically insignificant if the order of polynomial is equal to or larger than two. The value of Akaike information criteria (AIC) is smallest when the order of polynomial is two. Figure 10 graphically illustrates that there seems to be no kink at the threshold.

Table 6 RK estimates for the density of the assignment variable ($bin\ size=2,bandwidth\ |V|<50$)

Fig. 10

Density of the assignment variable ($bin\ size=2,bandwidth\ |V|<50$). Notes Bin size is 2 and fitted curves are based on the quadratic, cubic, and quartic RK models that impose continuity at the cutoff

1.2 Distributions of covariates against V

See Fig. 11.

Fig. 11

Covariates against need-capacity gap ($bandwidth\ |V|<50$). Notes Optimal data-driven plots are generated by the default setting of rdplot command with Stata 13, which is developed by Calonico et al. (2015). Quadratic fits are based on the quadratic RK model that imposes continuity at the cutoff

Appendix 6: OLS results from the empirical application

See Table 7.

Table 7 OLS estimates for total expenditure