Evaluating Long-Horizon Event Study Methodology

Abstract

We describe the fundamental issues that long-horizon event studies face in choosing the proper research methodology and summarize findings from existing simulation studies about the performance of commonly used methods. We document in details how to implement a simulation study and report our own findings on large-size samples. The findings have important implications for future research.

We examine the performance of more than 20 different testing procedures that fall into two categories. First, the buy-and-hold benchmark approach uses a benchmark to measure the abnormal buy-and-hold return for every event firm and tests the null hypothesis that the average abnormal return is zero. Second, the calendar-time portfolio approach forms a portfolio in each calendar month consisting of firms that have had an event within a certain time period prior to the month and tests the null hypothesis that the intercept is zero in the regression of monthly portfolio returns against the factors in an asset-pricing model. We find that using the sign test and the single most correlated firm being the benchmark provides the best overall performance for various sample sizes and long horizons. In addition, the Fama-French three-factor model performs better in our simulation study than the four-factor model, as the latter leads to serious over-rejection of the null hypothesis.

We evaluate the performance of bootstrapped Johnson’s skewness-adjusted t-test. This computation-intensive procedure is considered because the distribution of long-horizon abnormal returns tends to be highly skewed to the right. The bootstrapping method uses repeated random sampling to measure the significance of relevant test statistics. Due to the nature of random sampling, the resultant measurement of significance varies each time such a procedure is used. We also evaluate simple nonparametric tests, such as the Wilcoxon signed-rank test or the Fisher’s sign test, which are free from random sampling variation.

Appendix

This appendix includes the details on the benchmarks and the test statistics that are used in our simulation studies. We use five benchmarks. The first benchmark is a reference portfolio constructed on the basis of firm size and BE/ME. We follow Lyon et al. (1999) to form 70 reference portfolios at the end of June in each year from 1979 to 1997. At the end of June of year t, we calculate the size of every qualified firm as price per share multiplied by shares outstanding. We sort all NYSE firms by firm size into ten portfolios, each having the same number of firms, and then place all AMEX/NASDAQ firms into the ten portfolios based on firm size. Since a majority of NASDAQ firms are small, approximately 50 % of all firms fall in the smallest size decile. To obtain portfolios with the same number of firms, we further partition the smallest size decile into five subportfolios by firm size without regard to listing exchange. We now have 14 size portfolios. Next, we calculate each qualified firm’s BE/ME as the ratio of the book equity value (COMPUSTAT data item 60) of the firm’s fiscal year ending in year t − 1 to its market equity value at the end of December of year t − 1. We then divide each of the 14 portfolios into five subportfolios by BE/ME and conclude the procedure with 70 reference portfolios on the basis of size and BE/ME.

The size and BE/ME matched reference portfolio of an event firm is taken to be the one of the 70 reference portfolios constructed at the month of June prior to the event month that matches the event firm in size and BE/ME. The return on a size and BE/ME matched reference portfolio over τ months is calculated as

$$ B{R}_i^{SZBM}={\displaystyle \prod_{t=0}^{\tau -1}\left[1+\frac{{\displaystyle {\sum}_{j=1}^{n_t}{r}_{jt}}}{n_t}\right]}-1, $$

(14.5)

where month t = 0 is the event month, n _t is the number of firms in month t, and r _jt is the monthly return of firm j in month t. We use the label “SZBM” for the benchmark that is based on firm size and BE/ME.

The second benchmark is a reference portfolio constructed on the basis of firm size, BE/ME, and market beta. The Fama-French three-factor model suggests that expected stock returns are related to three factors: a market factor, a size-related factor, and a BE/ME-related factor. Reference portfolios constructed on the basis of size and BE/ME account for the systematic portion of expected stock returns due to the size and BE/ME factors, but not the portion due to the market factor. Our second benchmark is based on firm size, BE/ME, and market beta to take all three factors into account.

To build a three-factor reference portfolio for a given event firm, we first construct the 70 size and BE/ME reference portfolios as above and identify the one that matches the event firm. Next, we pick firms within the matched portfolio that have returns in CRSP monthly returns database for all 24 months prior to the event month and compute their market beta by regressing the 24 monthly returns on the value-weighted CRSP return index. Lastly, we divide these firms that have market beta into three portfolios by their rankings in beta and pick the one that matches the event firm in beta as the three-factor reference portfolio. The return on a three-factor portfolio over τ months is calculated as

$$ B{R}_i^{SZBMBT}={\displaystyle \prod_{t=0}^{\tau -1}\left[1+\frac{{\displaystyle {\sum}_{j=1}^{l_t}{r}_{jt}}}{n_t}\right]}-1, $$

(14.6)

where month t = 0 is the event month, n _t is the number of firms in month t, and r _jt is the monthly return of firm j in month t. We use the label “SZBMBT” to indicate that the benchmark is based on firm size, BE/ME, and market beta.

The third benchmark is a reference portfolio constructed on the basis of firm size, BE/ME, and pre-event correlation coefficient. The rational for using pre-event correlation coefficient as an additional dimension is that returns of highly correlated firms are likely to move in tandem in response to not only changes in “global” risk factors, such as the market factor, the size factor, and the BE/ME factor in the Fama-French model, but also changes in other “local” factors, such as the industry factor, the seasonal factor, liquidity factor, and the momentum factor. Over a long time period following an event, both global and local factors experience changes that affect stock returns. It is reasonable to expect more correlated stocks would be affected by these factors similarly and should have resulting stock return patterns that are closer to each other. Therefore, returns of a reference portfolio on the basis of pre-event size, BE/ME, and pre-event correlation coefficient are likely to be better estimate of the status quo (i.e., what if there was no event) return of an event firm.

To build a reference portfolio on the basis of size, BE/ME, and pre-event correlation coefficient, we first construct the same 70 size and BE/ME reference portfolios as above and identify the combination that matches the event firm. Next, we pick firms within the matched size and BE/ME reference portfolio that have returns in CRSP monthly returns database for all 24 months prior to the event month and compute their correlation coefficients with the event firm over the pre-event 24 months. Lastly, we choose the ten firms that have the highest pre-event correlation coefficient with the event firm to form the reference portfolio. Return of the portfolio over τ months is calculated as

$$ B{R}_i^{MC10}={\displaystyle \sum_{j=1}^{10}\frac{{\displaystyle {\prod}_{t=0}^{\tau -1}\left(1+{r}_{jt}\right)}-1}{10}}, $$

(14.7)

where month t = 0 is the event month and r _jt is the monthly return of firm j in month t. We use the label “MC10” to indicate that the benchmark consists of the most correlated ten firms. The benchmark return is the return of investing equally in the ten most correlated firms over the τ months beginning with the event month. The benchmark is to be considered as a hybrid between the reference portfolio discussed above and the matching firm approach shown below.

The fourth benchmark is a single firm matched to the event firm in size and BE/ME. Barber and Lyon (1997) report that using a size and BE/ME matched firm as benchmark gives measurements of long-term abnormal return that is free of the new listing bias, the rebalancing bias, and the skewness bias documented in Kothari and Warner (1997) and Barber and Lyon (1997). To select the size and BE/ME matched firm, we first identify all firms that have a market equity value between 70 % and 130 % of that of the event firm and then choose the firm with BE/ME closest to that of the event firm. The buy-and-hold return of the matched firm is computed as in Eq. 14.2. We use the label “SZBM1” to represent the single size and BE/ME matched firm.

The fifth and last benchmark is a single firm that has the highest pre-event correlation coefficient with the event firm. Specifically, to select the firm, we first construct the 70 size and BE/ME reference portfolios and identify the one that matches the event firm. Next, we pick firms within the matched size and BE/ME reference portfolio that have returns in CRSP monthly returns database for all 24 months prior to the event month and compute their correlation coefficients with the event firm over the pre-event 24 months. We choose the firm with the highest pre-event correlation coefficient with the event firm as the benchmark. The buy-and-hold return of the most correlated firm is computed as in Eq. 14.2. We use the label “MC1” to represent the most correlated single firm.

We apply four test statistics to test the null hypothesis of no abnormal returns: (a) Student’s t-test, (b) Fisher’s sign test, (c) Johnson’s skewness-adjusted t-test, and (d) bootstrapped Johnson’s t-test.

• (a) Student’s t-test

  Given the long-term buy-and-hold abnormal returns for a sample of n event firms, we compute Student’s t-statistic as follows:

  $$ t=\frac{\overline{ AR}}{s(AR)/\sqrt{n}}, $$

  (14.8)

  where $\overline{ AR}$ is the sample mean and s(AR) the sample standard deviation of the given sample of abnormal returns. The Student’s t-statistic tests the null hypothesis that the population mean of long-term buy-and-hold abnormal returns is equal to zero. The usual assumption for applying the Student’s t-statistic is that abnormal returns are mutually independent and follow the same normal distribution.

• (b) Fisher’s sign test

  To test the null hypothesis that the population median of long-term buy-and-hold abnormal returns is zero, we compute Fisher’s sign test statistic as follows:

  $$ B={\displaystyle \sum_{i=1}^n\mathrm{I}\left(A{R}_i>0\right)}, $$

  (14.9)

  where I(AR _i > 0) equals 1 if the abnormal return on the ith firm is greater than zero and 0 otherwise. At the chosen significance level of α, the null hypothesis is rejected in favor of the alternative of nonzero median if B ≥ b(α/2, n, 0.5) or B < [n − b(α/2, n, 0.5)], or in favor of positive median if B ≥ b(α, n, 0.5), or in favor of negative median if B < [n − b(α, n, 0.5)]. The constant b(α, n, 0.5) is the upper α percentile point of the binomial distribution with sample size n and success probability of 0.5. The usual assumption for applying the sign test is that abnormal returns are mutually independent and follow the same continuous distribution. Note that application of the sign test does not require the population distribution to be symmetric. When the population distribution is symmetric, the population mean equals the population median, and the sign test then indicates the significance of the population mean (see Hollander and Wolfe 2000, Chap. 3).

• (c) Johnson’s skewness-adjusted t-test

  Johnson (1978) developed the following skewness-adjusted t-test to correct the misspecification of Student’s t-test caused by the skewness of the population distribution. Johnson’s test statistic is computed as follows:

  $$ J=t+\frac{1}{3\sqrt{n}}{t}^2\gamma +\frac{1}{6\sqrt{n}}\gamma, $$

  (14.10)

  where t is Student’s t-statistic given in Eq. 14.8 and γ is an estimate of the coefficient of skewness given by $\gamma ={\displaystyle \sum_{i=1}^n{\left(A{R}_i-\overline{ AR}\right)}^3}/s{(AR)}^3n$. Johnson’s t-test is applied to test the null hypothesis of zero mean under the assumption that abnormal returns are mutually independent and follow the same continuous distribution. At the chosen significance level of α, the null hypothesis is rejected in favor of the alternative of nonzero mean if J > t(α/2, υ) or J < − t(α/2, υ), or in favor of positive mean if J > t(α, υ), or in favor of negative mean if J < −t(α, υ). The constant t(α, υ) is the upper α percentile point of the Student’s t distribution with the degrees of freedom υ = n − 1.

• (d) Bootstrapped Johnson’s skewness-adjusted t-test

  Sutton (1992) proposes to apply Johnson’s t-test with a computer-intensive bootstrap resampling technique when the population skewness is severe and the sample size is small. He demonstrates it by an extensive Monte Carlo study that the bootstrapped Johnson’s t-test reduces both type I and type II errors compared to Johnson’s t-test. Lyon et al. (1999) advocate the bootstrapped Johnson’s t-test in that long-term buy-and-hold abnormal returns are highly skewed when buy-and-hold reference portfolios are used as benchmarks. They report that the bootstrapped Johnson’s t-test is well specified and has considerable power in testing abnormal returns at the 1-year horizon. In this paper, we document its power at 3- and 5-year horizons.

  We apply the bootstrapped Johnson’s t-test as follows. From the given sample of n event firms, we draw m firms randomly with replacement counted as one resample until we have 250 resamples. We calculate Johnson’s test statistic as in Eq. 14.10 for each resample and end up with 250 J values, labeled as J ₁, ⋯, J ₂₅₀. Let J ₀ denotes the J value of the original sample. To test the null hypothesis of zero mean at the significance level of α, we first determine two critical values, C ₁ and C ₂, such that the percentage of J values less than C ₁ equals α/2 and the percentage of J values greater than c ₂ equals α/2, and then reject the null hypothesis if J ₀ < C ₁ or J ₀ > C ₂. We follow Lyon et al. (1999) to apply the bootstrapped Johnson’s t-test with m = 50.⁹