Review of Accounting Studies

, Volume 12, Issue 2, pp 217–225

Discussion of “biases in multi-year management financial forecasts: Evidence from private venture-backed U.S. companies”

Authors

Article

DOI: 10.1007/s11142-007-9027-2

Cite this article as:
Demers, E.A. Rev Acc Stud (2007) 12: 217. doi:10.1007/s11142-007-9027-2
  • 133 Views

Abstract

Armstrong, Dávila, Foster, and Hand (“ADFH”) use a proprietary venture capital database of revenue and profit projections submitted by young firms seeking financing to attempt to address a number of questions related to forecasts by managers of early stage, venture-backed, private entrepreneurial firms. The proprietary dataset together with the creative use of a “historically-grounded conditional projections” methodology are the most interesting features of ADFH’s study. However, these same aspects give rise to empirical design constraints that the study does not fully overcome. In addition, there are numerous leaps of logic required to arrive at some of ADFH’s conclusions and there are alternative explanations for ADFH’s findings that have not been entirely refuted. This leaves the reader with some doubt as to whether all of ADFH’s conclusions are fully substantiated. Nevertheless, the evidence presented makes an interesting contribution to our understanding of the forecasting behavior of young, private, rapidly growing, VCbacked firms, and provides some natural economic and methodological leads into further studies of these issues.

JEL Classifications

G24M13M41

Keywords

Management forecastsForecasting biasVenture capitalStart-ups

1 Overview

Armstrong, Dávila, Foster, and Hand (henceforth ≪ ADFH ≫) make an interesting contribution to the literatures related to management forecasting and the economics of young firms. Their study uses a proprietary venture capital database of revenue and profit projections submitted by young firms seeking financing to attempt to address the following questions:
  1. 1.

    Are managers’ long-term forecasts of revenues, expenses, and profits biased (and if so, by how much)?

     
  2. 2.

    Do the biases in managers’ forecasts strategically depend upon the forecast horizon?

     
  3. 3.

    Do the biases in managers’ forecasts strategically depend upon the verifiability of the firm’s assets?

     

A major strength of the paper is the proprietary dataset that allows the authors to investigate a much understudied but highly economically important sector of the economy: early stage, venture-backed, private entrepreneurial firms. Another nice feature of the study that is innovative, potentially very useful in other accounting research settings, but not without significant limitations in its current implementation, is ADFH’s “historically-grounded conditional projections” methodology. The weaknesses of the study relate primarily to the generalizability of its findings as well as empirical design constraints that are somewhat imposed upon the authors by the limitations of their novel dataset. In addition, there appear to be some logical leaps of faith involved in arriving at some of ADFH’s conclusions based upon the evidence they present.

Overall, the authors conclude with respect to their primary research questions that: 1) revenue forecasts beyond one year are optimistically biased, while expense forecasts are unconditionally pessimistic; 2) revenue forecasts are strategically increasing in optimism as the time horizon is extended; and 3) forecasts are strategically more optimistic for firms with higher asset intangibility. Although I am not convinced of all of the authors’ conclusions, I feel that ADFH have mined an interesting dataset and make a substantial contribution with this study. The authors are appropriately cautious in attempting to generalize their results beyond the sample studied, and thus I focus most of my discussion on empirical design issues and alternative interpretations of the results presented.

2 Research design

ADFH are interested in examining questions related to the potential bias in management forecasts. Accordingly, arriving at an estimate of forecast bias that has high construct validity is the most critical aspect of this study’s research design, and all of the authors’ subsequent economic inferences necessarily depend upon it. An extended prior literature related to analyst earnings forecasts typically sets out the following: \({\frac{Actual_{it} -Forecast_{it} }{scalar_{it} }\ne 0}\) implies bias, where the scalar may be either a firm-quarter-specific earnings or share price figure. In the analyst forecast literature, considerable attention has been paid to arriving at the correct proxy for analysts’ expectations (e.g., using mean versus median forecasts, dropping stale-dated estimates, etc.). In the current study we have the reverse problem; the forecasts are well-defined, but the actuals must be estimated. ADFH creatively develop their “historically-grounded conditional projections” (henceforth “HGCPs”) to obtain estimates of the firms’ realizations for revenues, expenses, and net income. Thus, they have: \({\frac{Forecast_{it} -HGCP_{it} }{HGCP_{it} }\ne 0}\) implies bias. Clearly, any systematic bias in the researchers’ estimated HGCP as a proxy for the actual financial realizations will induce a finding of “biased” forecasts.

2.1 Evaluating the historically-grounded conditional projection models

The authors use the fitted estimates from four sector-specific regressions of the following equations to estimate the unknown historical actual financial results (revenues, net income, and expenses, in turn), or “HGCPs”:
$$ REV_{it} =\alpha _k +\beta _k firmage_{it} +\sum\limits_{s=1}^4 {\upsilon _{sk} } DSTATE_{is} +\varepsilon _{it} $$
(1)
$$ REV_{it} =\alpha _k^{\prime} +\gamma _k^{\prime} REV_{i,t-1} +\beta _k^{\prime} firmage_{it} +\sum\limits_{s=1}^4 {\upsilon _{ks}^{\prime} } DSTATE_{is} +\varepsilon _{it}^{\prime} $$
(2)
Thus, for the first stage in Eq. (1) ADFH use firm age and four US state dummy variables to explain historical revenue levels. A theoretical—or even anecdotal—justification for this specification is noticeably lacking. Although it is quite conceivable that firms in technology-concentrated regions such as California or Massachusetts might evolve differently (e.g., more rapidly), and/or have different probabilities of success than firms scattered around less VC-rich and tech-centric areas of the US, it isn’t at all obvious that the level of a firm’s revenues should be simply some base amount determined by the firm’s geography plus a multiple of firm age. For example, it is not intuitively appealing to think that revenues are, as suggested in Table 4A, 2.59 * age plus some undisclosed estimated fixed dollar amount (is it $2 million? $4 million? $10 million?) for having headquarters in, say, CA, independent of last year’s revenues, the number of prior rounds of VC financing, or even any other more general fundamental economic indicators such as the growth realizations for the industry as a whole.

The empirical results of the Eq. 1 regressions, to the extent reported in the first columns of Table 4A, bear out my concerns. First, the mean adjusted-R2s from the sector-specific regressions are only 5% for each of revenues and expenses, and just 3% for net income. Although not explicitly disclosed, these means imply that some sector-specific regressions actually have adjusted-R2s of less than 3%. By comparison, the pioneering labor economics literature that the authors cite as the basis for their HGCP empirical technique report R2 figures in the range of 33% to 56% for their fully specified models. 1 By any reasonable standards, ADFH’s model (1) that explains less than 3% of the cross-sectional within-sample variation would not seem to provide a credible out-of-sample prediction model.

A second troubling aspect of the Table 4A HGCP results is that the reader is not provided with any information regarding the distribution of sector-specific coefficient estimates and adjusted R2’s. The authors’ sincere efforts at parsimony in presentation in this instance have obfuscated the reader’s ability to understand some critical aspects of the econometrics underlying their study. 2 Many obvious questions that are critical to our evaluation of the HGCP models arise as a result of this reporting opacity: Are all of the coefficients significant in each model? If not, are the insignificant coefficients used when fitting the estimated regression model to generate the HGCP estimates? How many of the coefficient estimates are negative? Just how bad (in terms of adjusted-R2s) are the worst-fitting models? Given the weak mean results reported in the first columns of Table 4A, could ADFH obtain more credible results by focusing on just those sectors for which the within-sample models are reasonably well-fitting? Why aren’t even the mean intercept and state dummy coefficient estimates reported? How much of the explanatory power in the regressions is coming from the unintuitive state dummy term that captures the assumed base level of revenues derived from geography? Are the coefficient estimates on these state dummies at least consistent with expectations? For example, are they all positive and is there an intuitively appealing rank order amongst them such as, e.g., the coefficient on CA exceeding the others? Because the results are reported in an overly-aggregated manner, the reader is unable to glean these necessary insights into the credibility of ADFH’s mission-critical HGCP models. And unfortunately the prima facie evidence that is presented is consistent with a lack of economic and statistical validity to this historical within-sample model.

The results reported in the second columns of Table 4A cast further doubt on the credibility of the estimates from the model (1) regressions. We see there that when an economically meaningful variable such as the prior year’s revenues or expenses is added to the model, the sign on the previously included age variable flips in each of the revenue and expense regressions, and the magnitudes of the coefficients are also significantly altered. The intuition for revenues (expenses) now being a declining function of age, is neither apparent nor explained. If a fixed state effect is warranted, then surely a state variable interacted with the prior year’s revenues is worthy of consideration? The interpretation of revenue growth rates varying by technology-concentration geographic indicators seems more intuitive than a model that assumes a fixed level of revenues depending upon the state of domicile. The design setup seems to suggest that multiple observations for the same firm are included in the Table 4A regressions, yet there isn’t any mention of the t-statistics having been adjusted to reflect this lack of independence. If the adjustment has not yet been made, the borderline significance of age in the revenue and expense regressions may be eliminated upon appropriate correction. Once again we are not provided with any information about the coefficients on the state dummies or the intercept term.

All of these questions and missing bits of information aside, the adjusted R2s from the within-sample regressions in the second columns of Table 4A are now up to an impressive level of 82 to 85% for expenses and revenues, respectively. Overall, the results from the second columns of Table 4A suggest that historical actual revenue, expense, and net income for ADFH’s sample firms may be well described by a simple auto-regressive process. Furthermore, the coefficients on these variables would appear to be reasonably economically interpretable in that revenues, expenses, and positive net income are explained as 118, 113 and 128% of the prior year’s revenues, expenses, and net income, respectively, and negative net income erodes by 36% of the prior year’s realization. However, if we refer back to Table 3 we see that actual revenues and expenses have grown by rates of 48% (at the median, and the data is right-skewed so the mean rates are higher), and thus the mean coefficient estimates reported in the second columns of Table 4A do not appear to correspond with historical growth rate realizations. Specifically, the regression estimates would appear to underestimate the realized rate of growth over the prior years’ revenues. This is significant because an underestimate in the projected actual revenues leads to a bias in favor of finding optimistic revenue forecasts.

2.2 Scaling issues

The OLS regression model reported in Table 3E uses net income scaled by expenses as the dependent variable, rather than the more traditional ratio of net income to revenues. ADFH sensibly chose this specification in order to minimize statistical distortions caused by zero or very small revenue observations. However, in their key Table 5 results, the authors report statistics that are based upon Eqs. 3 and 4, both of which use predicted historical revenues (“HPREDREV”) as the scalar. The authors do not provide support for this seemingly important inconsistency nor is there a discussion of its potential to admit “statistical distortions.” Furthermore, if the predicted historical revenues are underestimated as a result of the Eqs. 1 and 2 specification issues discussed above, then dividing by these underestimated HPREDREV figures will exacerbate the underestimation problems associated with HPREDREV in the numerator, and thereby further overstate the alleged biases in management’s forecasts.

3 Alternative interpretations of the empirical evidence

3.1 Question #1: Are managers’ forecasts biased?

The authors conclude that revenue forecasts beyond one year are highly optimistic. One alternative interpretation of the evidence presented is that the projected actual revenues are underestimated (see previous discussion on the HGCPs). An interesting finding that defies this alternative explanation is ADFH’s documentation that one-year ahead revenue forecasts are pessimistic. Unfortunately, this latter finding is not robust to ADFH’s more rigorous specification checks reported in Section 4.4.

The authors interpret the evidence they present to be consistent with expense forecasts that are unconditionally pessimistic. I find their logic to be a bit stretched as I’m not convinced that “unconditional” expense forecasts are a common business reality. First, it is notable that the database upon which ADFH rely does not explicitly contain expense forecasts and realizations. Rather, the researchers have calculated their expense figures as the difference between the reported values for revenues and net income. Thus, the very structure of the database suggests that expenses are not a directly, much less “unconditionally,” forecasted number for the VentureOne community of young growth firms and VC users. This is not surprising, as we teach students to develop pro forma income statements by starting with the “top line” for firms that have reached a revenue-producing stage and then to arrive at expense estimates based upon the operating structure that is required to support the projected level of revenues. Even under historical GAAP accounting expenses are “matched to revenues.” For technology firms that are in their earliest (i.e., pre- or low-revenue) stages of development, Joos and Zhdanov (2006) suggest that heavy spending is a positive sign, interpretable as successful research-related efforts that are being reinforced with further advances of capital for further development of the intangible assets. Thus, realizations of expenditures that are below predicted levels may be more likely to signal that the firm has not advanced through its R&D-related benchmarks rather than to provide an indication of spending efficiency, where the latter is, in any case, less likely to be the focus or priority of development stage high tech entities. Accordingly, I interpret lower-than-projected expense realizations to be adverse outcomes, and thus expense forecasts mirror the optimism found in the revenue-related results. And the two sets of interrelated results are entirely consistent with significant prior evidence in other literatures that managers of entrepreneurial firms tend to be overly optimistic about their firms’ projects.

3.2 Question #2: Are the biases strategically dependent on the forecast horizon?

ADFH conclude that biases are increasing over longer term horizons and the authors suggest that this increasing optimism is strategically motivated by the fact that higher perceived long-term net income will both give the firm a greater likelihood of securing its next round of venture financing and enhance the pre-money value of the firm in subsequent funding rounds (thereby minimizing the dilution to current equity holders). The results reported in Table 5 are cited as support for ADFH’s conclusion that forecasts are strategically biased over longer horizons.

One simple explanation for the authors’ findings of “strategically” optimistic revenue forecasts relates simply to the timing of their data; a large proportion of their forecast observations were generated during the “exuberant” market of the late 1990’s and early 2000’s. For example, Table 2C reports that over 63% (72%) of the 4-year-ahead (5-year-ahead) forecasts included in the study’s sample were made during the 1995–2000 period, an era that has come to be synonymous with “irrational exuberance” (Shiller (2000)). Given the precipitous decline in the NASDAQ market index that subsequently began in 2000 and continued through late 2002, it would be an extraordinary result if ADFH did not document a correspondingly high level of “over-optimism” in the later year revenue projections that correspond to this significant tech market correction period.

Another alternative explanation for the observation that bias is increasing over time is that there is greater natural variability over longer term horizons (e.g., Faul, Demers, Burke, and Poulter (1999)). Accordingly, there is greater uncertainty (which manifests in the current context as greater optimism) associated with longer-term forecasts. This phenomenon is not unique to early stage entity’s management forecasts for VC funding; as reported by Bruce and Bradshaw (2004), e.g., analysts’ five year growth forecasts are “notorious for their lack of quality” (p. 18), and were particularly so during the late 1990s in the high tech sector.

A related alternative explanation for ADFH’s findings of increasing bias over time is that the optimistic early years’ errors become compounded over time, resulting in a growing dispersion between forecasts and actuals as the later years unfold. For example, suppose that the firm’s managers project growth rates of 50, 110 and 160% in years 1 through 3, respectively. Further assume that growth realizations are slightly deferred such that a 10% rate of growth is realized in year 1, and thereafter the firm realizes its projected 50% and then 110% growth rate pattern. As shown in the table below, this missing of the first year’s target results in a pattern of increasingly optimistic errors in later years even though the firm is assumed to return to its ambitious projected growth path.

 

Estimated growth

Revenue forecast

Actual growth

Revenue actual

Revenue error

Error % of actual

Year 0

 

 

1.00

  

Year 1

50%

1.50

10%

1.10

−0.4

−36%

Year 2

110%

3.15

50%

1.65

−1.50

−91%

Year 3

160%

8.19

110%

3.47

−4.73

−136%

This pattern will arise if actual revenues miss the mark in an early year as in my simple example, or if the researchers’ estimated actuals underestimate the actual realizations in an early year as may result from ADFH’s research design. For example, ADFH’s Eq. (2) Year 2 estimated actuals rely upon Year 1 estimated actuals, Year 3 estimated actuals rely upon Year 2 estimated actuals, and so forth. Accordingly, any underestimate in the early years gets propagated through the balance of the panels in Table 5 as the authors report that virtually all firms that have forecasts for Year 5 (4) also report forecasts for Year 4 (3), and so forth.

Overall, although ADFH present evidence that their estimated forecast errors are greater over longer-term horizons, I don’t think they’ve satisfied us that this pattern is driven by strategic managerial behavior. There seem to be other simpler and more obvious alternative explanations for the pattern of forecast errors, and these have not been satisfactorily ruled out.

3.3 Question #3: Are the biases strategically related to asset intangibility?

ADFH examine whether a lower level of verifiability of a firm’s assets leads to a strategically greater (i.e., more optimistic) bias in management’s revenue and net income projections. Suspending our concerns regarding the estimation errors in ADFH’s measurement of bias, Table 5 presents solid evidence that there is a discernable rank ordering in the magnitudes of the forecast biases across industries, while Table 6 reports that these differences are statistically significant. Furthermore, the rank ordering of the bias corresponds to the assumed (but unmeasured and unsubstantiated) ranking of the intangibility of each segment’s assets. ADFH then make a great logical leap by concluding that the corresponding ranks of optimism and asset intangibility taken together are indicative of strategic behavior on the part of management. An obvious and simpler alternative explanation is that the rank orderings in ADFH’s estimated bias correspond to the rank orderings of the inherent uncertainty in the economics of these sectors, and this uncertainty in turn manifests in greater errors (i.e., optimistic bias) in forecast estimates. For example, Campbell, Lettau, Malkiel, and Xu (2001) document in their Table IV that the Computer and Retail sectors exhibit the first and second highest firm-specific volatilities, respectively. 3 In other words, the management forecast errors documented by ADFH are exactly what we would expect from the first-order effect of economic uncertainty even absent any strategic behavior. In order to convince the reader of bias strategically induced by asset intangibility, I think the researchers must first attempt to remove this first-order economic effect of uncertainty.

4 Summary and conclusion

Overall, I think ADFH’s study is a great attempt to find some new and potentially very interesting data that enables accounting researchers to examine issues related to an interesting and economically important but understudied segment of the economy. ADFH’s creative introduction of the “historically grounded conditional projection” methodology to the accounting literature may also lead to future meaningful applications in other settings. However, as evidenced by the limitations of the current study, the application of this methodology is not without caveats and its use should perhaps be limited to those settings where a within-sample model with fundamental economic validity and reliably high explanatory power can be developed. I find that there are numerous leaps of logic required to arrive at some of ADFH’s conclusions and there are alternative explanations for ADFH’s findings that have not been entirely refuted. This leaves me with some doubt as to whether all of ADFH’s conclusions are fully substantiated. Nevertheless, the evidence presented makes an interesting contribution to our understanding of the forecasting behavior of young, private, rapidly growing, VC-backed firms, and provides some natural economic and methodological leads into further studies of these issues.

Footnotes
1

Blinder (1973) reports R2s of 45% to 49% (Table A-1) and the models of Oaxaca (1973) generate R2s of 33% to 56% (Table 1).

 
2

In contrast, the previously referenced labor economics papers report separate and detailed regression results for each of their relevant labor groups. For example, the studies of Blinder (1973) and Oaxaca (1973) report the detailed results of each of their separate regressions (e.g., white males, black males, white females, black females, etc) that include the coefficient estimates on 35 to 45 explanatory variables.

 
3

Campbell et al. (2001) do not separately model the biotech industry, so the third of ADFH’s identifiable sectors’ firm-specific volatilities are not explicitly considered here.

 

Copyright information

© Springer Science+Business Media, LLC 2007