Review of Accounting Studies

, Volume 15, Issue 4, pp 779–807

The usefulness of accounting estimates for predicting cash flows and earnings

Authors

    • Stern School of BusinessNew York University
  • Siyi Li
    • Department of AccountancyUniversity of Illinois at Urbana-Champaign
  • Theodore Sougiannis
    • Department of AccountancyUniversity of Illinois at Urbana-Champaign
    • Athens Laboratory of Business Administration (ALBA)
Article

DOI: 10.1007/s11142-009-9107-6

Cite this article as:
Lev, B., Li, S. & Sougiannis, T. Rev Account Stud (2010) 15: 779. doi:10.1007/s11142-009-9107-6

Abstract

Accounting estimates and projections potentially improve the relevance of financial information by providing managers a venue to convey to investors forward-looking, inside information. The quality of financial information is, however, compromised by the increasing difficulty of making reliable estimates and forecasts and the frequent managerial misuse of estimates. Given the ever-increasing prevalence of estimates in accounting data, particularly due to the move to fair value accounting, whether these opposing forces result in an improvement in the quality of financial information is among the most fundamental issues in accounting. We examine the contribution of accounting estimates embedded in accruals to the quality of financial information, as reflected by their usefulness in the prediction of enterprise cash flows and earnings. Our out-of-sample prediction tests indicate that accounting estimates beyond those in working capital items (excluding inventory) do not improve the prediction of cash flows. Estimates do, however, improve the prediction of next year’s earnings, though not of subsequent years’ earnings. We conclude that the usefulness of accounting estimates to investors is limited and provide suggestions for improving the usefulness of estimates.

Keywords

Accounting estimatesIn-sample predictionOut-of-sample predictionEarningsCash flows

JEL Classification

G10 M41

1 Introduction

Financial statement information—be it balance sheet items such as net property, plant, and equipment; goodwill and other intangibles; accounts receivable and inventories; or key income statement figures such as revenues, pension expense, in-process R&D, or the recently expensed employee stock options—is largely based on managerial estimates and projections. The economic condition of the enterprise and the consequences of its operations as portrayed by quarterly and annual financial reports are therefore an intricate and ever changing web of facts and conjectures, where the dividing line between the two is largely unknown to information users. With the current move of accounting standard-setters in the United States and abroad toward increased fair-value measurement of assets and liabilities, the role of estimates and projections in financial reports will increase.

What is the effect of the multitude of managerial estimates embedded in accounting data on the usefulness of financial information? The answer is far from straightforward. On the one hand, estimates and projections are potentially useful to investors because they are the primary means for managers to convey credibly forward-looking proprietary information to investors. Thus, for example, the bad debt provision, if estimated properly, informs investors on expected future cash flows from customers; restructuring charges predict future employee severance payments and plant closing costs; and the capitalized portion of software development costs (SFAS 86) informs investors about development projects that passed successfully technological feasibility tests and are accordingly expected to enhance future revenues and earnings.1 This potential contribution of managerial estimates to investors’ assessment of future enterprise cash flows underlies the oft-quoted statement by the Financial Accounting Standard Board (FASB) in its Conceptual Framework about the superiority of accruals earnings—mostly based on estimates—over the largely fact-based cash flows in predicting future enterprise cash flows: “Information about enterprise earnings based on accruals accounting generally provides a better indication of an enterprise’s present and continuing ability to generate favorable cash flows than information limited to the financial aspects of cash receipts and payments (FASB 1978, p. IX).”

On the other hand, the contribution of estimates to the usefulness of financial information is counteracted by two major factors:
  1. 1.

    Objective difficulties. In the increasingly volatile business environment, due to fast-changing market conditions (deregulation, privatization, emerging economies) and rapid technological changes, it is difficult for managers to make reliable projections of business events. Consider, for example, the estimated future return on pension assets—a key component of the pension expense: this estimate is essentially a prediction of the long-term performance of capital markets. Are managers better predictors of market performance than investors?2 Or reflect on the generally large impairment charges of fixed assets and acquired intangibles (including goodwill) mandated by SFAS 144 and SFAS 142: the determination of these charges requires managers to estimate future cash flows from tangible and intangible assets. In today’s highly competitive and contested markets, the reliability of asset cash flows forecasted over several years is obviously questionable. Accordingly, the accounting estimates and projections underlying financial information introduce a considerable and unknown degree of noise, and perhaps bias, to financial information, detracting from their usefulness.

     
  2. 2.

    Manipulation. Add to the above objective difficulties in generating reliable estimates the expected and frequently documented susceptibility of accounting estimates to managerial manipulation; and the consequent adverse impact of estimates on the usefulness of financial information becomes apparent. Given that it is very difficult to “settle up” with manipulators of estimates—even if an estimate turns out ex post to be far off the mark, it is virtually impossible to prove that ex ante the estimate was intentionally manipulated—there are currently no effective disincentives for managers to manipulate accounting estimates. Indeed, many of the Securities and Exchange Commission (SEC) enforcement cases alleging financial reporting manipulation concern misuse of estimates underlying accruals (for example, Dechow et al. 1996).

     

Thus, the impact of estimates underlying accounting measurement and reporting procedures on the usefulness of financial information is an open question. The relevance of this examination cannot be overstated. Accounting estimates and projections underlie many Generally Accepted Accounting Principles (GAAP) and consume most of standard-setters’ time and efforts. Just consider the major issues addressed by the FASB in recent years—financial instruments, employee stock options, fixed assets and goodwill impairment, and the valuation of acquired intangibles, to name a few—all require major estimates and forecasts in the process of accounting measurement and reporting. If these and other accounting estimates do not contribute significantly to the usefulness of financial information, the efforts of accounting regulators, and even more importantly, the resources society devotes to the generation of estimates in the process of financial statement preparation and their auditing, are misdirected. Worse yet, if financial information users are led by the estimates-based accounting information to misallocate resources, an additional dead-weight cost is imposed on society.

We define and test the usefulness of estimates embedded in accrual earnings in terms of their ability to predict enterprise performance.3 This predictive use of financial information is central to security analysis and valuation and is also a fundamental premise of the FASB’s Conceptual Framework as indicated by the quote above. Future enterprise performance is mainly reflected by cash flows and earnings. Future cash flows are at the core of asset and liability valuation rules. Thus, for example, asset impairment (SFAS 144) is determined by expected cash flows, and the useful lives of acquired intangibles (SFAS 142) are a function of future cash flows. More fundamentally, enterprise cash flows are postulated by economic theory as the major determinants of asset values. Given a certain ambiguity about the specific definition of cash flows used by investors, we perform our tests with two widely used and frequently prescribed cash flow constructs: cash from operations (CFO) and free cash flows (FCF).

Despite the prominence of cash flows in economic asset valuation models, there is no denying that many investors and analysts are using financial data to predict earnings. The underlying heuristics are somewhat obscured; perhaps investors predict earnings first and derive future cash flow estimates from the predicted earnings. In any case, earnings prediction is prevalent in practice, and we therefore also examine the usefulness of accounting estimates for the prediction of earnings, both operating and net income.

The focus of this study is on accounting estimates, but many of the estimates underlying financial information are not disclosed in the financial reports.4 We, therefore, focus in this study on accruals, most of which are based on estimates, and distinguish between accruals that are largely unaffected by estimates (changes in working capital items, excluding inventory) and accruals that are primarily based on estimates (most nonworking capital accruals and inventory). This enables us to draw sharper inferences on the effect of estimates on the usefulness of financial information. We also analyze a smaller sample of firms with data on specific estimates, which we split into recurring and nonrecurring estimates to separate noise (the nonrecurring estimates) from information (the recurring estimates).

Our empirical analysis is based on a sample of all nonfinancial Compustat firms with the required data—ranging from roughly 1,500–3,200 companies per year—spanning the period 1988 through 2005. Our tests are conducted in three stages:
  1. 1.

    In-sample, industry-by-industry, predictions of future enterprise cash flows and earnings, based on (a) current cash flows only (the benchmark), (b) earnings, and (c) the set of cash flows, the change in working capital (excluding inventory), and various components of accruals based on estimates. Here we follow the regression procedures of Barth et al. (2001) and find, on more recent data, results that are generally consistent with theirs. This is our departure point.

     
  2. 2.

    Given that in-sample regressions are not prediction tests, we perform out-of-samplefirm-specific predictions of future cash flows and earnings, using the industry specific parameter estimates of the in-sample regressions. The focus of this analysis is on the improvement in the quality of predictions brought about by the addition of estimates (accruals) to the predictors. Our results show that accounting estimates do not improve the prediction of future cash flows (both operating and free cash flows), compared with predictions based on current cash from operations and the change in working capital excluding inventory. Notably, cash flow predictions based on current earnings only are significantly inferior to those generated by current cash from operations, contrary to Kim and Kross (2005). In our small sample analysis, neither recurring nor nonrecurring estimates improved significantly the predictions of either cash flows or earnings. The bottom line: accounting estimates beyond those in working capital items (except inventory) do not improve the prediction of cash flows. However, accruals do improve next year’s prediction of net and operating income.

     
  3. 3.

    Finally, we examine the economic significance of estimates. These tests complement stage two, which is based on the statistical significance of differences in the quality of alternative predictors. We perform various portfolio tests, where portfolios are constructed from predicted cash flows and earnings based on various predictors, some of which are based on estimates. The abnormal returns on these portfolios, generated by alternative predictors, are our gauge of economic significance. The focus here is on comparing the returns on portfolios constructed from predictions based on current cash flows only (the benchmark), with returns on portfolios constructed from predictions based on current earnings or on current cash flows plus changes in working capital items and estimates. The results from these tests generally corroborate the out-of-sample prediction tests. In practically all our portfolio tests, the model that uses current operating cash flows only to predict firm performance generates higher abnormal returns than models that add estimates to the prediction process used for the portfolio formation. Furthermore, the portfolios constructed from predictions based on current cash flows only yield abnormal returns with generally lower standard deviation than the alternative portfolios that include earnings or estimates among the predictors.

     

We caution against sweeping conclusions. We examine the usefulness of accounting estimates in terms of predictive ability with respect to future firm performance. Accounting information is used for other purposes too (contracting, national accounting), for which estimates may be useful. Furthermore, our prediction tests are based on fairly simple models. Users may be using different, more sophisticated models where estimates could prove to be useful. Nevertheless, we believe that our findings draw attention to the significant vulnerability of financial information from the multitude of underlying estimates and projections and to the urgent need for improving the reliability of estimates, on which we comment in the concluding section.

Concerning related research, a substantial number of studies examine the contribution of accruals to the prediction of future cash flows and other variables. These studies can be roughly classified into regression-based (in-sample) analyses, and out-of-sample prediction tests. An example of the former is Barth et al. (2001), who regress cash from operations on lagged values of cash from operations and components of accruals, reporting (p. 27) that “each accrual component reflects different information relating to future cash flows … [and] is significant with the predicted sign in predicting future cash flows, incremental to current cash flows.”5 Relatedly, Subramanyam and Venkatachalam (2007) conclude that operating cash flows are more strongly associated with future cash flows than earnings and that current earnings are more strongly associated with future earnings than cash flows. By and large, the in-sample regression studies suggest that accruals are associated with subsequent cash flows and contemporaneous equity values. However, as is argued in Sect. 4.1, in-sample regressions are not prediction tests and may even provide misleading inferences concerning prediction power.

An early out-of-sample prediction test is Finger (1994), who concludes from a sample of 50 companies with much historical data that cash flow is marginally superior to earnings for short-term predictions and performs similar to earnings in long-term cash flow predictions. However, time-series and cross-sectional out-of-sample short-term prediction tests by Lorek and Willinger (1996) and Kim and Kross (2005) show that current earnings predict more accurately future cash flows than current cash flows do. Barth et al. (2005, p. 5) “… find evidence of some reduction in mean prediction errors from disaggregating earnings into cash flows and total accruals, and some additional reduction from disaggregating total accruals into its four major components … median prediction errors generally support disaggregation of earnings only into cash flows and total accruals.” Overall, these findings vary considerably by industry and appear to indicate a more consistent success for the cash flows and total accruals model than for the cash flows and disaggregated accruals model.6 Thus, a mixed picture emerges from the out-of-sample tests.

The lack of convergence of the accruals’ usefulness research makes it difficult to draw firm conclusions. Some studies are in-sample, while others are out-of-sample. Some researchers relate accruals to contemporaneous returns or equity values, while others relate them to future values. Some predict cash flows, while others predict equity values based on models using forecasted or realized residual earnings. Our main contribution is the focus on the estimates embedded in accruals and the provision of certain closure to the usefulness of accruals issue. We distinguish between accruals that are largely based on facts and those primarily reflecting estimates, to focus on the usefulness of accounting estimates. Our main tests are out-of-sample predictions, replicating what most investors actually do—predict, with no ex post information (as implicitly assumed by in-sample studies), various versions of future earnings and cash flows. The comprehensiveness of our predicted performance measures (two versions of earnings and two of cash flows) and the number of future periods examined (years t+1, t+2, and aggregate next 2 years and next 3 years) enables us to draw general conclusions about the contribution of estimates to firm performance prediction. Furthermore, our study is the first to examine both the statistical and economic performance of accruals-based prediction models.7 The focus on accounting estimates, the out-of-sample methodology, and the examination of both statistical and economic significance—all bringing certain closure to the research question—are our main contributions.

The order of discussion is as follows: Sect. 2 outlines our research design, while Sect. 3 describes our sample. Section 4 reports our prediction tests’ results, and Sect. 5 explains our robustness checks. Section 6 focuses on a subsample with an extended set of accounting estimates. Section 7 reports our portfolio (economic significance) tests, while Sect. 8 concludes.

2 Research design

Our research design consists of three stages: (a) in-sample association tests of cash flows (earnings) regressed on lagged values of these variables and accruals, (b) out-of-sample forecasts of cash flows (earnings) based on these variables and accruals, and (c) calculation of hedge future excess returns on portfolios constructed from the out-of-sample predicted cash flows (earnings) in stage (b). For the out-of-sample tests, we use several prediction constructs, primarily to distinguish between accruals largely based on facts and those based on estimates. At one extreme of the accruals disaggregation, we classify all the accruals in the “operations” section of the cash flow statement into working capital changes excluding inventory (ΔWC*) and the remaining accruals, termed “estimates” (EST): https://static-content.springer.com/image/art%3A10.1007%2Fs11142-009-9107-6/MediaObjects/11142_2009_9107_Figa_HTML.gif Working capital items with the exception of inventory, such as accounts payable and short-term marketable securities, are generally not materially impacted by managerial estimates,8 while most of the remaining accruals are in fact pure estimates (for example, depreciation and amortization, bad debt provision, in-process R&D).

At the other end of the accruals disaggregation, we separate out the change in inventory (ΔINV) from the aggregate estimates (EST), given the evidence (for example, Thomas and Zhang 2002) that much of the accruals anomaly resides in inventory, probably due to intentional and unintentional misestimations of this item. We further break out depreciation and amortization and deferred taxes from other estimates because the identification of these items is possible from Compustat data over the entire sample period. This disaggregation is depicted thus: https://static-content.springer.com/image/art%3A10.1007%2Fs11142-009-9107-6/MediaObjects/11142_2009_9107_Figb_HTML.gif

The various components of accruals along with cash from operations,9 depicted in the two exhibits above are the independent variables in the estimation models underlying our in-sample predictions. We add to these variables the cash flow statement figure of capital expenditures (CAPEX), since the dependent variables in our models are future cash flows or earnings, which are generally affected by current investment (capital expenditures). We believe that the addition of capital expenditures to the regressors improves the specification of the in-sample prediction models and sharpens our focus on the relative performance of the accruals components, our focus of study. Indeed, the capital expenditures variable is statistically significant in most of our annual in-sample predictions models.10

2.1 Prediction tests

Our prediction tests take the following general form. We predict two versions of cash flows (cash from operations and free cash flows) and two constructs of earnings (net income before extraordinary items and operating income) in years t+1 and t+2, as well as in aggregate years t+1 and t+2, and t+1 through t+3. To gain insight into the usefulness of estimates in predicting firm performance, we use five prediction models with increasing disaggregation of accruals (regressors):
  • Model 1: current CFO only—the benchmark model;

  • Model 2: current net income (NI) only;

  • Model 3: current CFO and the change in working capital items excluding inventory (∆WC*)—namely, largely fact-based regressors;

  • Model 4: current CFO, the change in working capital items excluding inventory ∆WC*, and total remaining accruals, largely based on estimates (EST); and

  • Model 5: current CFO, the change in working capital items excluding inventory (∆WC*), the change in inventories (ΔINV), depreciation and amortization (D&A), the change in deferred taxes (DT), and all other estimates (EST*)—the most disaggregated model.

The purpose is to examine whether the gradual addition of components of accruals estimates to current cash flows (the benchmark) improves the prediction of future cash flows or earnings. Increasing the disaggregation of accruals should, in general, enhance the quality of prediction (from model 1 to 5), since the individual accrual components are allowed to have different effects (multiples) on the predicted values. We examine model 2 because the predictor, earnings, is a summary accounting variable that has been extensively investigated for its information content and has been used in most prior studies (for example, Barth, Cram, and Nelson 2001 and Kim and Kross 2005).

We use industry-specific estimated coefficients from each of the above five prediction models to calculate firm-specific predicted values for cash from operations, free cash flows, net income and operating income. We then calculate firm-specific prediction errors as the difference between the actual and predicted values of each variable examined. The following examples of the prediction of next year’s free cash flows (FCF) will clarify our prediction procedures.
  1. (a)
    Benchmark Model using CFO only (example for 1990):
    1. 1.

      Estimate cross-sectionally for each 2-digit industry the following regression: FCF(89) = α + βCFO(88) + ε.

       
    2. 2.

      Predict for each firm in a given 2-digit industry: EFCF(90) = α + βCFO(89), using the previously determined industry specific estimated coefficients.

       
    3. 3.

      Determine prediction error for each firm in a given 2-digit industry: FCF(90) − EFCF(90).

       
     
Here we predict 1990 free cash flows [EFCF(90)] from current cash from operations, [CFO (89)] (and capital expenditures). First, for each 2-digit industry, we regress cross-sectionally free cash flows of 1989 on CFO in 1988 and obtain the estimated coefficients α and β. Those coefficients are then used to predict firm-specific free cash flows (EFCF) in 1990, using the firm’s actual CFO of 1989. Then, a firm-specific prediction error is determined by comparing the firm’s actual 1990 FCF with the predicted one. The same procedure is repeated for every firm and sample year.
  1. (b)
    Expanded estimates, Model 5 (example for 1990):
    • Estimate cross-sectionally for each 2-digit industry:
      $$ {\text{FCF(}}89 )= \alpha + \beta_{1} {\text{CFO(}}88 )+ \beta_{2} \Updelta {\text{WC}}^{*} (88) + \beta_{3} \Updelta {\text{INV(}}88 )+ \beta_{4} {\text{D}}\& {\text{A(88)}} + \beta_{5} {\text{DT(}}88 )+ \beta_{6} {\text{EST}}^{*} (88) + \varepsilon $$
     

The prediction and error determinations are done as in (a) above.

Here we predict the 1990 free cash flows from 1989 cash from operations, capital expenditures, and the disaggregated set of estimates (see second diagram at the beginning of this section). Once more, we run by industry a cross-sectional regression of 1989 free cash flows on the 1988 values of the independent variables, estimating the α and β1 … β6 coefficients (and a β7 coefficient for 1988 capital expenditures). The firm-specific 1990 free cash flows are predicted using these industry-specific coefficients and the actual values of the independent variables in 1989. Computation of the 1990 free cash flows prediction error follows.

The prediction of free cash flows in t+2 follows the earlier procedure with one difference: the cross-sectional estimate (first equation) and the forecast (second equation) now involve a two-year lag (for example, free cash flows in 1990 regressed on cash from operations of 1988). The same procedure is performed for each firm and sample year. The expanded prediction models incorporating disaggregated accruals follow step (b) above. We also predict free cash flows for aggregate years t+1 plus t+2 and t+1 through t+3. These predictions are based on the procedures described above, except that aggregated future free cash flows are substituted for single year free cash flows as left-hand variables in the various models.

To evaluate the quality of the out-of-sample predictions, we compute summary measures of prediction errors derived from the firm- and year-specific estimated errors: the mean and median signed prediction errors indicating the bias in the forecasts, and the mean and median absolute prediction errors which abstract from the sign of the error, indicate forecast accuracy. The firm-specific prediction error in a given year is computed as the realized value of cash flow or earnings minus the predicted cash flow or earnings, divided by average total assets in year t.

2.2 Portfolio analysis

The third stage of our research design is motivated by Poon and Granger (2003, p. 491) who note: “Instead of striving to make some statistical inference, [prediction] model performance could be judged on some measures of economic significance.” We interpret their statement as saying that we should not rely solely on the statistical significance of our prediction errors calculated in stage two but should also examine and perhaps even rely more on measures of economic significance. To gauge the economic significance of the contribution of estimates to the usefulness of financial information, we perform a series of portfolio tests focusing on the incremental stock returns generated by the estimates-based prediction models.

Essentially, we use the out-of-sample predicted values of cash flows (cash from operations and free cash flows) and alternatively of earnings (net income and operating income), obtained in the second stage of our analysis, to form portfolios. Specifically, for each sample year, we rank all firms (across all industries) on predicted firm-specific cash flows or earnings (four rankings: two for cash flows and two for earnings). We then form 10 portfolios from each annual ranking and compute risk-adjusted (size and book-to-market adjusted) returns from holding these portfolios over several future periods. In assessing the performance of the various predictors (CFO, NI, ∆WC*, accruals of estimates), we primarily focus on a zero-investment (hedge) strategy: going long (investing) in the top portfolio—the 10% of firms with the largest (scaled) predicted cash flows or earnings—and shorting (selling) the bottom portfolio—10% of firms with the lowest predicted cash flows or earnings. The abnormal returns on these zero-investment portfolios indicate the economic contribution to investors of using accounting estimates as predictors. Thus, if estimates are useful to investors then portfolios constructed from predictions based on current cash flows and estimates-based accruals should consistently outperform portfolios formed from predictions based on current cash flows only.

It should be noted that if markets are efficient concerning the information in accruals—a big if, in light of Sloan (1996)—and if investors select securities using procedures similar to our industry-based prediction models specified above, then our subsequent portfolio abnormal returns should be roughly zero. Our purpose in these portfolio tests, however, is not to examine market efficiency, but rather to compare the performance of portfolio selection procedures with the estimates-based accruals against similar procedures without accruals (based on past cash flows only). We are thus focusing on the with- and without-accruals comparisons, being indifferent about market efficiency. Stated differently, the comparative abnormal hedge returns across the five prediction models, rather than the statistical significance of those returns, is our focus of analysis.

3 Sample selection and descriptive statistics

We obtain accounting data from the 2006 Compustat annual industrial, full coverage, and research files, and use data from the statement of cash flows because Collins and Hribar (2002) suggest that such data are preferable to accruals derived from the balance sheet. Since reporting a statement of cash flows was mandated by SFAS 95 in 1987, our accounting data span the period 1988–2005.11 In the in-sample regression analysis, each year from 1988 to 2004 is a predictor year (generating the independent variables), while each year from 1989 to 2005 is a predicted year (providing the dependent variables). Thus, 17 in-sample annual regressions are estimated for each industry.

Our sample selection procedure is as follows. We start with 75,571 observations with values for NI, CFO, ∆WC*, ∆INV, D&A, DT, EST, EST* and CAPEX for the current year, year t, and for NI over a three-year horizon, t−1 to t+1. Firms with all fiscal year ends are included. We control for outliers by following the procedures in Barth et al. (2001). Thus, after eliminating the top and bottom one percentile of current NI and CFO, we are left with 73,324 firm-year observations. By excluding observations with market value of equity or sales of less than $10 million, or with share prices below $1, to eliminate economically marginal firms, the number of observations decreases to 51,301. By deleting observations with studentized residuals greater than 3 or less than −3, we are left with 50,288 observations. Since we conduct industry-by-industry in-sample regression analysis we require each industry to have a minimum of 600 observations over the period 1988–2004. This criterion reduces the sample to its final size of 41,124 observations. We obtain stock returns data for the portfolio analysis from the 2006 CRSP files.12

Table 1 provides summary statistics (variables are scaled by average total assets) and a correlation matrix for out test variables. Panel A shows that depreciation and amortization (D&A) constitutes the bulk of the estimates underlying accruals (EST): the mean (median) of D&A is 0.054 (0.047), close to the mean (median) of EST, 0.059 (0.052). The mean of net estimates (EST*), excluding D&A and deferred taxes, is quite large, 0.019, and is driven mainly by large positive values, as the median value of 0.004, Q1 of 0.000 and Q3 of 0.019 imply. CFO has the lowest, while NI has the highest variability (standard deviations of 0.129 versus 0.149) among the various earnings and cash flow variables. In panel B all correlations are significant at the 5% level or better. We note the high negative correlations of our estimates variables, EST and EST*, with the income variables, NI and OI. However, the correlations of EST and EST* with both the cash flow variables, CFO and FCF, are much lower; positive for EST and negative for EST*.
Table 1

Descriptive statistics

Panel A: Distributional statistics

Variable

Mean

SD

Maximum

Q3

Median

Q1

Minimum

No. of observations

   

NI

0.017

0.149

0.323

0.088

0.043

−0.001

−1.626

41,124

   

CFO

0.066

0.129

0.390

0.136

0.079

0.019

−1.117

41,124

   

FCF

−0.004

0.140

0.387

0.072

0.018

−0.050

−2.095

41,124

   

OI

0.070

0.146

0.735

0.143

0.086

0.029

−1.382

41,122

   

∆WC*

−0.011

0.072

1.077

0.018

−0.006

−0.036

−1.280

41,124

   

∆INV

−0.014

0.053

0.526

0.002

−0.001

−0.022

−0.773

41,124

   

D&A

0.054

0.038

1.020

0.066

0.047

0.032

0.000

41,124

   

CAPEX

0.070

0.076

2.453

0.085

0.049

0.027

−0.005

41,124

   

DT

0.000

0.020

0.378

0.004

0.000

−0.003

−0.538

41,124

   

EST

0.059

0.101

1.629

0.090

0.052

0.020

−0.820

41,124

   

EST*

0.019

0.071

1.562

0.019

0.004

0.000

−0.906

41,124

   

Panel B: Pearson (Spearman) correlations above (below) the diagonal

Variable

NI

CFO

FCF

OI

ΔWC*

ΔINV

D&A

CAPEX

DT

EST

EST*

NI

 

0.640

0.564

0.883

−0.156

−0.142

−0.254

0.048

0.016

−0.547

−0.537

CFO

0.568

 

0.841

0.689

0.318

0.251

0.099

0.149

0.034

0.107

−0.096

FCF

0.467

0.807

 

0.606

0.272

0.268

−0.079

−0.410

0.006

0.049

−0.087

OI

0.889

0.585

0.480

 

−0.159

−0.154

−0.205

0.054

0.040

−0.308

−0.224

∆WC*

−0.113

0.341

0.308

−0.112

 

−0.061

0.023

0.040

−0.068

−0.077

−0.057

∆INV

−0.229

0.171

0.233

−0.222

−0.010

 

0.102

−0.066

0.026

0.575

0.014

D&A

−0.128

0.225

−0.021

−0.083

0.041

0.112

 

0.313

−0.034

0.486

0.089

CAPEX

0.180

0.246

−0.245

0.198

0.021

−0.150

0.430

 

0.046

0.091

−0.002

DT

0.081

0.076

0.020

0.086

−0.085

0.016

0.001

0.063

 

0.069

−0.182

EST

−0.351

0.232

0.124

−0.219

−0.049

0.590

0.543

0.119

0.093

 

0.712

EST*

−0.269

−0.035

−0.016

−0.123

−0.044

0.026

0.054

−0.039

−0.187

0.452

 

Panel A presents the distributional statistics of the primary pooled sample for the prediction of t+1 net income, which includes 41,124 firm-year observations that span 1988 and 2004. Our sample sizes vary slightly for different target variables and to certain degree for different forecasting horizons. Panel B presents the Pearson (Spearman) correlations among the key variables above (below) the diagonal. All correlations are significant at the 0.05 level

All variables are deflated by average total assets and are defined as follows (Compustat data item number in parentheses): NI net income, defined as income before extraordinary items (#18), CFO cash flow from operations (#308) less the accrual portion of extraordinary items and discontinued operations reported on the statement of cash flows (#124), FCF free cash flows, defined as CFO – CAPEX, OI Operating income after depreciation (#178), ∆WC* change in working capital excluding inventory per the statement of cash flows, namely the sum of the following items: change in accounts receivable (#302), change in accounts payable and accrued liabilities (#304), change in accrued income taxes (#305), change in other assets and liabilities (#307), ∆INV change (decrease) in inventory per the statement of cash flows (#303), D&A depreciation and amortizations per the statement of cash flows (#125), DT deferred taxes per the statement of cash flows (#126), EST estimates, calculated as CFO − NI − ∆WC*, EST* net estimates, defined as CFO − NI − ∆WC* − ∆INV − D&A – DT, CAPEX capital expenditures per the statement of cash flows (#128)

4 Empirical findings: prediction tests

4.1 Stage one: in-sample regressions

As a departure from available research, we essentially replicate the in-sample regressions of Barth et al. (2001) on recent data, regressing cash from operations on lagged values of cash from operations and earnings components (Model 5 in Sect. 2.1), and find that in each of the 23 industries examined the lagged cash from operations and change in working capital minus inventory are highly significant. In the majority of the industries, inventory change is also significant, as is depreciation and amortization. However, deferred taxes and other accruals estimates are significant in about half of the industries only. These results (available on request) are consistent with Barth et al. (2001).

However, it is important to note that a regression analysis of a given variable on lagged values of that variable along with other data, as frequently conducted in accounting and finance research, is not a conclusive test of predictive ability. As noted in Poon and Granger’s (2003, p. 492) survey:

In all forecast evaluations, it is important to distinguish in-sample and out-of-sample forecasts. In-sample forecast, which is based on parameters estimated using all data in the sample, implicitly assumes parameter estimates are stable through time. In practice, time variation of parameter estimates is a critical issue in forecasting. A good forecasting model should be one that can withstand the robustness of an out-of-sample test, a test design that is closer to reality. In our analyses of empirical findings … we focus our attention on studies that implement out-of-sample forecasts.

This important insight motivates our primary analysis, which focuses on out-of-sample prediction tests.

4.2 Stage two: out-of-sample prediction tests

Table 2 summarizes our main out-of-sample prediction findings. Recall that we predict four key performance indicators: cash from operations (CFO); free cash flows (FCF), defined as CFO minus capital expenditures (CAPEX); net income before extraordinary items (NI); and operating income (OI). There are four prediction horizons: next year, second year ahead, aggregate next 2 years, and aggregate next 3 years. Five prediction models are examined (as discussed and demonstrated in Sect. 2.1), where the predictive (independent) variables are (1) CFO only—the benchmark model, (2) NI only, (3) CFO and the annual change in working capital items excluding inventory (ΔWC*), (4) CFO plus the change in working capital items excluding inventory (ΔWC*), as well as the total remaining accruals (EST), which are largely estimates based, including the change in inventory, and (5) our most disaggregated model: CFO, ΔWC*, the change in inventories, depreciation and amortization, deferred taxes, and all remaining estimates. Current capital expenditure is included as an additional variable in each of the five models.
Table 2

Out-of-sample predictions of cash flows and earnings by current cash flows, earnings, and combinations of accruals

Prediction model

Cash from operations (CFO)

Free cash flows (FCF)

Net income (NI)

Operating income (OI)

Variables

MAER

MER

R2

Theil’s U

MAER

MER

R2

Theil’s U

MAER

MER

R2

Theil’s U

MAER

MER

R2

Theil’s U

Year 1

    Model 1

0.056&

0.001

0.46

0.58

0.062&

0.003

0.40

0.77

0.060&

0.004

0.38

0.75

0.061&

0.002

0.45

0.59

    Model 2

0.062

0.003

0.37

0.64

0.066

0.004

0.33

0.81

0.055

0.005

0.47

0.70

0.057

0.002

0.52

0.56

    Model 3

0.054*

0.001

0.50

0.56

0.061*

0.003

0.43

0.75

0.058*

0.004

0.42

0.72

0.058*

0.001

0.51

0.56

    Model 4

0.054#

0.002

0.50

0.57

0.061#

0.004

0.42

0.76

0.053#

0.004

0.51

0.67

0.054#

0.002

0.58

0.53

    Model 5

0.055#

0.002

0.49

0.57

0.062#

0.003

0.41

0.77

0.054#

0.004

0.50

0.69

0.054#

0.001

0.58

0.53

Year 2

    Model 1

0.062&

0.001

0.33

0.65

0.067&

0.004

0.24

0.87

0.069&

0.004

0.25

0.85

0.070

0.001

0.30

0.70

    Model 2

0.065

0.004

0.26

0.68

0.069

0.007

0.21

0.89

0.070

0.009

0.26

0.87

0.070

0.005

0.32

0.70

    Model 3

0.061*

0.001

0.34

0.64

0.067*

0.005

0.25

0.86

0.068*

0.005

0.26

0.85

0.069*

0.002

0.33

0.69

    Model 4

0.061#

0.002

0.34

0.65

0.067#

0.006

0.25

0.87

0.068#

0.007

0.29

0.84

0.067#

0.003

0.37

0.68

    Model 5

0.062#

0.002

0.34

0.65

0.068#

0.006

0.24

0.89

0.069#

0.007

0.28

0.86

0.068#

0.003

0.36

0.69

Years 1–2

    Model 1

0.109&

0.000

0.46

0.56

0.117&

0.008

0.38

0.78

0.128&

0.003

0.35

0.77

0.137&

−0.004

0.40

0.64

    Model 2

0.119

0.008

0.36

0.62

0.125

0.015

0.32

0.83

0.124

0.012

0.41

0.76

0.133

0.005

0.45

0.62

    Model 3

0.105*

0.002

0.50

0.55

0.116*

0.009

0.41

0.77

0.126*

0.005

0.39

0.76

0.132*

−0.002

0.45

0.61

    Model 4

0.105

0.003

0.50

0.55

0.116#

0.010

0.41

0.77

0.120#

0.009

0.45

0.73

0.126#

0.002

0.51

0.59

    Model 5

0.106

0.003

0.50

0.55

0.118#

0.010

0.39

0.78

0.121#

0.007

0.44

0.73

0.126#

0.002

0.50

0.59

Years 1–3

    Model 1

0.173&

−0.002

0.44

0.58

0.212&

0.021

0.28

0.85

0.232&

0.000

0.25

0.85

0.253&

−0.021

0.30

0.72

    Model 2

0.189

0.013

0.34

0.64

0.222

0.034

0.23

0.90

0.236

0.019

0.27

0.87

0.257

0.000

0.31

0.74

    Model 3

0.169*

0.002

0.46

0.57

0.212

0.025

0.30

0.85

0.231*

0.004

0.27

0.85

0.249*

−0.014

0.33

0.71

    Model 4

0.170

0.006

0.47

0.57

0.214#

0.030

0.29

0.86

0.229#

0.015

0.31

0.84

0.247#

−0.003

0.36

0.71

    Model 5

0.170

0.006

0.46

0.58

0.218#

0.029

0.28

0.88

0.235#

0.015

0.30

0.86

0.250

−0.003

0.35

0.72

Model 1: current cash flow from operations only (CFO), Model 2: current net income before extraordinary items only (NI), Model 3: current cash flow from operations and change in working capital excluding inventory (CFO, ∆WC*), Model 4: current cash flow from operations, change in working capital excluding inventory, and estimates (CFO, ∆WC*, EST), EST = CFO – NI − ∆WC*, Model 5: current cash flow from operations, change in working capital excluding inventory, and disaggregated components of estimates (CFO, ∆WC* and components of estimates). These components of estimates include the change in inventory (∆INV), depreciation & amortization (D&A), deferred taxes (DT), and other estimates (EST* = CFO – NI − ∆WC* − ∆INV − D&A − DT)

Capital expenditure is included in all models as a control variable. Models 4 and 5 both include estimates but at different levels of aggregation. Model 4 includes all estimates as a lump sum, while Model 5 includes components of estimates. All statistics are computed after trimming the top two percentiles of absolute forecast errors

MAER mean absolute forecast error from the pooled sample, MER mean signed forecast error from the pooled sample; R2 the average of the adjusted R2 from yearly regressions of actual values on predicted values over the sample period (1988–2004), Theil’sU-statistic the average of the yearly U-statistics. U is defined as the square root of ∑(actual − forecast)2/∑(actual)2

& Statistical significance of the mean absolute error (at the 0.05 level or better) of model 1 (CFO) compared with model 2 (NI)

# Statistical significance of the mean absolute error (at the 0.05 level or better) of model 3 (CFO and ∆WC*) compared with model 4 (CFO, ∆WC* and EST) and model 5 (CFO, ∆WC*, ∆INV, D&A, DT and EST*)

* Statistical significance of the mean absolute error (at the 0.05 level or better) of model 1 (CFO) compared with model 3 (CFO and ∆WC*)

We report in Table 2 four summary statistics for the prediction errors of our five models: the pooled firm-specific mean absolute error (MAER) of each of the five models; the pooled mean signed error, or bias (MER); the mean R2s from annual regressions of firm-specific actual values of future cash flows or earnings on the corresponding predicted values; and the average over the years of Theil’s U-statistics.13 We indicate with an ampersand (&), asterisk (*) or a hash (#) the pooled mean absolute prediction errors (MAER), which are significantly different between Models 1 and 2, Models 1 and 3, and Models 3 and 4, and Models 3 and 5, respectively.14 We have also computed the sample median signed errors, median absolute errors, and root mean square errors. Results from these errors indicators are very similar to those reported in Table 2. (We comment in the text on the occasional differences.) Below are the main inferences we draw from Table 2, and additional analyses:
  1. 1.

    Prediction of cash flows. Considering the prediction of cash from operations and free cash flows—left two quadruples of columns in Table 2—we note that the predictions derived from net income only (Model 2) are always significantly inferior to the predictions based on cash from operations only (Model 1). This is true across the four forecast horizons and the four error summary statistics. For example, in predicting one-year-ahead cash from operations (top left panel), the MAER, MER, and Theil’s U are lower for Model 1 than for Model 2 (0.056 vs. 0.062, 0.001 vs. 0.003, and 0.58 vs. 0.64, respectively), while the R2 of Model 1 is higher than that of Model 2 (0.46 vs. 0.37). The difference in the MAERs is statistically significant, as indicted by the ‘&’ sign. This pattern is evident across all eight panels reporting predictions of cash from operations and free cash flows for various horizons. Thus, for one- to three-year forecast horizons, current cash from operations is a better predictor of future cash from operations and free cash flows than current net income. This result is inconsistent with Kim and Kross (2005) findings that in one-year-ahead predictions of cash flows current earnings performs better than current cash flows.15

    Moving on to Model 3, (predictors: cash from operations and the change in working capital items minus inventory), we note that the CFO and FCF predictions derived from current CFO only (Model 1) under-perform predictions based on current CFO and the change in working capital items excluding inventory, ΔWC*. Thus, the mean absolute errors of Model 3 are significantly lower than those of Model 1 in all CFO and FCF panels, except in the FCF panel for the aggregate next 3 years horizon (bottom FCF panel).16 The reported R2s and Theil’s U statistics also indicate the under-performance of Model 1 relative to Model 3. For example, in predicting 1-year-ahead cash from operations (top left panel), the MAER and Theil’s U are lower for Model 3 than for Model 1 (0.054 vs. 0.056 and 0.56 vs. 0.58, respectively), while the R2 of Model 3 is higher than that of Model 1 (0.50 vs. 0.46). Thus, for 1–3-year forecast horizons, the total change in working capital items excluding inventory is incrementally informative over current cash flows. This is relevant for our focus on the usefulness of accounting estimates, because the working capital items, excluding inventory, and with the exception of accounts receivable, are largely free of estimates.

    We now move to examine the contribution of accounting estimates to cash flow prediction. We do this by comparing the performance of Models 4 and 5 with that of Model 3, where Model 3 becomes our benchmark given its superior performance up to this point. We note that CFO and FCF predictions derived from Model 4 (based on CFO, the change in working capital items excluding inventory (ΔWC*), as well as all other accruals including the change in inventory) and Model 5 (based on CFO, ΔWC*, the change in inventories, depreciation and amortization, deferred taxes, and all remaining accruals) equally perform or under-perform the predictions from Model 3 (based on CFO and ΔWC*). Specifically, the mean absolute errors of Model 3 are significantly lower than or equal to the mean absolute errors of Models 4 and 5 in all the CFO and FCF panels. Furthermore, the reported MERs, R2s, and Theil’s U statistics are also consistent with the under-performance of Models 4 and 5 relative to Model 3. For example, in predicting one-year-ahead cash from operations (top left panel), the MAER, MER, and Theil’s U for Model 3 are either equal to or lower than for Models 4 and 5 (0.054 vs. 0.054 and 0.055; 0.001 vs. 0.002 and 0.002; and 0.56 vs. 0.57 and 0.57, respectively), while the R2 of Model 3 is equal to or higher than the R2s of Models 4 and 5 (0.50 vs. 0.50 and 0.49). Accordingly, we conclude that for 1–3-year forecast horizons the accounting estimates embedded in accruals, either as a lump sum or disaggregated, do not improve cash flow predictions over current cash from operations and the change in working capital (excluding inventory).17

    Conclusions: Neither total earnings nor disaggregated estimates-based accruals systematically improve the prediction of cash flows (CFO or FCF) over the predictions based on current CFO and the change in working capital items (excluding inventory). This finding is inconsistent with the FASB’s conceptual stipulation that “Information about enterprise earnings … generally provides a better indication of an enterprise’s present and continuing ability to generate favorable cash flows than information limited to the financial aspects of cash receipts and payments” (FASB 1978, p. IX), though our data start 10 years after this statement was issued.

     
  1. 2.

    Prediction of earnings. The two quadruples of columns to the right of Table 2 report prediction performance statistics for net income (NI) and operating income (OI). Here, the predictions derived from net income (Model 2) significantly outperform those based on cash from operations only (Model 1), for the one-year-ahead forecasts. For example, in predicting next year’s operating income (top right panel), the MAER of Model 2 is significantly lower than that of Model 1 (0.057 vs. 0.061). The R2s and Theil’s Us confirm the stronger performance of Model 2, for one-year predictions. Interestingly, Model 2’s predictions are significantly inferior to Model 1’s in the 2-years-ahead and aggregate next 3 years predictions (second and bottom NI and OI panels). For example, in predicting aggregate 3-years-ahead operating income (bottom right panel), the MAER of Model 2 is significantly higher than that of Model 1 (0.257 vs. 0.253). Thus, for a one-year-ahead forecast horizon, current net income is a better predictor of future net income and operating income than current cash from operations.18

    Of the five models examined for earnings predictions, the best performer is Model 4—with three variables: CFO, ΔWC* (change in working capital excluding inventory), and EST (all other accruals)—for all forecast horizons. Intriguingly, Model 5, where EST is disaggregated to several estimates-based accruals, is somewhat inferior to Model 4. Apparently, predicting from disaggregated accruals results in noisy forecasts.

    Conclusions: Earnings is a better predictor of next year’s earnings than cash flow. Accounting accruals, when disaggregated to working capital items and other accruals, improve the prediction of operating and net income. No further improvement is achieved from a finer disaggregation of accruals.

     

5 Robustness checks

  1. 1.

    How good are our prediction models? Our models are admittedly simple—they obviously abstract from many of the complexities of real security analysis. Nevertheless, the R2s in Table 2—derived from annual regressions of actual values (future cash flows or earnings) on predicted values—are quite large. Thus, for example, for next year’s predictions (top panels of Table 2), the R2 range is 0.33–0.58. As expected, the R2s drop for second year predictions, yet they are still in the reasonable range of 0.21–0.37. Thus, despite their simplicity, our prediction models perform reasonably well.

     
  2. 2.

    Trimming extreme prediction errors. The results of Table 2 are after trimming the top 2% of the absolute forecast errors. We also computed prediction errors after trimming the top and bottom 1% of the forecast errors and without any trimming. The resulting patterns of prediction errors (not reported) are in both cases very similar to those of Table 2. As expected, Table 2 trimmed errors are substantially smaller than the nontrimmed errors, the R2s are larger, and the Theil’s U statistics are lower, yet our conclusions regarding the relative performance of the five models equally apply to the nontrimmed errors. Thus, outliers do not affect substantially our inferences.

     
  3. 3.

    Classification by size of accruals. Since the estimates we examine are components of total accruals, we classified the sample firms into three groups, by the size of accruals, to check whether accruals size affects our findings. Specifically, for each sample year we ranked the firms by the size of total accruals (scaled by total assets) and then formed three groups: the top 25% of firms (high accruals), the middle 50% (medium accruals), and the bottom 25% (low accruals). We then generated cash flow and earnings predictions for each of the three accruals groups in the same manner used for the total sample. The findings for all three accruals groups are essentially the same as those for the total sample: accounting estimates do not improve the prediction of cash flows (either CFO or FCF)19 and do improve the prediction of next year’s earnings. We note a pattern in the accruals classifications: for firms in the medium accruals category (middle 50% of the accruals ranking), the average MAERs are substantially lower and the R2s higher than the corresponding statistics of firms with large or small accruals. Thus, for example, in predicting next year’s net income, the MAER range of the firms in the top accruals quartile is 0.078–0.083 (for the five prediction models), while the corresponding MAER range for the medium accruals firms is 0.043–0.45 only. Thus, accruals, both high and low, adversely affect the performance of all our prediction models.

     
  4. 4.

    Industry effects. To examine whether the contribution of estimates to the prediction of cash flows and earnings varies across industries, we analyze our out-of-sample predictions for each of the 23 industries in the sample. This analysis identified several industries where accounting estimates did not improve even the prediction of next year’s earnings, relative to predictions based on CFO only: oil and gas (SIC#13), printing and publishing (27), fabricated metals (34), eating and drinking places (58), and health services (80). It is difficult, however, to find a common denominator to these industries. In addition, we note the expected difference in the predictive performance of all the five models due to stability of demand conditions: for industries with stable demand, such as electric and gas utilities, the MAERs were very low (range: 1.9–2.1%), while for volatile industries, such as software, the MAERs of all the models were relatively high (range: 9.9–10.8%). In general, stability of customer demand over time increases the accuracy of predicted firm performance, relative to unstable industries. Thus, our general finding that estimates do not improve the prediction of cash flows but do improve short-term earnings prediction, generally holds across industries.

     
  5. 5.

    Temporal changes. To examine for temporal changes in the contribution of estimates to the prediction of cash flows and earnings, we split the sample period—1988 through 1994 and 1995 through 2004—and compare the models’ performance across the two subperiods. The main finding standing out is the deterioration in predictive performance of all the five models in the recent period (1995 through 2004) relative to the early one (1988 through 1994). This significant decrease in the prediction performance of our models is probably caused by the general increase in business volatility (for example, Campbell et al. 2001), as well as by the increased manipulation of earnings via estimates. The latter cause (manipulation) is supported by the fact that the deterioration in the predictive performance of our models is substantially smaller for cash flows than for earnings (for example, for aggregate 3 years prediction of CFO, the MAERs of the five models in the early period are roughly 17%, increasing to only 20% in the latter period, while for earnings the increase is from 20 to 30%). As for the contribution of estimates to the prediction of earnings—it is smaller in the recent period than in the early one, likely reflecting once more the management of earnings via estimates which has increased substantially in the 1990s (for example, Lev and Nissim 2006).

     
  6. 6.

    Excluding firms with M&As, discontinued operations and foreign currency translations. Collins and Hribar (2002) show that the errors in estimating accruals are large in the presence of significant mergers and acquisitions, discontinued operations, and foreign currency translation. We accordingly examine the sensitivity of our findings to these effects by excluding firms with such events. We follow Collins and Hribar to identify firms with mergers and acquisitions, using Compustat’s annual footnote code 1. We use Compustat item #66—Discontinued Operations—to proxy for divestitures and Compustat item #150—Foreign Currency Adjustment (Income Account)—to proxy for foreign currency translation. Unlike Collins and Hribar, who used an absolute cutoff of $10,000 for discontinued operations and foreign currency translations, we use the absolute value of the ratio of item #66 or item #150 over item #18 (net income) as cutoff. If the absolute value of the ratio is greater than 10%, we regard those observations as having significant discontinued operations or foreign currency translations. To conserve sample size, we reduce the requirement of 600 observations (used in the main analysis) to 400 observations per each two-digit SIC code. We also control for outliers using the same procedures described in Sect. 3 above. The new sample consists of 29,500 observations in 25 two-digit SIC groups over the period of 1988 through 2004. The results from this reduced sample show that cash flow predictions derived from net income (Model 2) are significantly inferior to predictions based on cash from operations only (Model 1). However, the opposite is true for earnings predictions. Therefore, these results are consistent with those reported in Table 2.

     
  7. 7.

    Longer series of predictors. The cross-sectional estimation of parameters in the first stage of our predictions (demonstrated in Sect. 2.1) uses only the current values of cash flows and accruals as predictors. Analysts often use longer time series of historical data. Accordingly, we have also experimented with cross-sectional estimates based on the last 3 years of data on CFO, NI, the change in working capital excluding inventories, and remaining estimates. However, the predictive quality of these estimates, based on 3 years of historical data, is slightly inferior to those based on current data (reported in Table 2).

     

6 A sharper focus on accounting estimates

In the above analysis we focused on four components of accounting estimates available from Compustat for the entire sample period: the change in inventories (ΔINV), depreciation and amortization (D&A), deferred taxes (DT), and all other estimates embedded in accruals (EST*). However, in recent years Compustat expanded the information on estimates, allowing us to examine additional estimates: pension expense (#43), estimated doubtful receivables (#67), post retirement benefits (#292), restructuring costs (#377), in-process R&D (#388), stock compensation expense (#398), and asset writedowns (#381), all important estimates underlying financial information. These are components of the other estimates (EST*) in Model 5. To increase the power of our tests, we selected a sample with valid data on all of these estimates in a given year. Such a restriction was satisfied in the years 2001–2004, and the resulting sample consists of 305 firm-year observations.

Table 3 reports the out-of-sample prediction results for this sub-sample of firms with multiple specific estimates. Given the small number of observations in each individual year, we perform the analysis by pooling the data over the 4 years. We report in Table 3 statistics for cash from operations and net income predictions, estimated by seven prediction models: the original five, as in Table 2, and two additional models based on the specific estimates. Model 6 splits the estimates into recurring and nonrecurring items (recurring estimates (RECUR) consist of depreciation and amortization (D&A), deferred taxes (DT), pension expense (PENSION), provision for doubtful receivables (ARDBT), and post retirement benefits (PRB), while the nonrecurring estimates consist of restructuring costs (RSTRCST), in-process R&D (IPRD), stock compensation expense (STKCMP) and asset write-downs (WRTDWN)). Model 7 is the fully disaggregated model: predictors—current cash from operations, change in working capital excluding inventory, change in inventory, and the disaggregated components of recurring and nonrecurring estimates (D&A, DT, PENSION, ARDBT, PRB, RSTRCST, IPRD, STKCMP, and WRTDWN).
Table 3

Out-of-sample 1-year-ahead predictions of cash flows and earnings by current cash flows, earnings, and combinations of accounting estimates for a sample of 305 firm-years with detailed estimates over the period 2001–2004

Prediction model

Cash from operations (CFO)

Net income (NI)

Variables

MAER

MER

R2

Theil’s U

MAER

MER

R2

Theil’s U

Year 1

    Model 1

0.041

0.001

0.46

0.58

0.073

0.010

0.27

0.86

    Model 2

0.048

−0.010

0.31

0.66

0.067

−0.002

0.11

0.94

    Model 3

0.039*

0.000

0.48

0.57

0.070*

0.009

0.33

0.82

    Model 4

0.039

−0.001

0.48

0.57

0.070

0.012

0.27

0.86

    Model 5

0.040

−0.002

0.45

0.60

0.068

0.005

0.28

0.85

    Model 6

0.041#

−0.002

0.44

0.60

0.066

0.001

0.35

0.80

    Model 7

0.047#

−0.001

0.37

0.71

0.073

0.005

0.28

0.87

Model 1: current cash flow from operations only (CFO), Model 2: current income before extraordinary income only (NI), Model 3: current cash flow from operations and change in working capital excluding inventory (CFO, ∆WC*), Model 4: current cash flow from operations, change in working capital excluding inventory, and estimates (CFO, ∆WC*, EST), EST = CFO – NI − ∆WC*. Model 5: current cash flow from operations, change in working capital excluding inventory, and disaggregated components of estimates (CFO, ∆WC* and components of estimates). These components of estimates include the change in inventory (∆INV), depreciation and amortization (D&A), deferred taxes (DT), and other estimates (EST* = CFO – NI − ∆WC* − ∆INV − D&A − DT). Model 6: current cash flow from operations, change in working capital excluding inventory, change in inventory, recurring estimates, and nonrecurring estimates (CFO, ∆WC*, ∆INV, RECUR and NRECUR). Recurring estimates (RECUR) consist of depreciation and amortization (D&A), deferred taxes (DT), pension expense (PENSION), estimated bad receivables (ARDBT), and post retirement benefits (PRB). Nonrecurring estimates consist of restructuring costs (RSTRCST), in-process R&D (IPRD), stock compensation expense (STKCMP), and writedowns (WRTDWN). Model 7: current cash flow from operations, change in working capital excluding inventory, change in inventory, and disaggregated components of recurring and non-recurring estimates (CFO, ∆WC*, ∆INV, D&A, DT, PENSION, ARDBT, PRB, RSTRCST, IPRD, STKCMP, and WRTDWN). Capital expenditure is included in all models as a control variable. Each model is estimated by pooling the data over the years 2001–2004. Models 4, 5, 6, and 7 all include estimates but with different classifications and levels of aggregation. Model 4 includes the estimates as a lump sum, Model 5 disaggregates the estimates into four components. Model 6 splits the estimates into recurring and nonrecurring. Model 7 fully disaggregated the estimates into 10 components

MAER mean absolute forecast error from the pooled sample, MER mean signed forecast error from the pooled sample; R2 the adjusted R2 from the pooled regression of actual values on predicted values over the sample period (2001–2004)

Theil’s U-statistic is for the pooled sample. U is defined as the square root of ∑(actual − forecast)2/∑(actual)2. The additional Compustat variables used in Models 6 and 7 are defined as follows (Compustat data item number in parentheses). PENSION, pension and retirement expense (#43); ARDBT, receivable-estimated doubtful (#67); PRB, post retirement benefits (#292); RSTRCST, restructuring costs (#377); IPRD, in-process R&D (#388), STKCMP: stock compensation expense (#398); WRTDWN, writedowns (#381)

# Statistical significance (at the 0.05 level or better) of model 3 (CFO and ∆WC*) compared with model 4 (CFO, ∆WC*, and EST), model 5 (CFO, ∆WC*, ∆INV, D&A, DT, and EST*), model 6 (CFO, ∆WC*, ∆INV, RECUR, and NRECUR) or model 7 (CFO, ∆WC*, ∆INV, D&A, DT, PENSION, ARDBT, PRB, RSTRCST, IPRD, STKCMP, and WRTDWN)

* Statistical significance (at the 0.05 level or better) of model 1 (CFO) compared with model 3 (CFO and ∆WC*)

The splitting of estimates into recurring and nonrecurring items intends to separate prediction noise (the nonrecurring estimates) from information (the recurring estimates). The expectation is that the recurring estimates will improve the prediction of both earnings and cash flows. However, the statistics reported in Table 3 do not support this expectation. The data are consistent with the overall sample (Table 2): (1) Current cash flows generate the best predictions of future cash flows, while current earnings yield the best predictions of next year’s earnings. (2) Models 6 and 7—based on a fine disaggregation of estimates-based accruals—do not improve the prediction of cash flows or earnings over Model 3 (CFO plus the change in working capital items). For this expanded set of identified estimates, as for the entire sample (Table 2), the disaggregation of estimates doesn’t yield improved predictions of cash flows or earnings.

7 Economic significance tests: portfolio returns

To draw inferences about the economic significance of our predictions, we use the one-year-ahead cash flow and earnings predictions whose errors were analyzed above (Table 2). From these predictions we form portfolios based on cash flow or earnings scaled by total assets. Specifically, we rank the sample firms by the predicted cash flow (earnings) and form 10 portfolios from the ranked firms. Portfolios are formed at the end of April in each year, 1990 through 2004. Finally, we compute the hedge 12-month and 36-month abnormal returns from investing (going long) in the top—highest predicted cash flow (earnings)—portfolio and shorting the bottom—lowest performance—portfolio. The returns are computed for each firm starting from May of each year, 1990 through 2004, that is, the portfolios are aligned in calendar time.20 These returns are adjusted for both size and book-to-market factors in the conventional Fama and French (1992) manner. The hedge returns reported below are the means of the returns from the annual rankings, 1990 through 2004, and the t-statistics are based on the standard errors of the annual mean returns. For comparison, we also compute the perfect foresight returns, where the portfolio formation variables of t+1 cash flow or earnings, are the actual numbers for that year, rather than the predictions.

The hedge 12-month and 36-month returns are presented in Table 4 for portfolios based on predicted cash from operations, free cash flows, net income, and operating income. The four panels of Table 4 reveal an identical pattern: the hedge return on portfolios constructed from Model 1 predictions—using current cash from operations only—are higher than the returns from portfolios constructed from predictions based on Models 2 through 5, using earnings and combinations of accruals as predictors. For example, in Panel A (portfolios constructed from cash from operations predictions), Model 1’s 12-month ahead return is 6.8% (16.7% for 36-months), while all the returns from Models 2 through 5 are lower. (The 36-month return of Model 1 is statistically significant at the 0.05 level.) Surprisingly, the predictions based on cash from operations only (Model 1) also yield the lowest standard errors (not reported) of yearly returns (1990 through 2004) of the five models. Thus, Model 1 generates the highest mean returns with the lowest volatility. As expected, the returns of the perfect foresight model (left column)—a benchmark—are all large and significant.
Table 4

Mean 12-months and 36-months-ahead hedge abnormal returns based on the ranking of actual or predicted cash flow from operations (Panel A), free cash flow (Panel B), net income (Panel C), and operating income (Panel D)

 

Size- and B/M-adjusted hedge returns

Actual

Model 1

Model 2

Model 3

Model 4

Model 5

Panel A: ranking based on actual or predicted with models 1–5 cash flow from operations at t+1

    12-Month-ahead mean return (%)

28.4

6.8

−0.8

3.7

3.7

5.0

    t-Statistic

6.61

1.45

−0.13

0.67

0.66

0.89

    36-Month-ahead mean return (%)

53.0

16.7

−1.4

7.4

10.1

16.4

    t-Statistic

6.63

1.84

−0.14

0.60

0.92

1.84

Panel B: ranking based on actual or predicted with models 1–5 free cash flow at t+1

    12-Month-ahead mean return (%)

26.6

8.2

1.5

7.2

6.4

5.6

    t-Statistic

5.66

1.91

0.27

1.67

1.36

1.22

    36-Month-ahead mean return (%)

50.5

18.6

−2.0

9.7

9.1

8.7

    t-Statistic

7.77

3.21

−0.19

0.99

0.85

0.88

Panel C: ranking based on actual or predicted with models 1–5 net income at t+1

    12-Month-ahead mean return (%)

30.9

4.1

−4.0

2.6

0.4

−1.6

    t-Statistic

6.31

0.82

−0.61

0.44

0.06

−0.25

    36-Month-ahead mean return (%)

33.2

4.4

−5.7

1.0

−3.4

−5.8

    t-Statistic

2.86

0.56

−0.66

0.10

−0.40

−0.55

Panel D: ranking based on actual or predicted with models 1–5 operating income at t+1

    12-Month-ahead mean return (%)

29.8

5.8

−2.8

3.2

1.7

0.2

    t-Statistic

5.14

1.09

−0.47

0.56

0.28

0.03

    36-Month-ahead mean return (%)

29.8

5.8

−2.8

3.2

1.7

0.2

    t-Statistic

5.14

1.09

−0.47

0.56

0.28

0.03

For Panel A, in each year from 1990 to 2004 we rank the sample firms based on the actual cash from operations of the following year (year t+1) scaled by average total assets and form 10 portfolios. That is, we employ a perfect foresight strategy, as if we know the CFO of year t+1 that will be reported. For each portfolio we calculate abnormal (size and book-to-market adjusted) returns over the next 12-month period (36-month period) starting from May of the base year to April of the following year (April of year t+3). The portfolios are aligned in calendar time. The reported abnormal return (%) for each portfolio is the mean of the yearly mean portfolio abnormal returns over the 15-year period 1990–2004 (over the 13-year period, 1990–2002, for 36-month hedge abnormal return calculations). The t-statistics are based on the standard errors of the yearly mean portfolio returns as in Fama and MacBeth (1973). We repeat the ranking and abnormal returns calculations based on the predicted CFO (instead of actual CFO) from each of the prediction models, Model 1 through Model 5. Model 1 is based on current cash flow from operations only (CFO). Model 2 is based on current net income before extraordinary items only (NI). Model 3 is based on current cash flow from operations and change in working capital excluding inventory (CFO, ∆WC*). Model 4 is based on current cash flow from operations, change in working capital excluding inventory, and estimates (CFO, ∆WC*, EST). Model 5 is based on current cash flow from operations, change in working capital excluding inventory, and disaggregated components of estimates (CFO, ∆WC*, and components of estimates). These components of estimates include the change in inventory (∆INV), depreciation and amortization (D&A), deferred taxes (DT), and other estimates (EST* = CFO – NI − ∆WC* − ∆INV − D&A − DT). Panel B, Panel C, and Panel D repeat the analysis using actual and predicted free cash flows, net income, and operating income

Most of the returns in Table 4, except for Model 1’s in Panels A and B, are statistically insignificant. There are several reasons for that. Our significance tests (Fama and MacBeth 1973) are based on 15 years only (1990 through 2004) and exhibit a few very large outliers, both weakening the power of the tests.21 More fundamentally, our prediction models are based on information available to investors. If investors are basing investment decisions on models similar to ours, or better, and if markets are efficient, our subsequent periods portfolio returns should be insignificant. However, the efficiency of capital markets with respect to accruals—the focus of our analysis—was challenged by Sloan (1996) and the multitude of subsequent studies. We, therefore, believe that market efficiency per se should not invalidate our portfolio tests. The uniform return superiority of Model 1—constructed from cash from operations only—is consistent with and lends certain support to our extensive out-of-sample prediction tests (Table 2).

Given the somewhat surprising outcome of the portfolio tests, we subjected them to several of robustness checks.
  1. 1.

    Reflection of the accruals anomaly? This anomaly documents a unique pattern of stock returns: extreme accruals are negatively associated with subsequent returns. Is this phenomenon affecting our portfolio tests? To address this question we perform the following test. We classified the sample firms in each year to high, medium, and low current total accruals firms: top 25%, middle 50%, and bottom 25% of the yearly ranking by accruals scaled by total assets. We then performed the portfolio analysis reported above for each of the three accruals groups. The accruals indeed affect substantially the returns on the various portfolios, but the relative performance of the portfolios constructed from our five prediction models (CFO only, NI only, CFO, ΔWC*, and various combinations of estimates) is essentially the same as that portrayed in Table 4. The portfolio returns increase with the size of accruals (for example, in Model 1—CFO only—the return for low accruals firms is 3.5%, while the same model yields 4.7% for high accruals firms). Note that the pattern of increasing returns with the size of accruals is opposite to the decreasing returns of the accrual anomaly that is our returns results are the reverse of those to the anomaly. Thus, this test indicates that our portfolio return findings are not driven by the accruals anomaly.

     
  2. 2.

    Individual deciles and quintiles. The portfolio returns presented in Table 4 are based on a zero-investment strategy: long in the top decile of predicted cash flows or earnings and short in the bottom decile. Basing the zero-investment portfolios on quintiles (top and bottom) rather than on deciles does not change our conclusions. Also, an individual examination of the top decile (long) and bottom decile (short) underlying Table 4 does not reveal a particular contribution of either decile to the hedge portfolio returns.

     

8 Concluding remarks

Managerial estimates and projections are pervasive in accounting measurement and valuation procedures, affecting to an unknown (by investors) degree practically all income statement and balance sheet items. The contribution of these estimates and projections to the quality of financial data is increasingly challenged in the fast-changing and turbulent business environment, which makes it very difficult for managers to generate reliable projections. The quality of financial information is further compromised by the frequent use of estimates to manipulate financial information. What then is the contribution of accounting estimates to the quality and informativeness of financial information?

We investigate this question by evaluating the contribution of accounting estimates embedded in accruals to the prediction of both cash flows and earnings over various horizons. Our battery of tests, consisting of both out-of-sample prediction and investment portfolio analyses, indicate that estimates, in groups or by individual components, do not contribute appreciably to the prediction of cash flows, (cash from operations or free cash flows), yet do improve next year’s prediction of net and operating income beyond current operating cash flows. However, our investment portfolio tests, designed to reveal economically significant contribution, do not corroborate this improvement.

Our findings, however, are subject to several caveats. First, financial data are used in various contexts besides prediction (for example, contracting uses) for which estimates may be useful. Second, our prediction models—the core of our analyses—are admittedly simple. Perhaps estimates are useful in more sophisticated uses of financial information. Third, many accounting estimates are not reported separately in the financial reports, and we obviously could not test their usefulness. These caveats notwithstanding, we believe our findings cast sufficient doubt on the usefulness of accounting estimates to focus researchers and regulators’ attention on the ways to improve their reliability.

The major implication that we draw from our findings is the urgent need to enhance the reliability of accounting estimates, particularly as the FASB and IASB increase the role of estimates in their push to expand fair value accounting. This issue, however, is sparsely discussed by researchers and accounting regulators, and we are not aware of any current policy actions aimed at enhancing the reliability of accounting estimates/projections. There are several promising proposals in the literature, which, in our opinion, deserve further attention and development. One such proposal was advanced by Ijiri (2002), who calls for a separation by income statement line items of forecasts (estimates) from facts. Such a separation will provide users with an important reliability indicator of major income statement items (for example, compare the case where 5% of the firm’s revenues are based on estimates with the much more uncertain case where 25% of revenues are based on estimates).

Furthermore, Lundholm (1999) notes that, under current GAAP, accrual estimates for a given period are rarely compared with subsequent realizations, and managers, therefore, do not face the consequences of serious intentional or unintentional misestimates. To enhance the reliability of estimates, Lundholm proposes a requirement for an ex post report on the accuracy of prior estimates. It is highly likely that the specter of investors and board members focusing on large and consistent estimation errors (for example, the warranty provision was below the actual warranty costs in every quarter of the last 2 years) will provide managers with strong ex ante disincentives to manipulate the estimates and motivate them to spend more resources on improving the reliability of the estimation procedures. Finally, Lev and Zarowin (1999) and Lev et al. (2008) take the reconciliation of estimates with ex post realizations a step further by proposing that in case of large discrepancies, previous financial reports should be revised like the routine revisions of macro-economic data. Such a revision will improve the historical record of financial information, which has been shown to affect investors’ decisions (for example, Barth et al. 1999).

Footnotes
1

Indeed, Aboody and Lev (1998) document a positive association between capitalized software development costs and future earnings.

 
2

Consider, for example, the 2001 pension footnotes of three financial institutions, Merrill Lynch, Bank of New York, and Charles Schwab, which report the following estimates of the expected returns on pension assets: 6.60, 10.50, and 9.00%, respectively (Zion 2002). The wide range of estimates (6.6–10.5%) of the long term performance of capital markets reflects the inherently large uncertainty (unreliability) of the pension expense estimate.

 
3

There are, of course, other uses of financial data, such as in contracting arrangements, which are not aimed at predicting future enterprise performance.

 
4

For example, General Electric reports in its revenue recognition footnote that various components of revenues derived from long-term projects are based on the estimated profitability of these projects. GE, however, does not break down total revenues into estimates and “facts.”.

 
5

Bowen et al. (1986) and Greenberg et al. (1986) perform similar regression-based, in-sample predictions.

 
6

Studies such as Bathke et al. (1989) and Lorek et al. (1993) also perform out-of-sample prediction tests.

 
7

Examples of studies in related areas including economic significance tests are Ou and Penman (1989), Stober (1992), Abarbanell and Bushee (1998), and Piotroski (2000).

 
8

The accounts receivable change, net of the provision, is an exception, since it is subject to an estimate. But this estimate is included in our second accruals component, EST.

 
9

We measure CFO as in Barth et al. (2001), namely net cash flow from operating activities, adjusted for the accrual portion of extraordinary items and discontinued operations.

 
10

For robustness, we reran our predictions (reported in Table 2) without capital expenditures, and conclude that none of our inferences changes in the absence of capital expenditures.

 
11

Valid statement of cash flows data for the year 1987 are available for a relatively small number of firms not enough to do a meaningful industry-by-industry analysis. Thus, we do not use 1987 data.

 
12

We repeated all of our analyses with a sample without any outlier removal, namely where we only require non missing values for the key variables and at least 600 observations in each two-digit SIC over the sample period 1988 through 2004. This sample consists of 65,178 observations and is substantially larger than the sample of 41,124 observations used in the analysis reported below. We find that for many industries the R-squares in the in-sample regressions are higher for the un-truncated data than for the truncated data. The forecast error results are essentially identical to the results from the truncated sample in terms of inferences, but the errors are larger. The portfolio abnormal returns results exhibit similar patterns to the results from truncated data. Overall, the un-truncated data yield very similar results to those of the truncated data reported below.

 
13

The reported Theil's U-statistic is the average of the yearly U-statistics. Theil’s U is defined as the square root of ∑(actual-forecast)2/∑(actual)2. The U statistic can range from zero to one, with zero implying a perfect forecast. Thus, models generating better predictions should have lower U statistics.

 
14

All the absolute forecast errors (MAER) in Table 2 are statistically significant, with p-values of 0.01 or better. The majority of the signed errors (MER) is also significant at p-values of 0.01 or better, and many of the errors are statistically significant at least at p-values of 0.05. The following signed errors are insignificant: Model 1 in forecasting Years 1–2 CFO, Models 1 and 3 in forecasting Years 1–3 CFO, and Models 2, 4 and 5 in forecasting Years 1-3 OI.

 
15

Kim and Kross (2005) use balance sheet items to calculate cash from operations, while we use statement of cash flows data. We were able to replicate the out-of-sample prediction results of Kim and Kross using balance sheet items for our sample period. Accordingly, the difference in the results between the two studies is due to the data used. As shown by Collins and Hribar (2002), the cash from operations and accruals derivation from the statement of cash flows are preferable.

 
16

Note that despite the very small difference between the MAERs of Models 1 and 3, the mean differences are statistically significant at the 0.05 level or better (see asterisks).

 
17

These inferences do not change when we examine median signed and absolute prediction errors (available on request).

 
18

The median absolute errors are lower for Model 2 than for Model 1 in all NI and OI panels except in the bottom two panels (for the aggregate next two and three years horizons).

 
19

The exception: for the 25% of the sample firms with high accruals, the mean absolute errors of model 5 (CFO plus disaggregated accruals) are significantly lower than those of model 1, for both CFO and FCF in year 1.

 
20

For example, at the end of April 2000 all firms whose most recent fiscal year ended no later than December 1999 are ranked by predicted cash flows or earnings and assigned into portfolios.

 
21

Most of the return outliers are from the years 1999 and 2003, where the subsequent returns reflect significant market reversals: the burst of the tech bubble in 2000 and the economy’s emergence from recession in 2004. Extrapolation predictions like ours perform poorly in sharp reversal years.

 

Acknowledgments

The authors are indebted to the editor and reviewers of the Review of Accounting Studies for comments and suggestions and to Louis Chan, Ilia Dichev, John Hand, James Ohlson, Shiva Rajgopal, and Stephen Ryan for helpful comments, as well as to participants of seminars at Athens University of Economics and Business, London Business School, Penn State University, Purdue University, University of Illinois at Urbana-Champaign, University of Texas at Dallas, Washington University in St. Louis, the joint Columbia–NYU Seminar, the 16th Financial Economics and Accounting Conference, the 2006 AAA FARS Midyear Meeting, the 2007 Hellenic Finance and Accounting Association Conference, and the 2008 AAA Annual Meeting.

Copyright information

© Springer Science+Business Media, LLC 2009