False confidence: are we ignoring significant sources of uncertainty?

Abstract

Purpose

With the increasing use of stochastic simulation, also known as Monte Carlo simulation, to perform uncertainty analysis in life cycle assessment, it is important to consider whether the predominant methods and practices in the field accurately represent uncertainty in the results.

Methods

Two quantitative aspects of uncertainty characterization in ecoinvent, namely the derivation of additional uncertainty from the pedigree matrix and the use of static market activities to model consumption mixes, are reviewed with respect to their effects on stochastic simulation results. A discrete choice simulation is applied to model uncertainty in a consumption mix, and the results are compared to the conventional approach.

Results and discussion

Both practices studied are found to systematically underestimate uncertainty as measured by the size of the confidence interval. In markets with multiple suppliers, the uncertainty in the market average is dramatically narrower than the variability in the suppliers themselves.

Conclusions

The current state of practice leads to false inferences and may be misleading to the public. Life cycle assessment researchers should distinguish between synthetic variability models, such as those used in ecoinvent, and authentic estimates of uncertainty in foreground models. The community must continue to develop and critically evaluate methods for uncertainty characterization.