Abstract
Optimal fingerprinting is a standard method for detecting climate changes. Among the uncertainties taken into account by this method, one is the fact that the response to climate forcing is not known exactly, but in practice is estimated from ensemble averages of model simulations. This uncertainty can be taken into account using an Error-in-Variables model (or equivalently, the Total Least Squares method), and can be expressed through confidence intervals. Unfortunately, the predominant paradigm (likelihood ratio theory) for deriving confidence intervals is not guaranteed to work because the number of parameters that are estimated in the Error-in-Variables model grows with the number of observations. This paper discusses various methods for deriving confidence intervals and shows that the widely-used intervals proposed in the seminal paper by Allen and Stott are effectively equivalent to bias-corrected intervals from likelihood ratio theory. A new, computationally simpler, method for computing these intervals is derived. Nevertheless, these confidence intervals are incorrect in the “weak-signal regime”. This conclusion does not necessarily invalidate previous detection and attribution studies because many such studies lie in the strong-signal regime, for which standard methods give correct confidence intervals. A new diagnostic is introduced to check whether or not a data set lies in the weak-signal regime. Finally, and most importantly, a bootstrap method is shown to give correct confidence intervals in both strong- and weak-signal regimes, and always produces finite confidence intervals, in contrast to the likelihood ratio method which can give unbounded intervals that do not match the actual uncertainty.
Similar content being viewed by others
Notes
The similarity between our (46) and Allen and Stott’s (28) may not be obvious. First, there is a typo in Allen and Stott’s (28): the \(\varDelta\) on the right hand side of their (28) should be \({\mathbf{v}}\). Second, their (28) should be non-dimensional in order for it to have a chi-square distribution. In fact, (28) is implicitly non-dimensional because variables are pre-whitened. Later in their paper Allen and Stott normalize their (28) by an independent, unbiased estimate of the noise variance \({\hat{\sigma }}^2\), which yields an equation of the same form as (46).
References
Allen MR, Stott PA (2003) Estimating signal amplitudes in optimal fingerprinting, part I: theory. Clim Dyn 21:477–491
Allen MR, Tett SFB (1999) Checking for model consistency in optimal fingerprinting. Clim Dyn 15:419–434
Anderson TW (1976) Estimation of linear functional relationships: approximate distributions and connections with simultaneous equations in econometrics. J R Stat Soc Ser B (Methodol) 38(1):1–36
Barnett VD (1967) A note on linear structural relationships when both residual variances are known. Biometrika 54(3/4):670–672
Bell TL (1986) Theory of optimal weighting of data to detect climatic change. J Atmos Sci 43(16):1694–1710. https://doi.org/10.1175/1520-0469(1986)043<1694:TOOWOD>2.0.CO;2
Bindoff NL, Stott PA, AchutaRao KM, Allen MR, Gillett N, Gutzler D, Hansingo K, Hegerl G, Hu Y, Jain S, Mokhov II, Overland J, Perlwitz J, Webbari R, Zhang X (2013) Detection and attribution of climate change: from global to regional. In: Stocker T, Qin D, Plattner GK, Tignor M, Allen S, Boschung J, Nauels A, Xia Y, Bex V, Midgley P (eds) Climate change 2013: the physical science basis. Contribution of Working Group I to the fifth assessment report of the intergovernmental panel on climate change, chap 10. Cambridge University Press, Cambridge, pp 867–952
Chan LK, Mak TK (1984) Maximum likelihood estimation in multivariate structural relationships. Scand J Stat 11(1):45–50
Cheng CL, Van Ness JW (1999) Statistical regression with measurement error. Arnold (co-published by Oxford University Press), New York
Cordeiro GM, Cribari-Neto F (2014) An introduction to Bartlett correction and bias reduction. Springer, Berlin
Creasy MA (1956) Confidence limits for the gradient in the linear functional relationship. J R Stat Soc Ser B (Methodol) 18(1):65–69. http://www.jstor.org/stable/2983735
Cui H, Chen SX (2003) Empirical likelihood confidence region for parameter in the errors-in-variables models. J Multivar Anal 84(1):101–115. https://doi.org/10.1016/S0047-259X(02)00017-9
Efron B, Tibshirani RJ (1994) An introduction to the bootstrap. Chapman and Hall, New York
Emile-Geay J, Cobb KM, Carré M, Braconnot P, Leloup J, Zhou Y, Harrison SP, Corrège T, McGregor HV, Collins M, Driscoll R, Elliot M, Schneider B, Tudhope A (2015) Links between tropical Pacific seasonal, interannual and orbital variability during the Holocene. Nat Geosci 9:168–173
Fuller WA (1987) Measurement error models. Wiley, New York
Gleser LJ (1981) Estimation in a multivariate “errors in variables” regression model: large sample results. Ann. Stat. 9(1):24–44
Gleser LJ, Hwang JT (1987) The nonexistence of \(100(1 - \alpha )\%\) confidence sets of finite expected diameter in errors-in-variables and related models. Ann Stat 15(4):1351–1362. http://www.jstor.org/stable/2241678
Hannart A, Ribes A, Naveau P (2014) Optimal fingerprinting under multiple sources of uncertainty. Geophys Res Lett 41(4):1261–1268. https://doi.org/10.1002/2013GL058653
Hasselmann KF (1979) On the signal-to-noise problem in atmospheric response studies. In: Shaw DB (ed) Meteorology of the tropical ocean. Royal Meteorological Society, Reading, pp 251–259
Hasselmann K (1997) Multi-pattern fingerprint method for detection and attribution of climate change. Clim Dyn 13:601–611
Huffel SV, Lemmerling P (2002) Introduction to total least squares and errors-in-variables modeling. In: Huffel SV, Lemmerling P (eds) Total least squares and errors-in-variables modeling: analysis, algorithms, and applications. Kluwer Academic Publishers, Dordrecht, pp 1–11
Huntingford C, Stott PA, Allen MR, Lambert FH (2006) Incorporating model uncertainty into attribution of observed temperature change. Geophys Res Lett 33(L05):710
Lindley DV (1947) Regression lines and the linear functional relationship. Suppl J R Stat Soc 9(2):218–244
Markovsky I, Van Huffel S (2007) Overview of total least-squares methods. Signal Process 87(10):2283–2302. https://doi.org/10.1016/j.sigpro.2007.04.004
Nestares O, Fleet DJ, Heeger DJ (2000) Likelihood functions and confidence bounds for total-least-squares problems. In: Proceedings IEEE conference on computer vision and pattern recognition. CVPR 2000 (Cat. No.PR00662), vol 1, pp 523–530. https://doi.org/10.1109/CVPR.2000.855864
Neyman J, Scott EL (1948) Consistent estimates based on partially consistent observations. Econometrica 16(1):1–32
North GR, Bell TL, Cahalan RF, Moeng FJ (1982) Sampling errors in the estimation of empirical orthogonal functions. Mon Weather Rev 1982:699–706
North GR, Kim KY, Shen SSP, Hardin JW (1995) Detection of forced climate signals. Part I: filter theory. J Clim 8:401–408
Pešta M (2013) Total least squares and bootstrapping with applications in calibration. Statistics 47(5):966–991. https://doi.org/10.1080/02331888.2012.658806
Polson D, Hegerl GC, Zhang X, Osborn TJ (2013) Causes of robust seasonal land precipitation changes. J Clim 26(17):6679–6697. https://doi.org/10.1175/JCLI-D-12-00474.1
Ribes A, Azais JM, Planton S (2009) Adaptation of the optimal fingerprint method for climate change detection using a well-conditioned covariance matrix estimate. Clim Dyn 33:707–722
Santer BD, Brüggemann W, Cubasch U, Hasselmann K, Höck H, Maier-Reimer E, Mikolajewica U (1994) Signal-to-noise analysis of time-dependent greenhouse warming experiments. Clim Dyn 9(6):267–285. https://doi.org/10.1007/BF00204743
Van Huffel S (1991) The total least squares problem. Soc Ind Appl Math. https://doi.org/10.1137/1.9781611971002
Wilks SS (1962) Mathematical statistics. Wiley, New York
Wong MY (1989) Likelihood estimation of a simple linear regression model when both variables have error. Biometrika 76(1):141–148
Acknowledgements
The authors would like to acknowledge discussions with Richard Smith, which were helpful in clarifying important fine points in likelihood ratio theory, and constructive comments from two anonymous reviewers. This research was supported primarily by the National Science Foundation (AGS-1622295). Additional support was provided from National Science Foundation (AGS-1338427), National Aeronautics and Space Administration (NNX14AM19G), the National Oceanic and Atmospheric Administration (NA14OAR4310160). The views expressed herein are those of the authors and do not necessarily reflect the views of these agencies. MKT was partially supported by the Office of Naval Research (N00014-12-1-0911 and N00014-16-1-2073).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
DelSole, T., Trenary, L., Yan, X. et al. Confidence intervals in optimal fingerprinting. Clim Dyn 52, 4111–4126 (2019). https://doi.org/10.1007/s00382-018-4356-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00382-018-4356-3