Abstract
It has been proposed that classical filtering methods, like the Kalman filter and 3DVAR, can be used to solve linear statistical inverse problems. In the work of Iglesias, Lin, Lu, and Stuart (Commun. Math. Sci. 15(7):1867–1896, ??), error estimates were obtained for this approach. By optimally tuning a regularization parameter in the filters, the authors were able to show that the mean squared error could be systematically reduced. Building on the aforementioned work of Iglesias, Lin, Lu, and Stuart, we prove that by (i) considering the problem in a weaker norm and (ii) applying simple iterate averaging of the filter output, 3DVAR will converge in mean square, unconditionally on the choice of parameter. Without iterate averaging, 3DVAR cannot converge by running additional iterations with a fixed choice of parameter. We also establish that the Kalman filter’s performance in this setting cannot be improved through iterate averaging. We illustrate our results with numerical experiments that suggest our convergence rates are sharp.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cavalier, L.: Nonparametric statistical inverse problems. Inverse Probl. 24(3), 034004 (2008)
Da Prato, G., Zabczyk, J.: Stochastic equations in infinite dimensions. Cambridge University Press (2014)
Ding, L., Lu, S., Cheng, J.: Weak-norm posterior contraction rate of the 4dvar method for linear severely ill-posed problems. J. Complex. 46, 1–18 (2018)
Ghanem, R., Higdon, D., Owhadi, H. (eds.): Handbook of Uncertainty Quantification. Springer, Cham (2017)
Ghosal, S., van der Vaart, A.: Fundamentals of nonparametric bayesian. inference Cambridge University Press (2017)
Heinz, W.E., Hanke, M., Neubauer, A.: Regularization of inverse problems. Kluwer Academic Publishers (2000)
Humpherys, J., Redd, P., West, J.: A fresh look at the Kalman filter. SIAM Rev. 54(4), 801–823 (2012)
Iglesias, M.A.: A regularizing iterative ensemble Kalman method for PDE-constrained inverse problems. Inverse Probl. 32, 025002 (2016)
Iglesias, M.A., Law, K.J., Stuart, A.M.: Ensemble kalman methods for inverse problems. Inverse Probl. 29(4), 045001 (2013)
Iglesias, M.A., Lin, K., Lu, S., Stuart, A.M.: Filter based methods for statistical linear inverse problems. Commun. Math. Sci. 15(7), 1867–1896 (2017)
Jones, F.G.E.: High and infinite-dimensional filtering methods. PhD thesis, Drexel University (2020)
Knapik, B.T., van der Vaart, A.W., van Zanten, J.H., et al.: Bayesian inverse problems with gaussian priors. Ann. Stat. 39(5), 2626–2657 (2011)
Kushner, H.J., Yin, G.: Stochastic Approximation and Recursive Algorithms and Applications. Springer, New York (2003)
Law, K., Stuart, A., Zygalakis, K.: Data assimilation: a Mathematical Introduction. Springer International Publishing (2015)
Lu, S., Niu, P., Werner, F.: On the asymptotical regularization for linear inverse problems in presence of white noise. SIAM-ASA J. Uncertain. Quantif. 9(1), 1–28 (2021)
Mair, B.A., Ruymgaart, F.H.: Statistical inverse estimation in hilbert scales. SIAM J. Appl. Math. 56(5), 1424–1444 (1996)
Mathé, P., Pereverzev, S.V.: Optimal discretization of inverse problems in hilbert scales. regularization and self-regularization of projection methods. SIAM J. Numer. Anal. 38(6), 1999–2021 (2001)
Øksendal, B.: Stochastic differential equations. Springer (2003)
Pereverzev, S., Lu, S.: Regularization theory for Ill-posed problems. De Gruyter (2013)
Polyak, B.T., Juditsky, A.B.: Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30(4), 838–855 (1992)
Schillings, C., Stuart, A.: Convergence analysis of ensemble Kalman inversion: the linear, noisy case. Appl. Anal., pp. 1–17 (2017)
Schillings, C., Stuart, A.M.: Analysis of the ensemble Kalman filter for inverse problems. SIAM J. Numer. Anal. 55, 1264–1290 (2017)
Shumway, R.H., Stoffer, D.S.: Time series analysis and its applications. Springer (2011)
Stuart, A.M.: Inverse problems: A Bayesian perspective. Acta Numerica 19, 451–559 (2010)
Sullivan, T.J.: Introduction to uncertainty quantification. Springer, vol. 63 (2015)
van Rooij, A.C., Ruymgaart, F.H.: Asymptotic minimax rates for abstract linear estimators. J. Stat. Plan. Inference 53(3), 389–402 (1996)
Acknowledgements
The authors thank A.M. Stuart for suggesting an investigation of this problem. The content of this work originally appeared in [11] as a part of F.G. Jones’s PhD dissertation. Work reported here was run on hardware supported by Drexel’s University Research Computing Facility.
Funding
This work was supported by US National Science Foundation Grant DMS-1818716.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jones, F.G., Simpson, G. Iterate averaging, the Kalman filter, and 3DVAR for linear inverse problems. Numer Algor 92, 1105–1125 (2023). https://doi.org/10.1007/s11075-022-01332-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-022-01332-9