Statistics and Computing

, Volume 29, Issue 6, pp 1185–1201 | Cite as

De-noising by thresholding operator adapted wavelets

  • Gene Ryan YooEmail author
  • Houman Owhadi


Donoho and Johnstone (Ann Stat 26(3):879–921, 1998) proposed a method from reconstructing an unknown smooth function u from noisy data \(u+\zeta \) by translating the empirical wavelet coefficients of \(u+\zeta \) towards zero. We consider the situation where the prior information on the unknown function u may not be the regularity of u but that of \( {\mathcal {L}}u\) where \({\mathcal {L}}\) is a linear operator (such as a PDE or a graph Laplacian). We show that the approximation of u obtained by thresholding the gamblet (operator adapted wavelet) coefficients of \(u+\zeta \) is near minimax optimal (up to a multiplicative constant), and with high probability, its energy norm (defined by the operator) is bounded by that of u up to a constant depending on the amplitude of the noise. Since gamblets can be computed in \({\mathcal {O}}(N {\text {polylog}} N)\) complexity and are localized both in space and eigenspace, the proposed method is of near-linear complexity and generalizable to nonhomogeneous noise.


Probabilistic numerics Denoising Thresholding Wavelets Gamblet transform 



The authors gratefully acknowledges this work supported by the Air Force Office of Scientific Research and the DARPA EQUiPS Program under Award Number FA9550-16-1-0054 (Computational Information Games) and the Air Force Office of Scientific Research under Award Number FA9550-18-1-0271 (Games for Computation and Learning). We also thank two anonymous referees for detailed reviews and helpful comments.


  1. Boeing, G.: OSMnx: new methods for acquiring, constructing, analyzing, and visualizing complex street networks. Comput. Environ. Urban Syst. 65, 126–135 (2017)CrossRefGoogle Scholar
  2. Briol, F.X., Oates, C.J., Girolami, M., Osborne, M.A., Sejdinovic, D.: Probabilistic integration: a role for statisticians in numerical analysis? (2015). arXiv:1512.00933
  3. Chkrebtii, O.A., Campbell, D.A., Calderhead, B., Girolami, M.A.: Bayesian solution uncertainty quantification for differential equations. Bayesian Anal. 11(4), 1239–1267 (2016)MathSciNetCrossRefGoogle Scholar
  4. Cockayne, J., Oates, C.J., Sullivan, T., Girolami, M.: Probabilistic meshless methods for Bayesian inverse problems (2016). arXiv:1605.07811
  5. Cockayne, J., Oates, C., Sullivan, T., Girolami, M.: Bayesian probabilistic numerical methods (2017). arXiv:1702.03673
  6. Dasgupta, S., Gupta, A.: An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct. Algorithms 22, 60–65 (2003)MathSciNetCrossRefGoogle Scholar
  7. Diaconis, P.: Bayesian numerical analysis. In: Berger, J., Gupta, S. (eds.) Statistical Decision Theory and Related Topics, IV, Vol. 1 (West Lafayette, Ind., 1986), pp. 163–175. Springer, New York (1988) CrossRefGoogle Scholar
  8. Ding, L., Mathé, P.: Minimax rates for statistical inverse problems under general source conditions (2017). arxiv:1707.01706v2
  9. Donoho, D.L.: Statistical estimation and optimal recovery. Ann. Stat. 22, 238–270 (1994)MathSciNetCrossRefGoogle Scholar
  10. Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)MathSciNetCrossRefGoogle Scholar
  11. Donoho, D., Johnstone, I.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425–455 (1994)MathSciNetCrossRefGoogle Scholar
  12. Donoho, D., Liu, R., MacGibbon, B.: Minimax risk over hyperrectangles and implications. Ann. Stat. 18(3), 1416–1437 (1990)MathSciNetCrossRefGoogle Scholar
  13. Donoho, D.L., Johnstone, I.M., et al.: Minimax estimation via wavelet shrinkage. Ann. Stat. 26(3), 879–921 (1998)MathSciNetCrossRefGoogle Scholar
  14. Gazzola, F., Grunau, H.C., Sweers, G.: Polyharmonic Boundary Value Problems: Positivity Preserving and Nonlinear Higher Order Elliptic Equations in Bounded Domains. Springer, Berlin (2010)CrossRefGoogle Scholar
  15. Hennig, P., Osborne, M.A., Girolami, M.: Probabilistic numerics and uncertainty in computations. Proc. R. Soc. A. 471(2179), 20150142 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  16. Micchelli, C.A., Rivlin, T.J.: A survey of optimal recovery. In: Micchelli, C.A., Rivlin, T.J. (eds.) Optimal Estimation in Approximation Theory, pp. 1–54. Springer, Berlin (1977)CrossRefGoogle Scholar
  17. Oates, C., Cockayne, J., Aykroyd, R.G.: Bayesian probabilistic numerical methods for industrial process monitoring (2017). arXiv:1707.06107
  18. Owhadi, H.: Bayesian numerical homogenization. Multiscale Model. Simul. 13(3), 812–828 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  19. Owhadi, H.: Multigrid with rough coefficients and multiresolution operator decomposition from hierarchical information games. SIAM Rev. 59(1), 99–149 (2017)MathSciNetCrossRefGoogle Scholar
  20. Owhadi, H., Scovel, C.: Universal scalable robust solvers from computational information games and fast eigenspace adapted multiresolution analysis (2017). arXiv:1703.10761
  21. Owhadi, H., Scovel, C.: Operator Adapted Wavelets, Fast Solvers, and Numerical Homogenization from a Game Theoretic Approach to Numerical Approximation and Algorithm Design. Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, Cambridge (2020)zbMATHGoogle Scholar
  22. Owhadi, H., Zhang, L.: Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients. J. Comput. Phys. 347, 99–128 (2017)MathSciNetCrossRefGoogle Scholar
  23. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Inferring solutions of differential equations using noisy multi-fidelity data. J. Comput. Phys. 335, 736–746 (2017)MathSciNetCrossRefGoogle Scholar
  24. Schäfer, F., Sullivan, T.J., Owhadi, H.: Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity (2017). arXiv:1706.02205
  25. Schober, M., Duvenaud, D.K., Hennig, P.: Probabilistic ODE solvers with Runge–Kutta means. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K. (eds.) Advances in Neural Information Processing Systems 27, pp. 739–747. Curran Associates, Inc., Red Hook (2014)Google Scholar
  26. Woźniakowski, H.: Probabilistic setting of information-based complexity. J. Complex. 2(3), 255–269 (1986). MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.California Institute of TechnologyPasadenaUSA
  2. 2.California Institute of TechnologyPasadenaUSA

Personalised recommendations