Infinite-Dimensional Compressed Sensing and Function Interpolation

Abstract

We introduce and analyse a framework for function interpolation using compressed sensing. This framework—which is based on weighted \(\ell ^1\) minimization—does not require a priori bounds on the expansion tail in either its implementation or its theoretical guarantees and in the absence of noise leads to genuinely interpolatory approximations. We also establish a new recovery guarantee for compressed sensing with weighted \(\ell ^1\) minimization based on this framework. This guarantee conveys several benefits. First, unlike existing results, it is sharp (up to constants and log factors) for large classes of functions regardless of the choice of weights. Second, by examining the measurement condition in the recovery guarantee, we are able to suggest a good overall strategy for selecting the weights. In particular, when applied to the important case of multivariate approximation with orthogonal polynomials, this weighting strategy leads to provably optimal estimates on the number of measurements required, whenever the support set of the significant coefficients is a so-called lower set. Finally, this guarantee can also be used to theoretically confirm the benefits of alternative weighting strategies where the weights are chosen based on prior support information. This provides a theoretical basis for a number of recent numerical studies showing the effectiveness of such approaches.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    In some of the first presentations of the golfing scheme [8, 21], it was assumed that these random variables were independent, which is not the case in general. This issue was fixed in [2] via a more careful argument. Here we follow the approach of [22].

References

  1. 1.

    B. Adcock. Infinite-dimensional \(\ell ^1\) minimization and function approximation from pointwise data. Constr. Approx. (to appear), 2016.

  2. 2.

    B. Adcock and A. C. Hansen. Generalized sampling and infinite-dimensional compressed sensing. Found. Comput. Math., 16(5):1263–1323, 2016.

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    B. Adcock, R. Platte, and A. Shadrin. Optimal sampling rates for approximating analytic functions from pointwise samples. arXiv:1610.04769, 2016.

  4. 4.

    V. A. Antonov and K. V. Holšhevnikov. An estimate of the remainder in the expansion of the generating function for the Legendre polynomials (generalization and improvement of Bernstein’s inequality). Vestnik Leningrad. Univ. Mat., 13:163–166, 1981.

    MATH  Google Scholar 

  5. 5.

    B. Bah and R. Ward. The sample complexity of weighted sparse approximation. arxiv:1507.0673, 2015.

  6. 6.

    J. Bigot, C. Boyer, and P. Weiss. An analysis of block sampling strategies in compressed sensing. IEEE Trans. Inform. Theory (to appear), 2016.

  7. 7.

    A. Bourrier, M. E. Davies, T. Peleg, P. Pérez, and R. Gribonval. Fundamental performance limits for ideal decoders in high-dimensional linear inverse problems. IEEE Trans. Inform. Theory, 60(12):7928–7946, 2014.

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    E. J. Candès and Y. Plan. A probabilistic and RIPless theory of compressed sensing. IEEE Trans. Inform. Theory, 57(11):7235–7254, 2011.

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    E. J. Candès, J. Romberg, and T. Tao. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory, 52(2):489–509, 2006.

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    A. Chernov and D. Dũng. New explicit-in-dimension estimates for the cardinality of high-dimensional hyperbolic crosses and approximation of functions having mixed smoothness. J. Complexity, 32:92–121, 2016.

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    A. Chkifa, A. Cohen, G. Migliorati, F. Nobile, and R. Tempone. Discrete least squares polynomial approximation with random evaluations-application to parametric and stochastic elliptic PDEs. ESAIM Math. Model. Numer. Anal., 49(3):815–837, 2015.

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    A. Chkifa, N. Dexter, H. Tran, and C. Webster. Polynomial approximation via compressed sensing of high-dimensional functions on lower sets. Technical Report ORNL/TM-2015/497, Oak Ridge National Laboratory (also available as arXiv:1602.05823), 2015.

  13. 13.

    I.-Y. Chun and B. Adcock. Compressed sensing and parallel acquisition. arXiv:1601.06214, 2016.

  14. 14.

    A. Cohen, R. A. DeVore, and C. Schwab. Convergence rates of best \(N\)-term Galerkin approximations for a class of elliptic sPDEs. Found. Comput. Math., 10:615–646, 2010.

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    A. Cohen, R. A. DeVore, and C. Schwab. Analytic regularity and polynomial approximation of parametric and stochastic elliptic PDE’s. Analysis and Applications, 9:11–47, 2011.

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    D. L. Donoho. Compressed sensing. IEEE Trans. Inform. Theory, 52(4):1289–1306, 2006.

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    A. Doostan and H. Owhadi. A non-adapted sparse approximation of PDEs with stochastic inputs. J. Comput. Phys., 230(8):3015–3034, 2011.

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    S. Foucart and H. Rauhut. A Mathematical Introduction to Compressive Sensing. Birkhauser, 2013.

    Google Scholar 

  19. 19.

    M. Friedlander, H. Mansour, R. Saab, and I. Yilmaz. Recovering compressively sampled signals using partial support information. IEEE Trans. Inform. Theory, 58(2):1122–1134, 2012.

    MathSciNet  Article  MATH  Google Scholar 

  20. 20.

    W. Gautschi. How sharp is Bernstein’s inequality for Jacobi polynomials? Electron. Trans. Numer. Anal., 36:1–8, 2009.

    MathSciNet  MATH  Google Scholar 

  21. 21.

    D. Gross. Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inform. Theory, 57(3):1548–1566, 2011.

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    D. Gross, F. Krahmer, and R. Kueng. A partial derandomization of phaselift using spherical designs. J. Fourier Anal. Appl., 21(2):229–266, 2015.

    MathSciNet  Article  MATH  Google Scholar 

  23. 23.

    J. Hampton and A. Doostan. Compressive sampling of polynomial chaos expansions: Convergence analysis and sampling strategies. J. Comput. Phys., 280:363–386, 2015.

    MathSciNet  Article  MATH  Google Scholar 

  24. 24.

    J. D. Jakeman, A. Narayan, and T. Zhou. A generalized sampling and preconditioning scheme for sparse approximation of polynomial chaos expansions. arXiv:1602.06879, 2016.

  25. 25.

    T. Kühn, W. Sickel, and T. Ullrich. Approximation of mixed order Sobolev functions on the \(d\)-torus: Asymptotics, preasymptotics, and \(d\)-dependence. Constr. Approx., 42(3):353–398, 2015.

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    L. Lorch. Alternative proof of a sharpened form of Bernstein’s inequality for legendre polynomials. Appl. Anal., 14:237–240, 1982/3.

  27. 27.

    L. Mathelin and K. A. Gallivan. A compressed sensing approach for partial differential equations with random input data. Commun. Comput. Phys., 12(4):919–954, 2012.

    MathSciNet  Article  MATH  Google Scholar 

  28. 28.

    G. Migliorati. Polynomial approximation by means of the random discrete \(L^2\) projection and application to inverse problems for PDEs with stochastic data. PhD thesis, Politecnico di Milano, 2013.

  29. 29.

    G. Migliorati. Multivariate Markov-type and Nikolskii-type inequalities for polynomials associated with downward closed multi-index sets. J. Approx. Theory, 189:137–159, 2015.

    MathSciNet  Article  MATH  Google Scholar 

  30. 30.

    G. Migliorati, F. Nobile, E. von Schwerin, and R. Tempone. Analysis of the discrete \(L^2\) projection on polynomial spaces with random evaluations. Found. Comput. Math., 14:419–456, 2014.

    MathSciNet  MATH  Google Scholar 

  31. 31.

    A. Narayan, J. D. Jakeman, and T. Zhou. A Christoffel function weighted least squares algorithm for collocation approximations. arXiv:1412.4305, 2014.

  32. 32.

    A. Narayan and T. Zhou. Stochastic collocation on unstructured multivariate meshes. Commun. Comput. Phys., 18(1):1–36, 2015.

    MathSciNet  Article  MATH  Google Scholar 

  33. 33.

    J. Peng, J. Hampton, and A. Doostan. A weighted \(\ell _1\)-minimization approach for sparse polynomial chaos expansions. J. Comput. Phys., 267:92–111, 2014.

    MathSciNet  Article  MATH  Google Scholar 

  34. 34.

    R. Platte, L. N. Trefethen, and A. Kuijlaars. Impossibility of fast stable approximation of analytic functions from equispaced samples. SIAM Rev., 53(2):308–318, 2011.

    MathSciNet  Article  MATH  Google Scholar 

  35. 35.

    H. Rauhut and R. Ward. Sparse recovery for spherical harmonic expansions. In Proceedings of the 9th International Conference on Sampling Theory and Applications, 2011.

  36. 36.

    H. Rauhut and R. Ward. Sparse Legendre expansions via l1-minimization. J. Approx. Theory, 164(5):517–533, 2012.

    MathSciNet  Article  MATH  Google Scholar 

  37. 37.

    H. Rauhut and R. Ward. Interpolation via weighted \(\ell _1\) minimization. Appl. Comput. Harmon. Anal., 40(2):321–351, 2016.

    MathSciNet  Article  MATH  Google Scholar 

  38. 38.

    G. Szegö. Orthogonal Polynomials. American Mathematical Society, Providence, RI, 1975.

    Google Scholar 

  39. 39.

    G. Tang and G. Iaccarino. Subsampled Gauss quadrature nodes for estimating polynomial chaos expansions. SIAM/ASA J. Uncertain. Quantif., 2(1):423–443, 2014.

    MathSciNet  Article  MATH  Google Scholar 

  40. 40.

    H. Tran, C. Webster, and G. Zhang. Analysis of quasi-optimal polynomial approximations for parameterized PDEs with deterministic and stochastic coefficients. ORNL/TM-2014/468, Oak Ridge National Laboratory (also available as arXiv:1508.01821), 2015.

  41. 41.

    E. van den Berg and M. P. Friedlander. SPGL1: A solver for large-scale sparse reconstruction. http://www.cs.ubc.ca/labs/scl/spgl1, June 2007.

  42. 42.

    E. van den Berg and M. P. Friedlander. Probing the Pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput., 2(890–912), 31.

  43. 43.

    Z. Xu and T. Zhou. On sparse interpolation and the design of deterministic interpolation points. SIAM J. Sci. Comput., 36(4):1752–1769, 2014.

    MathSciNet  Article  MATH  Google Scholar 

  44. 44.

    L. Yan, L. Guo, and D. Xiu. Stochastic collocation algorithms using \(\ell _1\)-minimization. Int. J. Uncertain. Quantif., 2(3):279–293, 2012.

    MathSciNet  Article  Google Scholar 

  45. 45.

    X. Yang and G. E. Karniadakis. Reweighted \(\ell _1\) minimization method for stochastic elliptic differential equations. J. Comput. Phys., 248:87–108, 2013.

    Article  MATH  Google Scholar 

  46. 46.

    X. Yu and S. Baek. Sufficient conditions on stable recovery of sparse signals with partial support information. IEEE Signal Process. Letters, 20(5), 2013.

Download references

Acknowledgements

The work was supported in part by the Natural Sciences and Engineering Research Council of Canada through Grant 611675 and an Alfred P. Sloan Research Fellowship. The author would particularly like to thank Abdellah Chkifa, Clayton Webster, Hoang Tran and Guannan Zhang for introducing him to the concept of lower sets. The results of Sect. 7.3 are due to their insight. He would also like to like to thank Rick Archibald, Nilima Nigam, Clarice Poon and Tao Zhou for useful discussions and comments.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Ben Adcock.

Additional information

Communicated by Albert Cohen.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Adcock, B. Infinite-Dimensional Compressed Sensing and Function Interpolation. Found Comput Math 18, 661–701 (2018). https://doi.org/10.1007/s10208-017-9350-3

Download citation

Keywords

  • High-dimensional approximation
  • Interpolation
  • Compressed sensing
  • Structured sparsity
  • Orthogonal polynomials

Mathematics Subject Classification

  • 41A05
  • 41A10
  • 41A63
  • 65N12
  • 65N15