Skip to main content

Compression approaches for the regularized solutions of linear systems from large-scale inverse problems

Abstract

We introduce and compare new compression approaches to obtain regularized solutions of large linear systems which are commonly encountered in large scale inverse problems. We first describe how to approximate matrix vector operations with a large matrix through a sparser matrix with fewer nonzero elements, by borrowing from ideas used in wavelet image compression. Next, we describe and compare approaches based on the use of the low rank singular value decomposition (SVD), which can result in further size reductions. We describe how to obtain the approximate low rank SVD of the original matrix using the sparser wavelet compressed matrix. Some analytical results concerning the various methods are presented and the results of the proposed techniques are illustrated using both synthetic data and a very large linear system from a seismic tomography application, where we obtain significant compression gains with our methods, while still resolving the main features of the solutions.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. Akansu, A.N., Haddad, R.A.: Multiresolution Signal Decomposition: Transforms, Subbands, and Wavelets. Academic Press Inc, Orlando (1992)

    MATH  Google Scholar 

  2. Calvetti, D., Morigi, S., Reichel, L., Sgallari, F.: Tikhonov regularization and the L-curve for large discrete ill-posed problems. J. Comput. Appl. Math. 123 (1–2), 423–446 (2000)

  3. Chárlety, J., Voronin, S., Nolet, G., Loris, I., Simons, F.J., Sigloch, K., Daubechies, I.C.: Global seismic tomography with sparsity constraints: comparison with smoothing and damping regularization. J. Geophys. Res. Solid Earth (2013)

  4. Cohen, A., Daubechies, I.C., Feauveau, J.-C.: Biorthogonal bases of compactly supported wavelets. Commun. Pure Appl. Math. 45(5), 485–560 (1992)

    MATH  MathSciNet  Article  Google Scholar 

  5. Daubechies, I.C.: Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math. 41, 909–996 (1988)

    MATH  MathSciNet  Article  Google Scholar 

  6. Daubechies, I.C., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57(11), 1413–1457 (2004)

    MATH  Article  Google Scholar 

  7. Debayle, E., Sambridge, M.: Inversion of massive surface wave data sets: model construction and resolution assessment. J. Geophys. Res. 109, B02316 (2004)

    Google Scholar 

  8. Halko, N., Martinsson, P.G., Tropp, J.A.: Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 53(2), 217–288 (2011)

    MATH  MathSciNet  Article  Google Scholar 

  9. Härdle, W., Kerkyacharian, G., Picard, D., Tsybokov, A.: Wavelets, approximation, and statistical applications. Lecture Notes in Statistics, vol. 129. Springer, New York (1998)

  10. Lampe, J., Reichel, L., Voss, H.: Large-scale Tikhonov regularization via reduction by orthogonal projection. Linear Algebra Appl. 436(8), 2845–2865 (2012). (Special Issue dedicated to Danny Sorensen’s 65th birthday)

    MATH  MathSciNet  Article  Google Scholar 

  11. Markovsky, I.: Low rank approximation: algorithms, implementation, applications. In: Communications and Control Engineering. Springer, New York (2012)

  12. Marquering, H., Nolet, G., Dahlen, F.A.: Three-dimensional waveform sensitivity kernels. Geophys. J. Int. 132(3), 521–534 (1998)

    Article  Google Scholar 

  13. Meyer, Y.: Wavelets: Algorithms and Applications. Society for Industrial and Applied Mathematics, Philadelphia (1993). (Translated and revised by Robert D. Ryan)

  14. Nolet, G.: A Breviary of Seismic Tomography. Cambridge Univ. Press, Cambridge (2008)

    MATH  Book  Google Scholar 

  15. Paige, C.C., Saunders, M.A.: LSQR: an algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8(1), 43–71 (1982)

    MATH  MathSciNet  Article  Google Scholar 

  16. Ronchi, C., Iacono, R., Paolucci, P.S.: The cubed sphere: a new method for the solution of partial differential equations in spherical geometry. J. Comput. Phys. 124(1), 93–114 (1996)

    MATH  MathSciNet  Article  Google Scholar 

  17. Simons, F.J., Loris, I., Nolet, G., Daubechies, I.C., Voronin, S., Judd, J.S., Vetter, P.A., Chárlety, J., Vonesch, C.: Solving or resolving global tomographic models with spherical wavelets, and the scale and sparsity of seismic heterogeneity. Geophys. J. Int. 187(2), 969–988 (2011)

    Article  Google Scholar 

  18. Sweldens, W.: The lifting scheme: a new philosophy in biorthogonal wavelet constructions. In: Laine, A.F., Unser, M.A., Wickerhauser, M.V. (eds.) Wavelet applications in signal and image processing III. Proceedings of SPIE, vol. 2569, pp. 68–79 (1995)

  19. Tikhonov, A.N.: Solution of incorrectly formulated problems and the regularization method. Sov. Math. Dokl. (1963)

  20. Trefethen, L.N., Bau, D.: Numerical Linear Algebra. SIAM Philadelphia (1997)

  21. van Heijst, H.-J., Woodhouse, J.H.: Global high-resolution phase velocity distributions of overtone and fundamental mode surface waves determined by mode branch stripping. Geophys. J. Int. 137(3), 601–620 (1999)

    Article  Google Scholar 

  22. Voronin, S., Martinsson, P.-G.: RSVDPACK: subroutines for computing partial singular value decompositions via randomized sampling on single core, multi core, and GPU architectures (2015). ArXiv e-prints

  23. Wang, S., Zhang, Z.: Improving CUR matrix decomposition and the Nystrom approximation via adaptive sampling. J. Mach. Learn. Res. 14(1), 2729–2769 (2013)

    MATH  MathSciNet  Google Scholar 

  24. Woodbury, M.A.: Inverting modified matrices. In: Statistical Research Group, Memo. Rep. No. 42. Princeton University, Princeton (1950)

Download references

Acknowledgments

The authors would like to thank the referees for very clear and helpful comments which resulted in significant improvements. We also like to thank Ignace Loris, Gunnar Martinsson and Frederik Simons for very helpful discussion. Support from the ERC (Advanced Grant 226837), the Defense Advanced Projects Research Agency (Contract N66001-13-1-4050) and the National Science Foundation (Contracts 1320652 and 0748488) is greatly appreciated.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sergey Voronin.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Voronin, S., Mikesell, D. & Nolet, G. Compression approaches for the regularized solutions of linear systems from large-scale inverse problems. Int J Geomath 6, 251–294 (2015). https://doi.org/10.1007/s13137-015-0073-9

Download citation

Keywords

  • Ill-posedness
  • Regularization
  • Singular value decomposition
  • Wavelets
  • Data compression

Mathematics Subject Classification

  • 65F22
  • 86-08
  • 65T60
  • 15A18