Skip to main content
SpringerLink
Log in

Proof Methods for Robust Low-Rank Matrix Recovery

  • Chapter
  • First Online:
Compressed Sensing in Information Processing

Part of the book series: Applied and Numerical Harmonic Analysis ((ANHA))

  • 663 Accesses

Abstract

Low-rank matrix recovery problems arise naturally as mathematical formulations of various inverse problems, such as matrix completion, blind deconvolution, and phase retrieval. Over the last two decades, a number of works have rigorously analyzed the reconstruction performance for such scenarios, giving rise to a rather general understanding of the potential and the limitations of low-rank matrix models in sensing problems. In this chapter, we compare the two main proof techniques that have been paving the way to a rigorous analysis, discuss their potential and limitations, and survey their successful applications. On the one hand, we review approaches based on descent cone analysis, showing that they often lead to strong guarantees even in the presence of adversarial noise, but face limitations when it comes to structured observations. On the other hand, we discuss techniques using approximate dual certificates and the golfing scheme, which are often better suited to deal with practical measurement structures, but sometimes lead to weaker guarantees. Lastly, we review recent progress toward analyzing descent cones also for structured scenarios—exploiting the idea of splitting the cones into multiple parts that are analyzed via different techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Extensions to complex-valued inner product spaces are also possible, see, e.g., [46].

References

  1. Ahmed, A., Recht, B., Romberg, J.: Blind deconvolution using convex programming. IEEE Trans. Inform. Theory 60(3), 1711–1732 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  2. Amelunxen, D., Lotz, M., McCoy, M.B., Tropp, J.A.: Living on the edge: phase transitions in convex programs with random data. Inf. Inference 3(3), 224–294 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  3. Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Mach. Learn. 73(3), 243–272 (2008)

    Article  MATH  Google Scholar 

  4. Balan, R., Bodmann, B.G., Casazza, P.G., Edidin, D.: Painless reconstruction from magnitudes of frame coefficients. J. Fourier Anal. Appl. 15(4), 488–501 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bhatia, R.: Matrix Analysis. Springer, New York (2013)

    MATH  Google Scholar 

  6. Burer, S., Monteiro, R.D.C.: A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program. 95(2), 329–357 (2003). https://doi.org/10.1007/s10107-002-0352-8

    Article  MathSciNet  MATH  Google Scholar 

  7. Cai, T.T., Li, X., Ma, Z., et al.: Optimal rates of convergence for noisy sparse phase retrieval via thresholded wirtinger flow. Ann. Stat. 44(5), 2221–2251 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  8. Candès, E.J., Eldar, Y.C., Strohmer, T., Voroninski, V.: Phase retrieval via matrix completion. SIAM Rev. 57(2), 225–251 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  9. Candès, E.J., Li, X.: Solving quadratic equations via phaselift when there are about as many equations as unknowns. Found. Comput. Math. 14(5), 1017–1026 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  10. Candès, E.J., Li, X., Soltanolkotabi, M.: Phase retrieval from coded diffraction patterns. Appl. Comput. Harmon. Anal. 39(2), 277–299 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Candès, E.J., Li, X., Soltanolkotabi, M.: Phase retrieval via wirtinger flow: theory and algorithms. IEEE Trans. Inform. Theory 61(4), 1985–2007 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  12. Candès, E.J., Plan, Y.: Matrix completion with noise. Proc. IEEE 98(6), 925–936 (2010)

    Article  Google Scholar 

  13. Candès, E.J., Plan, Y.: A probabilistic and ripless theory of compressed sensing. IEEE Trans. Inform. Theory 57(11), 7235–7254 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  14. Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  15. Candès, E.J., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59, 1207–1223 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  16. Candès, E.J., Strohmer, T., Voroninski, V.: Phaselift: exact and stable signal recovery from magnitude measurements via convex programming. Commun. Pure Appl. Math. 66(8), 1241–1274 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  17. Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inform. Theory 51(12), 4203–4215 (2005). https://doi.org/10.1109/TIT.2005.858979

    Article  MathSciNet  MATH  Google Scholar 

  18. Candès, E.J., Tao, T.: The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Inform. Theory 56(5), 2053–2080 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  19. Chandrasekaran, V., Recht, B., Parrilo, P.A., Willsky, A.S.: The convex geometry of linear inverse problems. Found. Comput. Math. 12(6), 805–849 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  20. Chen, Y.: Incoherence-optimal matrix completion. IEEE Trans. Inf. Theory 61(5), 2909–2923 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  21. Chen, Y., Candès, E.J.: Solving random quadratic systems of equations is nearly as easy as solving linear systems. Commun. Pure Appl. Math. 70(5), 822–883 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  22. Chen, Y., Chi, Y., Fan, J., Ma, C., Yan, Y.: Noisy matrix completion: understanding statistical guarantees for convex relaxation via nonconvex optimization. arXiv preprint arXiv:1902.07698 (2019)

    Google Scholar 

  23. Choudhary, S., Mitra, U.: On identifiability in bilinear inverse problems. In: IEEE International Conference on Accoustics, Speech and Signal Processing, vol. 1(1) (2013)

    Google Scholar 

  24. Conca, A., Edidin, D., Hering, M., Vinzant, C.: An algebraic characterization of injectivity in phase retrieval. Appl. Comp. Harmonic Anal. 38(2), 346–356 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  25. Eldar, Y.C., Mendelson, S.: Phase retrieval: stability and recovery guarantees. Appl. Comput. Harmon. Anal. 36(3), 473–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  26. Fazel, M., Hindi, H., Boyd, S.P., et al.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the American Control Conference, vol. 6, pp. 4734–4739. Citeseer (2001)

    Google Scholar 

  27. Fengler, A., Haghighatshoar, S., Jung, P., Caire, G.: Non-Bayesian activity detection, large-scale fading coefficient estimation, and unsourced random access with a massive MIMO receiver. IEEE Trans. Inform. Theory 1–1 (2021). https://doi.org/10.1109/TIT.2021.3065291

  28. Fienup, C., Dainty, J.: Phase retrieval and image reconstruction for astronomy. Image Recovery Theory Appl. 231, 275 (1987)

    Google Scholar 

  29. Fienup, J.R.: Phase retrieval algorithms: a comparison. Appl. Opt. 21(15), 2758–2769 (1982)

    Article  Google Scholar 

  30. Fornasier, M., Rauhut, H., Ward, R.: Low-rank matrix recovery via iteratively reweighted least squares minimization. SIAM J. Optim. 21(4), 1614–1640 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  31. Foucart, S., Rauhut, H.: A Mathematical Introduction to Compressive Sensing, vol. 1. Birkhäuser, Basel (2013)

    Google Scholar 

  32. Ge, R., Lee, J.D., Ma, T.: Matrix completion has no spurious local minimum. In: Advances in Neural Information Processing Systems, pp. 2973–2981 (2016)

    Google Scholar 

  33. Geppert, J., Krahmer, F., Stöger, D.: Sparse power factorization: balancing peakiness and sample complexity. Adv. Comput. Math. 45, 1711–1728 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  34. Godard, G.H.: Self-recovering equalization and carrier tracking in two dimensional data communication systems. IEEE Trans. Commun. 28(11), 1867–1875 (1980). https://doi.org/10.1109/TCOM.1980.1094608

    Article  Google Scholar 

  35. Gordon, Y.: On Milman’s inequality and random subspaces which escape through a mesh in Rn. In: Lindenstrauss, J., Milman, V.D. (eds.) Geometric Aspects of Functional Analysis, pp. 84–106. Springer Berlin Heidelberg, Berlin, Heidelberg (1988)

    Chapter  Google Scholar 

  36. Gross, D.: Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inform. Theory 57(3), 1548–1566 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  37. Gross, D., Krahmer, F., Kueng, R.: A partial derandomization of phaselift using spherical designs. J. Fourier Anal. Appl. 21(2), 229–266 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  38. Gross, D., Krahmer, F., Kueng, R.: Improved recovery guarantees for phase retrieval from coded diffraction patterns. Appl. Comput. Harmon. Anal. 42(1), 37–64 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  39. Gross, D., Nesme, V.: Note on sampling without replacing from a finite collection of matrices. arXiv preprint arXiv:1001.2738 (2010)

    Google Scholar 

  40. Harrison, R.W.: Phase problem in crystallography. JOSA A 10(5), 1046–1055 (1993)

    Article  Google Scholar 

  41. Haykin: Blind Deconvolution. Prentice Hall, New Jersey (1994). http://www.getcited.org/pub/103095818

  42. Horstmeyer, R., Chen, R.Y., Ou, X., Ames, B., Tropp, J.A., Yang, C.: Solving ptychography with a convex relaxation. New J. of Phys. 17(5), 053044 (2015). https://doi.org/10.1088/1367-2630/17/5/053044

    Article  Google Scholar 

  43. Jain, P., Netrapalli, P., Sanghavi, S.: Low-rank matrix completion using alternating minimization. In: Proceedings of the Forty-fifth Annual ACM Symposium on Theory of Computing, STOC ’13, pp. 665–674. ACM, New York (2013). https://doi.org/10.1145/2488608.2488693

  44. Javanmard, A., Montanari, A.: Localization from incomplete noisy distance measurements. Found. Comput. Math. 13(3), 297–345 (2013). https://doi.org/10.1007/s10208-012-9129-5

    Article  MathSciNet  MATH  Google Scholar 

  45. Jung, P., Krahmer, F., Stöger, D.: Blind demixing and deconvolution at near-optimal rate. IEEE Trans. Inform. Theory 64(2), 704–727 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  46. Jung, P., Kueng, R., Mixon, D.G.: Derandomizing compressed sensing with combinatorial design. Front. Appl. Math. Stat. 5, 26 (2019). https://doi.org/10.3389/fams.2019.00026

    Article  Google Scholar 

  47. Kabanava, M., Kueng, R., Rauhut, H., Terstiege, U.: Stable low-rank matrix recovery via null space properties. Inf. Inference 5(4), 405–441 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  48. Kech, M.: Explicit frames for deterministic phase retrieval via phaselift. CoRR abs/1508.00522 (2015). http://arxiv.org/abs/1508.00522

  49. Kech, M., Krahmer, F.: Optimal injectivity conditions for bilinear inverse problems with applications to identifiability of deconvolution problems. SIAM J. Appl. Alg. Geom. 1(1), 20–37 (2017). https://doi.org/10.1137/16M1067469

    MathSciNet  MATH  Google Scholar 

  50. Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from a few entries. IEEE Trans. Inf. Theory 56(6), 2980–2998 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  51. Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from noisy entries. J. Mach. Learn. Res. 11, 2057–2078 (2010)

    MathSciNet  MATH  Google Scholar 

  52. Kliesch, M., Szarek, S.J., Jung, P.: Simultaneous structures in convex signal recovery—revisiting the convex combination of norms. Front. Appl. Math. Stat. 5 (2019). https://doi.org/10.3389/fams.2019.00023

  53. Klopp, O.: Noisy low-rank matrix completion with general sampling distribution. Bernoulli 20(1), 282–303 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  54. Koltchinskii, V., Lounici, K., Tsybakov, A.B., et al.: Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. Ann. Stat. 39(5), 2302–2329 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  55. Koltchinskii, V., Mendelson, S.: Bounding the smallest singular value of a random matrix without concentration. Int. Math. Res. Not. IMRN 2015(23), 12991–13008 (2015)

    MathSciNet  MATH  Google Scholar 

  56. Krahmer, F., Liu, Y.K.: Phase retrieval without small-ball probability assumptions. IEEE Trans. Inform. Theory 64(1), 485–500 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  57. Krahmer, F., Mendelson, S., Rauhut, H.: Suprema of chaos processes and the restricted isometry property. Commun. Pure Appl. Math. 67(11), 1877–1904 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  58. Krahmer, F., Stöger, D.: Blind deconvolution: Convex geometry and noise robustness. In: 52nd Annual Asilomar Conference on Signals, Systems, and Computers (2018)

    Google Scholar 

  59. Krahmer, F., Stöger, D.: Complex phase retrieval from Subgaussian measurements. J. Fourier Anal. Appl. 26(6), 27 (2020). Id/No 89

    Google Scholar 

  60. Krahmer, F., Stöger, D.: On the convex geometry of blind deconvolution and matrix completion. Commun. Pure Appl. Math. (2020)

    Google Scholar 

  61. Kueng, R.: Low rank matrix recovery from few orthonormal basis measurements. In: 2015 International Conference on Sampling Theory and Applications (SampTA), pp. 402–406 (2015)

    Google Scholar 

  62. Kueng, R., Gross, D., Krahmer, F.: Spherical designs as a tool for derandomization: The case of phaselift. In: 2015 International Conference on Sampling Theory and Applications (SampTA), pp. 192–196 (2015). https://doi.org/10.1109/SAMPTA.2015.7148878

  63. Kueng, R., Jung, P.: Robust nonnegative sparse recovery and the nullspace property of 0/1 measurements. IEEE Trans. Inf. Theory 64(2), 689–703 (2018). https://doi.org/10.1109/TIT.2017.2746620

    Article  MathSciNet  MATH  Google Scholar 

  64. Kueng, R., Rauhut, H., Terstiege, U.: Low rank matrix recovery from rank one measurements. Appl. Comput. Harmon. Anal. 42(1), 88–116 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  65. Kueng, R., Zhu, H., Gross, D.: Low rank matrix recovery from Clifford orbits. arXiv preprint arXiv:1610.08070 (2016)

    Google Scholar 

  66. Kümmerle, C., Sigl, J.: Harmonic mean iteratively reweighted least squares for low-rank matrix recovery. J. Mach. Learn. Res. 19, 49 (2018)

    MathSciNet  MATH  Google Scholar 

  67. Lee, K., Li, Y., Junge, M., Bresler, Y.: Blind recovery of sparse signals from subsampled convolution. IEEE Trans. Inform. Theory 63(2), 802–821 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  68. Li, Y., Lee, K., Bresler, Y.: A unified framework for identifiability analysis in bilinear inverse problems with applications to subspace and sparsity models. IEEE Trans. Inf. Theory 63(2), 822–842 (2017)

    Article  MATH  Google Scholar 

  69. Ling, S., Strohmer, T.: Blind deconvolution meets blind demixing: algorithms and performance bounds. IEEE Trans. Inform. Theory 63(7), 4497–4520 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  70. Ma, C., Wang, K., Chi, Y., Chen, Y.: Implicit regularization in nonconvex statistical estimation: Gradient descent converges linearly for phase retrieval, matrix completion and blind deconvolution. arXiv preprint arXiv:1711.10467 (2017)

    Google Scholar 

  71. Mendelson, S.: Learning without concentration. In: Conference on Learning Theory, pp. 25–39 (2014)

    Google Scholar 

  72. Miao, J., Charalambous, P., Kirz, J., Sayre, D.: Extending the methodology of x-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens. Nature 400(6742), 342–344 (1999)

    Article  Google Scholar 

  73. Millane, R.P.: Phase retrieval in crystallography and optics. JOSA A 7(3), 394–411 (1990)

    Article  Google Scholar 

  74. Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  75. Oymak, S., Jalali, A., Fazel, M., Eldar, Y.C., Hassibi, B.: Simultaneously structured models with application to sparse and low-rank matrices. IEEE Trans. Inform. Theory 61(5), 2886–2908 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  76. Recht, B.: A simpler approach to matrix completion. J. Mach. Learn. Res. 12(Dec), 3413–3430 (2011)

    MathSciNet  MATH  Google Scholar 

  77. Rennie, J.D.M., Srebro, N.: Fast maximum margin matrix factorization for collaborative prediction. In: Proceedings of the 22Nd International Conference on Machine Learning, ICML ’05, pp. 713–719. ACM, New York (2005). https://doi.org/10.1145/1102351.1102441

  78. Rodenburg, J.M.: Ptychography and related diffractive imaging methods. Adv. Imaging Electron Phys. 150, 87–184 (2008)

    Article  Google Scholar 

  79. Stockham, T., Cannon, T., Ingebretsen, R.: Blind deconvolution through digital signal processing. Proc. IEEE 63(4), 678–692 (1975). https://doi.org/10.1109/PROC.1975.9800

    Article  Google Scholar 

  80. Stöger, D., Jung, P., Krahmer, F.: Blind deconvolution and compressed sensing. In: 4th International Workshop on Compressed Sensing Theory and Its Applications to Radar, Sonar and Remote Sensing (CoSeRa), pp. 24–27. IEEE (2016)

    Google Scholar 

  81. Stöger, D., Jung, P., Krahmer, F.: Blind demixing and deconvolution with noisy data at near optimal rate. In: Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 10394 (2017)

    Google Scholar 

  82. Stöger, D., Jung, P., Krahmer, F.: Blind demixing and deconvolution with noisy data: near-optimal rate. In: WSA 2017; 21th International ITG Workshop on Smart Antennas, pp. 1–5. VDE (2017)

    Google Scholar 

  83. Sun, R., Luo, Z.Q.: Guaranteed matrix completion via non-convex factorization. IEEE Trans. Inform. Theory 62(11), 6535–6579 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  84. Tropp, J.A.: User-friendly tail bounds for sums of random matrices. Found. Comput. Math. 12(4), 389–434 (2012). https://doi.org/10.1007/s10208-011-9099-z

    Article  MathSciNet  MATH  Google Scholar 

  85. Tropp, J.A.: Convex recovery of a structured signal from independent random linear measurements. In: Sampling Theory, A Renaissance. Compressive Sensing and Other Developments, pp. 67–101. Birkhäuser/Springer, Cham (2015)

    Google Scholar 

  86. Tropp, J.A., Yurtsever, A., Udell, M., Cevher, V.: Practical sketching algorithms for low-rank matrix approximation. SIAM J. Matrix Anal. Appl. 38(4), 1454–1485 (2017). https://doi.org/10.1137/17M1111590

    Article  MathSciNet  MATH  Google Scholar 

  87. Walk, P., Jung, P., Hassibi, B.: MOCZ for blind short-packet communication: basic principles. IEEE Trans. Wirel. Commun. 18(11), 5080–5097 (2019). https://doi.org/10.1109/TWC.2019.2932668

    Article  Google Scholar 

  88. Walk, P., Jung, P., Pfander, G.E., Hassibi, B.: Ambiguities on convolutions with applications to phase retrieval. In: Matthews, M.B. (ed.) 50th Asilomar Conference on Signals, Systems and Computers, ACSSC 2016, Pacific Grove, CA, USA, November 6–9, 2016, pp. 1228–1234. IEEE (2016). https://doi.org/10.1109/ACSSC.2016.7869569

  89. Walther, A.: The question of phase retrieval in optics. J. Mod. Opt. 10(1), 41–49 (1963)

    MathSciNet  Google Scholar 

  90. Watson, G.: Characterization of the subdifferential of some matrix norms. Linear Algebra Appl. 170, 33–45 (1992). https://doi.org/10.1016/0024-3795(92)90407-2

    Article  MathSciNet  MATH  Google Scholar 

  91. Yurtsever, A., Udell, M., Tropp, J.A., Cevher, V.: Sketchy decisions: convex low-rank matrix optimization with optimal storage. In: Singh, A., Zhu, X.J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20–22 April 2017, Fort Lauderdale, FL, USA, Proceedings of Machine Learning Research, vol. 54, pp. 1188–1196. PMLR (2017). http://proceedings.mlr.press/v54/yurtsever17a.html

Download references

Acknowledgements

This work was prepared as part of the Priority Programme Compressed Sensing in Information Processing (SPP 1798) of the German Research Foundation (DFG). The authors would like to thank Julia Kostina for finding a minor mistake in the first version of the manuscript.

Appendix: Descent Cone Elements Are Effectively Low Rank

Lemma 2.2

Suppose that \(Z \in \mathbb {C}^{n_1 \times n_2}\) is contained in the nuclear norm descent cone of a rank-r matrix \(X \in \mathbb {C}^{n_1 \times n_2}\) . Then,

$$\displaystyle \begin{aligned} \|Z\|{}_* \leq \left(1+\sqrt{2}\right) \sqrt{r} \|Z\|{}_F. \end{aligned}$$

The constant \(1+\sqrt {2}\) is not optimal and could be further improved by a more refined analysis. The argument presented here is novel and inspired by dual certificate arguments reviewed in Sect. 2.3. It also requires a rectangular generalization of the pinching inequality for Hermitian matrices, see, e.g., [5, Problem II.5.4]

Theorem 2.13 ((Hermitian) Pinching Inequality)

Let \(P_1,\ldots ,P_L \subset \mathbb {H}_n\) be a resolution of the identity ( \(P_l^2=P_l\) and ∑l P l = Id). Then,

$$\displaystyle \begin{aligned} \| X \|{}_* \geq \sum_{l=1}^L \left\| P_l X P_l \right\|{}_* \quad \mathit{\text{for every}}\ X \in \mathbb{H}_n. \end{aligned}$$

We can extend pinching to general rectangular matrices by embedding them within a larger block matrix. The self-adjoint dilation of \(Z \in \mathbb {C}^{n_1 \times n_2}\) is

$$\displaystyle \begin{aligned} \mathcal{T}(Z) = \left( \begin{array}{cc} 0 & Z \\ Z^* & 0 \end{array} \right) \in \mathbb{H}_{n_1 + n_2}. \end{aligned}$$

Dilations preserve spectral information. In particular,

$$\displaystyle \begin{aligned} \| \mathcal{T}(Z) \|{}_* =&\mathrm{tr} \left( \sqrt{\mathcal{T}(Z)^* \mathcal{T}(Z)} \right) = \mathrm{tr} \left( \begin{array}{cc} \sqrt{ZZ^*} & 0 \\ 0 & \sqrt{Z^* Z} \end{array} \right) \\ =& \mathrm{tr} (\sqrt{Z Z^*}) + \mathrm{tr}(\sqrt{Z^* Z}) = 2 \| Z \|{}_*. {} \end{aligned} $$
(2.38)

For simplicity, we only formulate and prove our generalization of the Hermitian pinching inequality for identity resolutions with two elements each. Statement and proof do, however, readily extend to more general resolutions with compatible dimensions.

Corollary 2.3 (Pinching for Non-symmetric Matrices)

Let \(P,P^\perp \in \mathbb {H}_{n_1}\) and \(Q,Q^\perp \in \mathbb {H}_{n_2}\) be two resolutions of the identity. Then,

$$\displaystyle \begin{aligned} \|X \|{}_* \geq \|P X Q \|{}_* + \left\| P^\perp X Q^\perp \right\|{}_* \quad \mathit{\text{for all}}\ X \in \mathbb{C}^{n_1 \times n_2}. \end{aligned}$$

Proof (Corollary 2.3)

Use Eq. (2.38) to relate the nuclear norm of X to the nuclear norm of its self-adjoint dilation:

$$\displaystyle \begin{aligned} 2 \|X \|{}_* = \| \mathcal{T}(X) \|{}_* = \left\| \left( \begin{array}{cc} 0 & X \\ X^* & 0 \end{array} \right) \right\|{}_*. \end{aligned}$$

Next, we combine \(P,P^\perp \in \mathbb {H}_{n_1}\) and \(Q,Q^\perp \in \mathbb {H}_{n_2}\) to obtain a resolution of the identity with compatible dimension:

$$\displaystyle \begin{aligned} \left( \begin{array}{cc} P & 0 \\ 0 & Q \end{array} \right), \; \left( \begin{array}{cc} P^\perp & 0 \\ 0 & Q^\perp \end{array} \right) \in \mathbb{H}_{n_1+n_2}. \end{aligned} $$

Since everything is Hermitian, we can apply Theorem 2.13 (original pinching) with respect to this resolution of the identity to the nuclear norm of the s.a. dilation:

$$\displaystyle \begin{aligned} \left\| \left( \begin{array}{cc} 0 & X \\ X^* & 0 \end{array} \right) \right\|{}_* \geq & \left\| \left( \begin{array}{cc} P & 0 \\ 0 & Q \end{array} \right) \left( \begin{array}{cc} 0 & X \\ X^* & 0 \end{array} \right) \left( \begin{array}{cc} P & 0 \\ 0 & Q \end{array} \right) \right\|{}_* + \left\| \left( \begin{array}{cc} P^\perp & 0 \\ 0 & Q^\perp \end{array} \right) \left( \begin{array}{cc} 0 & X \\ X^* & 0 \end{array} \right) \left( \begin{array}{cc} P^\perp & 0 \\ 0 & Q^\perp \end{array} \right) \right\|{}_*\\ = & \left\| \left( \begin{array}{cc} 0 & PXQ \\ QX^*P & 0 \end{array} \right) \right\|{}_* + \left\| \left( \begin{array}{cc} 0 & P^{\perp}XQ^{\perp} \\ Q^{\perp}X^*P^{\perp} & 0 \end{array} \right) \right\|{}_*. \end{aligned} $$

We can now recognize self-adjoint dilations of two rectangular matrices. Using Eq. (2.38) implies

$$\displaystyle \begin{aligned} \| \mathcal{T}(X) \|{}_* \geq & \| \mathcal{T}(PXQ) \|{}_* + \| \mathcal{T}(P^\perp X Q^\perp) \|{}_* = 2 \| PXQ \|{}_* + 2 \|P^\perp X Q^\perp \|{}_*. \end{aligned} $$

□Next, the concept of sign functions of real numbers is extendable to non-Hermitian matrices. Let \(X \in \mathbb {C}^{n_1 \times n_2}\) be a rectangular matrix with SVD X = U ΣV ∗. We define its sign matrix to be \(\mathrm {sign}(X) = U V^* \in \mathbb {C}^{n_1 \times n_2}\). Note that this sign matrix is unitary and obeys

$$\displaystyle \begin{aligned} \langle \mathrm{sign}(X),X \rangle_F = \mathrm{tr} \left( (UV^*)^* U \Sigma V^* \right) = \mathrm{tr}(\Sigma) = \|X \|{}_*. \end{aligned}$$

The last ingredient is the dual formulation of the nuclear norm:

$$\displaystyle \begin{aligned} \| X \|{}_* = \max_{\|U\| \leq 1}\left| \langle U,X \rangle \right| = \max_{U \mathrm{unitary}} \left| \langle U, X \rangle \right|. \end{aligned}$$

Proof (Lemma 2.2)

By assumption, \(Z \in \mathbb {C}^{n_1 \times n_2}\) is contained in the descent cone of a rank-r matrix X. This implies that there exists τ > 0 such that ∥X∥∗≥∥X + τZ∥∗. Apply an SVD X = U ΣV ∗ and use it to define r-dimensional orthoprojectors \(P=UU^* \in \mathbb {H}_{n_1}\), \(Q=VV^* \in \mathbb {H}_{n_2}\), as well as their orthocomplements P ⊥ = Id − P and Q ⊥ = Id − Q. Use them to define the matrix-valued projections

$$\displaystyle \begin{aligned} \mathcal{P}_{T_X}^\perp:\; Z \mapsto P^\perp Z Q^\perp \quad \text{and} \quad \mathcal{P}_{T_X}:\; \mapsto Z-\mathcal{P}_{T_X}^\perp (Z) = PZ + ZQ - PZQ \end{aligned} $$

such that \(Z=\mathcal {P}_{T_X}^\perp (Z) + \mathcal {P}_{T_X} (Z) = Z_{T_X}^\perp + Z_{T_X}\) and, in particular, \(X_{T_X}^\perp =0\) and \(X_{T_X} = X\). In words, \(\mathcal {P}_{T_X}\) projects \(\mathbb {C}^{n_1 \times n_2}\) onto a subspace whose compression to the kernel of X vanishes identically, namely the tangent space of X (as defined in (2.23)). Moreover, for every \(Z \in \mathbb {C}^{n_1 \times n_2}\),

$$\displaystyle \begin{aligned} \mathrm{rk} \left( Z_{T_X} \right)&= \mathrm{rk} \left( PZ+(P+P^\perp) ZQ-PZQ \right) = \mathrm{rk} \left( PZ + P^\perp ZQ \right) \\ &\leq \mathrm{rk} \left( PZ \right) + \mathrm{rk} \left( P^\perp Z Q \right) \leq \mathrm{rk}(P) + \mathrm{rk}(Q) =2r, {} \end{aligned} $$
(2.39)

because matrix rank is subadditive and cannot increase under matrix products. Corollary 2.3 (pinching)—with respect to P and Q—and the descent cone property of Z together imply

$$\displaystyle \begin{aligned} \| X \|{}_* \geq & \left\| X + \tau Z \right\|{}_* \geq \left\| P (X+\tau Z) Q \right\|{}_* + \left\| P^\perp (X + \tau Z) Q^\perp \right\|{}_* \\ =& \left\| X + \tau P Z Q \right\|{}_* + \tau \left\| P^\perp Z Q^\perp \right\|{}_* \\ =& \left| \langle \mathrm{sign}(X+\tau PZQ), X + \tau PZQ \rangle_F \right| + \tau \left\| P^\perp Z Q^\perp \right\|{}_* \\ \geq & \left| \langle \mathrm{sign} (X), X \rangle_F + \tau \langle \mathrm{sign}(X),P ZQ \rangle_F \right| + \tau \|P^\perp Z Q^\perp \|{}_* \\ \geq &\| X \|{}_* + \tau \left(- \left| \langle \mathrm{sign}(X),PZQ\rangle_F \right|+ \left\| P^\perp Z Q^\perp \right\|{}_* \right). \end{aligned} $$

Since τ > 0, this chain of inequalities can only be valid if

$$\displaystyle \begin{aligned} \left\| Z_{T_X}^\perp \right\|{}_* = \left\| P^\perp Z Q^\perp \right\|{}_* \leq \left| \langle \mathrm{sign}(X),P Z Q \rangle_F \right| \leq \| \mathrm{sign}(X) \| \| PZQ \|{}_* \leq \sqrt{r} \| PZQ \|{}_F \end{aligned}$$

because both P and Q are rank-r projectors. We can combine this with a decomposition \(Z=Z_{T_X}^\perp + Z_{T_X}\) and Eq. (2.39) to conclude

$$\displaystyle \begin{aligned} \| Z \|{}_* \leq & \left\| Z_{T_X}^\perp \right\|{}_* + \left\| Z_{T_X} \right\|{}_* \leq \sqrt{r} \|P Z Q \|{}_F + \sqrt{\mathrm{rank}(Z_{T_X})} \|Z_{T_X} \|{}_F \\ \leq & \sqrt{r} \|Z \|{}_F + \sqrt{2r} \|Z \|{}_F = \left(1+\sqrt{2}\right) \sqrt{r} \|Z \|{}_F \end{aligned} $$

because both Z↦PZQ and \(Z \mapsto Z_{T_X}\) are contractions with respect to the Frobenius norm. □

Author information

Authors and Affiliations

  1. Department of Mathematics, Technical University of Munich, Garching, Germany

    Tim Fuchs

  2. Institute for Theoretical Physics, University of Cologne, Cologne, Germany

    David Gross

  3. Communications and Information Theory Group, Technische Universität Berlin, Berlin, Germany

    Peter Jung

  4. Data Science in Earth Observation, Technical University of Munich, Munich, Germany

    Peter Jung

  5. Department of Mathematics, Technical University of Munich, Garching, Germany

    Felix Krahmer

  6. Munich Data Science Institute, Technical University of Munich, Garching, Germany

    Felix Krahmer

  7. Institute for Integrated Circuits, Johannes Kepler University Linz, Linz, Austria

    Richard Kueng

  8. Department of Mathematics, KU Eichstätt-Ingolstadt, Eichstätt, Germany

    Dominik Stöger

Authors
  1. Tim Fuchs
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Felix Krahmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Richard Kueng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dominik Stöger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tim Fuchs .

Editor information

Editors and Affiliations

  1. Mathematisches Institut, Ludwig Maximilian University of Munich, München, Bayern, Germany

    Gitta Kutyniok

  2. Lehrstuhl für Mathematik, RWTH Aachen University, Aachen, Nordrhein-Westfalen, Germany

    Holger Rauhut

  3. Lehrstuhl für Mathematik, RWTH Aachen University, Aachen, Nordrhein-Westfalen, Germany

    Robert J. Kunsch

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Fuchs, T., Gross, D., Jung, P., Krahmer, F., Kueng, R., Stöger, D. (2022). Proof Methods for Robust Low-Rank Matrix Recovery. In: Kutyniok, G., Rauhut, H., Kunsch, R.J. (eds) Compressed Sensing in Information Processing. Applied and Numerical Harmonic Analysis. Birkhäuser, Cham. https://doi.org/10.1007/978-3-031-09745-4_2

Download citation

Publish with us

Policies and ethics

Search

Navigation