Comparison of Matrix Norm Sparsification

Abstract

A well-known approach in the design of efficient algorithms, called matrix sparsification, approximates a matrix A with a sparse matrix $A'$. Achlioptas and McSherry (J ACM 54(2):9-es, 2007) initiated a long line of work on spectral-norm sparsification, which aims to guarantee that $\Vert A'-A\Vert \le \epsilon \Vert A\Vert$ for error parameter $\epsilon >0$. Various forms of matrix approximation motivate considering this problem with a guarantee according to the Schatten p-norm for general p, which includes the spectral norm as the special case $p=\infty$. We investigate the relation between fixed but different $p\ne q$, that is, whether sparsification in the Schatten p-norm implies (existentially and/or algorithmically) sparsification in the Schatten $q\text {-norm}$ with similar sparsity. An affirmative answer could be tremendously useful, as it will identify which value of p to focus on. Our main finding is a surprising contrast between this question and the analogous case of $\ell _p$-norm sparsification for vectors: For vectors, the answer is affirmative for $p<q$ and negative for $p>q$, but for matrices we answer negatively for almost all sufficiently distinct $p\ne q$. In addition, our explicit constructions may be of independent interest.

Appendices

Appendix

A Proof of Lemma 1.6

In this section, we prove Lemma 1.6.

Lemma 1.6

For all PSD matrices $A\in \mathbb {R}^{n\times n}$ and $\epsilon >0$, every $\epsilon$-spectral approximation $A'$ of A is also an $(\epsilon ,S_p)$-norm approximation of A, simultaneously for all $p\ge 1$.

Proof

Let $A'\in \mathbb {R}^{n\times n}$ be an $\epsilon$-spectral approximation of A, i.e., $-\epsilon A \preceq A'-A \preceq \epsilon A$. Observe that the matrix $A'-A$ is symmetric. Let the eigendecomposition of $A'-A$ be $UDU^\top$, including zero eigenvalues so that $U\in \mathbb {R}^{n\times n}$ is unitary. Denote the i-th column of U by $u_i$ (which is a normalized eigenvector). Then

$$\begin{aligned} \Vert A'-A\Vert _{S_p}^p&= \sum _i \left| u_i^\top (A'-A)u_i \right| ^p \le \sum _i |u_i^\top (\epsilon A)u_i|^p = \epsilon ^p \sum _i \left( u_i^\top A u_i \right) ^p\\&= \epsilon ^p \left\| {\text {diag}}\left( U^\top A U \right) \right\| _{S_p}^p \le \epsilon ^p \Vert U^\top A U \Vert _{S_p}^p = \epsilon ^p \Vert A\Vert _{S_p}^p, \end{aligned}$$

where ${\text {diag}}(U^\top A U)$ is a diagonal matrix with the same diagonal as $U^\top A U$ (and zeros otherwise), and the last inequality holds by Lemma 3.4 (pinching inequality [6]). $\square$