A Universal Expectation Bound on Empirical Projections of Deformed Random Matrices

Abstract

Let $C$ be a real-valued $M\times M$ matrix with singular values $\lambda _1\ge \cdots \ge \lambda _M$, and $E$ a random matrix of centered i.i.d. entries with finite fourth moment. In this paper, we give a universal upper bound on the expectation of $||\hat{\pi }_rX||_{S_2}^2-||\pi _rX||^2_{S_2}$, where $X:=C+E$ and $\hat{\pi }_r$ (resp. $\pi _r$) is a rank-$r$ projection maximizing the Hilbert–Schmidt norm $||{\tilde{\pi }}_rX||_{S_2}$ (resp. $||{\tilde{\pi }}_rC||_{S_2}$) over the set $\mathcal{S }_{M,r}$ of all orthogonal rank-$r$ projections. This result is a generalization of a theorem for Gaussian matrices due to [7]. Our approach differs substantially from the techniques of the mentioned article. We analyze $||\hat{\pi }_rX||_{S_2}^2-||\pi _rX||^2_{S_2}$ from a rather deterministic point of view by an upper bound on $||\hat{\pi }_rX||_{S_2}^2-||\pi _rX||^2_{S_2}$, whose randomness is totally determined by the largest singular value of $E$.