Advertisement

An Elementary Approach to the Problem of Column Selection in a Rectangular Matrix

  • Stéphane ChrétienEmail author
  • Sébastien Darses
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10710)

Abstract

The problem of extracting a well conditioned submatrix from any rectangular matrix (with e.g. normalized columns) has been a subject of extensive research with applications to machine learning (rank revealing factorization, sparse solutions to least squares regression problems, clustering, \(\cdots \)), optimisation (low stretch spanning trees, \(\cdots \)), and is also connected with problems in functional and harmonic analysis (Bourgain-Tzafriri restricted invertibility problem).

In this paper, we provide a deterministic algorithm which extracts a submatrix \(X_S\) from any matrix X with guaranteed individual lower and upper bounds on each singular value of \(X_S\). We are also able to deduce a slightly weaker (up to a \(\log \)) version of the Bourgain-Tzafriri theorem as an immediate side result.

We end the paper with a description of how our method applies to the analysis of a large data set and how its numerical efficiency compares with the method of Spieman and Srivastava.

Keywords

Bourgain Tzafriri theorem Restricted invertibility Column selection problems 

References

  1. 1.
    Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027–1035. Society for Industrial and Applied Mathematics (2007)Google Scholar
  2. 2.
    Avron, H., Boutsidis, C.: Faster subset selection for matrices and applications. SIAM J. Matrix Anal. Appl. 34(4), 1464–1499 (2013)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bourgain, J., Tzafriri, L.: Invertibility of “large” submatrices with applications to the geometry of Banach spaces and harmonic analysis. Israel J. Math. 57(2), 137–224 (1987)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Boutsidis, C., Drineas, P., Magdon-Ismail, M.: Near-optimal column-based matrix reconstruction. SIAM J. Comput. 43(2), 687–717 (2014)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Boutsidis, C., Drineas, P., Mahoney, M.: On selecting exactly k columns from a matrix (2008, in press)Google Scholar
  6. 6.
    d’Aspremont, A., Ghaoui, L.E., Jordan, M.I., Lanckriet, G.R.: A direct formulation for sparse PCA using semidefinite programming. In: Advances in Neural Information Processing Systems, pp. 41–48 (2005)Google Scholar
  7. 7.
    Farahat, A.K., Elgohary, A., Ghodsi, A., Kamel, M.S.: Greedy column subset selection for large-scale data sets. Knowl. Inform. Syst. 45(1), 1–34 (2015)CrossRefGoogle Scholar
  8. 8.
    Mallat, S.: Group invariant scattering. Commun. Pure Appl. Math. 65(10), 1331–1398 (2012)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Bruna, J., Mallat, S.: Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1872–1886 (2013)CrossRefGoogle Scholar
  10. 10.
    Naor, A.: Sparse quadratic forms and their geometric applications [following Batson, Spielman and Srivastava]. Séminaire Bourbaki: Vol. 2010/2011. Exposés 1027–1042. Astérisque No. 348 (2012), Exp. No. 1033, viii, 189–217Google Scholar
  11. 11.
    Nikolov, A.: Randomized rounding for the largest simplex problem. In: Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, pp. 861–870 (2015)Google Scholar
  12. 12.
    Nikolov, A., Singh, M.: Maximizing determinants under partition constraints. In: STOC 2016, pp. 192–201 (2016)Google Scholar
  13. 13.
    Spielman, D.A., Srivastava, N.: An elementary proof of the restricted invertibility theorem. Israel J. Math. 190, 83–91 (2012)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Tropp, J.A.: The random paving property for uniformly bounded matrices. Studia Math. 185(1), 67–82 (2008)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Tropp, J.A.: Norms of random submatrices and sparse approximation. C. R. Acad. Sci. Paris, Ser. I 346, 1271–1274 (2008)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Tropp, J.A.: Column subset selection, matrix factorization, and eigenvalue optimization. In: Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 978–986. Society for Industrial and Applied Mathematics (2009)Google Scholar
  17. 17.
    Vershynin, R.: John’s decompositions: selecting a large part. Israel J. Math. 122, 253–277 (2001)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Youssef, P.: A note on column subset selection. Int. Math. Res. Not. IMRN 23, 6431–6447 (2014)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.National Physical LaboratoryTeddingtonUK
  2. 2.Aix Marseille Université, CNRS, Centrale Marseille, I2M UMR 7373MarseilleFrance

Personalised recommendations