pp 1–17 | Cite as

On Using Toeplitz and Circulant Matrices for Johnson–Lindenstrauss Transforms

  • Casper Benjamin FreksenEmail author
  • Kasper Green Larsen
Part of the following topical collections:
  1. Special Issue: Algorithms and Computation


The Johnson–Lindenstrauss lemma is one of the cornerstone results in dimensionality reduction. A common formulation of it, is that there exists a random linear mapping \(f : {\mathbb {R}}^n \rightarrow {\mathbb {R}}^m\) such that for any vector \(x \in {\mathbb {R}}^n\), f preserves its norm to within \((1 {\pm } \varepsilon )\) with probability \(1 - \delta \) if \(m = \varTheta (\varepsilon ^{-2} \lg (1/\delta ))\). Much effort has gone into developing fast embedding algorithms, with the Fast Johnson–Lindenstrauss transform of Ailon and Chazelle being one of the most well-known techniques. The current fastest algorithm that yields the optimal \(m = {\mathcal {O}}(\varepsilon ^{-2}\lg (1/\delta ))\) dimensions has an embedding time of \({\mathcal {O}}(n \lg n + \varepsilon ^{-2} \lg ^3 (1/\delta ))\). An exciting approach towards improving this, due to Hinrichs and Vybíral, is to use a random \(m \times n\) Toeplitz matrix for the embedding. Using Fast Fourier Transform, the embedding of a vector can then be computed in \({\mathcal {O}}(n \lg m)\) time. The big question is of course whether \(m = {\mathcal {O}}(\varepsilon ^{-2} \lg (1/\delta ))\) dimensions suffice for this technique. If so, this would end a decades long quest to obtain faster and faster Johnson–Lindenstrauss transforms. The current best analysis of the embedding of Hinrichs and Vybíral shows that \(m = {\mathcal {O}}(\varepsilon ^{-2}\lg ^2 (1/\delta ))\) dimensions suffice. The main result of this paper, is a proof that this analysis unfortunately cannot be tightened any further, i.e., there exist vectors requiring \(m = \varOmega (\varepsilon ^{-2} \lg ^2 (1/\delta ))\) for the Toeplitz approach to work.


Dimensionality reduction Johnson–Lindenstrauss Toeplitz matrices 

Mathematics Subject Classification

68Q25 68W40 



  1. 1.
    Achlioptas, D.: Database-friendly random projections: Johnson–Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66(4), 671–687 (2003). MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Ailon, N., Chazelle, B.: The fast Johnson–Lindenstrauss transform and approximate nearest neighbors. SIAM J. Comput. 39(1), 302–322 (2009). MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Ailon, N., Liberty, E.: Fast dimension reduction using Rademacher series on dual BCH codes. Discrete Comput. Geom. 42(4), 615–630 (2009). MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Ailon, N., Liberty, E.: An almost optimal unrestricted fast Johnson–Lindenstrauss transform. ACM Trans. Algorithms 9(3), 21:1–21:12 (2013). MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Blocki, J., Blum, A., Datta, A., Sheffet, O.: The Johnson–Lindenstrauss transform itself preserves differential privacy. In: Proceedings of the 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science, FOCS ’12, pp. 410–419. IEEE Computer Society, Washington, DC (2012).
  6. 6.
    Boutsidis, C., Zouzias, A., Mahoney, M.W., Drineas, P.: Randomized dimensionality reduction for \(k\)-means clustering. IEEE Trans. Inf. Theory 61(2), 1045–1062 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Candes, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006). MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Cohen, M.B., Elder, S., Musco, C., Musco, C., Persu, M.: Dimensionality reduction for \(k\)-means clustering and low rank approximation. In: Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing, STOC ’15, pp. 163–172. ACM, New York (2015).
  9. 9.
    Dasgupta, A., Kumar, R., Sarlos, T.: A sparse Johnson–Lindenstrauss transform. In: Proceedings of the Forty-Second ACM Symposium on Theory of Computing, STOC ’10, pp. 341–350. ACM, New York (2010).
  10. 10.
    Dasgupta, S., Gupta, A.: An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct. Algorithms 22(1), 60–65 (2003). MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Har-Peled, S., Indyk, P., Motwani, R.: Approximate nearest neighbor: towards removing the curse of dimensionality. Theory Comput 8(14), 321–350 (2012). MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Hinrichs, A., Vybíral, J.: Johnson–Lindenstrauss lemma for circulant matrices. Random Struct. Algorithms 39(3), 391–398 (2011). MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Indyk, P.: Algorithmic applications of low-distortion geometric embeddings. In: Proceedings of the 42nd IEEE Symposium on Foundations of Computer Science, FOCS ’01, pp. 10–33. IEEE Computer Society, Washington, DC (2001).
  15. 15.
    Johnson, W.B., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. Contemp. Math. 26, 189–206 (1984). MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Kane, D.M., Nelson, J.: Sparser Johnson–Lindenstrauss transforms. J. ACM 61(1), 4:1–4:23 (2014). MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Krahmer, F., Ward, R.: New and improved Johnson–Lindenstrauss embeddings via the restricted isometry property. SIAM J. Math. Anal. 43(3), 1269–1281 (2011)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Larsen, K.G., Nelson, J.: Optimality of the Johnson–Lindenstrauss lemma. In: Proceedings of the 2017 IEEE 58th Annual Symposium on Foundations of Computer Science, FOCS ’17. IEEE Computer Society, Washington, DC (2017)Google Scholar
  19. 19.
    Muthukrishnan, S.: Data streams: algorithms and applications. In: Foundations and Trends™ in Theoretical Computer Science, vol. 1, no 2. Publishers Inc., Hanover (2005). MathSciNetCrossRefGoogle Scholar
  20. 20.
    Spielman, D.A., Srivastava, N.: Graph sparsification by effective resistances. SIAM J. Comput. 40(6), 1913–1926 (2011). MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Vempala, S.S.: The random projection method. In: DIMACS—Series in Discrete Mathematics and Theoretical Computer Science, vol. 65. American Mathematical Society, Providence (2004). CrossRefGoogle Scholar
  22. 22.
    Vu, K., Poirion, P.L., Liberti, L.: Using the Johnson–Lindenstrauss lemma in linear and integer programming. ArXiv e-prints (2015)Google Scholar
  23. 23.
    Vybíral, J.: A variant of the Johnson–Lindenstrauss lemma for circulant matrices. J. Funct. Anal. 260(4), 1096–1105 (2011). MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Weinberger, K., Dasgupta, A., Langford, J., Smola, A., Attenberg, J.: Feature hashing for large scale multitask learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, pp. 1113–1120. ACM, New York (2009).
  25. 25.
    Woodruff, D.P.: Sketching as a tool for numerical linear algebra. In: Foundations and Trends™ in Theoretical Computer Science, vol. 10, no 1–2. Publishers Inc., Hanover (2014). CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Aarhus UniversityAarhus NDenmark

Personalised recommendations