# Random Projection

• Jianzhong Wang
Chapter

## Abstract

Principal component analysis (PCA) is a very important linear method for dimensionality reduction. It measures data distortion globally by the Frobenius norm of the matrix of data difference. The reduced data of PCA consists of several leading eigenvectors of the covariance matrix of the data set. Hence, PCA may not preserve the local separation of the original data. To respect local properties of data in dimensionality reduction (DR), we employ Lipschitz embedding. Random projection is a powerful method to construct Lipschitz mappings to realize dimensionality reduction with a high probability. Random projection does not introduce a significant distortion when the dimension and cardinality of data both are large. It randomly projects the original high-dimensional data into a lower-dimensional subspace. Because the projection costs linear computational time, the method is computationally efficient, yet produces sufficient accuracy with a high probability. In Section 7.1, we give a review of Lipschitz embedding. In Section 7.2, we introduce random matrices and random projection algorithms. In Section 7.3, the justification of the validity of random projection is presented in detail. Particularly, Johnson and Lindenstrauss Lemma will be proved in this section. The applications of random projection are given in Section 7.4.

## Keywords

Principal Component Analysis Dimensionality Reduction Face Recognition Singular Value Decomposition Random Matrice
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## References

1. [1]
Johnson, W.B., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. AMS Contemp. Math. 26, 189–206 (1984).
2. [2]
Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proc. 30th Symp. on Theory of Computing, vol. 13, pp. 604–613 (1998).Google Scholar
3. [3]
Achlioptas, D.: Database-friendly random projections. Proc. 20th PODS pp. 274–281 (2001).Google Scholar
4. [4]
Arriaga, R.I., Vempala, S.: An algorithmic theory of learning: Robust concepts and random projection. Machine Learning 63(2), 161–182 (2006). URL http://dx.doi.org/10.1007/s10994-006-6265-7. Accessed 1 December 2011.
5. [5]
Klivans, A.R., Servedio, R.A.: Learning intersections of halfspaces with a margin. Proc. 17th COLT 100, 348–362 (2004).
6. [6]
Szlam, A.: Non-stationary analysis on datasets and applications. Ph.D. thesis, Yale University (2006).Google Scholar
7. [7]
Frankl, P., Maehara, H.: The Johnson-Lindenstrauss lemma and the sphericity of some graphs. Journal of Combinatorial Theory 44, 355–362 (1988).
8. [8]
Dasgupta, S., Gupta, A.: An elementary proof of a theorem of Johnson and Lindenstrauss. Random Structures and Algorithms 22(1), 60–65 (1999).
9. [9]
Goel, N., Bebis, G., Nefian, A.: Face recognition experiments with random projection (2004).Google Scholar
10. [10]
Dasgupta, S.: Experiments with random projection. Proc. Uncertainty in Artificial Intelligence (2000).Google Scholar
11. [11]
Martinez, A.M., Benavente, R.: Cvc technical report 24. Ph.D. Thesis, University of Cambridge and ATT Laboratories Cambridge (1998).Google Scholar
12. [12]
Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, KDD 01, pp. 245–250. ACM, New York, NY, USA (2001).
13. [13]
Kaski, S.: Dimensionality reduction by random mapping:fast similarity computation for clustering. In: Proc. Int. Joint Conf. on Neural Networks, vol. 1, pp. 413–418 (1998).Google Scholar
14. [14]
Lin, J., Gunopulos, D.: Dimensionality reduction by random projection and latent semantic indexing. In: Proc. of SDM (2003).Google Scholar
15. [15]
Sulić, V., J. Perš, Kristan, M., Kovačič, S.: Dimensionality reduction for distributed vision systems using random projection. In: International Conference on Pattern Recognition (2010).Google Scholar
16. [16]
Berry, M.W., Pulatova, S.A., Stewart, G.W.: Algorithm 844: computing sparse reduced-rank approximations to sparse matrices. ACM Trans Math Softw 31(2), 252–269 (2005).
17. [17]
Chan, T.F., Hansen, P.C.: Some applications of the rank revealing QR factorization. SIAM J. on Sci. Statist. Comput. 13, 727–741 (1992).
18. [18]
Chen, Z., Dongarra, J.J.: Condition numbers of Gaussian random matrices. SIAM J. on Matrix Anal. Appl. 27, 603–620 (2005).
19. [19]
Goldstine, H.H., von Neumann, J.: Numerical inverting of matrices of high order II. Amer. Math. Soc. Proc. 2, 188–202 (1951).
20. [20]
Goreinov, S.A., Tyrtyshnikov, E.E., Zamarashkin, N.L.: A theory of pseudo-skeleton approximations. Linear Algebra and Its Applications 261, 1–21 (1997).
21. [21]
Gu, M., Eisenstat, S.C.: Efficient algorithms for computing a strong rank-revealing QR factorization. SIAM J. on Sci. Comput. 17, 848–869 (1996).
22. [22]
Martinsson, P.G., Rokhlin, V., Tygert, M.: A randomized algorithm for the approximation of matrices. Tech. Rep. 1361, Dept. of Computer Science, Yale University (2006).Google Scholar
23. [23]
Tyrtyshnikov, E.: Matrix bruhat decompositions with a remark on the QR (GR) algorithm. Linear Algebra Appl. 250, 61–68 (1997).
24. [24]
Tyrtyshnikov, E., Zamarashkin, N.: Thin structure of eigenvalue clusters for non-Hermitian Toeplitz matrices. Linear Algebra Appl. 292, 297–310 (1999).
25. [25]
Woolfe, F., Liberty, E., Rokhlin, V., Tygert, M.: A fast randomized algorithm for the approximation of matrices. Appl. Comput. Harmon. Anal. 25(3), 335–366 (2008).
26. [26]
Zamarashkin, N., Tyrtyshnikov, E.: Eigenvalue estimates for Hankel matrices. Sbornik: Mathematics 192, 59–72 (2001).