Abstract
Principal component analysis (PCA) is a commonly used statistical technique for unsupervised dimensionality reduction, with a drawback of high-computational cost. Random projection (RP) is a matrix-based dimensionality reduction (DR) technique, which projects data by using a projection matrix i.e., constructed with random vectors. Random projection projects the high-dimensional data into low-dimensional feature space with the help of a projection matrix, which is constructed independent of input data. RP uses randomly generated matrices for projection purpose, even though it is computationally more advantageous than PCA, it has been giving unstable results, due to its randomness and data-independence property. Here in this work, we propose a via-medium solution which captures the structure-preserving feature of PCA and the pair-wise distance preserving feature from RP, and also takes less computational cost compared to PCA. Extensive experiments on low and high-dimensional data sets illustrate the efficiency and effectiveness of our proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Diaconis, P., Freedman, D.: Asymptotics of graphical projection pursuit. Ann. Statist. 12(3), 793–815 (1984). https://doi.org/10.1214/aos/1176346703. https://projecteuclid.org/euclid.aos/1176346703
Dasgupta, S.: Experiments with random projection. In: Uncertainity in Artificial Intelligence: Proceedings of the 16th Conference (UAI-2000), pp. 143–151 (2000). Morgan Kaufmann
Shen, Y., Hu, W., Yang, M., Wei, B., Chou, C.T.: Projection matrix optimisation for compressive sensing based applications in embedded systems. In: Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems (SenSys) (2013), Art. ID 22
Duarte-Carvajalino, J.M., Sapiro, G.: Learning to sense sparse signals: simultaneous sensing matrix and sparsifying dictionary optimization. IEEE Trans. Image Process. 18(7), 1395–1408 (2009)
Zhou, M., et al.: Nonparametric Bayesian dictionary learning for analysis of noisy and incomplete images. IEEE Trans. Image Process. 21(1), 130–144 (2012)
Elad, M.: Optimized projections for compressed sensing. Singnal Process. IEEE Trans. 55(12), 5695–5702 (2007)
Papadimitriou, C.H., Raghavan, P., Tamaki, H., Vempala, S.: Latent semantic indexing: a probabilistic analysis. In: 17th Annual Symposium on Principles of Database Systems, Seattle, WA, pp. 159–168 (1998)
Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: 30th Annual ACM Symposium on Theory of Computing (Dallas. TX), ACM, New York, pp. 604–613 (1998)
Schulman, L.J.: Clustering for edge-cost minimization. In: 32nd Annual ACM Symposium on Theory of Computing (Portland, OR, 2000), pp. 547–555. ACM, New York (2000)
Indyk, P.: Stable distributions, pseudorandom generators, embeddings and data stream computation. In: 41st Annual Symposium on Foundations on Computer Science (Redondo Beach, CA, 2000), pp. 189–197. IEEE Comput. Soc. Press, Los Alamitos, CA (2000)
Achlioptas D.: Database-friendly random projections: Johnson-lindenstrauss with binary coins. J. Comput. Syst. Sci. 66, 671–687. Special Issue on PODS (2001)
Frankl, P., Maehara, H.: The Johnson-Lindenstrauss Lemma and the sphericity of some graphs. J. Comb. Theory Ser. A 44, 355–362 (1987)
Dasgupta, S., Gupta, A.: An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct. Algorithms 22, 60–65 (2003)
Kleinberg, J.M.: Two algorithms for nearest neighbor search in high dimensions. Proceedings of 29th STOC, pp. 599-608 (1997)
Bingham, E., Mannila, H.: Random projection in dimensionality reduction: application to image and text data. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data mining. ACM, pp. 245-250 (2001)
Deegalla, S., Bostrom, H.: Reducing high-dimensional data by principal component analysis vs. random projection for nearest neighbor classification. In: Proceedings of the 5th International Conference on Machine Learning and Applications (ICMLA), Fl, pp. 245–250 (2006)
Fern, X.Z.., Brodley, C.E.: Random projection for high dimensional data clustering: a cluster ensemble approach. In: Proceedings of the Twentieth International Conference of Machine Learning, In: ICML. vol. 3, pp. 186–193 (2003)
Xie, H., Wang, Y.: Comparison among dimensionality reduction techniques based on Random Projection for cancer classification. Comput. Biol. Chem. 65(2016), 165–172 (2016). https://doi.org/10.1016/j.compbiolchem.2016.09.010
Michal, A., Michael, E., Alfred, B.: K-svd: design of dictionaries for sparse representation. In: Proceedings of SPARS, vol. 5, pp. 9–12 (2005)
Rana, R., Yang, M., Wark, T., Chou, C.T., Hu, W.: A deterministic construction of projection matrix for adaptive trajectory compression. IEEE Trans. Parallel Distrib. Syst. (2013)
Johnson, W., Lindenstrauss, J.: Extensions of lipschitz mappings into a hilbert space. Contemp. Math. 26, 189–206 (1984)
Cardoso, A., Wichert, A.: Iterative random projections for high-dimensional data clustering. Pattern Recognit. Lett. 33, 1749–1755 (2012)
Li, P., Hastie, T.J., Church, K.W.: Vary sparse random projections. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, New York, NY, USA. pp. 287–296 (2006)
Hecht-Nielsen, R.: Context vectors: general purpose approximate meaning representations self-organized from raw data. Comput. Intell. Imitating Life 43–56 (1994)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient based learning applied to document recognition. In: Proc. IEEE 86(11), 2278–2324 (1998)
Elad, M.: Optimized projections for compressed sensing. IEEE Trans. Signal Process. 55(12), 5695–5702 (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Pasunuri, R., Venkaiah, V.C. (2020). A Computationally Efficient Data-Dependent Projection for Dimensionality Reduction. In: Bansal, J., Gupta, M., Sharma, H., Agarwal, B. (eds) Communication and Intelligent Systems. ICCIS 2019. Lecture Notes in Networks and Systems, vol 120. Springer, Singapore. https://doi.org/10.1007/978-981-15-3325-9_26
Download citation
DOI: https://doi.org/10.1007/978-981-15-3325-9_26
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-3324-2
Online ISBN: 978-981-15-3325-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)