Skip to main content

Advertisement

SpringerLink
Log in

Big data dimensionality reduction techniques in IoT: review, applications and open research challenges

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In the age of big data, all forms of data with increasing samples and high-dimensional characteristics are demonstrating their importance in a variety of fields, including data mining, pattern recognition, machine learning, and the Internet of Things (IoTs), to name a few. The complexity of data processing increases as the dataset rises in size. The term “complexity” refers to the difficulty of finding and exploiting correlations between distinct dataset aspects. Therefore, using dimensionality reduction (DR) approach the complexity between distinct features can be eliminated. Keeping in view the betterment that can be achieved in storage and processing of big data in different IoT applications, this article reviews the literature on DR techniques with their advantages, properties, taxonomy, and parameters of evaluation. Further, the article elaborates on future research challenges, and an insight into applications of DR in different domains offers readers with information about the applicability of a certain data reduction technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

None.

References

  1. Marjani, M., Nasaruddin, F., Gani, A., Karim, A., Hashem, I.A.T., Siddiqa, A., Yaqoob, I.: Big IoT data analytics: architecture, opportunities, and open research challenges. IEEE Access 5, 5247–5261 (2017)

    Article  Google Scholar 

  2. Panarello, A., Tapas, N., Merlino, G., Longo, F., Puliafito, A.: Blockchain and IoT integration: a systematic survey. Sensors 18(8), 2275 (2018)

    Article  Google Scholar 

  3. Internet of Things outlook (2017). https://www.ericsson.com/en/mobility-report/reports

  4. Mehmood, R., Alturki, R., Zeadally, S.: Multimedia applications over metropolitan area networks (MANs). J. Netw. Comput. Appl. 34(5), 1518–1529 (2011)

    Article  Google Scholar 

  5. Mehmood, R., Lu, J.A.: Computational Markovian analysis of large systems. J. Manuf. Technol. Manag. 22(6), 804–817 (2011)

    Article  Google Scholar 

  6. Chhikara, P., Jain, N., Tekchandani, R., Kumar, N.: Data dimensionality reduction techniques for industry 4.0: research results, challenges, and future research directions. Software: Practice and Experience (2020). https://doi.org/10.1002/spe.2876

    Article  Google Scholar 

  7. Kumar, A., Bawa., S.: Distributed and big data storage management in grid computing. arXiv preprint (2012). arXiv:1207.2867

  8. Xu, X., Liang, T., Zhu, J., Zheng, D., Sun, T.: Review of classical dimensionality reduction and sample selection methods for large-scale data processing. Neurocomputing 328, 5–15 (2019)

    Article  Google Scholar 

  9. Rani, R., Kashyap, V., Khurana, M.: Role of IoT-cloud ecosystem in smart cities: review and challenges. Mater. Today Proc. 49(8), 2994–2998 (2020)

    Google Scholar 

  10. Kaur, D., Aujla, G.S., Kumar, N., Zomaya, A.Y., Perera, C., Ranjan, R.: Tensor-based big data management scheme for dimensionality reduction problem in smart grid systems: SDN perspective. IEEE Trans. Knowl. Data Eng. 30(10), 1985–1998 (2018)

    Article  Google Scholar 

  11. Rani, R., Khurana, M., Sharma, D., Moudgil, A.: Comparative study on various storage optimization techniques in IoT-cloud ecosystem. In: 2021 International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), pp. 659–663 (2021)

  12. Sarveniazi, A.: An actual survey of dimensionality reduction. Am. J. Comput. Math. 4(2), 55–72 (2014)

    Article  Google Scholar 

  13. Kumar, A., Bawa, S.: Virtualization of large-scale data storage system to achieve dynamicity and scalability in grid computing. In: Advances in Computer Science, Engineering and Applications, pp. 323–331. Springer, Berlin (2012)

  14. Ur Rehman, M.H., Liew, C.S., Abbas, A., Jayaraman, P.P., Wah, T.Y., Khan, S.U.: Big data reduction methods: a survey. Data Sci. Eng. 1, 265–284 (2016)

    Article  Google Scholar 

  15. Kumar, A., Bawa, S., Sharma, V.: Dynamic and scalable data storage management in grid environments. In: National Conference on Emerging Trend in Engineering and Sciences, Samrat Ashok Technological Institute, MP, India (2010)

  16. Xu, X., Liang, T., Zhu, J., Zheng, D., Sun, T.: Review of classical dimensionality reduction and sample selection methods for large-scale data processing. Neurocomputing 328, 5–15 (2019)

    Article  Google Scholar 

  17. Platzer, A.: Visualization of SNPs with t-SNE. PLoS ONE 8(2), e56883 (2013)

    Article  Google Scholar 

  18. Van Der Maaten, L., Postma, E., Van den Herik, J.: Dimensionality reduction: a comparative. J. Mach. Learn. Res. 10(66–71), 13 (2009)

    Google Scholar 

  19. Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Application of Dimensionality Reduction in Recommender System—A Case Study. Technical Report. University of Minnesota, Department of Computer Science and Engineering (2000)

  20. Kalyan Chakravarthy, S., Sudhakar, N., Srinivasa Reddy, E., Venkata Subramanian, D., Shankar, P.: Dimension reduction and storage optimization techniques for distributed and big data cluster environment. In: Soft Computing and Medical Bioinformatics, pp. 47–54. Springer, Singapore (2019)

  21. Hu, P., Ning, H., Qiu, T., Zhang, Y., Luo, X.: Fog computing based face identification and resolution scheme in Internet of Things. IEEE Trans. Ind. Inform. 13(4), 1910–1920 (2017)

    Article  Google Scholar 

  22. Rani, R., Kumar, N., Khurana, M., Kumar, A., Barnawi, A.: Storage as a service in fog computing: a systematic review. J. Syst. Archit. 116, 102033 (2021)

    Article  Google Scholar 

  23. Lieberman, J., Leidner, A., Percivall, G., Rönsdorf, C.: Using big data analytics and IoT principles to keep an eye on underground infrastructure. In: 2017 IEEE International Conference on Big Data (Big Data), pp. 4592–4601. IEEE (2017)

  24. Kumar, A., Bawa, S.: Dais: dynamic access and integration services framework for cloud-oriented storage systems. Clust. Comput. 23, 3289–3308 (2020)

    Article  Google Scholar 

  25. Hajjaji, Y., Boulila, W., Farah, I.R., Romdhani, I., Hussain, A.: Big data and IoT-based applications in smart environments: a systematic review. Comput. Sci. Rev. 39, 100318 (2021)

    Article  Google Scholar 

  26. Boulila, W., Farah, I.R., Hussain, A.: A novel decision support system for the interpretation of remote sensing big data. Earth Sci. Inform. 11(1), 31–45 (2018)

    Article  Google Scholar 

  27. Boulila, W., Ayadi, Z., Farah, I.R.: Application to land cover change prediction model: sensitivity analysis approach to model epistemic and aleatory imperfection. J. Comput. Sci. 23, 58–70 (2017)

    Article  Google Scholar 

  28. Jennath, H.S., Adarsh, S., Anoop, V.S.: Distributed IoT and applications: a survey. In: Integrated Intelligent Computing, Communication and Security, pp. 333–341. Springer, Singapore (2019)

  29. Camastra, F.: Data dimensionality estimation methods: a survey. Pattern Recognit. 36(12), 2945–2954 (2003)

    Article  MATH  Google Scholar 

  30. Cunningham, J.P., Yu, B.M.: Dimensionality reduction for large-scale neural recordings. Nat. Neurosci. 17(11), 1500–1509 (2014)

    Article  Google Scholar 

  31. Becht, E., McInnes, L., Healy, J., Dutertre, C.-A., Kwok, I.W.H., Ng, L.G., Ginhoux, F., Newell, E.W.: Dimensionality reduction for visualizing single-cell data using UMAP. Nat. Biotechnol. 37(1), 38–44 (2019)

    Article  Google Scholar 

  32. Wei, H.-L., Billings, S.A.: Feature subset selection and ranking for data dimensionality reduction. IEEE Trans. Pattern Anal. Mach. Intell. 29(1), 162–166 (2007)

    Article  Google Scholar 

  33. Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: KDD ’01: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2001, pp. 245-250 (2001)

  34. Tan, S., Mayrovouniotis, M.L.: Reducing data dimensionality through optimizing neural network inputs. AIChE J. 41(6), 1471–1480 (1995)

    Article  Google Scholar 

  35. Hu, X., Luo, P., Zhang, X., Wang, J., Zhou, T.: Research on the effectiveness evaluation of big data in combat simulation. In: ICBDR 2018, pp. 70–75 (2018)

  36. An, J., Zhang, X., Jiao, L.C.: Dimensionality reduction based on group-based tensor model for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 13(10), 1497–1501 (2016)

    Article  Google Scholar 

  37. Sorzano, C.O.S., Vargas, J., Pascual Montano, A.: A survey of dimensionality reduction techniques. arXiv (2014)

  38. Wang, F., Sun, J.: Survey on distance metric learning and dimensionality reduction in data mining. Data Min. Knowl. Discov. 29(2), 534–564 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  39. Ficuciello, F., Calinon, S., Falco, P.: A brief survey on the role of dimensionality reduction in manipulation learning and control. IEEE Robot. Autom. Lett. 3(3), 2608–2615 (2018)

    Article  Google Scholar 

  40. Li, W., Feng, F., Li, H., Qian, D.: Discriminant analysis-based dimension reduction for hyperspectral image classification: a survey of the most recent advances and an experimental comparison of different techniques. IEEE Geosci. Remote Sens. Mag. 6(1), 15–34 (2018)

    Article  Google Scholar 

  41. Cichocki, A., Lee, N., Oseledets, I., Phan, A.-H., Zhao, Q., Sugiyama, M., Mandic, D.P.: Tensor networks for dimensionality reduction and large-scale optimization: Part 2: applications and future perspectives. Found. Trends Mach. Learn. 9(6), 249–429 (2017)

    Article  MATH  Google Scholar 

  42. Abdulhammed, R., Musafer, H., Alessa, A., Faezipour, M., Abuzneid, A.: Features dimensionality reduction approaches for machine learning based network intrusion detection. Electronics 8(3), 322 (2019)

    Article  Google Scholar 

  43. Peng, G., Wang, Z., Wei, Z., Yuri, G., Yuriy, K., Oleg, A., Oleksandr, R., Sergii, S.: Dimensionality reduction in deep learning for chest X-ray analysis of lung cancer. In: 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI), pp. 878–883 (2018)

  44. Kiarashinejad, Y., Abdollahramezani, S., Adibi, A.: Deep learning approach based on dimensionality reduction for designing electromagnetic nanostructures. NPJ Comput. Mater. 6(1), 1–12 (2020)

    Article  Google Scholar 

  45. Raymer, M.L., Punch, W.F., Goodman, E.D., Kuhn, L.A., Jain, A.K.: Dimensionality reduction using genetic algorithms. IEEE Trans. Evol. Comput. 4(2), 164–171 (2000)

    Article  Google Scholar 

  46. Kaski, S.: Dimensionality reduction by random mapping: fast similarity computation for clustering. In: 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence, vol. 1, pp. 413–418 (1998)

  47. Noh, H., Araujo, A., Sim, J., Weyand, T., Han, B.: Large-scale image retrieval with attentive deep local features. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), October 2017

  48. Zhong, X., Enke, D.: Forecasting daily stock market return using dimensionality reduction. Expert Syst. Appl. 67, 126–139 (2017)

    Article  Google Scholar 

  49. Boutsidis, C., Zouzias, A., Mahoney, M.W., Drineas, P.: Randomized dimensionality reduction for \(k\)-means clustering. IEEE Trans. Inf. Theory 61(2), 1045–1062 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  50. Wu, Z., Li, Y., Plaza, A., Li, J., Xiao, F., Wei, Z.: Parallel and distributed dimensionality reduction of hyperspectral data on cloud computing architectures. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 9(6), 2270–2278 (2016)

    Article  Google Scholar 

  51. Cichocki, A., Lee, N., Oseledets, I., Phan, A.-H., Zhao, Q., Mandic, D.P.: Tensor networks for dimensionality reduction and large-scale optimization: Part 1: low-rank tensor decompositions. Found. Trends Mach. Learn. 9(4–5), 249–429 (2016)

    Article  MATH  Google Scholar 

  52. Williamson, R.C., Doiron, B., Smith, M.A., Yu, B.M.: Bridging large-scale neuronal recordings and large-scale network models using dimensionality reduction. Curr. Opin. Neurobiol. 55, 40–47 (2019)

    Article  Google Scholar 

  53. Ali, L., Wajahat, I., Golilarz, N.A., Keshtkar, F., Bukhari, S.A.C.: LDA-GA-SVM: improved hepatocellular carcinoma prediction through dimensionality reduction and genetically optimized support vector machine. Neural Comput. Appl. 33(7), 2783–2792 (2021)

    Article  Google Scholar 

  54. Mardani, A., Liao, H., Nilashi, M., Alrasheedi, M., Cavallaro, F.: A multi-stage method to predict carbon dioxide emissions using dimensionality reduction, clustering, and machine learning techniques. J. Clean. Prod. 275, 122942 (2020)

    Article  Google Scholar 

  55. Elhenawy, M., Masoud, M., Glaser, S., Rakotonirainy, A.: A new approach to improve the topological stability in non-linear dimensionality reduction. IEEE Access 8, 33898–33908 (2020)

    Article  Google Scholar 

  56. Tomar, D., Tomar, P.: Dimensionality reduction techniques for IoT based data. Rec. Adv. Comput. Sci. Commun. (Formerly Rec. Patents Comput. Sci.) 14(3), 724–735 (2021)

    Article  Google Scholar 

  57. Kaya, I.E., Pehlivanlı, A.Ç., Sekizkardeş, E.G., Ibrikci, T.: PCA based clustering for brain tumor segmentation of T1W MRI images. Comput. Methods Programs Biomed. 140, 19–28 (2017)

    Article  Google Scholar 

  58. Bahşi, H., Nõmm, S., La Torre, F.B.: Dimensionality reduction for machine learning based IoT botnet detection. In: 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1857–1862. IEEE (2018)

  59. Zhang, T., Yang, B.: Dimension reduction for big data. Stat. Interface 11(2), 295–306 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  60. Qummar, S., Khan, F.G., Shah, S., Khan, A., Shamshirband, S., Ur Rehman, Z., Khan, I.A., Jadoon, W.: A deep learning ensemble approach for diabetic retinopathy detection. IEEE Access 7, 150530–150539 (2019)

    Article  Google Scholar 

  61. Pour, M.S., Bou-Harb, E., Varma, K., Neshenko, N., Pados, D.A., Choo, K.-K.R.: Comprehending the IoT cyber threat landscape: a data dimensionality reduction technique to infer and characterize Internet-scale IoT probing campaigns. Digit. Investig. 28, S40–S49 (2019)

    Article  Google Scholar 

  62. Thippa Reddy, G., Praveen Kumar Reddy, M., Lakshmanna, K., Kaluri, R., Rajput, D.S., Srivastava, G., Baker, T.: Analysis of dimensionality reduction techniques on big data. IEEE Access 8, 54776–54788 (2020)

    Article  Google Scholar 

  63. Bhattacharya, S., Siva Rama Krishnan, S., Praveen Kumar Reddy, M., Kaluri, R., Singh, S., Thippa Reddy, G., Alazab, M., Tariq, U.: A novel PCA-firefly based XGBoost classification model for intrusion detection in networks using GPU. Electronics 9(2), 219 (2020)

    Article  Google Scholar 

  64. Martins, I.D., Bahiense, L., Infante, C.E.D., Arruda, E.F.: Dimensionality reduction for multi-criteria problems: an application to the decommissioning of oil and gas installations. Expert Syst. Appl. 148, 113236 (2020)

    Article  Google Scholar 

  65. Zebari, R., Abdulazeez, A., Zeebaree, D., Zebari, D., Saeed, J.: A comprehensive review of dimensionality reduction techniques for feature selection and feature extraction. J. Appl. Sci. Technol. Trends 1(2), 56–70 (2020)

    Article  Google Scholar 

  66. Vizarraga, J., Casas, R., Marco, Á., David Buldain, J.: Dimensionality reduction for smart IoT sensors. Electronics 9(12), 2035–2051 (2020)

    Article  Google Scholar 

  67. Reyna-Orta, A., Andrade, Á.G.: Dimensionality reduction to solve resource allocation problem in 5G UDN using genetic algorithm. Soft Comput. 25(6), 4629–4642 (2021)

    Article  Google Scholar 

  68. Gavel, S., Raghuvanshi, A.S., Tiwari, S.: Distributed intrusion detection scheme using dual-axis dimensionality reduction for Internet of Things (IoT). J. Supercomput. 77, 1–24 (2021)

    Article  Google Scholar 

  69. Ali, F., El-Sappagh, S., Riazul Islam, S.M., Ali, A., Attique, M., Imran, M., Kwak, K.-S.: An intelligent healthcare monitoring framework using wearable sensors and social networking data. Future Gener. Comput. Syst. 114, 23–43 (2021)

    Article  Google Scholar 

  70. Vu-Ngoc, H., Elawady, S.S., Mehyar, G.M., Abdelhamid, A.H., Mattar, O.M., Halhouli, O., Vuong, N.L., Ali, C.D.M., Hassan, U.H., Kien, N.D., et al.: Quality of flow diagram in systematic review and/or meta-analysis. PLoS ONE 13(6), 1–16 (2018)

    Article  Google Scholar 

  71. Shea, B.J., Reeves, B.C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al.: AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (2017). https://doi.org/10.1136/bmj.j4008

    Article  Google Scholar 

  72. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., Prisma Group: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6(7), e1000097 (2009)

    Article  Google Scholar 

  73. Ayesha, S., Hanif, M.K., Talib, R.: Overview and comparative study of dimensionality reduction techniques for high dimensional data. Inf. Fusion 59, 44–58 (2020)

    Article  Google Scholar 

  74. Mohamed, H.H., Belaid, S., Naanaa, W., Romdhane, L.B.: Local commute-time guided MDS for 3D non-rigid object retrieval. Appl. Intell. 48(9), 2873–2883 (2018)

    Article  Google Scholar 

  75. Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(86), 2579–2605 (2008)

    MATH  Google Scholar 

  76. Devassy, B.M., George, S.: Dimensionality reduction and visualisation of hyperspectral ink data using t-SNE. Forensic Sci. Int. 311, 110194 (2020)

    Article  Google Scholar 

  77. Das, G., Chattopadhyay, M., Gupta, S.: A comparison of self-organising maps and principal components analysis. Int. J. Market Res. 58(6), 815–834 (2016)

    Article  Google Scholar 

  78. Fujiwara, T., Chou, J.-K., Shilpika, S., Panpan, X., Ren, L., Ma, K.-L.: An incremental dimensionality reduction method for visualizing streaming multidimensional data. IEEE Trans. Vis. Comput. Graph. 26(1), 418–428 (2020)

    Article  Google Scholar 

  79. Nascimento, M., Silva, F.F., Sáfadi, T., Nascimento, A.C.C., Ferreira, T.E.M., Barroso, L.M.A., Azevedo, C.F., Guimarães, S.E.F., Serão, N.V.L.: Independent component analysis (ICA) based-clustering of temporal RNA-Seq data. PLoS ONE 12(7), e0181195 (2017)

    Article  Google Scholar 

  80. Uysal, A.K., Gunal, S.: Text classification using genetic algorithm oriented latent semantic features. Expert Syst. Appl. 41(13), 5938–5947 (2014)

    Article  Google Scholar 

  81. Hao, S., Xu, Y., Peng, H., Su, K., Ke, D.: Automated Chinese essay scoring from topic perspective using regularized latent semantic indexing. In: 2014 22nd International Conference on Pattern Recognition, pp. 3092–3097 (2014)

  82. Raunak, V., Gupta, V., Metze, F.: Effective dimensionality reduction for word embeddings. In: Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pp. 235–243 (2019)

  83. Cheng, J., Liu, Q., Lu, H., Chen, Y.-W.: Supervised kernel locality preserving projections for face recognition. Neurocomputing 67, 443–449 (2005)

    Article  Google Scholar 

  84. Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: MPCA: multilinear principal component analysis of tensor objects. IEEE Trans. Neural Netw. 19(1), 18–39 (2008)

    Article  Google Scholar 

  85. Yu, W., Teng, X., Liu, C.: Face recognition using discriminant locality preserving projections. Image Vis. Comput. 24(3), 239–248 (2006)

    Article  Google Scholar 

  86. Chen, S., Zhao, H., Kong, M., Luo, B.: 2D-LPP: a two-dimensional extension of locality preserving projections. Neurocomputing 70(4), 912–921 (2007)

    Article  Google Scholar 

  87. Wan, M., Yang, G., Sun, C., Liu, M.: Sparse two-dimensional discriminant locality-preserving projection (S2DDLPP) for feature extraction. Soft Comput. 23(14), 5511–5518 (2019)

    Article  Google Scholar 

  88. Zhu, L., Zhu, S.: Face recognition based on orthogonal discriminant locality preserving projections. Neurocomputing 70(7), 1543–1546 (2007)

    Article  Google Scholar 

  89. Lu, G.-F., Lin, Z., Jin, Z.: Face recognition using discriminant locality preserving projections based on maximum margin criterion. Pattern Recognit. 43(10), 3572–3579 (2010)

    Article  MATH  Google Scholar 

  90. Beckmann, C.F., Smith, S.M.: Probabilistic independent component analysis for functional magnetic resonance imaging. IEEE Trans. Med. Imaging 23(2), 137–152 (2004)

    Article  Google Scholar 

  91. Li, J.-B., Pan, J.-S., Chen, S.-M.: Kernel self-optimized locality preserving discriminant analysis for feature extraction and recognition. Neurocomputing 74(17), 3019–3027 (2011)

    Article  Google Scholar 

  92. Zhang, D., Zhao, Y., Du, M.: A new supervised dimensionality reduction algorithm using linear discriminant analysis and locality preserving projection. WSEAS Trans. Inf. Sci. Appl. 10(4), 2224–3402 (2013)

    Google Scholar 

  93. Ye, J., Janardan, R., Li, Q.: Two-dimensional linear discriminant analysis. Adv Neural Inf. Process. Syst. 17, 1569–1576 (2004)

    Google Scholar 

  94. Wang, B., Hu, Y., Gao, J., Sun, Y., Chen, H., Yin, B.: Locality preserving projections for Grassmann manifold. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17) (2017)

  95. Peterfreund, E., Gavish, M.: Multidimensional scaling of noisy high dimensional data. Appl. Comput. Harmon. Anal. 51, 333–373 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  96. Sacha, D., Kraus, M., Bernard, J., Behrisch, M., Schreck, T., Asano, Y., Keim, D.A.: SOMFlow: guided exploratory cluster analysis with self-organizing maps and analytic provenance. IEEE Trans. Vis. Comput. Graph. 24(1), 120–130 (2018)

    Article  Google Scholar 

  97. Ramamurthy, M., Harold Robinson, Y., Vimal, S., Suresh, A.: Auto encoder based dimensionality reduction and classification using convolutional neural networks for hyperspectral images. Microprocess. Microsyst. 79, 103280 (2020)

    Article  Google Scholar 

  98. Krasoulis, A., Nazarpour, K., Vijayakumar, S.: Use of regularized discriminant analysis improves myoelectric hand movement classification. In: 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 395–398 (2017)

  99. Ran, R., Fang, B., Wu, X., Zhang, S.: A simple and effective generalization of exponential matrix discriminant analysis and its application to face recognition. IEICE Trans. Inf. Syst. 101(1), 265–268 (2018)

    Article  Google Scholar 

  100. Rabin, N., Kahlon, M., Malayev, S., Ratnovsky, A.: Classification of human hand movements based on EMG signals using nonlinear dimensionality reduction and data fusion techniques. Expert Syst. Appl. 149, 113281 (2020)

    Article  Google Scholar 

  101. Hyvarinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10(3), 626–634 (1999)

    Article  Google Scholar 

  102. Wang, Y., Zhu, L.: Research and implementation of SVD in machine learning. In: 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), pp. 471–475 (2017)

  103. Radüntz, T., Scouten, J., Hochmuth, O., Meffert, B.: Automated EEG artifact elimination by applying machine learning algorithms to ICA-based features. J. Neural Eng. 14(4), 046004 (2017)

    Article  Google Scholar 

  104. Ren, W., Wen, G., Luan, R., Yang, Z., Zhang, Z.: Single-channel blind source separation and its application on arc sound signal processing. In: Transactions on Intelligent Welding Manufacturing, pp. 115–126. Springer, Singapore (2018)

  105. Fitria, D., Ma’sum, M.A., Imah, E.M., Gunawan, A.A.: Automatic arrhythmias detection using various types of artificial neural network based learning vector quantization (LVQ). J. Ilmu Komput. Inform. 7(2), 90–100 (2014)

    Google Scholar 

  106. Berry, M.W., Dumais, S.T., O’Brien, G.W.: Using linear algebra for intelligent information retrieval. SIAM Rev. 37(4), 573–595 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  107. Billsus, D., Pazzani, M.J., et al.: Learning collaborative information filters. In: ICML 98, pp. 46–54 (1998)

  108. Bhattacharyya, S.: Direct marketing response models using genetic algorithms. In: KDD, 1998, pp. 144–148 (1998)

  109. Santello, M., Flanders, M., Soechting, J.F.: Postural hand synergies for tool use. J. Neurosci. 18(23), 10105–10115 (1998)

    Article  Google Scholar 

  110. Lataniotis, C., Marelli, S., Sudret, B.: Extending classical surrogate modeling to high dimensions through supervised dimensionality reduction: a data-driven approach. Int. J. Uncertain. Quantif. 10(1), 55–82 (2020)

    Article  MathSciNet  Google Scholar 

  111. Egbue, O., Long, S.: A Socio-technical Analysis of Widespread Electric Vehicle Adoption, p. 6. Department of Engineering Management and Systems Engineering, Missouri University of Science and Technology, St Rolla (2012)

    Google Scholar 

  112. Zhong, X., Enke, D.: Predicting the daily return direction of the stock market using hybrid machine learning algorithms. Financ. Innov. 5(1), 1–20 (2019)

    Article  Google Scholar 

  113. Plaza, A., Bioucas-Dias, J.M., Simic, A., Blackwell, W.J.: Foreword to the special issue on hyperspectral image and signal processing. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 5(2), 347–353 (2012)

    Article  Google Scholar 

  114. Plaza, A., Martinez, P., Plaza, J., Perez, R.: Dimensionality reduction and classification of hyperspectral image data using sequences of extended morphological transformations. IEEE Trans. Geosci. Remote Sens. 43(3), 466–479 (2005)

    Article  Google Scholar 

  115. Plaza, A., Martínez, P., Plaza, J., Pérez, R.: Dimensionality reduction and classification of hyperspectral image data using sequences of extended morphological transformations. IEEE Trans. Geosci. Remote Sens. 43(3), 466–479 (2005)

    Article  Google Scholar 

  116. Faheem, M., Shah, S.B.H., Butt, R.A., Raza, B., Anwar, M., Ashraf, M.W., Ngadi, Md.A., Gungor, V.C.: Opportunities and challenges: smart grid communication and information technologies in the perspective of industry 4.0. Comput. Sci. Rev. 30, 1–30 (2018)

    Article  Google Scholar 

  117. Houari, R., Bounceur, A., Kechadi, M.-T., Tari, A.-K., Euler, R.: Dimensionality reduction in data mining: a copula approach. Expert Syst. Appl. 64, 247–260 (2016)

    Article  Google Scholar 

  118. Lee, C., Luo,Z., Ngiam, K.Y., Zhang, M., Zheng, K., Chen, G., Ooi, B.C., Yip, W.L.J.: Big healthcare data analytics: challenges and applications. In: Handbook of Large-Scale Distributed Computing in Smart Healthcare, pp. 11–41. Springer, Cham (2017)

  119. Muhammad, A.N., Aseere, A.M., Chiroma, H., Shah, H., Gital, A.Y., Hashem, I.A.T.: Deep learning application in smart cities: recent development, taxonomy, challenges and research prospects. Neural Comput. Appl. 33, 1–37 (2020)

    Google Scholar 

  120. Soomro, K., Bhutta, M.N.M., Khan, Z., Tahir, M.A.: Smart city big data analytics: an advanced review. WIREs Data Min. Knowl. Discov. 9(5), e1319 (2019)

    Google Scholar 

  121. Arsa, D.M.S., Jati, G., Soleh, M., Jatmiko, W.: Vehicle detection using dimensionality reduction based on deep belief network for intelligent transportation system. In: 2017 6th IEEE International Conference on Advanced Logistics and Transport (ICALT), pp. 199–204 (2017)

Download references

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura, Punjab, India

    Ridhima Rani & Meenu Khurana

  2. Department of Computer Engineering, Marwadi University, Rajkot, Gujarat, India

    Ajay Kumar

  3. Computer Science and Engineering Department, Thapar University, Patiala, Punjab, India

    Neeraj Kumar

  4. Department of Computer Science and Information Engineering, Asia University, Taichung, Taiwan

    Neeraj Kumar

  5. King Abdul Aziz University, Jidda, Saudi Arabia

    Neeraj Kumar

  6. Department of Computer Science, University of Petroleum and Energy Studies, Dehra Dun, Uttarakhand, India

    Neeraj Kumar

Authors
  1. Ridhima Rani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meenu Khurana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ajay Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Neeraj Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have equally contributed in this paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Neeraj Kumar.

Ethics declarations

Conflict of interest

We have no conflicts of interest to disclose.

Ethical approval

We confirm that this work is original and has not been published elsewhere, nor it is currently under consideration for publication elsewhere. This article does not contain any studies with animals performed by any of the authors. This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rani, R., Khurana, M., Kumar, A. et al. Big data dimensionality reduction techniques in IoT: review, applications and open research challenges. Cluster Comput 25, 4027–4049 (2022). https://doi.org/10.1007/s10586-022-03634-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-022-03634-y

Keywords

Search

Navigation