Abstract
We are living in an era of big data, where the process of generating data is continuously been taking place with each coming second. Data that is more varied and extremely complex in structure (unstructured/semi-structured) with problems of indexing, sorting, searching, analyzing and visualizing are major challenges of today’s organizations. Big data is always defined by its 5-v characteristics which are Volume, Velocity, Veracity, Variety, and Value. Almost each data model comprising big data is dependent on these 5-v characteristics. A large number of researches have been done on velocity and volume, but the complete and efficient solution for the variety is still not available in the markets. Traditional solutions provided by DBMS generally use multidimensional data type. However, many new data types cannot be compatible with these traditional systems. Big Data is a general problem affecting different fields, whether it is business, economic, social security or scientific research. To analyze huge data sets in order to get insights and find patterns in data is called big data analytics. Big data analytics is the need of every corporate and state of the art organization to look forward and make useful decisions. This paper comprises of discussion on current issues, opportunities, trends, and challenges of big data aimed to discuss variety in more detail. An efficient solution for the big data variety problem will be discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jagadish, H.V., Gehrke, J., Labrinidis, A., Papakonstantinou, Y., Patel, J.M., Ramakrishnan, R., Shahabi, C.: Big data and technical challenges. Commun. ACM 57(7), 86–94 (2014)
Fan, J., Han, F, Liu, H.: Challenges of big data analysis. Nat. Sci. Rev. 1(2), 293–314 (2014)
Rabl, T., Gmez-Villamor, S., Sadoghi, M., Munts-Mulero, V., Jacobsen, H.-A., Mankovskii, S.: Solving big data challenges for enterprise application performance management. Proc. VLDB Endowment 5(12), 1724–1735 (2012)
http://hpccsystems.com/. Last Accessed on 18 May 2019
Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: the next frontier for innovation, competition, and productivity (2011)
Gerhardt, B., Griffin, K., Klemann, R.: Unlocking value in the fragmented world of big data analytics. Cisco Int. Bus. Sol. Group 7 (2012)
Luo, J., Wu, M., Gopukumar, D., Zhao, Y.: Big data application in biomedical research and health care: a literature review. Biomed. Inf. Insights 8, BII-S31559 (2016)
Shan, S., Gary Wang, G.: Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct. Multi. Optim. 41(2), 219–241 (2010)
Di Ciaccio, A., Coli, M., Ibanez, J.M.A. (eds.): Advanced Statistical Methods for the Analysis of Large Data-Sets. Springer Science Business Media (2012)
Pbay, P., Thompson, D., Bennett, J., Mascarenhas, A.: Design and performance of a scalable, parallel statistics toolkit. In: 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum, pp. 1475–1484. IEEE (2011)
Zhou, J., Chen, C.L.P., Chen, L, Li, H.-X.: A collaborative fuzzy clustering algorithm in distributed network environments. IEEE Trans. Fuzzy Syst. 22(6), 1443–1456 (2013)
Pentaho, B.I.: Getting Started with Pentaho Business Analytics. Pentaho Corporation (2012)
Ranka, S., Sahni, S.: Clustering on a hypercube multicomputer. IEEE Trans. Parallel Distrib. Syst. 2(2), 129–137 (1991)
Cai, D., He, X., Han, J.: SRDA: an efficient algorithm for large-scale discriminant analysis. IEEE Trans. Knowl. Data Eng. 20(1), 1–12 (2007)
Bertone, P., Gerstein, M.: Integrative data mining: the new direction in bioinformatics. IEEE Eng. Med. Biol. Mag. 20(4), 33–40 (2001)
Hinton, G.E.: Learning multiple layers of representation. Trends Cogn. Sci. 11(10), 428–434 (2007)
Bekkerman, R., Bilenko, M., Langford, J. (eds.): Scaling up Machine Learning: Parallel and Distributed Approaches. Cambridge University Press (2011)
Simoff, S., Bhlen, M.H., Mazeika, A. (eds.): Visual Data Mining: Theory, Techniques and Tools for Visual Analytics, vol. 4404. Springer Science and Business Media (2008)
Keim, D.A., Panse, C., Sips, M., North, S.C.: Visual data mining in large geospatial point sets. IEEE Comput. Graphics Appl. 24(5), 36–44 (2004)
Thompson, D., Levine, J.A., Bennett, J.C., Bremer, P.T., Gyulassy, A., Pascucci, V., Pbay, P.P.: Analysis of large-scale scalar data using hixels. In: 2011 IEEE Symposium on Large Data Analysis and Visualization, pp. 23–30. IEEE (2011)
Kolari, P., Joshi, A.: Web mining: research and practice. Comput. Sci. Eng. 6(4), 49–53 (2004)
Acharjya, D.P., Ahmed, K.: A survey on big data analytics: challenges, open research issues, and tools. Int. J. Adv. Comput. Sci. Appl. 7(2), 511–518 (2016)
Che, D., Safran, M., Peng, Z.: From big data to big data mining: challenges, issues, and opportunities. In: International Conference on Database Systems for Advanced Applications, pp. 1–15. Springer, Berlin, Heidelberg (2013)
Michael, K., Miller, K.W.: Big data: new opportunities and new challenges [guest editors’ introduction]. Computer 46(6), 22–24 (2013)
Bologa, A.R., Bologa, R., Florea, A.: Big data and specific analysis methods for insurance fraud detection. Database Syst. J. 4(4), 30–39 (2013)
Chung, P.T., Chung, S.H.: On data integration and data mining for developing business intelligence. In: IEEE Long Island Systems, Applications, and Technology Conference (LISAT), pp. 1–6. IEEE (2013)
Pattnaik, K., Mishra, B.S.P.: Introduction to big data analysis. In: Techniques and Environments for Big Data Analysis, pp. 1–20. Springer, Cham (2016)
Wu, X., Zhu, X., Wu, G.Q., Ding, W.: Data mining with big data. IEEE Trans. Knowl. Data Eng. 26(1), 97–107 (2014)
Chen, C.P., Zhang, C.Y.: Data-intensive applications, challenges, techniques, and technologies: a survey on Big Data. Inf. Sci. 275, 314–347 (2014)
Tariq, M.I., Tayyaba, S., Ashraf, M.W., Rasheed, H.: Risk based NIST effectiveness analysis for cloud security. Bahria University J. Inf. Commun. Technol. (BUJICT) 10(Special Is) (2017)
Kaur, N., Sood, S.K.: Dynamic resource allocation for big data streams based on data characteristics (5 V s). Int. J. Netw. Manage. 27(4), e1978 (2017)
Sagiroglu, S., Sinanc, D.: Big data: a review. In: International Conference on Collaboration Technologies and Systems (CTS), pp. 42–47. IEEE (2013)
Khan, N., Yaqoob, I., Hashem, I.A.T., Inayat, Z., Ali, M., Kamaleldin, W., Alam, M., Shiraz, M., Gani, A.: Big data: survey, technologies, opportunities, and challenges. Sci. World J. (2014)
Mao, R., Xu, H., Wu, W., Li, J., Li, Y., Lu, M.: Overcoming the challenge of variety: big data abstraction, the next evolution of data management for AAL communication systems. IEEE Commun. Mag. 53(1), 42–47 (2015)
Robak, S., Franczyk, B., Robak, M.: Research Problems Associated with Big Data Utilization in Logistics and Supply Chains Design and Management. In: FedCSIS Position Papers, pp. 245–249 (2014)
Osman, A.M.S.: A novel big data analytics framework for smart cities. Futur. Gener. Comput. Syst. 91, 620–633 (2019)
Butt, S.A., Jamal, T., Azad, M.A., Ali, A., Safa, N.S.: A multivariant secure framework for smart mobile health application. Trans. Emerg. Telecommun. Technol. e3684 (2019)
Butt, S.A., Jamal, T.: IoT smart health security threats. In: 19th International Conference on Computational Science and Its Applications (ICCSA) IEEE, At Pittsburgh, Russia. https://doi.org/10.1109/ICCSA.000-8 (2019)
Jamal, T., Butt, S.A.: Cooperative cloudlet for pervasive networks. Proc. Asia Pacific J. Multi. Res. 5(3), 42–26 (2017)
Tariq, M.I.: Agent based information security framework for hybrid cloud computing. KSII Trans. Internet Inf. Syst. 13(1) (2019)
De-La-Hoz-Franco, E., Ariza-Colpas, P., Quero, J.M., Espinilla, M.: Sensor-based datasets for human activity recognition–a systematic review of literature. IEEE Access 6, 59192–59210 (2018)
Jamal, T., Butt, S.A.: Malicious node analysis in MANETS. Int. J. Inf. Technol. 1–9 (2018)
De la Hoz, E., de la Hoz, E., Ortiz, A., Ortega, J., Martínez-Álvarez, A.: Feature selection by multi-objective optimisation: application to network anomaly detection by hierarchical self-organising maps. Knowl.-Based Syst. 71, 322–338 (2014)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Naeem, M. et al. (2022). Trends and Future Perspective Challenges in Big Data. In: Pan, JS., Balas, V.E., Chen, CM. (eds) Advances in Intelligent Data Analysis and Applications. Smart Innovation, Systems and Technologies, vol 253. Springer, Singapore. https://doi.org/10.1007/978-981-16-5036-9_30
Download citation
DOI: https://doi.org/10.1007/978-981-16-5036-9_30
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-5035-2
Online ISBN: 978-981-16-5036-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)