Abstract
Major online sites such as Amazon, eBay, and Yahoo are now adopting Spark. Many organizations run Spark in thousands of nodes available in the clusters. Spark is a “rapid cluster computing” and a broader data processing platform. It has a thirsty and active open-source community. Spark core is the Apache Spark kernel. We discuss in this paper the use and applications of Apache Spark, the mainstream of popular organization. These organizations extract, collect event data from the users’ daily use, and engage in real-time interactions with such data. As a result, Apache Spark is a big data next-generation tool. It offers both batch and streaming capabilities to process data more quickly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lu X, Shankar D, Gugnani S, Panda DKDK (2016) High-performance design of Apache Spark with RDMA and its benefits on various workloads. In: Proceedings of 2016 IEEE international conference on big data, Big Data 2016, pp 253–262
Domoney WF, Ramli N, Alarefi S, Walker SD (2016) Smart city solutions to water management using self-powered, low-cost, water sensors and Apache Spark data aggregation. In: Proceedings of 2015 IEEE international renewable and sustainable energy conference, IRSEC 2015
Carlini E, Dazzi P, Esposito A, Lulli A, Ricci L (2014) Balanced graph partitioning with Apache Spark. In: Euro-Par 2014: parallel Processing workshop, pp 129–140
Triguero I, Galar M, Merino D, Maillo J, Bustince H, Herrera F (2016) Evolutionary undersampling for extremely imbalanced big data classification under Apache Spark. In: 2016 IEEE congress on evolutionary computation, CEC 2016, pp 640–647
Yan Y, Huang L, Yi L (2015) Is Apache Spark scalable to seismic data analytics and computations? In: Proceedings of 2015 IEEE international conference on big data, IEEE Big Data 2015, pp 2036–2045 (2015)
Chiba T, Onodera T (2015) Workload characterization and optimization of TPC-H queries on Apache Spark. IBM Research—Tokyo, Japan, pp 1–12 (2015)
Alsheikh MA, Niyato D, Lin S, Tan H-P, Han Z (2016) Mobile Big data analytics using deep learning and Apache Spark. IEEE Netw 31:21–29
Mushtaq H, Al-Ars Z (2015) Cluster-based Apache Spark implementation of the GATK DNA analysis pipeline. Proceedings of 2015 IEEE international conference on bioinformatics and biomedicine, BIBM 2015, pp 1471–1477
Zadeh RB, Meng X, Staple A, Yavuz B, Pu L, Venkataraman S, Sparks E, Ulanov A, Zaharia M (2016) Matrix computations and optimization in Apache Spark. In: KDD’ 16, pp 31–38
Maarala AI, Rautiainen M, Salmi M, Pirttikangas S, Riekki J (2015) Low latency analytics for streaming traffic data with Apache Spark. In: Proceedings of 2015 IEEE international conference on big data, IEEE Big Data 2015, pp 2855–2858
Graux D, Jachiet L, Genev P, Graux D, Jachiet L, Genev P, Graux D, Jachiet L, Genevès P, Layaïda N (2016) SPARQLGX in action: efficient distributed evaluation of SPARQL with Apache Spark. In: 15th international semantic web conference
Gopalani S, Arora R (2015) Comparing Apache Spark and Map Reduce with performance analysis using K-means. Int J Comput Appl 113:8887
Acknowledgements
The authors express gratitude toward the assistance provided by Accendere Knowledge Management Services Pvt. Ltd., in preparing the manuscripts. We also thank our mentors and faculty members who guided us throughout the research and helped us in achieving the desired results.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sahana, H.P., Sanjana, M.S., Mohammed Muddasir, N., Vidyashree, K.P. (2020). Apache Spark Methods and Techniques in Big Data—A Review. In: Ranganathan, G., Chen, J., Rocha, Á. (eds) Inventive Communication and Computational Technologies. Lecture Notes in Networks and Systems, vol 89. Springer, Singapore. https://doi.org/10.1007/978-981-15-0146-3_67
Download citation
DOI: https://doi.org/10.1007/978-981-15-0146-3_67
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-0145-6
Online ISBN: 978-981-15-0146-3
eBook Packages: EngineeringEngineering (R0)