Abstract
Given the difficulty of browsing and analyzing big data via web browsers, using Spark technology, we took the big-data analysis of steel companies as an example to propose a framework for big-data visualization technology. We used HDFS for data storage, Spark for data analysis, Django for web systems, and ECharts for data visualization, ultimately providing a complete visualization solution. Using visualization, we realized price forecasting, sales analysis and production process quality traceability, help enterprises to make decisions and provide support for technological process improvement.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yin, S., Kaynak, O.: Big data for modern industry: challenges and trends [point of view]. Proc. IEEE 103, 143–146 (2015)
Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. HotCloud 10, 95 (2010)
Shvachko, K., Kuang, H., Radia, S., Chansler, R. (eds.): The Hadoop Distributed File System. IEEE (2010)
Franklin, M. (ed.): The Berkeley Data Analytics Stack: Present and Future. IEEE (2013)
Shi, J., et al.: Clash of the titans: Mapreduce vs. spark for large scale data analytics. Proc. VLDB Endow. 8, 2110–2121 (2015)
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51, 107–113 (2008)
Zaharia, M., et al. (eds.): Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. USENIX Association (2012)
Chen, H., Chiang, R.H.L., Storey, V.C.: Business intelligence and analytics: from big data to big impact. MIS Q. 1165–1188 (2012)
Fiaz, A.S., Asha, N., Sumathi, D., Navaz, A.S.: Data visualization: enhancing big data more adaptable and valuable. Int. J. Appl. Eng. Res. 11, 2801–2804 (2016)
Holovaty, A., Kaplan-Moss, J.: The Definitive Guide to Django: Web Development Done Right. Apress, New York (2009)
Liu, J., Qin, S.J.: Perspectives on big data modeling of process industries. Acta Automatica Sinica 42, 161–171 (2016)
Acknowledgements
The authors acknowledge the financial support from the National Key Research and Development Program of China (No. 2016YFB0700500), and the National Science Foundation of China (No. 61572075, No. 61702036), and Fundamental Research Funds for the Central Universities (No. FRF-TP-17-012A1), and China Postdoctoral Science Foundation (No. 2017M620619).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Ban, X., Wang, B., Cheng, C., Taghzouit, S. (2018). Iron and Steel Enterprises Big Data Visualization Analysis Based on Spark. In: Luo, Y. (eds) Cooperative Design, Visualization, and Engineering. CDVE 2018. Lecture Notes in Computer Science(), vol 11151. Springer, Cham. https://doi.org/10.1007/978-3-030-00560-3_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-00560-3_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00559-7
Online ISBN: 978-3-030-00560-3
eBook Packages: Computer ScienceComputer Science (R0)