Abstract
Industry 4.0 boosted the generation of large volumes of sensor data in manufacturing production lines. When adequately mined, this information can anticipate failures and launch maintenance actions increasing quality and productivity. This paper explores the integration of real-time big data techniques in industry. Specifically, this work contributes with a framework for real-time predictive maintenance supported by big data technologies. The proposed framework is composed of: (i) Apache Kafka as messaging system to manage sensor data; (ii) Spark as Machine Learning engine for large-scale data processing; and (iii) Cassandra as NoSQL distributed database. We showcase the synergy of these cutting-edge technologies in a predictive maintenance system tailored for the request. By leveraging advanced data analysis methods, we reveal hidden patterns and insights valuable for researchers across various disciplines. The experiments were performed with the NASA turbofan jet engine dataset, which includes run-to-failure simulated data from turbo fan jet engines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Available at https://www.kaggle.com/datasets/behrad3d/nasa-cmaps, November 2023.
- 2.
Available at https://spark.apache.org/docs/latest/ml-classification-regression.html, November 2023.
References
Ahmad, S., Purdy, S.: Real-time anomaly detection for streaming analytics. arXiv preprint arXiv:1607.02480 (2016)
Asgari, M., Farnaghi, M., Ghaemi, Z.: Predictive mapping of urban air pollution using apache spark on a hadoop cluster. In: Proceedings of the 2017 International Conference on Cloud and Big Data Computing, pp. 89–93 (2017)
Canizo, M., Onieva, E., Conde, A., Charramendieta, S., Trujillo, S.: Real-time predictive maintenance for wind turbines using big data frameworks. In: 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), pp. 70–77. IEEE (2017)
Chatterjee, A., Ahmed, B.S.: IoT anomaly detection methods and applications: a survey. Internet Things 19, 100568 (2022). https://www.sciencedirect.com/science/article/pii/S2542660522000622
Ed-daoudy, A., Maalmi, K., El Ouaazizi, A.: A scalable and real-time system for disease prediction using big data processing. Multimedia Tools Appl. 82, 30405–30434 (2023)
Haarman, M., Mulders, M., Vassiliadis, C.: Predictive maintenance 4.0: predict the unpredictable. PwC and Mainnovation 4 (2017)
Han, J., Pei, J., Tong, H.: Data Mining: Concepts and Techniques. Morgan Kaufmann, Waltham (2022)
Herlocker, J.L., Konstan, J.A., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative filtering. In: Proceedings of the 22nd Annual International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 230–237. ACM (1999)
Jin, X., Wah, B.W., Cheng, X., Wang, Y.: Significance and challenges of big data research. Big Data Res. 2(2), 59–64 (2015). https://www.sciencedirect.com/science/article/pii/S2214579615000076, visions on Big Data
Kanavos, A., Trigka, M., Dritsas, E., Vonitsanos, G., Mylonas, P.: A regularization-based big data framework for winter precipitation forecasting on streaming data. Electronics 10(16), 1872 (2021)
Kaur, J., Mann, K.S.: AI based healthcare platform for real time, predictive and prescriptive analytics using reactive programming. In: 10th International Conference on Computer and Electrical Engineering, Univ Alberta, Edmonton, CANADA, 11–13 October 2017 (2018). Journal of Physics Conference Series, vol. 933
Nair, L.R., Shetty, S.D., Shetty, S.D.: Applying spark based machine learning model on streaming big data for health status prediction. Comput. Electr. Eng. 65, 393–399 (2018)
Oussous, A., Benjelloun, F.Z., Ait Lahcen, A., Belfkih, S.: Big data technologies: a survey. J. King Saud Univ. Comput. Inf. Sci. 30(4), 431–448 (2018). https://www.sciencedirect.com/science/article/pii/S1319157817300034
O’donovan, P., Leahy, K., Bruton, K., O’Sullivan, D.T.: Big data in manufacturing: a systematic mapping study. J. Big Data 2, 1–22 (2015)
Shaqrah, A., Almars, A.: Examining the internet of educational things adoption using an extended unified theory of acceptance and use of technology. Internet Things 19, 100558 (2022)
Takács, G., Pilászy, I., Németh, B., Tikk, D.: Scalable collaborative filtering approaches for large recommender systems. J. Mach. Learn. Res. 10, 623–656 (2009)
Zonta, T., da Costa, C.A., da Rosa Righi, R., de Lima, M.J., da Trindade, E.S., Li, G.P.: Predictive maintenance in the Industry 4.0: a systematic literature review. Comput. Ind. Eng. 150, 106889 (2020). https://www.sciencedirect.com/science/article/pii/S0360835220305787
Acknowledgements
This work was supported by the UIDB/05105/2020 Program Contract, funded by national funds through the FCT I.P.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Teixeira, M., Thierstein, F., Entringer, P., Sá, H., Leitão, J.D., Leal, F. (2024). Framework for Real-Time Predictive Maintenance Supported by Big Data Technologies. In: Rocha, Á., Adeli, H., Dzemyda, G., Moreira, F., Poniszewska-Marańda, A. (eds) Good Practices and New Perspectives in Information Systems and Technologies. WorldCIST 2024. Lecture Notes in Networks and Systems, vol 985. Springer, Cham. https://doi.org/10.1007/978-3-031-60215-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-60215-3_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-60214-6
Online ISBN: 978-3-031-60215-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)