Skip to main content
SpringerLink
Log in

Failure prediction in production line based on federated learning: an empirical study

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Data protection across organizations is limiting the application of centralized learning (CL) techniques. Federated learning (FL) enables multiple participants to build a learning model without sharing data. Nevertheless, there is very few research works on FL in intelligent manufacturing. This paper presents the results of an empirical study on failure prediction in the production line based on FL. This paper (1) designs Federated Support Vector Machine and federated random forest algorithms for the horizontal FL and vertical FL scenarios, respectively; (2) proposes an experiment process for evaluating the effectiveness between the FL and CL algorithms; (3) finds that the performance of FL and CL are not significantly different on the global testing data, on the random partial testing data, and on the estimated unknown Bosch data, respectively. The fact that the testing data is heterogeneous enhances our findings. Our study reveals that FL can replace CL for failure prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://www.kaggle.com/c/bosch-production-line-performance.

  2. https://www.intel.com/content/www/us/en/artificial-intelligence/posts/federated-learning-for-medical-imaging.html.

References

  • Aussel, N., Chabridon, S., & Petetin, Y. (2020). Combining federated and active learning for communication-efficient distributed failure prediction in aeronautics. arXiv preprintarXiv:2001.07504.

  • Bakopoulou, E., Tillman, B., & Markopoulou, A. (2019). A federated learning approach for mobile packet classification. arXiv preprintarXiv:1907.13113.

  • Boughorbel, S., Jarray, F., Venugopal, N., Moosa, S., Elhadi, H., & Makhlouf, M. (2019a). Federated uncertainty-aware learning for distributed hospital EHR data. arXiv preprintarXiv:1910.12191.

  • Boughorbel, S., Jarray, F., Venugopal, N., Moosa, S., Elhadi, H., & Makhlouf, M. (2019b). Federated uncertainty-aware learning for distributed hospital ehr data. arXiv preprintarXiv:1910.12191.

  • Brisimi, T. S., Chen, R., Mela, T., Olshevsky, A., Paschalidis, I. C., & Shi, W. (2018). Federated learning of predictive models from federated electronic health records. International Journal of Medical Informatics, 112, 59–67.

    Article  Google Scholar 

  • Carbery, C. M, Woods, R., & Marshall, A. H. (2018). A bayesian network based learning system for modelling faults in large-scale manufacturing. In 2018 IEEE international conference on industrial technology (ICIT) (pp. 1357–1362). IEEE.

  • Carbery, C. M., Woods, R., & Marshall, A. H. (2019). A new data analytics framework emphasising preprocessing of data to generate insights into complex manufacturing systems. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 233(19–20), 6713–6726.

  • Chen, D., Xie, L. J., Kim, B., Wang, L., Hong, C. S., Wang, L.C., & Han, Z. (2020). Federated learning based mobile edge computing for augmented reality applications. In 2020 international conference on computing, networking and communications (ICNC) (pp. 767–773). IEEE.

  • Dianbo, L., Timothy, M., Raheel, S., & Mandl, K. D. (2018). Fadl: Federated-autonomous deep learning for distributed electronic health record. arXiv preprintarXiv:1811.11400.

  • Moming, D., Duo, L., Xianzhang, C., Yujuan, T., Jinting, R., Lei, Q., & Liang, L. (2019). Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications. In 37th IEEE international conference on computer design, ICCD 2019, Abu Dhabi, United Arab Emirates, November 17–20, 2019 (pp. 246–254). IEEE.

  • Ester, M., Kriegel, H.-P., Sander, J., Xiaowei, X., et al. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. Kdd, 96, 226–231.

    Google Scholar 

  • Feng, J., Rong, C., Sun, F., Guo, D., & Li, Y. (2020). PMF: A privacy-preserving human mobility prediction framework via federated learning. IMWUT, 4(1), 10:1–10:21.

    Google Scholar 

  • Roy, A. G., Siddiqui, S., Pölsterl, S., Navab, N., & Wachinger, C. (2019). Braintorrent: A peer-to-peer environment for decentralized federated learning. arXiv preprint. arXiv:1905.06731.

  • Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C. & Ramage, D. (2018). Federated learning for mobile keyboard prediction. arXiv preprintarXiv:1811.03604.

  • Hebert, J. (2016). Predicting rare failure events using classification trees on large scale manufacturing data with complex interactions. In 2016 IEEE international conference on big data (big data) (pp. 2024–2028). IEEE.

  • Hu, B., Gao, Y., Liu, L., & Ma, H. (2018). Federated region-learning: An edge computing based framework for urban environment sensing. In IEEE global communications conference, GLOBECOM 2018, Abu Dhabi, United Arab Emirates, December 9-13, 2018 (pp 1–7). IEEE.

  • Huang, L., Shea, A. L., Qian, H., Masurkar, A., Deng, H., & Liu, D. (2019). Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records. Journal of Biomedical Informatics, 99, 103291.

    Article  Google Scholar 

  • Huang, X., Zanni-Merk, C., & Crémilleux, B. (2019). Enhancing deep learning with semantics: an application to manufacturing time series analysis. Procedia Computer Science, 159, 437–446.

    Article  Google Scholar 

  • Ickin, S., Vandikas, K., & Fiedler, M. (2019). Privacy preserving qoe modeling using collaborative learning. In P. Casas, F. Wamser, F. E. Bustamante & D. R. Choffnes (Eds.), Proceedings of the 4th Internet-QoE workshop on QoE-based analysis and management of data communication networks, internet-QoE@MobiCom 2019, Los Cabos, Mexico, October 21, 2019 (pp. 13–18). ACM.

  • Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Nitin Bhagoji, A., Bonawitz, K., Charles, Z., Cormode,G., Cummings, R., et al. (2019). Advances and open problems in federated learning. arXiv preprintarXiv:1912.04977.

  • Khoza, S. C, & Grobler, J. (2019). Comparing machine learning and statistical process control for predicting manufacturing performance. In EPIA conference on artificial intelligence (pp. 108–119). Springer.

  • Kotenko, I., Saenko, I., & Branitskiy, A. (2019). Improving the performance of manufacturing technologies for advanced material processing using a big data and machine learning framework. Materials Today: Proceedings, 11, 380–385.

    Google Scholar 

  • Kusiak, A. (2017). Smart manufacturing must embrace big data. Nature, 544(7648), 23–25.

    Article  Google Scholar 

  • Li, B., Hou, B., Wen-tao, Y., Xiao-bing, L., & Yang, C. (2017). Applications of artificial intelligence in intelligent manufacturing: A review. Frontiers of Information Technology & Electronic Engineering, 18(1), 86–96.

    Article  Google Scholar 

  • Tian, L., Kumar, A., Sahu, A. K., Talwalkar, A., & Virginia, S. (2020a). Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3), 50–60.

    Article  Google Scholar 

  • Li, W., Milletarì, F., Xu, D., Rieke, N., Hancox, J., Zhu, W., Baust, M., Cheng, Y., Ourselin, S., Jorge Cardoso, M., & Feng, A. (2019). Privacy-preserving federated brain tumour segmentation. In H.-I. Suk, M. Liu, P. Yan, & C. Lian, (Eds.) Machine learning in medical imaging - 10th international workshop, MLMI 2019, held in conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Proceedings, volume 11861 of Lecture Notes in Computer Science (pp. 133–141). Springer.

  • Li, Z., Sharma, V., & Mohanty, S. P. (2020b). Preserving data privacy via federated learning: Challenges and solutions. IEEE Consumer Electronics Magazine, 9(3), 8–16.

    Article  Google Scholar 

  • Liu, Y., Liu, Y., Liu, Z., Liang, Y., Meng, C., Zhang, J., & Zheng, Y. (2020a). Federated forest. IEEE Transactions on Big Data.

  • Liu, Y., James, J. Q., Kang, J., Niyato, D., & Zhang, S. (2020b). A federated learning approach. IEEE Internet of Things Journal: Privacy-Preserving Traffic Flow Prediction.

    Google Scholar 

  • Liu, Y., Zhang, L., Ge, N., & Li, G. (2020c). A systematic literature review on federated learning: From a model quality perspective. arXiv preprintarXiv:2012.01973.

  • Liu, Z., Zhang, D., Jia, W., Lin, X., & Liu, H. (2020d). An adversarial bidirectional serial-parallel LSTM-based QTD framework for product quality prediction. Journal of Intelligent Manufacturing, 31, 1–19.

    Article  Google Scholar 

  • Lu, S., Yao, Y., & Shi, W. (2019). Collaborative learning on the edges: A case study on connected vehicles. In I. Ahmad & S. Sundararaman (Eds.), 2nd USENIX workshop on hot topics in edge computing, HotEdge 2019, Renton, WA, USA, July 9, 2019. USENIX Association.

  • Yunlong, L., Huang, X., Dai, Y., Maharjan, S., & Zhang, Y. (2020). Differentially private asynchronous federated learning for mobile edge computing in urban informatics. IEEE Transactions on Industrial Informatics, 16(3), 2134–2143.

    Article  Google Scholar 

  • Corinzia, L., & Buhmann, J. M. (2019). Variational federated multi-task learning. arXiv preprintarXiv:1906.06268.

  • Mangal, A. & Kumar, N. (2016). Using big data to enhance the bosch production line performance: A kaggle challenge. In 2016 IEEE international conference on big data (big data) (pp. 2029–2035). IEEE.

  • Maurya, A. (2016). Bayesian optimization for predicting rare internal failures in manufacturing processes. In 2016 IEEE international conference on big data (big data) (pp. 2036–2045). IEEE.

  • Brendan McMahan, H, Moore, E., Ramage, D., Hampson, S., et al. (2016). Communication-efficient learning of deep networks from decentralized data. arXiv preprintarXiv:1602.05629.

  • Moldovan, D., Anghel, I., Cioara, T., & Salomie, I. (2019). Time series features extraction versus lstm for manufacturing processes performance prediction. In 2019 international conference on speech technology and human-computer dialogue (SpeD) (pp. 1–10). IEEE.

  • Montgomery, D. C. (2007). Introduction to statistical quality control. Hoboken.

  • Mowla, N. I, Tran, N. H, Doh, I., & Chae, K. (2019). Federated learning-based cognitive detection of jamming attack in flying ad-hoc network. IEEE Access.

  • Nguyen, T. D., Marchal, S., Miettinen, M., Fereidooni, H., Asokan, N, & Sadeghi, A.-R. (2019). Dïot: A federated self-learning anomaly detection system for iot. In 2019 IEEE 39th international conference on distributed computing systems (ICDCS) (pp. 756–767). IEEE.

  • Olivia, C., Aris, G.-D., Theodoros, S., Issa, S., Yoonyoung, P., Grace, H., & Amar, D. (2019). Differential privacy-enabled federated learning for sensitive health data. arXiv preprintarXiv:1910.02578.

  • Pfohl Stephen, R., Dai Andrew, M., & Heller, K. (2019). Federated and differentially private learning for electronic health records. arXiv preprintarXiv:1911.05861.

  • Einat Ronen, A. C., & Burns, K. (2013). Utilizing predictive sales analytics with big data. Technical report, Intel, Tech. Rep.[Online]. Available: http://www. intel. com/content ....

  • Runeson, P., & Höst, M. (2009). Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering, 14(2), 131.

    Article  Google Scholar 

  • Samarakoon, S., Bennis, M., Saad, W., & Debbah, M. (2019). Distributed federated learning for ultra-reliable low-latency vehicular communications. IEEE Transactions on Communications.

  • Mulya Saputra, Y., Thai Hoang, D., Nguyen, D. N., Dutkiewicz, E., Dominik Mueck, M., & Srikanteswara, S.. (2019). Energy demand prediction with federated learning for electric vehicle networks. arXiv preprintarXiv:1909.00907.

  • Shaoqi, C., Dongyu, X., Guohui, C., Qiang, Y., & Qi, L.. (2020). Fl-QSAR: A federated learning based QSAR prototype for collaborative drug discovery. bioRxiv.

  • Sheller, M. J., Anthony Reina, G., Edwards, B., Martin, J., & Bakas, S. (2018a). Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation. In A. Crimi, S. Bakas, H. J. Kuijf, F. Keyvan, M. Reyes, & T. van Walsum (Eds.) Brainlesion: Glioma, multiple sclerosis, stroke and traumatic brain injuries - 4th international workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised selected papers, Part I, volume 11383 of Lecture Notes in Computer Science (pp. 92–104). Springer.

  • Sheller, M. J., Anthony Reina, G, Edwards, B., Martin, J., & Bakas, S. (2018b). Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation. In International MICCAI Brainlesion workshop (pp. 92–104). Springer.

  • Sozinov, K., Vlassov, V., & Girdzijauskas, S. (2018). Human activity recognition using federated learning. In J. Chen & L. T. Yang (Eds.), IEEE international conference on parallel & distributed processing with applications, ubiquitous computing & communications, big data & cloud computing, social computing & networking, sustainable computing & communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018, Melbourne, Australia, December 11–13, 2018 (pp 1103–1111). IEEE.

  • Stoica, I., Song, D., Ada Popa, R., Patterson, D., Mahoney, M. W., Katz, R., Joseph, A. D, Jordan, M., Hellerstein, J. M, Gonzalez, J. E., et al. (2017). A berkeley view of systems challenges for ai. arXiv preprintarXiv:1712.05855.

  • Suzumura, T., Zhou, Y., Barcardo, N., Ye, G., Houck, K., Kawahara, R., Anwar, A., Larise Stavarache, L., Klyashtorny, D., Ludwig, H., et al. (2019). Towards federated graph learning for collaborative financial crimes detection. arXiv preprintarXiv:1909.12946.

  • Tao, F., Qi, Q., Liu, A., & Kusiak, A. (2018). Data-driven smart manufacturing. Journal of Manufacturing Systems, 48, 157–169.

    Article  Google Scholar 

  • Tao, Q., Fangzhao, W., Chuhan, W., Yongfeng, H., & Xing, X. (2020). Fedrec: Privacy-preserving news recommendation with federated learning. arXiv preprintarXiv:2003.09592.

  • Xinle, L., Yang, L., Tianjian, C., Ming, L., & Qiang, Y. (2019). Federated transfer reinforcement learning for autonomous driving. arXiv preprintarXiv:1910.06001.

  • Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019a). Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2), 1–19.

  • Yang, W., Zhang, Y., Ye, K., Li, L., & Xu, C.-Z. (2019b). Ffd: A federated learning based method for credit card fraud detection. In International Conference on Big Data (pp. 18–32). Springer.

  • Yin, R. K. (2017). Case study research and applications: Design and methods. Thousand Oaks.

  • Zhang, D., Xu, B., & Wood, J. (2016). Predict failures in production lines: A two-stage approach with clustering and supervised learning. In 2016 IEEE international conference on big data (big data) (pp. 2070–2074). IEEE.

  • Zhong, R. Y., Xu, X., Klotz, E., & Newman, S. T. (2017). Intelligent manufacturing in the context of industry 40: a review. Engineering, 3(5), 616–630.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Key Research and Development Program of China Grant No. 2019YFB1703903, National Natural Science Foundation of China Grant No. 61732019 and No. 61902011, and the Grant No. NJ2018014 of the Key Laboratory of Safety-Critical Software (Nanjing University of Aeronautics and Astronautics), Ministry of Industry and Information Technology.

Author information

Authors and Affiliations

  1. School of Software, Beihang University, Beijing, China

    Ning Ge & Guanghao Li

  2. Key Laboratory of Safety-Critical Software (Nanjing University of Aeronautics and Astronautics), Ministry of Industry and Information Technology, Beijing, China

    Ning Ge

  3. School of Computer Science and Engineering, Beihang University, Beijing, China

    Li Zhang & Yi Liu

Authors
  1. Ning Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guanghao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ge, N., Li, G., Zhang, L. et al. Failure prediction in production line based on federated learning: an empirical study. J Intell Manuf 33, 2277–2294 (2022). https://doi.org/10.1007/s10845-021-01775-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-021-01775-2

Keywords

Search

Navigation