Skip to main content

Advertisement

Log in

Predictions-on-chip: model-based training and automated deployment of machine learning models at runtime

For multi-disciplinary design and operation of gas turbines

  • Theme Section Paper
  • Published:
Software and Systems Modeling Aims and scope Submit manuscript

Abstract

The design of gas turbines is a challenging area of cyber-physical systems where complex model-based simulations across multiple disciplines (e.g., performance, aerothermal) drive the design process. As a result, a continuously increasing amount of data is derived during system design. Finding new insights in such data by exploiting various machine learning (ML) techniques is a promising industrial trend since better predictions based on real data result in substantial product quality improvements and cost reduction. This paper presents a method that generates data from multi-paradigm simulation tools, develops and trains ML models for prediction, and deploys such prediction models into an active control system operating at runtime with limited computational power. We explore the replacement of existing traditional prediction modules with ML counterparts with different architectures. We validate the effectiveness of various ML models in the context of three (real) gas turbine bearings using over 150,000 data points for training, validation, and testing. We introduce code generation techniques for automated deployment of neural network models to industrial off-the-shelf programmable logic controllers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/. Software available from tensorflow.org

  2. Alexopoulos, K., Nikolakis, N., Chryssolouris, G.: Digital twin-driven supervised machine learning for the development of artificial intelligence applications in manufacturing. Int. J. Comput. Integr. Manuf. 33(5), 429–439 (2020). https://doi.org/10.1080/0951192X.2020.1747642

    Article  Google Scholar 

  3. Bencomo, N., Paucar, L.H.G.: RaM: causally-connected and requirements-aware runtime models using bayesian learning. In: 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 216–226. IEEE (2019)

  4. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)

    MATH  Google Scholar 

  5. Boschert, S., Rosen, R.: Digital Twin-The Simulation Aspect, pp. 59–74. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32156-1_5

    Book  Google Scholar 

  6. Breuker, D.: Towards model-driven engineering for big data analytics—an exploratory analysis of domain-specific languages for machine learning. In: 2014 47th Hawaii International Conference on System Sciences, pp. 758–767. IEEE (2014)

  7. Burgueño, L., Cabot, J., Gérard, S.: An LSTM-based neural network architecture for model transformations. In: 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 294–299. IEEE (2019)

  8. Cengarle, M.V., Bensalem, S., McDermid, J., Passerone, R., Sangiovanni-Vincentelli, A., Törngren, M.: CyPhERS: cyber-physical European roadmap and strategy characteristics, capabilities, potential applications of cyber-physical systems: a preliminary analysis. Technical Report 611430 (2013)

  9. Chbat, N.W., Rajamani, R., Ashley, T.A.: Estimating gas turbine internal cycle parameters using a neural network. In: ASME 1996 International Gas Turbine and Aeroengine Congress and Exhibition, pp. V005T15A023–V005T15A023. American Society of Mechanical Engineers (1996)

  10. Darvas, D., Viñuela, E.B., Majzik, I.: PLC code generation based on a formal specification language. In: 2016 IEEE 14th International Conference on Industrial Informatics (INDIN), pp. 389–396. IEEE (2016)

  11. Fan, C., Xiao, F., Zhao, Y.: A short-term building cooling load prediction method using deep learning algorithms. Appl. Energy 195, 222–233 (2017)

    Article  Google Scholar 

  12. Fast, M.: Artificial neural networks for gas turbine monitoring. Division of Thermal Power Engineering, Department of Energy Sciences (2010)

  13. Gregory, B.: Turbine preliminary design using artificial intelligence and numerical optimization techniques. J. Turbomach. 114, 1 (1992)

    Article  Google Scholar 

  14. Huyck, B., Ferreau, H.J., Diehl, M., De Brabanter, J., Van Impe, J.F., De Moor, B., Logist, F.: Towards online model predictive control on a programmable logic controller: practical considerations. Mathematical Problems in Engineering, Vol. 2012 (2012)

  15. Ibrahem, I., Akhrif, O., Moustapha, H., Staniszewski, M.: Neural networks modelling of aero-derivative gas turbine engine: a comparison study, pp. 738–745 (2019). https://doi.org/10.5220/0007928907380745

  16. Kanelopoulos, K., Stamatis, A., Mathioudakis, K.: Incorporating neural networks into gas turbine performance diagnostics. In: ASME 1997 International Gas Turbine and Aeroengine Congress and Exhibition, pp. V004T15A011–V004T15A011. American Society of Mechanical Engineers (1997)

  17. Kiakojoori, S., Khorasani, K.: Dynamic neural networks for gas turbine engine degradation prediction, health monitoring and prognosis. Neural Comput. Appl. 27(8), 2157–2192 (2016)

    Article  Google Scholar 

  18. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. ArXiv preprint arXiv:1412.6980 (2014)

  19. Kusiak, A., Li, M., Zhang, Z.: A data-driven approach for steam load prediction in buildings. Appl. Energy 87(3), 925–933 (2010)

    Article  Google Scholar 

  20. Kusmenko, E., Nickels, S., Pavlitskaya, S., Rumpe, B., Timmermanns, T.: Modeling and training of neural processing systems. In: 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 283–293. IEEE (2019)

  21. Lafdani, E.K., Nia, A.M., Ahmadi, A.: Daily suspended sediment load prediction using artificial neural networks and support vector machines. J. Hydrol. 478, 50–62 (2013)

    Article  Google Scholar 

  22. Lazzaretto, A., Toffolo, A.: Analytical and neural network models for gas turbine design and off-design simulation. Int. J. Appl. Thermodyn. 4(4), 173–182 (2001)

    Google Scholar 

  23. Lee, E.A., Hartmann, B., Kubiatowicz, J., Rosing, T.S., Wawrzynek, J., Wessel, D., Rabaey, J.M., Pister, K., Sangiovanni-Vincentelli, A.L., Seshia, S.A., Blaauw, D., Dutta, P., Fu, K., Guestrin, C., Taskar, B., Jafari, R., Jones, D.L., Kumar, V., Mangharam, R., Pappas, G.J., Murray, R.M., Rowe, A.: The swarm at the edge of the cloud. IEEE Des. Test 31(3), 8–20 (2014). https://doi.org/10.1109/MDAT.2014.2314600

    Article  Google Scholar 

  24. Li, Q., Meng, Q., Cai, J., Yoshino, H., Mochida, A.: Applying support vector machine to predict hourly cooling load in the building. Appl. Energy 86(10), 2249–2256 (2009)

    Article  Google Scholar 

  25. Luo, W., Hu, T., Zhang, C., Wei, Y.: Digital twin for CNC machine tool: modeling and using strategy. J. Ambient Intell. Humaniz. Comput. (2018). https://doi.org/10.1007/s12652-018-0946-5

    Article  Google Scholar 

  26. Madni, A.M., Madni, C.C., Lucero, S.D.: Leveraging digital twin technology in model-based systems engineering. Systems 7(1), 7 (2019)

    Article  Google Scholar 

  27. Nascimento, R.G., Viana, F.A.: Fleet prognosis with physics-informed recurrent neural networks. ArXiv preprint arXiv:1901.05512 (2019)

  28. Nguyen, P.T., Di Rocco, J., Di Ruscio, D., Pierantonio, A., Iovino, L.: Automated classification of metamodel repositories: a machine learning approach. In: 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 272–282. IEEE (2019)

  29. Ogaji, S., Singh, R.: Artificial neural networks in fault diagnosis: a gas turbine scenario. In: Computational Intelligence in Fault Diagnosis, pp. 179–207. Springer (2006)

  30. Pilarski, S., Staniszewski, M., Villeneuve, F., Varró, D.: On artificial intelligence for simulation and design space exploration in gas turbine design. In: Burgueño, L., Pretschner, A., Voss, S., Chaudron, M., Kienzle, J., Völter, M., Gérard, S., Zahedi, M., Bousse, E., Rensink, A., Polack, F., Engels, G., Kappel, G. (eds.) In: 22nd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion, MODELS Companion 2019, Munich, Germany, September 15–20, 2019, pp. 170–174. IEEE (2019). https://doi.org/10.1109/MODELS-C.2019.00029

  31. Prechelt, L.: Early Stopping—But When?, pp. 55–69. Springer, Berlin (1998). https://doi.org/10.1007/3-540-49430-8_3

  32. Puschel, M., Moura, J.M., Johnson, J.R., Padua, D., Veloso, M.M., Singer, B.W., Xiong, J., Franchetti, F., Gacic, A., Voronenko, Y., et al.: SPIRAL: code generation for DSP transforms. Proc. IEEE 93(2), 232–275 (2005)

    Article  Google Scholar 

  33. Qiao, Q., Wang, J., Ye, L., Gao, R.X.: Digital twin for machining tool condition prediction. Proc. CIRP 81, 1388–1393 (2019). https://doi.org/10.1016/j.procir.2019.04.049. 52nd CIRP Conference on Manufacturing Systems (CMS), Ljubljana, Slovenia, June 12-14, 2019

  34. Rauber, T.W., de Assis Boldt, F., Varejão, F.M.: Heterogeneous feature models and feature selection applied to bearing fault diagnosis. IEEE Trans. Ind. Electron. 62(1), 637–646 (2015)

    Article  Google Scholar 

  35. Sacha, K.: Automatic code generation for PLC controllers. In: International Conference on Computer Safety, Reliability, and Security, pp. 303–316. Springer (2005)

  36. Scikit-Learn Bayesian Ridge Regression. https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.BayesianRidge.html#sklearn.linear_model.BayesianRidge

  37. SGT-A65: Aeroderivative gas turbine: Gas turbines: Manufacturer: Siemens energy global. https://www.siemens-energy.com/global/en/offerings/power-generation/gas-turbines/sgt-a65-tr.html

  38. Sobie, C., Freitas, C., Nicolai, M.: Simulation-driven machine learning: bearing fault classification. Mech. Syst. Signal Process. 99, 403–419 (2018)

    Article  Google Scholar 

  39. Steinegger, M., Zoitl, A.: Automated code generation for programmable logic controllers based on knowledge acquisition from engineering artifacts: concept and case study. In: Proceedings of 2012 IEEE 17th International Conference on Emerging Technologies and Factory Automation (ETFA 2012), pp. 1–8. IEEE (2012)

  40. Thapa, D., Park, C.M., Park, S.C., Wang, G.N.: Auto-generation of IEC standard PLC code using t-MPSG. Int. J. Control Autom. Syst. 7(2), 165–174 (2009)

    Article  Google Scholar 

  41. Thomas, G., Cabaret, S., Barillère, R., Kulman, N., Rochez, J., Pons, X., Azarov, K.: LHC-GCS: a model-driven approach for automatic PLC and SCADA code generation. Technical Report (2005)

  42. Tipping, M.E.: Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 1, 211–244 (2001)

    MathSciNet  MATH  Google Scholar 

  43. Wang, Z., Hong, T., Piette, M.A.: Building thermal load prediction through shallow machine learning and deep learning. Appl. Energy 263, 114683 (2020)

    Article  Google Scholar 

  44. Widodo, A., Kim, E.Y., Son, J.D., Yang, B.S., Tan, A.C., Gu, D.S., Choi, B.K., Mathew, J.: Fault diagnosis of low speed bearing based on relevance vector machine and support vector machine. Expert Syst. Appl. 36(3), 7252–7261 (2009)

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by the Digital Multidisciplinary Analysis and Design Optimization Platform for Aeroderivative GasTurbines (Siemens Ca CRDPJ 513922-17 X-247371 and NSERC CRDPJ 513922-17 X-247323 funds).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sebastian Pilarski.

Additional information

Communicated by Eugene Syriani and Manuel Wimmer.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pilarski, S., Staniszewski, M., Bryan, M. et al. Predictions-on-chip: model-based training and automated deployment of machine learning models at runtime. Softw Syst Model 20, 685–709 (2021). https://doi.org/10.1007/s10270-020-00856-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-020-00856-9

Keywords

Navigation