Skip to main content
SpringerLink
Log in

Multibody dynamics and control using machine learning

  • Review
  • Published:
Multibody System Dynamics Aims and scope Submit manuscript

Abstract

Artificial intelligence and mechanical engineering are two mature fields of science that intersect more and more often. Computer-aided mechanical analysis tools, including multibody dynamics software, are very versatile and have revolutionalized many industries. However, as shown by the literature presented in this review, combining the advantages of multibody system dynamics and machine learning creates new and exciting possibilities. For example, the multibody method can assist machine learning by providing synthetic data, while machine learning can provide fast and accurate subsystem models. The intersection of both approaches results in surrogate and hybrid modeling techniques, advanced control algorithms, and optimal design applications. A notable example is the development of autonomous systems for vehicles, robots, and mobile machinery. In our review we have found nontrivial, innovative, and even surprising applications of machine learning and multibody dynamics. This review focuses on applying neural networks, mainly deep learning, in connection with the multibody system method. Over one hundred and fifty papers are covered, and three main research areas are identified and introduced: data-driven modeling, model-based control and estimation, and data-driven control. The paper starts with a primer on machine learning and concludes with future research directions. The main goal is to provide a comprehensive and up-to-date review of existing literature to inspire further research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Availability of data and materials

Not applicable (review paper).

References

  1. Kurinov, I., Orzechowski, G., Hamalainen, P., Mikkola, A.: Automated excavator based on reinforcement learning and multibody system dynamics. IEEE Access 8, 213998–214006 (2020)

    Article  Google Scholar 

  2. Kalashnikov, D., Varley, J., Chebotar, Y., Swanson, B., Jonschkowski, R., Finn, C., Levine, S., Hausman, K.: MT-Opt: continuous multi-task robotic reinforcement learning at scale (2021). arXiv:2104.08212 [cs]

  3. Cordeschi, R.: AI turns fifty: revisiting its origins. Appl. Artif. Intell. 21(4–5), 259–279 (2007)

    Article  Google Scholar 

  4. Nwankpa, C., Ijomah, W., Gachagan, A., Marshall, S.: Activation functions: comparison of trends in practice and research for deep learning, pp. 1–20 (2018)

  5. Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., Chen, T.: Recent advances in convolutional neural networks. Pattern Recognit. 77, 354–377 (2018)

    Article  Google Scholar 

  6. LeCun, Y., Haffner, P., Bottou, L., Bengio, Y.: Object recognition with gradient-based learning. In: Shape, Contour and Grouping in Computer Vision, pp. 319–345. Springer, Berlin (1999)

    Chapter  Google Scholar 

  7. Dhillon, A., Verma, G.K.: Convolutional neural network: a review of models, methodologies and applications to object detection. Prog. Artif. Intell. 9(2), 85–112 (2020)

    Article  Google Scholar 

  8. Li, H.C., Deng, Z.Y., Chiang, H.H.: Lightweight and resource-constrained learning network for face recognition with performance optimization. Sensors (Switzerland) 20(21), 1–20 (2020)

    Article  Google Scholar 

  9. Salman, H., Singhal, P., Shankar, T., Yin, P., Salman, A., Paivine, W., Sartoretti, G., Travers, M.: Learning to sequence robot behaviors for visual navigation (2018). arXiv:1907.02057

  10. Laschowski, B., McNally, W., Wong, A., McPhee, J.: Preliminary design of an environment recognition system for controlling robotic lower-limb prostheses and exoskeletons. In: IEEE International Conference on Rehabilitation Robotics, vol. 2019–June, pp. 868–873 (2019)

    Google Scholar 

  11. McNally, W., Vats, K., Wong, A., McPhee, J.: EvoPose2D: pushing the boundaries of 2D human pose estimation using neuroevolution (2020)

  12. Palaz, D., Magimai-Doss, M., Collobert, R.: End-to-end acoustic modeling using convolutional neural networks for HMM-based automatic speech recognition. Speech Commun. 108, 15–32 (2019)

    Article  Google Scholar 

  13. Batur Dinler, Ö., Aydin, N.: An optimal feature parameter set based on gated recurrent unit recurrent neural networks for speech segment detection. Appl. Sci. (Switzerland) 10(4), 1273 (2020).

    Google Scholar 

  14. Yin, W., Kann, K., Yu, M., Schütze, H.: Comparative study of CNN and RNN for natural language processing (2017)

  15. Chandar, S., Sunder, H.: Dynamic systems simulation and control using consecutive recurrent neural networks. Commun. Comput. Inf. Sci. 1290, 92–103 (2020)

    Google Scholar 

  16. Trischler, A.P., D’Eleuterio, G.M.: Synthesis of recurrent neural networks for dynamical system simulation. Neural Netw. 80, 67–78 (2016)

    Article  MATH  Google Scholar 

  17. Hu, Y., Huber, A., Anumula, J., Liu, S.-C.: Overcoming the vanishing gradient problem in plain recurrent networks, Sect. 2, pp. 1–20 (2018)

  18. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  19. Gao, C., Yan, J., Zhou, S., Varshney, P.K., Liu, H.: Long short-term memory-based deep recurrent neural networks for target tracking. Inf. Sci. 502, 279–296 (2019)

    Article  Google Scholar 

  20. Hewamalage, H., Bergmeir, C., Bandara, K.: Recurrent neural networks for time series forecasting: current status and future directions. Int. J. Forecast. 37(1), 388–427 (2021)

    Article  Google Scholar 

  21. Wright, S., Nocedal, J., et al.: Numerical optimization. Springer Sci. 35(67–68), 7 (1999)

    MATH  Google Scholar 

  22. Bottou, L.: Stochastic gradient descent tricks. In: Neural Networks: Tricks of the Trade, pp. 421–436. Springer, Berlin (2012)

    Chapter  Google Scholar 

  23. Zhang, Z.: Improved Adam optimizer for deep neural networks. In: 2018 IEEE/ACM 26th International Symposium on Quality of Service, IWQoS 2018, pp. 1–2 (2019)

    Google Scholar 

  24. Such, F.P., Madhavan, V., Conti, E., Lehman, J., Stanley, K.O., Clune, J.: Deep neuroevolution: genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning (2017)

  25. Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms, pp. 1–9

  26. Wu, X., Kumar, V., Ross, Q.J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z.H., Steinbach, M., Hand, D.J., Steinberg, D.: Top 10 algorithms in data mining. Knowl. Inf. Syst. 14, 1–37 (2008)

    Article  Google Scholar 

  27. Xu, X., Liang, T., Zhu, J., Zheng, D., Sun, T.: Review of classical dimensionality reduction and sample selection methods for large-scale data processing. Neurocomputing 328, 5–15 (2019)

    Article  Google Scholar 

  28. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  29. Bellmann, R.: The theory of dynamic programming. Bull. Am. Math. Soc. 60, 503–515 (1954)

    Article  MathSciNet  Google Scholar 

  30. Lagoudakis, M.G., Parr, R.: Least-squares policy iteration. J. Mach. Learn. Res. 4(6), 1107–1149 (2004)

    MathSciNet  MATH  Google Scholar 

  31. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., Hassabis, D.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)

    Article  Google Scholar 

  32. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    Article  Google Scholar 

  33. Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous learning control with deep reinforcement (2016)

  34. Fujimoto, S., Van Hoof, H., Meger, D.: Addressing function approximation error in actor-critic methods. In: 35th International Conference on Machine Learning ICML 2018, vol. 4, pp. 2587–2601 (2018)

    Google Scholar 

  35. Goodfellow, I.J., Pouget-abadie, J., Mirza, M., Xu, B., Warde-farley, D.: Generative adversarial nets, pp. 1–9

  36. Ren, H., Ben-Tzvi, P.: Learning inverse kinematics and dynamics of a robotic manipulator using generative adversarial networks. Robot. Auton. Syst. 124, 103386 (2020)

    Article  Google Scholar 

  37. Ardeh, H.A., Tupy, M., Negrut, D.: On the construction and use of surrogate models for the dynamic analysis of multibody systems. In: Volume 13: New Developments in Simulation Methods and Software for Engineering Applications; Safety Engineering, Risk Analysis and Reliability Methods; Transportation Systems, ASMEDC, vol. 13, pp. 17–26 (2009)

    Google Scholar 

  38. Azzam, B., Schelenz, R., Roscher, B., Baseer, A., Jacobs, G.: Development of a wind turbine gearbox virtual load sensor using multibody simulation and artificial neural networks. Forsch. Ingenieurwes./Eng. Res. 85, 241–250 (2021)

    Article  Google Scholar 

  39. García Peyrano, O., Vignolo, J., Mayer, R., Marticorena, M.: Online unbalance detection and diagnosis on large flexible rotors by SVR and ANN trained by dynamic multibody simulations. J. Dyn. Mon. Diag. 1(3) (2022). https://doi.org/10.37965/jdmd.2022.58

  40. Han, S., Choi, H.S., Choi, J., Choi, J.H., Kim, J.G.: A DNN-based data-driven modeling employing coarse sample data for real-time flexible multibody dynamics simulations. Comput. Methods Appl. Mech. Eng. 373, 113480 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  41. Kahr, M., Kovács, G., Loinig, M., Brückl, H.: Condition monitoring of ball bearings based on machine learning with synthetically generated data. Sensors 22(7), 2490 (2022)

    Article  Google Scholar 

  42. Ma, J., Dong, S., Chen, G., Peng, P., Qian, L.: A data-driven normal contact force model based on artificial neural network for complex contacting surfaces. Mech. Syst. Signal Process. 156, 107612 (2021)

    Article  Google Scholar 

  43. Sobie, C., Freitas, C., Nicolai, M.: Simulation-driven machine learning: bearing fault classification. Mech. Syst. Signal Process. 99, 403–419 (2018)

    Article  Google Scholar 

  44. Ye, Y., Zhu, B., Huang, P., Peng, B.: Oornet: a deep learning model for on-board condition monitoring and fault diagnosis of out-of-round wheels of high-speed trains. Measurement 199, 111268 (2022)

    Article  Google Scholar 

  45. Nasr, A., Inkol, K.A., Bell, S., McPhee, J.: InverseMuscleNET: alternative machine learning solution to static optimization and inverse muscle modeling. Front. Comput. Neurosci. 15 (2021). https://doi.org/10.3389/fncom.2021.759489

  46. Polydoros, A.S., Nalpantidis, L., Kruger, V.: Real-time deep learning of robotic manipulator inverse dynamics. In: IEEE International Conference on Intelligent Robots and Systems, vol. 2015–Decem, no. October, pp. 3442–3448 (2015)

    Google Scholar 

  47. Rane, L., Ding, Z., McGregor, A.H., Bull, A.M.: Deep learning for musculoskeletal force prediction. Ann. Biomed. Eng. 47(3), 778–789 (2019)

    Article  Google Scholar 

  48. Zhou, S., Schoellig, A.P.: Active training trajectory generation for inverse dynamics model learning with deep neural networks. In: Proceedings of the IEEE Conference on Decision and Control, vol. 2019–Decem, no. i, pp. 1784–1790 (2019)

    Google Scholar 

  49. Byravan, A., Fox, D.: SE3-nets: learning rigid body motion using deep neural networks. In: 2017 IEEE International Conference on Robotics and Automation, ICRA, pp. 173–180. IEEE, Singapore (2017)

    Google Scholar 

  50. Choi, H.S., An, J., Han, S., Kim, J.G., Jung, J.Y., Choi, J., Orzechowski, G., Mikkola, A., Choi, J.H.: Data-driven simulation for general-purpose multibody dynamics using Deep Neural Networks. Multibody Syst. Dyn. 51(4), 419–454 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  51. Hegedüs, F., Gáspár, P., Bécsi, T.: Fast motion model of road vehicles with artificial neural networks. Electronics 10, 928 (2021)

    Article  Google Scholar 

  52. Kraft, S., Causse, J., Martinez, A.: Black-box modelling of nonlinear railway vehicle dynamics for track geometry assessment using neural networks. Veh. Syst. Dyn. 57(9), 1241–1270 (2019)

    Article  Google Scholar 

  53. Martin, T.P., Zaazaa, K.E., Whitten, B., Tajaddini, A.: Using a multibody dynamic simulation code with neural network technology to predict railroad vehicle-track interaction performance in real time. In: Volume 5: 6th International Conference on Multibody Systems, Nonlinear Dynamics, and Control, Parts A, B, and C, ASMEDC, vol. 5 PART C, pp. 1881–1891 (2007)

    Google Scholar 

  54. Nasr, A., Bell, S., He, J., Whittaker, R.L., Jiang, N., Dickerson, C.R., McPhee, J.: Musclenet: mapping electromyography to kinematic and dynamic biomechanical variables by machine learning. J. Neural Eng. 18(4), 0460d3 (2021)

    Article  Google Scholar 

  55. Pan, Y., Nie, X., Li, Z., Gu, S.: Data-driven vehicle modeling of longitudinal dynamics based on a multibody model and deep neural networks. Measurement 180, 109541 (2021)

    Article  Google Scholar 

  56. Urda, P., Aceituno, J.F., Muñoz, S., Escalona, J.L.: Artificial neural networks applied to the measurement of lateral wheel-rail contact force: a comparison with a harmonic cancellation method. Mech. Mach. Theory 153, 103968 (2020)

    Article  Google Scholar 

  57. Angeli, A., Desmet, W., Naets, F.: Deep learning for model order reduction of multibody systems to minimal coordinates. Comput. Methods Appl. Mech. Eng. 373, 113517 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  58. Hosking, B.A., McPhee, J.: Powertrain modeling and model predictive longitudinal dynamics control for hybrid electric vehicles. In: SAE Technical Papers, vol. 2018–April, pp. 1–10 (2018)

    Google Scholar 

  59. Oishi, A., Yagawa, G.: Computational mechanics enhanced by deep learning. Comput. Methods Appl. Mech. Eng. 327, 327–351 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  60. Ye, Y., Huang, P., Sun, Y., Shi, D.: MBSNet: a deep learning model for multibody dynamics simulation and its application to a vehicle-track system. Mech. Syst. Signal Process. 157, 107716 (2021)

    Article  Google Scholar 

  61. Saxby, D.J., Killen, B.A., Pizzolato, C., Carty, C.P., Diamond, L.E., Modenese, L., Fernandez, J., Davico, G., Barzan, M., Lenton, G., da Luz, S.B., Suwarganda, E., Devaprakash, D., Korhonen, R.K., Alderson, J.A., Besier, T.F., Barrett, R.S., Lloyd, D.G.: Machine learning methods to support personalized neuromusculoskeletal modelling. Biomech. Model. Mechanobiol. 19(4), 1169–1185 (2020)

    Article  Google Scholar 

  62. Brunton, S.L., Kutz, J.N., Manohar, K., Aravkin, A.Y., Morgansen, K., Klemisch, J., Goebel, N., Buttrick, J., Poskin, J., Blom-Schieber, A.W., Hogan, T., McDonald, D.: Data-driven aerospace engineering: reframing the industry with machine learning. AIAA J. 59(8), 1–26 (2021)

    Article  Google Scholar 

  63. Hashemi, A., Lin, Y., McNally, W., Laschowski, B., Hosking, B., Wong, A., McPhee, J.: Integration of machine learning with dynamics and control: from autonomous cars to biomechatronics. Canadian Society for Mechanical Engineering (CSME) Bulletin, 9–10 (2019)

  64. Nasr, A., Hashemi, A., McPhee, J.: Model-based mid-level regulation for assist-as-needed hierarchical control of wearable robots: a computational study of human-robot adaptation. Robotics 11(1), 20 (2022)

    Article  Google Scholar 

  65. Spielberg, N.A., Brown, M., Kapania, N.R., Kegelman, J.C., Gerdes, J.C.: Neural network vehicle models for high-performance automated driving. Sci. Robot. 4(28) (2019). https://doi.org/10.1126/scirobotics.aaw1975

  66. Bertsekas, D.: Reinforcement Learning and Optimal Control. Athena Scientific, United States (2019)

    Google Scholar 

  67. Richards, A.G.: Robust constrained model predictive control. PhD thesis, Massachusetts Institute of Technology (2005)

  68. De Boer, P.-T., Kroese, D.P., Mannor, S., Rubinstein, R.Y.: A tutorial on the cross-entropy method. Ann. Oper. Res. 134(1), 19–67 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  69. Kelly, M.: An introduction to trajectory optimization: how to do your own direct collocation. SIAM Rev. 59(4), 849–904 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  70. Pal, C.-V., Leon, F.: Brief survey of model-based reinforcement learning techniques. In: 2020 24th International Conference on System Theory, Control and Computing, ICSTCC, pp. 92–97. IEEE, Romania (2020)

    Google Scholar 

  71. Wang, T., Bao, X., Clavera, I., Hoang, J., Wen, Y., Langlois, E., Zhang, S., Zhang, G., Abbeel, P., Ba, J.: Benchmarking model-based reinforcement learning (2019). arXiv preprint. arXiv:1907.02057

  72. Kwakernaak, H., Sivan, R., Tyreus, B.N.D.: Linear Optimal Control Systems (1974)

    Book  Google Scholar 

  73. Tassa, Y., Erez, T., Todorov, E.: Synthesis and stabilization of complex behaviors through online trajectory optimization. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4906–4913. IEEE, Portugal (2012)

    Chapter  Google Scholar 

  74. Lee, J.H.: Model predictive control: review of the three decades of development. Int. J. Control. Autom. Syst. 9(3), 415–424 (2011)

    Article  MathSciNet  Google Scholar 

  75. Schwenzer, M., Ay, M., Bergs, T., Abel, D.: Review on model predictive control: an engineering perspective. Int. J. Adv. Manuf. Technol. 117(5), 1327–1349 (2021)

    Article  Google Scholar 

  76. Roveda, L., Maskani, J., Franceschi, P., Abdi, A., Braghin, F., Molinari Tosatti, L., Pedrocchi, N.: Model-based reinforcement learning variable impedance control for human-robot collaboration. J. Intell. Robot. Syst. 100(2), 417–433 (2020)

    Article  Google Scholar 

  77. Nagabandi, A., Kahn, G., Fearing, R.S., Levine, S.: Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 7579–7586 (2018)

    Google Scholar 

  78. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: OpenAI Gym, pp. 1–4 (2016)

  79. Williams, G., Wagener, N., Goldfain, B., Drews, P., Rehg, J.M., Boots, B., Theodorou, E.A.: Information theoretic MPC for model-based reinforcement learning. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 1714–1721 (2017)

    Google Scholar 

  80. Richards, S., Azizan, N., Slotine, J.-J., Pavone, M.: Adaptive-control-oriented meta-learning for nonlinear systems (2021)

  81. Punjani, A., Abbeel, P.: Deep learning helicopter dynamics models. In: 2015 IEEE International Conference on Robotics and Automation, ICRA, pp. 3223–3230 (2015)

    Google Scholar 

  82. Bansal, S., Jiang, F.J., Tomlin, C.J., Oct, S.Y.: Learning quadrotor dynamics using neural network for flight control, no. 0931843

  83. Rutherford, S.J., Cole, D.J.: Modelling nonlinear vehicle dynamics with neural networks. Int. J. Veh. Des. 53(4), 260–287 (2010)

    Article  Google Scholar 

  84. Angeli, A., Desmet, W., Naets, F.: Deep learning of multibody minimal coordinates for state and input estimation with Kalman filtering. Multibody Syst. Dyn. 53(2), 205–223 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  85. Alessandri, A., Baglietto, M., Battistelli, G., Zavala, V.: Advances in moving horizon estimation for nonlinear systems, pp. 5681–5688 (2010)

  86. Jin, C., Maitland, A., McPhee, J.: Hierarchical Nonlinear Moving Horizon Estimation of Vehicle Lateral Speed and Road Friction Coefficient. Letters Dyn. Sys. Control. 1(1), 011005 (2021)

    Google Scholar 

  87. Sun, D., Liao, Q., Stoyanov, T., Kiselev, A., Loutfi, A.: Bilateral telerobotic system using Type-2 fuzzy neural network based moving horizon estimation force observer for enhancement of environmental force compliance and human perception. Automatica 106, 358–373 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  88. Alessio, A., Bemporad, A.: A survey on explicit model predictive control. Lect. Notes Control Inf. Sci. 384, 345–369 (2009)

    MATH  Google Scholar 

  89. Winqvist, R., Venkitaraman, A., Wahlberg, B.: On training and evaluation of neural network approaches for model predictive control (2020)

  90. Diamond, S., Boyd, S.: Cvxpy: a python-embedded modeling language for convex optimization. J. Mach. Learn. Res. 17(1), 2909–2913 (2016)

    MathSciNet  MATH  Google Scholar 

  91. Varshney, P., Nagar, G., Saha, I.: DeepControl: energy-efficient control of a quadrotor using a deep neural network. In: IEEE International Conference on Intelligent Robots and Systems, pp. 43–50 (2019)

    Google Scholar 

  92. Pan, Y., Wang, J.: Nonlinear model predictive control using a recurrent neural network. In: Proceedings of the International Joint Conference on Neural Networks, no. July, pp. 2296–2301 (2008)

    Google Scholar 

  93. Cao, Y., Gopaluni, R.B.: Deep neural network approximation of nonlinear model predictive control. IFAC-PapersOnLine 53(2), 11319–11324 (2020)

    Article  Google Scholar 

  94. Roberts, J.W., Manchester, I.R., Tedrake, R.: Feedback controller parameterizations for reinforcement learning. In: IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, pp. 310–317 (2011)

    Google Scholar 

  95. Lewis, F.L., Vrabie, D.: Adaptive dynamic programming for feedback control. In: Proceedings of 2009 7th Asian Control Conference ASCC 2009, pp. 1402–1409 (2009)

    Google Scholar 

  96. Kober, J., Bagnell, J.A., Peters, J.: Kober_IJRR_2013, pp. 1–73 (2013)

  97. Sun, C., Orbik, J., Devin, C., Yang, B., Gupta, A., Berseth, G., Levine, S.: Fully autonomous real-world reinforcement learning for mobile manipulation, pp. 1–16 (2021)

  98. Li, M., Wen, Y., Gao, X., Si, J., Huang, H.: Toward expedited impedance tuning of a robotic prosthesis for personalized gait assistance by reinforcement learning control. IEEE Trans. Robot. 38(1), 407–420 (2021)

    Article  Google Scholar 

  99. Song, S., Kidziński, Ł., Peng, X.B., Ong, C., Hicks, J., Levine, S., Atkeson, C.G., Delp, S.L.: Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation. J. NeuroEng. Rehabil. 18(1), 1–17 (2021)

    Article  Google Scholar 

  100. Lee, S., Park, M., Lee, K., Lee, J.: Scalable muscle-actuated human simulation and control. ACM Trans. Graph. 38(4), 73 (2019)

    Article  Google Scholar 

  101. Weng, J., Hashemi, E., Arami, A.: Natural walking with musculoskeletal models using deep reinforcement learning. IEEE Robot. Autom. Lett. 6(2), 4156–4162 (2021)

    Article  Google Scholar 

  102. Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 41–48 (2009)

    Chapter  Google Scholar 

  103. Lin, Y., McPhee, J., Azad, N.L.: Comparison of deep reinforcement learning and model predictive control for adaptive cruise control. IEEE Trans. Intell. Veh. 6(2), 221–231 (2021)

    Article  Google Scholar 

  104. Buşoniu, L., de Bruin, T., Tolić, D., Kober, J., Palunko, I.: Reinforcement learning for control: performance, stability, and deep approximators. Annu. Rev. Control 46, 8–28 (2018)

    Article  MathSciNet  Google Scholar 

  105. Hewing, L., Wabersich, K.P., Menner, M., Zeilinger, M.N.: Learning-based model predictive control: toward safe learning in control. Annu. Rev. Control Robot. Auton. Syst. 3(1), 269–296 (2020)

    Article  Google Scholar 

  106. Hussein, A., Gaber, M.M., Elyan, E., Jayne, C.: Imitation learning: a survey of learning methods. ACM Comput. Surv. 50(2), 1–35 (2017)

    Article  Google Scholar 

  107. Wu, Y., Zhao, F., Tao, T., Ajoudani, A.: A framework for autonomous impedance regulation of robots based on imitation learning and optimal control. IEEE Robot. Autom. Lett. 6(1), 127–134 (2020)

    Article  Google Scholar 

  108. Huang, Y., Rozo, L., Silverio, J., Caldwell, D.G.: Non-parametric imitation learning of robot motor skills. In: Proceedings - IEEE International Conference on Robotics and Automation, vol. 2019–May, pp. 5266–5272 (2019)

    Google Scholar 

  109. Finn, C., Yu, T., Zhang, T., Abbeel, P., Levine, S.: One-shot visual imitation learning via meta-learning. In: CoRL, pp. 1–12 (2017)

    Google Scholar 

  110. Kebria, P.M., Khosravi, A., Salaken, S.M., Nahavandi, S.: Deep imitation learning for autonomous vehicles based on convolutional neural networks. IEEE/CAA J. Autom. Sin. 7(1), 82–95 (2020)

    Article  Google Scholar 

  111. Azar, N.A., Shahmansoorian, A., Davoudi, M.: From inverse optimal control to inverse reinforcement learning: a historical review. Annu. Rev. Control 50, 119–138 (2020)

    Article  MathSciNet  Google Scholar 

  112. You, C., Lu, J., Filev, D., Tsiotras, P.: Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning. Robot. Auton. Syst. 114, 1–18 (2019)

    Article  Google Scholar 

  113. Wu, Z., Sun, L., Zhan, W., Yang, C., Tomizuka, M.: Efficient sampling-based maximum entropy inverse reinforcement learning with application to autonomous driving. IEEE Robot. Autom. Lett. 5(4), 5355–5362 (2020)

    Article  Google Scholar 

  114. Sharifzadeh, S., Chiotellis, I., Triebel, R., Cremers, D.: Learning to drive using inverse reinforcement learning and deep Q-networks. In: Nips, pp. 1–7 (2016)

    Google Scholar 

  115. Woodworth, B., Ferrari, F., Zosa, T.E., Riek, L.D.: Preference learning in assistive robotics: observational repeated inverse reinforcement learning. Proc. Mach. Learn. Res. 85, 1–19 (2018)

    Google Scholar 

  116. Bing, Z., Lemke, C., Cheng, L., Huang, K., Knoll, A.: Energy-efficient and damage-recovery slithering gait design for a snake-like robot based on reinforcement learning and inverse reinforcement learning. Neural Netw. 129, 323–333 (2020)

    Article  Google Scholar 

  117. Okafor, E., Udekwe, D., Ibrahim, Y., Mu’azu, M.B., Okafor, E.G.: Heuristic and deep reinforcement learning-based PID control of trajectory tracking in a ball-and-plate system. J. Inf. Telecommun. 5(2), 179–196 (2021)

    Google Scholar 

  118. Bilgic, H.H., Sen, M.A., Kalyoncu, M.: Tuning of LQR controller for an experimental inverted pendulum system based on the bees algorithm. J. Vibroeng. 18(6), 3684–3694 (2016)

    Article  Google Scholar 

  119. Marco, A., Hennig, P., Bohg, J., Schaal, S., Trimpe, S.: Automatic LQR tuning based on Gaussian process global optimization. In: Proceedings - IEEE International Conference on Robotics and Automation, vol. 2016–June, pp. 270–277 (2016)

    Google Scholar 

  120. Hussein, S.U., Muhammed, B.M., Sikiru, T.H., Umoh, I.J., Salawudeen, A.T.: Trajectory tracking control of ball on plate system using weighted artificial fish swarm algorithm based PID. In: 2017 IEEE 3rd International Conference on Electro-Technology for National Development, NIGERCON 2017, vol. 2018–Janua, no. June 2018, pp. 554–561 (2018)

    Google Scholar 

  121. Hashemi, A., McPhee, J.: Assistive sliding mode control of a rehabilitation robot with automatic weight adjustment. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, EMBC, pp. 4891–4896 (2021)

    Google Scholar 

  122. Shipman, W.J., Coetzee, L.C.: Reinforcement learning and deep neural networks for PI controller tuning. IFAC-PapersOnLine 52(14), 111–116 (2019)

    Article  Google Scholar 

  123. Carlucho, I., De Paula, M., Acosta, G.G.: An adaptive deep reinforcement learning approach for MIMO PID control of mobile robots. ISA Trans. 102, 280–294 (2020)

    Article  Google Scholar 

  124. El Hakim, A., Hindersah, H., Rijanto, E.: Application of reinforcement learning on self-tuning PID controller for soccer robot multi-agent system. In: Proceedings of the 2013 Joint International Conference on Rural Information and Communication Technology and Electric-Vehicle Technology, rICT and ICEV-T 2013 (2013)

    Google Scholar 

  125. Ure, N.K., Yavas, M.U., Alizadeh, A., Kurtulus, C.: Enhancing situational awareness and performance of adaptive cruise control through model predictive control and deep reinforcement learning. In: IEEE Intelligent Vehicles Symposium, Proceedings, vol. 2019–June, no. IV, pp. 626–631 (2019)

    Google Scholar 

  126. Jardine, P.T., Givigi, S.N., Yousefi, S.: Experimental results for autonomous model-predictive trajectory planning tuned with machine learning. In: 11th Annual IEEE International Systems Conference, SysCon 2017 - Proceedings (2017)

    Google Scholar 

  127. Jardine, P.T., Kogan, M., Givigi, S.N., Yousefi, S.: Adaptive predictive control of a differential drive robot tuned with reinforcement learning. Int. J. Adapt. Control Signal Process. 33(2), 410–423 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  128. Mehndiratta, M., Camci, E., Kayacan, E.: Automated tuning of nonlinear model predictive controller by reinforcement learning. In: IEEE International Conference on Intelligent Robots and Systems, no. October, pp. 3016–3021 (2018)

    Google Scholar 

  129. Liu, C., Wen, G., Zhao, Z., Sedaghati, R.: Neural-network-based sliding-mode control of an uncertain robot using dynamic model approximated switching gain. IEEE Trans. Cybern. 51(5), 2339–2346 (2021)

    Article  Google Scholar 

  130. Wen, Y., Si, J., Brandt, A., Gao, X., Huang, H.H.: Online reinforcement learning control for the personalization of a robotic knee prosthesis. IEEE Trans. Cybern. 50(6), 2346–2356 (2020)

    Article  Google Scholar 

  131. Wu, W., Saul, K.R., Huang, H.: Using reinforcement learning to estimate human joint moments from electromyography or joint kinematics: an alternative solution to musculoskeletal-based biomechanics. J. Biomech. Eng. 143(4), 1–9 (2021)

    Article  Google Scholar 

  132. Tu, X., Li, M., Liu, M., Si, J., Huang, H.: A data-driven reinforcement learning solution framework for optimal and adaptive personalization of a hip exoskeleton (2020)

  133. Wu, Q., Chen, B., Wu, H.: Adaptive admittance control of an upper extremity rehabilitation robot with neural-network-based disturbance observer. IEEE Access 7, 123807–123819 (2019)

    Article  Google Scholar 

  134. Abdollahi, F., Talebi, H.A., Patel, R.V.: A stable neural network observer with application to flexible-joint manipulators. In: ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age, vol. 4, no. 1, pp. 1910–1914 (2002)

    Google Scholar 

  135. He, W., Sun, Y., Yan, Z., Yang, C., Li, Z., Kaynak, O.: Disturbance observer-based neural network control of cooperative multiple manipulators with input saturation. IEEE Trans. Neural Netw. Learn. Syst. 31(5), 1735–1746 (2020)

    Article  MathSciNet  Google Scholar 

  136. Luan, F., Na, J., Huang, Y., Gao, G.: Adaptive neural network control for robotic manipulators with guaranteed finite-time convergence. Neurocomputing 337, 153–164 (2019)

    Article  Google Scholar 

  137. Yen, V.T., Nan, W.Y., Van Cuong, P.: Robust adaptive sliding mode neural networks control for industrial robot manipulators. Int. J. Control. Autom. Syst. 17(3), 783–792 (2019)

    Article  Google Scholar 

  138. Zhou, Q., Zhao, S., Li, H., Lu, R., Wu, C.: Robotic manipulators with dead zone. IEEE Trans. Neural Netw. Learn. Syst. PP(12), 1–10 (2018)

    Google Scholar 

  139. Ouyang, Y., Dong, L., Xue, L., Sun, C.: Adaptive control based on neural networks for an uncertain 2-DOF helicopter system with input deadzone and output constraints. IEEE/CAA J. Autom. Sin. 6(3), 807–815 (2019)

    Article  MathSciNet  Google Scholar 

  140. Wu, Y., Huang, R., Li, X., Liu, S.: Adaptive neural network control of uncertain robotic manipulators with external disturbance and time-varying output constraints. Neurocomputing 323, 108–116 (2019)

    Article  Google Scholar 

  141. Pan, Y., Cheng, C.-A., Saigol, K., Lee, K., Yan, X., Theodorou, E., Boots, B.: Agile autonomous driving using end-to-end deep imitation learning (2018)

  142. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  143. Lutter, M., Ritter, C., Peters, J.: Deep Lagrangian networks: using physics as model prior for deep learning. In: 7th International Conference on Learning Representations, ICLR 2019, pp. 1–17 (2019)

    Google Scholar 

  144. Greydanus, S., Dzamba, M., Yosinski, J.: Hamiltonian neural networks. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Red Hook (2019)

    Google Scholar 

  145. Cranmer, M., Greydanus, S., Hoyer, S., Battaglia, P., Spergel, D., Ho, S.: Lagrangian neural networks (2020). arXiv:2003.04630 [physics, stat]

  146. Krishnapriyan, A., Gholami, A., Zhe, S., Kirby, R., Mahoney, M.W.: Characterizing possible failure modes in physics-informed neural networks. In: Advances in Neural Information Processing Systems, vol. 34, pp. 26548–26560. Curran Associates, Red Hook (2021)

    Google Scholar 

  147. Lea, C., Flynn, M.D., Vidal, R., Reiter, A., Hager, G.D.: Temporal convolutional networks for action segmentation and detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 156–165 (2017)

    Google Scholar 

  148. Andersson, C., Ribeiro, A.H., Tiels, K., Wahlstrom, N., Schon, T.B.: Deep convolutional networks in system identification. In: Proceedings of the IEEE Conference on Decision and Control, vol. 2019–Decem, pp. 3670–3676 (2019)

    Google Scholar 

  149. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  150. Lin, T., Wang, Y., Liu, X., Qiu, X.: A survey of transformers (2021). arXiv preprint. arXiv:2106.04554

  151. Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., Ying, R., Leskovec, J., Battaglia, P.: Learning to simulate complex physics with graph networks. In: International Conference on Machine Learning, PMLR, pp. 8459–8468 (2020)

    Google Scholar 

Download references

Funding

The first and last authors gratefully acknowledge funding provided by the Canada Research Chairs program and the Natural Sciences and Engineering Research Council of Canada. The second and third authors gratefully acknowledge funding provided by Business Finland’s “Kumppanuusmalli – SANTTU – LUT” project under grant 8859/31/2021.

Author information

Authors and Affiliations

  1. University of Waterloo, Waterloo, Canada

    Arash Hashemi & John McPhee

  2. LUT University, Lappeenranta, Finland

    Grzegorz Orzechowski & Aki Mikkola

Authors
  1. Arash Hashemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Grzegorz Orzechowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aki Mikkola
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. John McPhee
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Arash Hashemi: conceptualization, visualization, writing (Sects. 2, 4, 5, 6, 7), review and editing. Grzegorz Orzechowski: conceptualization, visualization, writing (Abstract, Sects. 1, 3), review and editing. Aki Mikkola: conceptualization, funding acquisition, supervision, writing (Abstract, Sects. 1, 3), review and editing. John McPhee: conceptualization, funding acquisition, supervision, writing (Sects. 2, 4, 5, 6, 7), review and editing.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hashemi, A., Orzechowski, G., Mikkola, A. et al. Multibody dynamics and control using machine learning. Multibody Syst Dyn 58, 397–431 (2023). https://doi.org/10.1007/s11044-023-09884-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11044-023-09884-x

Keywords

Search

Navigation