Skip to main content
SpringerLink
Log in

Computational Sensing, Understanding, and Reasoning: An Artificial Intelligence Approach to Physics-Informed World Modeling

  • Review article
  • Published:
Archives of Computational Methods in Engineering Aims and scope Submit manuscript

Abstract

This work offers a discussion on how computational mechanics and physics-informed machine learning can be integrated into the process of sensing, understanding, and reasoning of physical phenomena. A foundation in physics can leverage interpretability, data efficiency, and generalization of the models sought for the dynamics of complex physical systems. Consequently, this synergy results in promising approaches to develop world models that are capable of performing accurate and reliable simulations (reasoning) in low-data regimes. Among the possible alternative formulations, we highlight how thermodynamics offers a general framework to construct inductive biases, demonstrating its potential in applications where physics-consistent predictions are essential.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

Not applicable.

Code Availability

Not applicable.

References

  1. Tamkin A, Brundage M, Clark J, Ganguli D (2021) Understanding the capabilities, limitations, and societal impact of large language models. arXiv preprint arXiv:2102.02503

  2. Li K, Hopkins AK, Bau D, Viégas F, Pfister H, Wattenberg M (2022) Emergent world representations: exploring a sequence model trained on a synthetic task. arXiv preprint arXiv:2210.13382

  3. Lake BM, Ullman TD, Tenenbaum JB, Gershman SJ (2017) Building machines that learn and think like people. Behav Brain Sci 40:253

    Google Scholar 

  4. Kubricht JR, Holyoak KJ, Lu H (2017) Intuitive physics: current research and controversies. Trends Cogn Sci 21(10):749–759

    Google Scholar 

  5. McCloskey M (1983) Intuitive physics. Sci Am 248(4):122–131

    Google Scholar 

  6. Reynolds CR, Fletcher-Janzen E (2007) Encyclopedia of special education: a reference for the education of children, adolescents, and adults with disabilities and other exceptional individuals, vol 3. Wiley, New York

    Google Scholar 

  7. Piloto LS, Weinstein A, Battaglia P, Botvinick M (2022) Intuitive physics learning in a deep-learning model inspired by developmental psychology. Nature human behaviour 6(9):1257–1267

    Google Scholar 

  8. Allen KR, Lopez-Guevara T, Stachenfeld K, Sanchez-Gonzalez A, Battaglia P, Hamrick J, Pfaff T (2022) Physical design using differentiable learned simulators. arXiv preprint arXiv:2202.00728

  9. Sanchez-Gonzalez A, Godwin J, Pfaff T, Ying R, Leskovec J, Battaglia P (2020) Learning to simulate complex physics with graph networks. In: International conference on machine learning, pp 8459–8468. PMLR

  10. Hernandez Q, Badias A, Gonzalez D, Chinesta F, Cueto E (2021) Deep learning of thermodynamics-aware reduced-order models from data. Comput. Methods Appl. Mech. Eng. 379:113763

    MathSciNet  Google Scholar 

  11. Finn C, Levine S (2017) Deep visual foresight for planning robot motion. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp 2786–2793. IEEE

  12. Liu CK, Negrut D (2021) The role of physics-based simulators in robotics. Ann Rev Control Robot Autonom Syst 4:35–58

    Google Scholar 

  13. Allen KR, Smith KA, Tenenbaum JB (2020) Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning. Proc Natl Acad Sci 117(47):29302–29310

    Google Scholar 

  14. Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R, et al (2018) Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261

  15. Zheng N, Liu Z, Ren P, Ma Y, Chen S, Yu S, Xue J, Chen B, Wang F (2017) Hybrid-augmented intelligence: collaboration and cognition. Front Inf Technol Electron Eng 18(2):153–179

    Google Scholar 

  16. Rui Y (2017) From artificial intelligence to augmented intelligence. IEEE MultiMed 24(1):4–5

    Google Scholar 

  17. Chinesta F, Cueto E, Abisset-Chavanne E, Duval JL, Khaldi FE (2020) Virtual, digital and hybrid twins: a new paradigm in data-based engineering and engineered data. Arch Comput Methods Eng 27:105–134

    MathSciNet  Google Scholar 

  18. Moya B, Badías A, Alfaro I, Chinesta F, Cueto E (2022) Digital twins that learn and correct themselves. Int J Numer Methods Eng 123(13):3034–3044

    Google Scholar 

  19. Cerf VG (2013) Augmented intelligence. IEEE Internet Comput 17(5):96–96

    Google Scholar 

  20. Bar M (2009) The proactive brain: memory for predictions. Philos Trans R Soc B 364(1521):1235–1243

    Google Scholar 

  21. Hamrick J, Battaglia P, Tenenbaum JB (2011) Internal physics models guide probabilistic judgments about object dynamics. In: Proceedings of the 33rd annual conference of the Cognitive Science Society, vol 2. Cognitive Science Society, Austin, TX

  22. Battaglia PW, Hamrick JB, Tenenbaum JB (2013) Simulation as an engine of physical scene understanding. Proc Natl Acad Sci 110(45):18327–18332

    Google Scholar 

  23. Traylor A, Feiman R, Pavlick E (2022) Can neural networks learn implicit logic from physical reasoning? In: The eleventh international conference on learning representations

  24. Murphy KP, Torralba A, Freeman W (2003) Using the forest to see the trees: a graphical model relating features, objects, and scenes. In: Advances in neural information processing systems 16

  25. Gupta A, Efros AA, Hebert M (2010) Blocks world revisited: Image understanding using qualitative geometry and mechanics. In: Computer Vision–ECCV 2010: 11th European conference on computer vision, Heraklion, Crete, Greece, September 5–D11, 2010, Proceedings, Part IV 11, pp 482–496. Springer

  26. Schenck C, Fox D (2016) Detection and tracking of liquids with fully convolutional networks. arXiv preprint arXiv:1606.06266

  27. Shen B, Yan X, Qi CR, Najibi M, Deng B, Guibas L, Zhou Y, Anguelov D (2023) Gina-3d: learning to generate implicit neural assets in the wild. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4913–4926

  28. Kandukuri R, Achterhold J, Moeller M, Stueckler J (2020) Learning to identify physical parameters from video using differentiable physics. In: DAGM German conference on pattern recognition, pp 44–57. Springer

  29. Wu J, Yildirim I, Lim JJ, Freeman B, Tenenbaum J (2015) Galileo: Perceiving physical object properties by integrating a physics engine with deep learning. In: Advances in neural information processing systems 28

  30. Mrowca D, Zhuang C, Wang E, Haber N, Fei-Fei LF, Tenenbaum J, Yamins DL (2018) Flexible neural representation for physics prediction. In: Advances in neural information processing systems 31

  31. Bender J, Erleben K, Trinkle J (2014) Interactive simulation of rigid body dynamics in computer graphics. In: Computer graphics forum, vol. 33, pp 246–270. Wiley Online Library

  32. Rath L, Geist AR, Trimpe S (2022) Using physics knowledge for learning rigid-body forward dynamics with gaussian process force priors. In: Conference on robot learning, pp 101–111. PMLR

  33. Huang S, Cheng Z-Q, Li X, Wu X, Zhang Z, Hauptmann A (2018) Perceiving physical equation by observing visual scenarios. arXiv preprint arXiv:1811.12238

  34. Li Y, Wu J, Tedrake R, Tenenbaum JB, Torralba A (2018) Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. arXiv preprint arXiv:1810.01566

  35. Schenck C, Fox D (2018) Spnets: differentiable fluid dynamics for deep neural networks. In: Conference on robot learning, pp 317–335. PMLR

  36. Duchaine V, Gosselin C (2009) Safe, stable and intuitive control for physical human-robot interaction. In: 2009 IEEE international conference on robotics and automation, pp 3383–3388. IEEE

  37. Koppula HS, Saxena A (2015) Anticipating human activities using object affordances for reactive robotic response. IEEE Trans Pattern Anal Mach Intell 38(1):14–29

    Google Scholar 

  38. Liu X-Y, Wang J-X (2021) Physics-informed dyna-style model-based deep reinforcement learning for dynamic control. Proc R Soc A 477(2255):20210618

    MathSciNet  Google Scholar 

  39. Driess D, Schubert I, Florence P, Li Y, Toussaint M (2022) Reinforcement learning with neural radiance fields. Adv Neural Inf Process Syst 35:16931–16945

    Google Scholar 

  40. Badias A, Alfaro I, Gonzalez D, Chinesta F, Cueto E (2021) Morph-dslam: model order reduction for physics-based deformable slam. IEEE Trans Pattern Anal Mach Intell 44(11):7764–7777

    Google Scholar 

  41. Wei B, Zhao Y, Hao K, Gao L (2021) Visual sensation and perception computational models for deep learning: state of the art, challenges and prospects. arXiv preprint arXiv:2109.03391

  42. Assen JJR, Nishida S, Fleming RW (2020) Visual perception of liquids: insights from deep neural networks. PLoS Comput Biol 16(8):1008018

    Google Scholar 

  43. Zhang Y, Dong Z, Obaidat MS, Ban X (2023) Non-Newtonian fluid simulation and reconstruction from monocular videos. Simul Model Pract Theory 123:102688

    Google Scholar 

  44. Lopez-Guevara T, Pucci R, Taylor NK, Gutmann MU, Ramamoorthy S, Suhr K (2020) Stir to pour: Efficient calibration of liquid properties for pouring actions. In: 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 5351–5357. IEEE

  45. Della Santina C, Truby RL, Rus D (2020) Data-driven disturbance observers for estimating external forces on soft robots. IEEE Robot Autom Lett 5(4):5717–5724

    Google Scholar 

  46. Richter F, Orosco RK, Yip MC (2022) Image based reconstruction of liquids from 2d surface detections. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13811–13820

  47. Schenck C, Fox D (2018) Perceiving and reasoning about liquids using fully convolutional networks. Int J Robot Res 37(4–5):452–471

    Google Scholar 

  48. Kloss A, Schaal S, Bohg J (2022) Combining learned and analytical models for predicting action effects from sensory data. Int J Robot Res 41(8):778–797

    Google Scholar 

  49. Degrave J, Hermans M, Dambre J et al (2019) A differentiable physics engine for deep learning in robotics. Front Neurorobot 6:1

    Google Scholar 

  50. Avila Belbute-Peres F, Smith K, Allen K, Tenenbaum J, Kolter JZ (2018) End-to-end differentiable physics for learning and control. In: Advances in neural information processing systems 31

  51. Ding M, Chen Z, Du T, Luo P, Tenenbaum J, Gan C (2021) Dynamic visual reasoning by learning differentiable physics models from video and language. Adv Neural Inf Process Syst 34:887–899

    Google Scholar 

  52. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems 30

  53. Rao C, Sun H, Liu Y (2021) Physics-informed deep learning for computational elastodynamics without labeled data. J Eng Mech 147(8):04021043

    Google Scholar 

  54. Callaham JL, Maeda K, Brunton SL (2019) Robust flow reconstruction from limited measurements via sparse representation. Phys Rev Fluids 4(10):103907

    Google Scholar 

  55. Yang T-Y, Rosca J, Narasimhan K, Ramadge PJ (2022) Learning physics constrained dynamics using autoencoders. Adv Neural Inf Process Syst 35:17157–17172

    Google Scholar 

  56. Marisca I, Cini A, Alippi C (2022) Learning to reconstruct missing data from spatiotemporal graphs with sparse observations. Adv Neural Inf Process Syst 35:32069–32082

    Google Scholar 

  57. Sun L, Wang J-X (2020) Physics-constrained Bayesian neural network for fluid flow reconstruction with sparse and noisy data. Theoret Appl Mech Lett 10(3):161–169

    Google Scholar 

  58. Tong Z, Li Y (2020) Real-time reconstruction of contaminant dispersion from sparse sensor observations with gappy pod method. Energies 13(8):1956

    MathSciNet  Google Scholar 

  59. Li T, Buzzicotti M, Biferale L, Bonaccorso F, Chen S, Wan M (2022) Data reconstruction of turbulent flows with gappy pod, extended pod and generative adversarial networks. arXiv preprint arXiv:2210.11921

  60. Mainini L, Willcox K (2015) Surrogate modeling approach to support real-time structural assessment and decision making. AIAA J 53(6):1612–1626

    Google Scholar 

  61. Demo N, Tezzele M, Rozza G (2023) A deeponet multi-fidelity approach for residual learning in reduced order modeling. arXiv preprint arXiv:2302.12682

  62. Salam T, Hsieh MA (2019) Adaptive sampling and reduced-order modeling of dynamic processes by robot teams. IEEE Robot Automat Lett 4(2):477–484

    Google Scholar 

  63. Rovina H, Salam T, Kantaros Y, Hsieh MA (2020) Asynchronous adaptive sampling and reduced-order modeling of dynamic processes by robot teams via intermittently connected networks. In: 2020 IEEE/RSJ international conference on Intelligent Robots and Systems (IROS), pp 4798–4805. IEEE

  64. Ebert C, Ruwisch C, Weiss J, Uijt De Haag M, Silvestre F (2022) Trajectory planning in windy urban environment–a gappy pod approach for wind field estimates with sparse sensors. In: AIAA AVIATION 2022 Forum, p 3757

  65. Raben SG, Charonko JJ, Vlachos PP (2012) Adaptive gappy proper orthogonal decomposition for particle image velocimetry data reconstruction. Meas Sci Technol 23(2):025303

    Google Scholar 

  66. Kelshaw D, Rigas G, Magri L (2022) Physics-informed cnns for super-resolution of sparse observations on dynamical systems. arXiv preprint arXiv:2210.17319

  67. Yu L, Yousif MZ, Zhang M, Hoyas S, Vinuesa R, Lim H-C (2022) Three-dimensional esrgan for super-resolution reconstruction of turbulent flows with tricubic interpolation-based transfer learning. Phys Fluids 34:12

    Google Scholar 

  68. Vinuesa R, Brunton SL, McKeon BJ (2023) The transformative potential of machine learning for experiments in fluid mechanics. arXiv preprint arXiv:2303.15832

  69. Gao H, Sun L, Wang J-X (2021) Super-resolution and denoising of fluid flow using physics-informed convolutional neural networks without high-resolution labels. Phys Fluids 33:7

    Google Scholar 

  70. t Wang X, Xie L, Dong C, Shan Y (2021) Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1905–1914

  71. Chen H, He X, Qing L, Wu Y, Ren C, Sheriff RE, Zhu C (2022) Real-world single image super-resolution: a brief review. Inf Fusion 79:124–145

    Google Scholar 

  72. Saharia C, Ho J, Chan W, Salimans T, Fleet DJ, Norouzi M (2022) Image super-resolution via iterative refinement. IEEE Trans Pattern Anal Mach Intell 45(4):4713–4726

    Google Scholar 

  73. Nair A, Chen D, Agrawal P, Isola P, Abbeel P, Malik J, Levine S (2017) Combining self-supervised learning and imitation for vision-based rope manipulation. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp 2146–2153. IEEE

  74. Nava M, Paolillo A, Guzzi J, Gambardella LM, Giusti A (2021) Uncertainty-aware self-supervised learning of spatial perception tasks. IEEE Robot Automat Lett 6(4):6693–6700

    Google Scholar 

  75. Yan M, Zhu Y, Jin N, Bohg J (2020) Self-supervised learning of state estimation for manipulating deformable linear objects. IEEE Robot Automat Lett 5(2):2372–2379

    Google Scholar 

  76. Flaschel M, Kumar S, De Lorenzis L (2021) Unsupervised discovery of interpretable hyperelastic constitutive laws. Comput Methods Appl Mech Eng 381:113852

    MathSciNet  Google Scholar 

  77. Lye KO, Mishra S, Ray D (2020) Deep learning observables in computational fluid dynamics. J Comput Phys 410:109339

    MathSciNet  Google Scholar 

  78. Shin Y-S, Kim J (2023) Sensor data reconstruction for dynamic responses of structures using external feedback of recurrent neural network. Sensors 23(5):2737

    Google Scholar 

  79. Moya B, Badias A, Gonzalez D, Chinesta F, Cueto E (2022) Physics perception in sloshing scenes with guaranteed thermodynamic consistency. IEEE Trans Pattern Anal Mach Intell 45(2):2136–2150

    Google Scholar 

  80. Sun C, Karlsson P, Wu J, Tenenbaum JB (2019) Murphy, K.: Stochastic prediction of multi-agent interactions from partial observations. arXiv preprint arXiv:1902.09641

  81. Antonova R, Yang J, Sundaresan P, Fox D, Ramos F, Bohg J (2022) A Bayesian treatment of real-to-sim for deformable object manipulation. IEEE Robot Automat Lett 7(3):5819–5826

    Google Scholar 

  82. Lim V, Huang H, Chen LY, Wang J, Ichnowski J, Seita D, Laskey M, Goldberg K (2022) Real2sim2real: self-supervised learning of physical single-step dynamic actions for planar robot casting. In: 2022 International Conference on Robotics and Automation (ICRA), pp 8282–8289. IEEE

  83. Li Y, Torralba A, Anandkumar A, Fox D, Garg A (2020) Causal discovery in physical systems from videos. Adv Neural Inf Process Syst 33:9180–9192

    Google Scholar 

  84. Bai Z, Brunton SL, Brunton BW, Kutz JN, Kaiser E, Spohn A (2017) Noack, B.R.: Data-driven methods in fluid dynamics: sparse classification from experimental data. Springer, Berlin

  85. Rodríguez-Ocampo P, Ring M, Hernández-Fontes J, Alcérreca-Huerta J, Mendoza E, Gallegos-Diez-Barroso G, Silva R (2020) A 2d image-based approach for cfd validation of liquid mixing in a free-surface condition. J Appl Fluid Mech 13(5):1487–1500

    Google Scholar 

  86. Bieker K, Peitz S, Brunton SL, Kutz JN, Dellnitz M (2020) Deep model predictive flow control with limited sensor data and online learning. Theoret Comput Fluid Dyn 34:577–591

    MathSciNet  Google Scholar 

  87. Thuruthel TG, Shih B, Laschi C, Tolley MT (2019) Soft robot perception using embedded soft sensors and recurrent neural networks. Sci Robot 4(26):1488

    Google Scholar 

  88. Tariverdi A, Venkiteswaran VK, Richter M, Elle OJ, Tørresen J, Mathiassen K, Misra S, Martinsen ØG (2021) A recurrent neural-network-based real-time dynamic model for soft continuum manipulators. Front Robot AI 8:631303

    Google Scholar 

  89. Bonassi F, Farina M, Xie J, Scattolini R (2022) On recurrent neural networks for learning-based control: recent results and ideas for future developments. J Process Control 114:92–104

    Google Scholar 

  90. Ehrhardt S, Monszpart A, Mitra NJ, Vedaldi A (2019) Taking visual motion prediction to new heightfields. Comput Vis Image Understand 181:14–25

    Google Scholar 

  91. Buschoff LMS, Schulz E, Binz M (2023) The acquisition of physical knowledge in generative neural networks

  92. Doerr A, Daniel C, Schiegg M, Duy N-T, Schaal S, Toussaint M, Sebastian T (2018) Probabilistic recurrent state-space models. In: International conference on machine learning, pp 1280–1289. PMLR

  93. Rai R, Sahu CK (2020) Driven by data or derived through physics? a review of hybrid physics guided machine learning techniques with cyber-physical system (cps) focus. IEEE Access 8:71050–71073

    Google Scholar 

  94. Achterhold J, Tobuschat P, Ma H, Büchler D, Muehlebach M, Stueckler J (2023) Black-box vs. gray-box: A case study on learning table tennis ball trajectory prediction with spin and impacts. In: Learning for Dynamics and Control Conference (L4DC). accepted

  95. Driess D, Huang Z, Li Y, Tedrake R, Toussaint M (2023) Learning multi-object dynamics with compositional neural radiance fields. In: Conference on robot learning, pp 1755–1768. PMLR

  96. Badías A, González D, Alfaro I, Chinesta F, Cueto E (2020) Real-time interaction of virtual and physical objects in mixed reality applications. Int J Numer Methods Eng 121(17):3849–3868

    MathSciNet  Google Scholar 

  97. Zhong YD, Han J, Dey B, Brikis GO (2023) Improving gradient computation for differentiable physics simulation with contacts. In: Learning for dynamics and control conference, pp 128–141. PMLR

  98. Jiang Y, Sun J, Liu CK (2022) Data-augmented contact model for rigid body simulation. In: Learning for dynamics and control conference, pp 378–390. PMLR

  99. Strecke M, Stueckler J (2021) Diffsdfsim: differentiable rigid-body dynamics with implicit shapes. In: 2021 international conference on 3D Vision (3DV), pp 96–105. IEEE

  100. Hernández Q, Badías A, Chinesta F, Cueto E (2023) Port-metriplectic neural networks: thermodynamics-informed machine learning of complex physical systems. Comput Mech 1:1–9

    MathSciNet  Google Scholar 

  101. Um K, Brand R, Fei YR, Holl P, Thuerey N (2020) Solver-in-the-loop: Learning from differentiable physics to interact with iterative pde-solvers. Adv Neural Inf Process Syst 33:6111–6122

    Google Scholar 

  102. Wiewel S, Kim B, Azevedo VC, Solenthaler B, Thuerey N (2020) Latent space subdivision: stable and controllable time predictions for fluid flow. In: Computer graphics forum, vol. 39, pp 15–25. Wiley Online Library

  103. Takahashi T, Liang J, Qiao Y-L, Lin MC (2021) Differentiable fluids with solid coupling for learning and control. In: Proceedings of the AAAI conference on artificial intelligence, vol. 35, pp 6138–6146

  104. Karniadakis GE, Kevrekidis IG, Lu L, Perdikaris P, Wang S, Yang L (2021) Physics-informed machine learning. Nat Rev Phys 3(6):422–440

    Google Scholar 

  105. Ayensa-Jimenez J, Doweidar MH, Sanz-Herrera JA, Doblare M (2021) Prediction and identification of physical systems by means of physically-guided neural networks with meaningful internal layers. Comput Methods Appl Mech Eng 381:113816

    MathSciNet  Google Scholar 

  106. Ajay A, Wu J, Fazeli N, Bauza M, Kaelbling LP, Tenenbaum JB, Rodriguez A (2018) Augmenting physical simulators with stochastic neural networks: Case study of planar pushing and bouncing. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 3066–3073. IEEE

  107. Zeng A, Song S, Lee J, Rodriguez A, Funkhouser T (2020) Tossingbot: learning to throw arbitrary objects with residual physics. IEEE Trans Robot 36(4):1307–1319

    Google Scholar 

  108. Allevato A, Pryor M, Thomaz AL (2021) Multiparameter real-world system identification using iterative residual tuning. J Mech Robot 13(3):031021

    Google Scholar 

  109. Rudin C, Chen C, Chen Z, Huang H, Semenova L, Zhong C (2022) Interpretable machine learning: fundamental principles and 10 grand challenges. Stat Surv 16:1–85

    MathSciNet  Google Scholar 

  110. Lu L, Pestourie R, Yao W, Wang Z, Verdugo F, Johnson SG (2021) Physics-informed neural networks with hard constraints for inverse design. SIAM J Sci Comput 43(6):1105–1132

    MathSciNet  Google Scholar 

  111. Liu S, Zhongkai H, Ying C, Su H, Zhu J, Cheng Z (2022) A unified hard-constraint framework for solving geometrically complex pdes. Adv Neural Inf Process Syst 35:20287–20299

    Google Scholar 

  112. Alkhadhr S, Almekkawy M (2023) Wave equation modeling via physics-informed neural networks: models of soft and hard constraints for initial and boundary conditions. Sensors 23(5):2792

    Google Scholar 

  113. Wu W, Daneker M, Jolley MA, Turner KT, Lu L (2023) Effective data sampling strategies and boundary condition constraints of physics-informed neural networks for identifying material properties in solid mechanics. Appl Math Mech 44(7):1039–1068

    MathSciNet  Google Scholar 

  114. Prantl L, Ummenhofer B, Koltun V, Thuerey N (2022) Guaranteed conservation of momentum for learning particle-based fluid dynamics. Adv Neural Inf Process Syst 35:6901–6913

    Google Scholar 

  115. Tang J, Kim B, Azevedo VC, Solenthaler B (2023) Physics-informed neural corrector for deformation-based fluid control. In: Computer Graphics Forum, vol. 42, pp 161–173. Wiley Online Library

  116. Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707

    MathSciNet  Google Scholar 

  117. Huang Z, Sun Y, Wang W (2023) Generalizing graph ode for learning complex system dynamics across environments. arXiv preprint arXiv:2307.04287

  118. Huang Z, Sun Y, Wang W (2021) Coupled graph ode for learning interacting system dynamics. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp 705–715

  119. Huang Z, Sun Y, Wang W (2020) Learning continuous system dynamics from irregularly-sampled partial observations. Adv Neural Inf Process Syst 33:16177–16187

    Google Scholar 

  120. Weinan E (2017) A proposal on machine learning via dynamical systems. Commun Math Stat 1(5):1–11

    MathSciNet  Google Scholar 

  121. Wen G, Li D, Qin F (2022) Learning symplectic dynamics via generating recurrent neural network. In: Proceedings of the 2022 5th international conference on machine learning and machine intelligence, pp 65–70

  122. Jin P, Zhang Z, Zhu A, Tang Y, Karniadakis GE (2020) Sympnets: intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems. Neural Netw 132:166–179

    Google Scholar 

  123. Chen Z, Feng M, Yan J, Zha H (2022) Learning neural Hamiltonian dynamics: a methodological overview. arXiv preprint arXiv:2203.00128

  124. Cranmer M, Greydanus S, Hoyer S, Battaglia P, Spergel D, Ho S (2020) Lagrangian neural networks. arXiv preprint arXiv:2003.04630

  125. Trask N, Huang A, Hu X (2022) Enforcing exact physics in scientific machine learning: a data-driven exterior calculus on graphs. J Comput Phys 456:110969

    MathSciNet  Google Scholar 

  126. Patel RG, Manickam I, Trask NA, Wood MA, Lee M, Tomas I, Cyr EC (2022) Thermodynamically consistent physics-informed neural networks for hyperbolic systems. J Comput Phys 449:110754

    MathSciNet  Google Scholar 

  127. Vlassis NN, Sun W (2021) Sobolev training of thermodynamic-informed neural networks for interpretable elasto-plasticity models with level set hardening. Comput Methods Appl Mech Eng 377:113695

    MathSciNet  Google Scholar 

  128. Kaltenbach S, Koutsourelakis P-S (2020) Incorporating physical constraints in a deep probabilistic machine learning framework for coarse-graining dynamical systems. J Comput Phys 419:109673

    MathSciNet  Google Scholar 

  129. Masi F, Stefanou I, Vannucci P, Maffi-Berthier V (2021) Thermodynamics-based artificial neural networks for constitutive modeling. J Mech Phys Solids 147:104277

    MathSciNet  Google Scholar 

  130. He X, Chen J-S (2022) Thermodynamically consistent machine-learned internal state variable approach for data-driven modeling of path-dependent materials. Comput Methods Appl Mech Eng 402:115348

    MathSciNet  Google Scholar 

  131. Yu H, Tian X, Weinan E, Li Q (2021) Onsagernet: learning stable and interpretable dynamics using a generalized onsager principle. Phys Rev Fluids 6(11):114402

    Google Scholar 

  132. Huang S, He Z, Reina C (2022) Variational onsager neural networks (vonns): a thermodynamics-based variational learning strategy for non-equilibrium pdes. J Mech Phys Solids 163:104856

    MathSciNet  Google Scholar 

  133. Biot M (1955) Variational principles in irreversible thermodynamics with application to viscoelasticity. Phys Rev 97(6):1463

    MathSciNet  Google Scholar 

  134. Morrison PJ (1986) A paradigm for joined Hamiltonian and dissipative systems. Physica D 18(1–3):410–419

    MathSciNet  Google Scholar 

  135. Öttinger HC, Grmela M (1997) Dynamics and thermodynamics of complex fluids. ii. Illustrations of a general formalism. Phys Rev E 56(6):6633

    MathSciNet  Google Scholar 

  136. Lee K, Trask N, Stinis P (2021) Machine learning structure preserving brackets for forecasting irreversible processes. Adv Neural Inf Process Syst 34:5696–5707

    Google Scholar 

  137. Zhang Z, Shin Y, Em Karniadakis G (2022) Gfinns: generic formalism informed neural networks for deterministic and stochastic dynamical systems. Philos Trans R Soc A 380(2229):20210207

    MathSciNet  Google Scholar 

  138. Hernández Q, Badías A, González D, Chinesta F, Cueto E (2021) Structure-preserving neural networks. J Comput Phys 426:109950

    MathSciNet  Google Scholar 

  139. Hernández Q, Badías A, Chinesta F, Cueto E (2023) Thermodynamics-informed neural networks for physically realistic mixed reality. Comput Methods Appl Mech Eng 407:115912

    MathSciNet  Google Scholar 

  140. Hernández Q, Badías A, Chinesta F, Cueto E (2022) Thermodynamics-informed graph neural networks. arXiv preprint arXiv:2203.01874

  141. Liu J, Shen H, Wang D, Kang Y, Tian Q (2021) Unsupervised domain adaptation with dynamics-aware rewards in reinforcement learning. Adv Neural Inf Process Syst 34:28784–28797

    Google Scholar 

  142. Liu J, Zhang H, Zhuang Z, Kang Y, Wang D, Wang B (2023) Design from policies: conservative test-time adaptation for offline policy optimization. arXiv preprint arXiv:2306.14479

  143. Short ES, Allevato A, Thomaz AL (2019) Sail: simulation-informed active in-the-wild learning. In: 2019 14th ACM/IEEE international conference on Human-Robot Interaction (HRI), pp 468–477. IEEE

  144. Allevato A, Short ES, Pryor M, Thomaz A (2020) Tunenet: one-shot residual tuning for system identification and sim-to-real robot task transfer. In: Conference on robot learning, pp 445–455. PMLR

  145. Lutter M, Peters J (2023) Combining physics and deep learning to learn continuous-time dynamics models. Int J Robot Res 42(3):83–107

    Google Scholar 

  146. Hussein A, Gaber MM, Elyan E, Jayne C (2017) Imitation learning: a survey of learning methods. ACM Comput Surv 50(2):1–35

    Google Scholar 

  147. Ho J, Ermon S (2016) Generative adversarial imitation learning. Adv Neural Inf Process Syst 29

  148. Chen S, Ma X, Xu Z (2023) Imitation learning as state matching via differentiable physics. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7846–7855

  149. Adams M, Li X, Boucinha L, Kher SS, Banerjee P, Gonzalez J-L (2021) Hybrid digital twins: a primer on combining physics-based and data analytics approaches. IEEE Softw 39(2):47–52

    Google Scholar 

  150. Sancarlos A, Cameron M, Abel A, Cueto E, Duval J-L, Chinesta F (2021) From rom of electrochemistry to ai-based battery digital and hybrid twin. Arch Comput Methods Eng 28:979–1015

    MathSciNet  Google Scholar 

  151. Desai S, Durugkar I, Karnan H, Warnell G, Hanna J, Stone P (2020) An imitation from observation approach to transfer learning with dynamics mismatch. Adv Neural Inf Process Syst 33:3917–3929

    Google Scholar 

  152. Mohebujjaman M, Rebholz LG, Iliescu T (2019) Physically constrained data-driven correction for reduced-order modeling of fluid flows. Int J Numer Methods Fluids 89(3):103–122

    MathSciNet  Google Scholar 

  153. Goswami S, Anitescu C, Chakraborty S, Rabczuk T (2020) Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theoret Appl Fract Mech 106:102447

    Google Scholar 

  154. Laroche R, Barlier M (2017) Transfer reinforcement learning with shared dynamics. In: Proceedings of the AAAI conference on artificial intelligence, vol. 31

  155. Guastoni L, Güemes A, Ianiro A, Discetti S, Schlatter P, Azizpour H, Vinuesa R (2021) Convolutional-network models to predict wall-bounded turbulence from wall quantities. J Fluid Mech 928:27

    MathSciNet  Google Scholar 

  156. Moya B, Badías A, González D, Chinesta F, Cueto E (2023) A thermodynamics-informed active learning approach to perception and reasoning about fluids. Comput Mech 1:1–15

    MathSciNet  Google Scholar 

  157. Monaghan JJ (1992) Smoothed particle hydrodynamics. Ann Rev Astron Astrophys 30(1):543–574

    Google Scholar 

  158. Bates C, Battaglia PW, Yildirim I, Tenenbaum JB (20185) Humans predict liquid dynamics using probabilistic simulation. In: CogSci

  159. Espanol P, Serrano M, Öttinger HC (1999) Thermodynamically admissible form for discrete hydrodynamics. Phys Rev Lett 83(22):4542

    Google Scholar 

  160. Moya B, Alfaro I, Gonzalez D, Chinesta F, Cueto E (2020) Physically sound, self-learning digital twins for sloshing fluids. PLoS ONE 15(6):0234569

    Google Scholar 

  161. Brunton SL, Proctor JL, Kutz JN (2016) Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc Natl Acad Sci 113(15):3932–3937

    MathSciNet  Google Scholar 

  162. Pascanu R, Mikolov T, Bengio Y (2013) On the difficulty of training recurrent neural networks. In: International conference on machine learning, pp 1310–1318. Pmlr

  163. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555

  164. Kubo R (1966) The fluctuation-dissipation theorem. Rep Prog Phys 29(1):255

    Google Scholar 

  165. Kraus M (2021) Metriplectic integrators for dissipative fluids. In: Geometric science of information: 5th international conference, GSI 2021, Paris, France, July 21–23, 2021, Proceedings 5, pp 292–301. Springer

  166. Grmela M, Öttinger HC (1997) Dynamics and thermodynamics of complex fluids. i. Development of a general formalism. Phys Rev E 56(6):6620

    MathSciNet  Google Scholar 

  167. Reyes B, Howard AA, Perdikaris P, Tartakovsky AM (2021) Learning unknown physics of non-Newtonian fluids. Phys Rev Fluids 6(7):073301

    Google Scholar 

  168. Cranmer M, Sanchez Gonzalez A, Battaglia P, Xu R, Cranmer K, Spergel D, Ho S (2020) Discovering symbolic models from deep learning with inductive biases. Adv Neural Inf Process Syst 33:17429–17442

    Google Scholar 

  169. Arrieta AB, Díaz-Rodríguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, García S, Gil-López S, Molina D, Benjamins R (2020) Explainable artificial intelligence (xai): concepts, taxonomies, opportunities and challenges toward responsible ai. Inf fusion 58:82–115

    Google Scholar 

  170. Hao Z, Liu S, Zhang Y, Ying C, Feng Y, Su H, Zhu J (2022) Physics-informed machine learning: a survey on problems, methods and applications. arXiv preprint arXiv:2211.08064

  171. Kadambi A, Melo C, Hsieh C-J, Srivastava M, Soatto S (2023) Incorporating physics into data-driven computer vision. Nat Mach Intell 1:1–9

    Google Scholar 

  172. Banerjee C, Nguyen K, Fookes C, Karniadakis G (2023) Physics-informed computer vision: a review and perspectives. arXiv preprint arXiv:2305.18035

  173. LeCun Y (2022) A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review 62

  174. Silver D, Singh S, Precup D, Sutton RS (2021) Reward is enough. Artif Intell 299:103535

    MathSciNet  Google Scholar 

Download references

Acknowledgements

This research is part of the DesCartes program and is supported by the National Research Foundation, Prime Minister Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme. This material is also based upon work supported in part by the Army Research Laboratory and the Army Research Office under contract/Grant No. W911NF2210271. This work has been partially funded by the Spanish Ministry of Science and Innovation, AEI /10.13039/501100011033, through Grant No. TED2021-130105B-I00. The authors also acknowledge the support of ESI Group through the chairs at the University of Zaragoza and at ENSAM Institute of Technology.

Funding

This research has been supported by the Army Research Laboratory and the Army Research Office under contract/grant number W911NF2210271, the Spanish Ministry of Science and Innovation, AEI /10.13039/501100011033, through Grant No. TED2021-130105B-I00, ESI Group through the chairs at the University of Zaragoza and at ENSAM Institute of Technology and he National Research Foundation, Prime Minister Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme.

Author information

Authors and Affiliations

  1. CNRS@CREATE LTD., CNRS, 1 CREATE Way, Singapore, 138602, Singapore

    Beatriz Moya & Francisco Chinesta

  2. ETSIAE, Polytechnic University of Madrid, C. de José Gutiérrez Abascal, 2, Madrid, 28006, Spain

    Alberto Badías

  3. ESI Group Chair, Aragon Institute of Engineering Research, Universidad de Zaragoza, Maria de Luna, s.n., Zaragoza, 50018, Spain

    David González & Elias Cueto

  4. ESI Group Chair, PIMM Lab, ENSAM Institute of Technology, 151 Bvd. de l’Hôpital, Paris, 75013, France

    Francisco Chinesta

Authors
  1. Beatriz Moya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alberto Badías
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David González
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francisco Chinesta
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elias Cueto
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors declare they have contributed equally to this work and there is no interest conflict.

Corresponding author

Correspondence to Beatriz Moya.

Ethics declarations

Conflict of interest

The authors have no financial or non-financial interests to disclose.

Ethical Approval

Not applicable.

Consent for Publication

All authors consent the publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moya, B., Badías, A., González, D. et al. Computational Sensing, Understanding, and Reasoning: An Artificial Intelligence Approach to Physics-Informed World Modeling. Arch Computat Methods Eng 31, 1897–1914 (2024). https://doi.org/10.1007/s11831-023-10033-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11831-023-10033-y

Search

Navigation