Computational Mechanics

, Volume 64, Issue 2, pp 467–499 | Cite as

A cooperative game for automated learning of elasto-plasticity knowledge graphs and models with AI-guided experimentation

  • Kun Wang
  • WaiChing SunEmail author
  • Qiang Du
Original Paper


We introduce a multi-agent meta-modeling game to generate data, knowledge, and models that make predictions on constitutive responses of elasto-plastic materials. We introduce a new concept from graph theory where a modeler agent is tasked with evaluating all the modeling options recast as a directed multigraph and find the optimal path that links the source of the directed graph (e.g. strain history) to the target (e.g. stress) measured by an objective function. Meanwhile, the data agent, which is tasked with generating data from real or virtual experiments (e.g. molecular dynamics, discrete element simulations), interacts with the modeling agent sequentially and uses reinforcement learning to design new experiments to optimize the prediction capacity. Consequently, this treatment enables us to emulate an idealized scientific collaboration as selections of the optimal choices in a decision tree search done automatically via deep reinforcement learning.


Directed multigraph Data-driven constitutive modeling Multi-agent deep reinforcement learning Combinatorial optimization Computational combinatorics 



The work of KW and WCS is supported by the Earth Materials and Processes program from the US Army Research Office under Grant Contract W911NF-18-2-0306, the Dynamic Materials and Interactions Program from the Air Force Office of Scientific Research under Grant Contract FA9550-17-1-0169, the nuclear energy university program from Department of Energy under Grant Contract DE-NE0008534, the Mechanics of Material program at National Science Foundation under Grant Contract CMMI-1462760, and the Columbia SEAS Interdisciplinary Research Seed Grant. The work of QD is supported in part by NSF CCF-1704833, DMS-1719699, DMR-1534910, and ARO MURI W911NF-15-1-0562. These supports are gratefully acknowledged. The views and conclusions contained in this document are those of the authors, and should not be interpreted as representing the official policies, either expressed or implied, of the sponsors, including the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.


  1. 1.
    Andrade JE, Borja RI (2006) Capturing strain localization in dense sands with random density. Int J Numer Methods Eng 67(11):1531–1564zbMATHGoogle Scholar
  2. 2.
    Asaro RJ (1983) Crystal plasticity. J Appl Mech 50(4b):921–934zbMATHGoogle Scholar
  3. 3.
    Aydin A, Borja RI, Eichhubl P (2006) Geological and mathematical framework for failure modes in granular rock. J Struct Geol 28(1):83–98Google Scholar
  4. 4.
    Bang-Jensen J, Gutin GZ (2008) Digraphs: theory, algorithms and applications. Springer, BerlinzbMATHGoogle Scholar
  5. 5.
    Bardet JP, Choucair W (1991) A linearized integration technique for incremental constitutive equations. Int J Numer Anal Methods Geomech 15(1):1–19zbMATHGoogle Scholar
  6. 6.
    Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R et al (2018) Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261
  7. 7.
    Been K, Jefferies MG, Hachey J (1991) Critical state of sands. Geotechnique 41(3):365–381Google Scholar
  8. 8.
    Bonabeau E (2002) Agent-based modeling: methods and techniques for simulating human systems. Proc Natl Acad Sci 99(suppl 3):7280–7287Google Scholar
  9. 9.
    Borja RI (2013) Plasticity: modeling and computation. Springer, BerlinzbMATHGoogle Scholar
  10. 10.
    Borja RI, Sama KM, Sanz PF (2003) On the numerical integration of three-invariant elastoplastic constitutive models. Comput Methods Appl Mech Eng 192(9–10):1227–1258zbMATHGoogle Scholar
  11. 11.
    Boyce BL, Kramer SLB, Fang HE, Cordova TE, Neilsen MK, Dion K, Kaczmarowski AK, Karasz E, Xue L, Gross AJ (2014) The sandia fracture challenge: blind round robin predictions of ductile tearing. Int J Fract 186(1–2):5–68Google Scholar
  12. 12.
    Casagrande A (1976) Liquefaction and cyclic deformation of sands—a critical review. Harvard soil mechanics series (88). Harvard University, CambridgeGoogle Scholar
  13. 13.
    Chomsky N (2014) Aspects of the theory of syntax, vol 11. MIT pressGoogle Scholar
  14. 14.
    Choo J, Sun WC (2018a) Coupled phase-field and plasticity modeling of geological materials: from brittle fracture to ductile flow. Comput Methods Appl Mech Eng 330:1–32MathSciNetGoogle Scholar
  15. 15.
    Choo J, Sun WC (2018b) Cracking and damage from crystallization in pores: coupled chemo-hydro-mechanics and phase-field modeling. Comput Methods Appl Mech Eng 335:347–379MathSciNetGoogle Scholar
  16. 16.
    Coussy O (2004) Poromechanics. Wiley, New YorkzbMATHGoogle Scholar
  17. 17.
    Cundall PA, Strack ODL (1979) A discrete numerical model for granular assemblies. Geotechnique 29(1):47–65Google Scholar
  18. 18.
    Dabrowski R, Stencel K, Timoszuk G (2011) Software is a directed multigraph. In: European conference on software architecture. Springer, pp 360–369Google Scholar
  19. 19.
    Dafalias YF, Manzari MT (2004) Simple plasticity sand model accounting for fabric change effects. J Eng Mech 130(6):622–634Google Scholar
  20. 20.
    de Borst R, Heeres OM (2002) A unified approach to the implicit integration of standard, non-standard and viscous plasticity models. Int J Numer Anal Methods Geomech 26(11):1059–1070zbMATHGoogle Scholar
  21. 21.
    Foerster J, Assael IA, de Freitas N, Whiteson S (2016) Learning to communicate with deep multi-agent reinforcement learning. In: 30th conference on neural information processing systems (NIPS 2016). Advances in neural information processing systems, Barcelona, Spain, pp 2137–2145Google Scholar
  22. 22.
    Furukawa T, Yagawa G (1998) Implicit constitutive modelling for viscoplasticity using neural networks. Int J Numer Methods Eng 43(2):195–219zbMATHGoogle Scholar
  23. 23.
    Gens A, Potts DM (1988) Critical state models in computational geomechanics. Eng Comput 5(3):178–197Google Scholar
  24. 24.
    Ghaboussi J, Garrett JH Jr, Wu X (1991) Knowledge-based modeling of material behavior with neural networks. J Eng Mech 117(1):132–153Google Scholar
  25. 25.
    Ghaboussi J, Pecknold DA, Zhang M, Haj-Ali RM (1998) Autoprogressive training of neural network constitutive models. Int J Numer Methods Eng 42(1):105–126zbMATHGoogle Scholar
  26. 26.
    Ghavamzadeh M, Mannor S, Pineau J, Tamar A (2015) Bayesian reinforcement learning: a survey. Found Trends® Mach Learn 8(5–6):359–483zbMATHGoogle Scholar
  27. 27.
    Graham RL, Knuth DE, Patashnik O, Liu S (1989) Concrete mathematics: a foundation for computer science. Comput Phys 3(5):106–107zbMATHGoogle Scholar
  28. 28.
    Hibbitt, Karlsson, Sorensen (2001) ABAQUS/standard user’s manual, vol 1. Hibbitt, Karlsson & Sorensen, PawtucketGoogle Scholar
  29. 29.
    Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366zbMATHGoogle Scholar
  30. 30.
    Humboldt W (1999) On language: on the diversity of human language construction and its influence on the mental development of the human species. Cambridge University PressGoogle Scholar
  31. 31.
    Ibañez R, Abisset-Chavanne E, Aguado JV, Gonzalez D, Cueto E, Chinesta F (2018) A manifold learning approach to data-driven computational elasticity and inelasticity. Arch Comput Methods Eng 25(1):47–57MathSciNetzbMATHGoogle Scholar
  32. 32.
    Jefferies MG (1993) Nor-sand: a simle critical state model for sand. Géotechnique 43(1):91–103Google Scholar
  33. 33.
    Kendall MG et al (1946) The advanced theory of statistics. , 5th edn. Charles Griffin & Company, LondonGoogle Scholar
  34. 34.
    Kirchdoerfer T, Ortiz M (2016) Data-driven computational mechanics. Comput Methods Appl Mech Eng 304:81–101MathSciNetzbMATHGoogle Scholar
  35. 35.
    Kirchdoerfer T, Ortiz M (2017) Data driven computing with noisy material data sets. Comput Methods Appl Mech Eng 326:622–641MathSciNetGoogle Scholar
  36. 36.
    Koeppe A, Bamer F, Padilla CAH, Markert B (2017) Neural network representation of a phase-field model for brittle fracture. PAMM 17(1):253–254Google Scholar
  37. 37.
    Kuhn MR, Sun WC, Wang Q (2015) Stress-induced anisotropy in granular materials: fabric, stiffness, and permeability. Acta Geotech 10(4):399–419Google Scholar
  38. 38.
    Lake Brenden M, Ullman Tomer D, Tenenbaum Joshua B, Gershman Samuel J (2017) Building machines that learn and think like people. Behav Brain Sci 40:e253Google Scholar
  39. 39.
    Lange M (2012) What makes a scientific explanation distinctively mathematical? Br J Philos Sci 64(3):485–511MathSciNetzbMATHGoogle Scholar
  40. 40.
    Lefik M, Schrefler BA (2002) Artificial neural network for parameter identifications for an elasto-plastic model of superconducting cable under cyclic loading. Comput Struct 80(22):1699–1713Google Scholar
  41. 41.
    Lefik M, Schrefler BA (2003) Artificial neural network as an incremental non-linear constitutive model for a finite element code. Comput Methods Appl Mech Eng 192(28–30):3265–3283zbMATHGoogle Scholar
  42. 42.
    Li XS, Dafalias YF (2011) Anisotropic critical state theory: role of fabric. J Eng Mech 138(3):263–275Google Scholar
  43. 43.
    Ling HI, Liu H (2003) Pressure-level dependency and densification behavior of sand through generalized plasticity model. J Eng Mech 129(8):851–860Google Scholar
  44. 44.
    Ling HI, Yang S (2006) Unified sand model based on the critical state and generalized plasticity. J Eng Mech 132(12):1380–1391Google Scholar
  45. 45.
    Liu Y, Sun WC, Fish J (2016) Determining material parameters for critical state plasticity models based on multilevel extended digital database. J Appl Mech 83(1):011003Google Scholar
  46. 46.
    Liu Y, Sun WC, Yuan Z, Fish J (2016) A nonlocal multiscale discrete-continuum model for predicting mechanical behavior of granular materials. Int J Numer Methods Eng 106(2):129–160MathSciNetzbMATHGoogle Scholar
  47. 47.
    Liu Z, Fleming M, Liu WK (2018) Microstructural material database for self-consistent clustering analysis of elastoplastic strain softening materials. Comput Methods Appl Mech Eng 330:547–577MathSciNetGoogle Scholar
  48. 48.
    Lubliner J, Auricchio F (1996) Generalized plasticity and shape-memory alloys. Int J Solids Struct 33(7):991–1003zbMATHGoogle Scholar
  49. 49.
    Malcher L, Pires FMA, de Sá JMAC, Andrade FXC (2009) Numerical integration algorithm of a new model for metal plasticity and fracture including pressure and lode angle dependence. Int J Mater Form 2(1):443–446Google Scholar
  50. 50.
    Malmgren RD, Ottino JM, Amaral LAN (2010) The role of mentorship in protégé performance. Nature 465(7298):622Google Scholar
  51. 51.
    Manzari MT, Dafalias YF (1997) A critical state two-surface plasticity model for sands. Geotechnique 47(2):255–272Google Scholar
  52. 52.
    Miehe C, Schröder J (2001) A comparative study of stress update algorithms for rate-independent and rate-dependent crystal plasticity. Int J Numer Methods Eng 50(2):273–298zbMATHGoogle Scholar
  53. 53.
    Mira P, Tonni L, Pastor M, Merodo JAF (2009) A generalized midpoint algorithm for the integration of a generalized plasticity model for sands. Int J Numer Methods Eng 77(9):1201–1223MathSciNetzbMATHGoogle Scholar
  54. 54.
    Mooney MA, Finno RJ, Viggiani MG (1998) A unique critical state for sand? J Geotech Geoenviron Eng 124(11):1100–1108Google Scholar
  55. 55.
    Munafò MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, du Sert NP, Simonsohn U, Wagenmakers E-J, Ware JJ, Ioannidis JPA (2017) A manifesto for reproducible science. Nat Hum Behav 1:0021Google Scholar
  56. 56.
    Na SH, Sun WC (2017) Computational thermo-hydro-mechanics for multiphase freezing and thawing porous media in the finite deformation range. Comput Methods Appl Mech Eng 318:667–700MathSciNetGoogle Scholar
  57. 57.
    Na SH, Sun WC (2018) Computational thermomechanics of crystalline rock, part I: a combined multi-phase-field/crystal plasticity approach for single crystal simulations. Comput Methods Appl Mech Eng 338:657–691MathSciNetGoogle Scholar
  58. 58.
    Olivier A, Smyth AW (2018) A marginalized unscented kalman filter for efficient parameter estimation with applications to finite element models. Comput Methods Appl Mech Eng 339:615–643MathSciNetGoogle Scholar
  59. 59.
    Ortiz M, Pandolfi A (1999) Finite-deformation irreversible cohesive elements for three-dimensional crack-propagation analysis. Int J Numer Methods Eng 44(9):1267–1282zbMATHGoogle Scholar
  60. 60.
    Pack K, Luo M, Wierzbicki T (2014) Sandia fracture challenge: blind prediction and full calibration to enhance fracture predictability. Int J Fract 186(1–2):155–175Google Scholar
  61. 61.
    Pandolfi ANNA, Krysl P, Ortiz M (1999) Finite element simulation of ring expansion and fragmentation: the capturing of length and time scales through cohesive models of fracture. Int J Fract 95(1–4):279–297Google Scholar
  62. 62.
    Park K, Paulino GH (2011) Cohesive zone models: a critical review of traction-separation relationships across fracture surfaces. Appl Mech Rev 64(6):060802Google Scholar
  63. 63.
    Pastor M, Zienkiewicz OC, Chan AHC (1990) Generalized plasticity and the modelling of soil behaviour. Int J Numer Anal Methods Geomech 14(3):151–190zbMATHGoogle Scholar
  64. 64.
    Pestana JM, Whittle AJ, Salvati LA (2002) Evaluation of a constitutive model for clays and sands: part I—sand behaviour. Int J Numer Anal Methods Geomech 26(11):1097–1121zbMATHGoogle Scholar
  65. 65.
    Raileanu R, Denton E, Szlam A, Fergus R (2018) Modeling others using oneself in multi-agent reinforcement learning. arXiv preprint arXiv:1802.09640
  66. 66.
    Rutqvist J, Ijiri Y, Yamamoto H (2011) Implementation of the barcelona basic model into tough-flac for simulations of the geomechanical behavior of unsaturated soils. Comput Geosci 37(6):751–762Google Scholar
  67. 67.
    Salinger AG, Bartlett RA, Bradley AM, Chen Q, Demeshko IP, Gao X, Hansen GA, Mota A, Muller RP, Nielsen E et al (2016) Albany: using component-based design to develop a flexible, generic multiphysics analysis code. Int J Multiscale Comput Eng 14(4):415–438Google Scholar
  68. 68.
    Sánchez M, Gens A, Guimarães LN, Olivella S (2005) A double structure generalized plasticity model for expansive materials. Int J Numer Anal Methods Geomech 29(8):751–787zbMATHGoogle Scholar
  69. 69.
    Schofield A, Wroth P (1968) Critical state soil mechanics, vol 310. McGraw-Hill, LondonGoogle Scholar
  70. 70.
    Silver D, Hubert T, Schrittwieser J, Antonoglou I, Lai M, Guez A, Lanctot M, Sifre L, Kumaran D, Graepel T et al (2017) Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815
  71. 71.
    Silver D, Hubert T, Schrittwieser J, Antonoglou I, Lai M, Guez A, Lanctot M, Sifre L, Kumaran D, Graepel T et al (2017) Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815
  72. 72.
    Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A et al (2017c) Mastering the game of go without human knowledge. Nature 550(7676):354Google Scholar
  73. 73.
    Simo JC, Hughes TJR (2006) Computational inelasticity, vol 7. Springer, BerlinzbMATHGoogle Scholar
  74. 74.
    Sloan SW (1987) Substepping schemes for the numerical integration of elastoplastic stress–strain relations. Int J Numer Methods Eng 24(5):893–911zbMATHGoogle Scholar
  75. 75.
    Sloan SW, Abbo AJ, Sheng D (2001) Refined explicit integration of elastoplastic models with automatic error control. Eng Comput 18(1/2):121–194zbMATHGoogle Scholar
  76. 76.
    Šmilauer V, Catalano E, Chareyre B, Dorofeenko S, Duriez J, Gladky A, Kozicki J, Modenese C, Scholtès L, Sibille L et al (2010) Yade reference documentation. Yade Doc 474(1):1–161Google Scholar
  77. 77.
    Smith J, Xiong W, Yan W, Lin S, Cheng P, Kafka OL, Wagner GJ, Cao J, Liu WK (2016) Linking process, structure, property, and performance for metal-based additive manufacturing: computational approaches with experimental support. Comput Mech 57(4):583–610zbMATHGoogle Scholar
  78. 78.
    Sun Q, Tao Y, Du Q (2018) Stochastic training of residual networks: a differential equation viewpoint. arXiv preprintarXiv:1812.00174Google Scholar
  79. 79.
    Sun WC, Kuhn MR, Rudnicki JW et al (2014) A micromechanical analysis on permeability evolutions of a dilatant shear band. In: 48th US rock mechanics/geomechanics symposium. American Rock Mechanics AssociationGoogle Scholar
  80. 80.
    Sun WC (2013) A unified method to predict diffuse and localized instabilities in sands. Geomech Geoeng 8(2):65–75Google Scholar
  81. 81.
    Sun WC (2015) A stabilized finite element formulation for monolithic thermo-hydro-mechanical simulations at finite strain. Int J Numer Methods Eng 103(11):798–839MathSciNetzbMATHGoogle Scholar
  82. 82.
    Sun WC, Kuhn MR, Rudnicki JW (2013) A multiscale DEM-LBM analysis on permeability evolutions inside a dilatant shear band. Acta Geotech 8(5):465–480Google Scholar
  83. 83.
    Sun WC, Ostien JT, Salinger AG (2013) A stabilized assumed deformation gradient finite element formulation for strongly coupled poromechanical simulations at finite strain. Int J Numer Anal Methods Geomech 37(16):2755–2788Google Scholar
  84. 84.
    Tampuu A, Matiisen T, Kodelja D, Kuzovkin I, Korjus K, Aru J, Aru J, Vicente R (2017) Multiagent cooperation and competition with deep reinforcement learning. PLoS ONE 12(4):e0172395Google Scholar
  85. 85.
    Tan M (1993) Multi-agent reinforcement learning: independent vs. cooperative agents. In: Proceedings of the tenth international conference on machine learning, pp 330–337Google Scholar
  86. 86.
    Tang S, Zhang L, Liu WK (2018) From virtual clustering analysis to self-consistent clustering analysis: a mathematical study. Comput Mech 62(6):1443–1460MathSciNetzbMATHGoogle Scholar
  87. 87.
    Truesdell C (1959) The rational mechanics of materials—past, present, future. Appl Mech Rev 12:75–80MathSciNetGoogle Scholar
  88. 88.
    Truesdell C, Noll W (2004) The non-linear field theories of mechanics. Springer, Berlin, Heidelberg, pp 1–579Google Scholar
  89. 89.
    Tu X, Andrade JE, Chen Q (2009) Return mapping for nonsmooth and multiscale elastoplasticity. Comput Methods Appl Mech Eng 198(30–32):2286–2296zbMATHGoogle Scholar
  90. 90.
    Tvergaard V (1990) Effect of fibre debonding in a whisker-reinforced metal. Mater Sci Eng A 125(2):203–213Google Scholar
  91. 91.
    Ulven OI, Sun WC (2018) Capturing the two-way hydromechanical coupling effect on fluid-driven fracture in a dual-graph lattice beam model. Int J Numer Anal Methods Geomech 42(5):736–767Google Scholar
  92. 92.
    Wang K, Sun WC (2016) A semi-implicit discrete-continuum coupling method for porous media based on the effective stress principle at finite strain. Comput Methods Appl Mech Eng 304:546–583MathSciNetzbMATHGoogle Scholar
  93. 93.
    Wang K, Sun WC (2017) Data-driven discrete-continuum method for partially saturated micro-polar porous media. In: Poromechanics VI, pp 571–578Google Scholar
  94. 94.
    Wang K, Sun WC (2018) A multiscale multi-permeability poroplasticity model linked by recursive homogenizations and deep learning. Comput Methods Appl Mech Eng 334:337–380MathSciNetGoogle Scholar
  95. 95.
    Wang K, Sun WC (2019) Meta-modeling game for deriving theory-consistent, microstructure-based traction-separation laws via deep reinforcement learning. Comput Methods Appl Mech Eng 346:216–241MathSciNetGoogle Scholar
  96. 96.
    Wang K, Sun WC (2019) An updated Lagrangian LBM–DEM–FEM coupling model for dual-permeability fissured porous media with embedded discontinuities. Comput Methods Appl Mech Eng 344:276–305MathSciNetGoogle Scholar
  97. 97.
    Wang K, Sun W, Salager S, Na S, Khaddour G (2016) Identifying material parameters for a micro-polar plasticity model via X-ray micro-CT images: lessons learned from the curve-fitting exercises. Int J Multiscale Comput Eng 14(4):389–413Google Scholar
  98. 98.
    West DB et al (2001) Introduction to graph theory, vol 2. Prentice Hall, Upper Saddle RiverGoogle Scholar
  99. 99.
    Wollny I, Sun WC, Kaliske M (2017) A hierarchical sequential ale poromechanics model for tire–soil–water interaction on fluid-infiltrated roads. Int J Numer Methods Eng 112(8):909–938MathSciNetGoogle Scholar
  100. 100.
    Wood DM (1990) Soil behaviour and critical state soil mechanics. Cambridge University Press, CambridgezbMATHGoogle Scholar
  101. 101.
    Xin H, Sun WC, Fish J (2017) Discrete element simulations of powder-bed sintering-based additive manufacturing. Int J Mech Sci 149:373–392Google Scholar
  102. 102.
    Zhao J, Guo N (2013) Unique critical state characteristics in granular media considering fabric anisotropy. Géotechnique 63(8):695Google Scholar
  103. 103.
    Zienkiewicz OC, Mroz Z (1984) Generalized plasticity formulation and applications to geomechanics. Mech Eng Mater 44(3):655–680Google Scholar
  104. 104.
    Zienkiewicz Olgierd C, Chan AHC, Pastor M, Schrefler BA, Shiomi T (1999) Computational geomechanics. Citeseer, New YorkzbMATHGoogle Scholar
  105. 105.
    Zohdi TI (2013) Rapid simulation of laser processing of discrete particulate materials. Arch Comput Methods Eng 20(4):309–325Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Civil Engineering and Engineering MechanicsColumbia UniversityNew YorkUSA
  2. 2.Department of Applied Physics and Applied Mathematics, and Data Science InstituteColumbia UniversityNew YorkUSA

Personalised recommendations