Adaptive Simulation Selection for the Discovery of the Ground State Line of Binary Alloys with a Limited Computational Budget

Part of the Fields Institute Communications book series (FIC, volume 79)


First principles calculations are computationally expensive. This information acquisition cost, combined with an exponentially high number of possible material configurations, constitutes an important roadblock towards the ultimate goal of materials by design. To overcome this barrier, one must devise schemes for the automatic and maximally informative selection of simulations. Such information acquisition decisions are task-dependent, in the sense that an optimal information acquisition policy for learning about a specific material property will not necessarily be optimal for learning about another. In this work, we develop an information acquisition policy for learning the ground state line (GSL) of binary alloys. Our approach is based on a Bayesian interpretation of the cluster expanded energy. This probabilistic surrogate of the energy enables us to quantify the epistemic uncertainty induced by the limited number of simulations which, in turn, is the key to defining a function of the configuration space that quantifies the expected improvement to the GSL resulting from a hypothetical simulation. We show that optimal information acquisition policies should balance the maximization of the expected improvement of the GSL and the minimization of the size of the simulated structure. We validate our approach by learning the GSLs of NiAl and TiAl binary alloys, where to establish the ground truth GSL we use the embedded-atom method (EAM) for the calculation of the energy of a given alloy configuration. Note that the proposed policies are directly applicable to the discovery of generic phase diagrams, if one can construct a probabilistic surrogate of the relevant thermodynamic potential.



The work of N.Z. was supported by EPSRC (Grant No. EP/L027682/1). N.Z. thanks the Royal Society for support through a Wolfson Research Merit Award and the Technische Universität München, Institute for Advanced Study for support through a Hans Fisher Senior Fellowship, funded by the German Excellence Initiative and the European Union Seventh Framework Programme under Grant Agreement No. 291763. The work of J.K. at Cornell was supported by the US Department of Energy, Office of Science, Advanced Scientific Computing Research and the Computational Mathematics program of the National Science Foundation (NSF) (award DMS-1214282). I.B. acknowledges the startup support provided by the School of Mechanical Engineering at Purdue University.


  1. 1.
    Bahn SR, Jacobsen KW (2002) An object-oriented scripting interface to a legacy electronic structure code. Computing in Science and Engineering 4(3):56–66CrossRefGoogle Scholar
  2. 2.
    Bayes M, Price M (1763) An essay towards solving a problem in the doctrine of chances. by the late rev. Mr. Bayes, frs communicated by Mr. Price, in a letter to John Canton, amfrs. Philosophical Transactions (1683–1775) pp 370–418Google Scholar
  3. 3.
    Bertsekas D (2007) Dynamic Programming and Optimal Control, 4th edn. Athena ScientificzbMATHGoogle Scholar
  4. 4.
    Bilionis I, Zabaras N (2012) Multi-output local Gaussian process regression: Applications to uncertainty quantification. Journal of Computational Physics 231(17):5718–5746, DOI Doi10.1016/J.Jcp.2012.04.047, URL <GotoISI>://WOS:000305915400009 Scholar
  5. 5.
    Bilionis I, Zabaras N (2012) Multidimensional adaptive relevance vector machines for uncertainty quantification. SIAM Journal on Scientific Computing 34(6):B881–B908, DOI Doi 10.1137/120861345, URL <GotoISI>://WOS:000312737900020 Scholar
  6. 6.
    Bilionis I, Zabaras N, Konomi BA, Lin G (2013) Multi-output separable Gaussian process: Towards an efficient, fully Bayesian paradigm for uncertainty quantification. Journal of Computational Physics 241:212–239, DOI Doi 10.1016/J.Jcp.2013.01.011, URL <GotoISI>://WOS:000317186100012 Scholar
  7. 7.
    Bishop CM (2006) Pattern Recognition and Machine Learning, vol 4. Springer New YorkzbMATHGoogle Scholar
  8. 8.
    Boyer R (1996) An overview on the use of titanium in the aerospace industry. Materials Science and Engineering: A 213(1):103–114MathSciNetCrossRefGoogle Scholar
  9. 9.
    Brent RP (1971) An algorithm with guaranteed convergence for finding a zero of a function. The Computer Journal 14(4):422–425MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Broyden C (1969) A new double-rank minimization algorithm. Notices Amer Math Soc p 670Google Scholar
  11. 11.
    Ceder G (1993) A derivation of the ising model for the computation of phase diagrams. Computational Materials Science 1(2):144–150, DOI 10.1016/0927-0256(93)90005-8, URL
  12. 12.
    Christen JA, Sansó B (2011) Advances in the sequential design of computer experiments based on active learning. Communications in Statistics-Theory and Methods 40(24):4467–4483MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Currin C, Mitchell T, Morris M, Ylvisaker D (1988) A Bayesian approach to the design and analysis of computer experiments. Report, Oak Ridge LaboratoryCrossRefGoogle Scholar
  14. 14.
    Daw MS, Baskes MI (1984) Embedded-atom method: Derivation and application to impurities, surfaces, and other defects in metals. Physical Review B 29(12):6443CrossRefGoogle Scholar
  15. 15.
    Daw MS, Foiles SM, Baskes MI (1993) The embedded-atom method: a review of theory and applications. Materials Science Reports 9(7):251–310CrossRefGoogle Scholar
  16. 16.
    Dekker T (1969) Finding a zero by means of successive linear interpolation. Constructive aspects of the fundamental theorem of algebra pp 37–51Google Scholar
  17. 17.
    Dreyssé H, Berera A, Wille L, De Fontaine D (1989) Determination of effective-pair interactions in random alloys by configurational averaging. Physical Review B 39(4):2442CrossRefGoogle Scholar
  18. 18.
    Ducastelle F, Ducastelle F (1991) Order and phase stability in alloys. North-Holland AmsterdamzbMATHGoogle Scholar
  19. 19.
    Durand-Charre M (1997) The microstructure of superalloys. Gordon and Breach Science Publishers, Amsterdam, The NetherlandsGoogle Scholar
  20. 20.
    Fletcher R (1970) A new approach to variable metric algorithms. The computer journal 13(3):317–322CrossRefzbMATHGoogle Scholar
  21. 21.
    Frazier PI, Powell WB, Dayanik S (2008) A Knowledge-Gradient Policy for Sequential Information Collection. SIAM Journal on Control and Optimization 47(5):2410–2439, DOI Doi 10.1137/070693424, URL <GotoISI>://WOS:000260848200008 Scholar
  22. 22.
    Frenkel D, Smit B (2001) Understanding molecular simulation: from algorithms to applications, vol 1. Academic pressGoogle Scholar
  23. 23.
    Garbulsky G, Ceder G (1995) Linear-programming method for obtaining effective cluster interactions in alloys from total-energy calculations: Application to the fcc Pd-V system. Physical Review B 51(1):67CrossRefGoogle Scholar
  24. 24.
    Goldfarb D (1970) A family of variable-metric methods derived by variational means. Mathematics of computation 24(109):23–26MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Hart GL, Forcade RW (2008) Algorithm for generating derivative structures. Physical Review B 77(22):224,115CrossRefGoogle Scholar
  26. 26.
    Hohenberg P, Kohn W (1964) Inhomogeneous electron gas. Physical review 136(3B):B864MathSciNetCrossRefGoogle Scholar
  27. 27.
    Huang W, Chang Y (1998) A thermodynamic analysis of the Ni-Al system. Intermetallics 6(6):487–498CrossRefGoogle Scholar
  28. 28.
    Hunter JD (2007) Matplotlib: A 2D graphics environment. Computing in science and engineering 9(3):90–95CrossRefGoogle Scholar
  29. 29.
    Jaynes ET (2003) Probability Theory: The Logic of Science. Cambridge university pressGoogle Scholar
  30. 30.
    Jones D (2001) A Taxonomy of Global Optimization Methods Based on Response Surfaces. Journal of Global Optimization 21(4):345–383, DOI 10.1023/A:1012771025575, URL
  31. 31.
    Jones D, Schonlau M, Welch W (1998) Efficient global optimization of expensive black-box functions. Journal of Global Optimization 13(4):455–492, DOI 10.1023/A:1008306431147, URL
  32. 32.
    Jones E, Oliphant T, Peterson P (2014) SciPy: Open source scientific tools for PythonGoogle Scholar
  33. 33.
    Knowles J (2006) ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. Evolutionary Computation, IEEE Transactions on 10(1):50–66CrossRefGoogle Scholar
  34. 34.
    Kohan A, Tepesch P, Ceder G, Wolverton C (1998) Computation of alloy phase diagrams at low temperatures. Computational Materials Science 9(3–4):389–396, DOI 10.1016/S0927-0256(97)00168-7, URL
  35. 35.
    Kohn W, Sham LJ (1965) Self-consistent equations including exchange and correlation effects. Physical Review 140(4A):A1133MathSciNetCrossRefGoogle Scholar
  36. 36.
    Kresse G, Furthmüller J (1996) Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set. Computational Materials Science 6(1):15–50, URL
  37. 37.
    Kresse G, Furthmüller J (1996) Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set. Phys Rev B 54:11,169–11,186, DOI 10.1103/PhysRevB.54.11169, URL
  38. 38.
    Kresse G, Hafner J (1993) Ab initio molecular dynamics for liquid metals. Physical Review B 47(1):558CrossRefGoogle Scholar
  39. 39.
    Kresse G, Hafner J (1994) Ab initio molecular-dynamics simulation of the liquid-metal–amorphous-semiconductor transition in germanium. Physical Review B 49(20):14,251CrossRefGoogle Scholar
  40. 40.
    Kristensen J, Zabaras NJ (2014) Bayesian uncertainty quantification in the evaluation of alloy properties with the cluster expansion method. Computer Physics Communications 185(11):2885–2892MathSciNetCrossRefGoogle Scholar
  41. 41.
    Kristensen J, Bilionis I, Zabaras N (2013) Relative entropy as model selection tool in cluster expansions. Physical Review B 87(17):174,112CrossRefGoogle Scholar
  42. 42.
    Landau LD, Lifshitz E (1980) Statistical Physics. Part 1: Course of Theoretical PhysicsGoogle Scholar
  43. 43.
    Lizotte D (2008) Practical Bayesian Optimization. ThesisGoogle Scholar
  44. 44.
    Locatelli M (1997) Bayesian algorithms for one-dimensional global optimization. Journal of Global Optimization 10(1):57–76, URL <GotoISI>://WOS:A1997WJ71300004Google Scholar
  45. 45.
    MacKay DJC (1992) Information-based objective functions for active data selection. Neural Computation 4(4):590–604, URL x003C;GotoISI>://WOS:A1992JF87200009Google Scholar
  46. 46.
    McKinney W (2010) Data structures for statistical computing in Python. In: Proceedings of the 9th Python in Science Conference, pp 51–56Google Scholar
  47. 47.
    Millman KJ, Aivazis M (2011) Python for scientists and engineers. Computing in Science and Engineering 13(2):9–12CrossRefGoogle Scholar
  48. 48.
    Mishin Y (2004) Atomistic modeling of the γ and γ?-phases of the Ni–Al system. Acta Materialia 52(6):1451–1467CrossRefGoogle Scholar
  49. 49.
    Mockus J (1972) On bayesian methods for seeking the extremum. Automatics and Computers (Avtomatika i Vychislitelnayya Tekchnika) 4(1):53–52Google Scholar
  50. 50.
    Mockus J (1994) Application of Bayesian approach to numerical methods of global and stochastic optimization. Journal of Global Optimization 4(4):347–365, DOI 10.1007/bf01099263, URL <GotoISI>://WOS:A1994NM81800001Google Scholar
  51. 51.
    Mueller T, Ceder G (2009) Bayesian approach to cluster expansions. Physical Review B 80(2):024,103CrossRefGoogle Scholar
  52. 52.
    Nelson LJ, Hart GL, Zhou F, Ozoliņš V (2013) Compressive sensing as a paradigm for building physics models. Physical Review B 87(3):035,125CrossRefGoogle Scholar
  53. 53.
    Nelson LJ, Ozoliņš V, Reese CS, Zhou F, Hart GL (2013) Cluster expansion made easy with Bayesian compressive sensing. Physical Review B 88(15):155,105CrossRefGoogle Scholar
  54. 54.
    Oliphant TE (2007) Python for scientific computing. Computing in Science and Engineering 9(3):10–20CrossRefGoogle Scholar
  55. 55.
    Ong SP, Richards WD, Jain A, Hautier G, Kocher M, Cholia S, Gunter D, Chevrier VL, Persson KA, Ceder G (2013) Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis. Computational Materials Science 68:314–319CrossRefGoogle Scholar
  56. 56.
    Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, et al (2011) Scikit-learn: Machine learning in python. The Journal of Machine Learning Research 12:2825–2830MathSciNetzbMATHGoogle Scholar
  57. 57.
    Pollock TM, Tin S (2006) Nickel-based superalloys for advanced turbine engines: chemistry, microstructure and properties. Journal of propulsion and power 22(2):361–374CrossRefGoogle Scholar
  58. 58.
    Powell WB, Ryzhov IO (2012) Optimal Learning. Wiley Series in Probability and Statistics, WileyCrossRefGoogle Scholar
  59. 59.
    Purja Pun G, Mishin Y (2009) Development of an interatomic potential for the Ni-Al system. Philosophical Magazine 89(34–36):3245–3267, URL NISTInteratomicPotentialsRepository: Scholar
  60. 60.
    Raghavan V (2009) Al-Ni-Ti (Aluminum-Nickel-Titanium). Journal of Phase Equilibria and Diffusion 30(1):77–78CrossRefGoogle Scholar
  61. 61.
    Raghavan V (2010) Al-Fe-Ni (Aluminum-Iron-Nickel). Journal of Phase Equilibria and Diffusion 31:455–458, DOI 10.1007/s11669-010-9745-1CrossRefGoogle Scholar
  62. 62.
    Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. Adaptive computation and machine learning, MIT Press, Cambridge, MA, URL Tableofcontentsonly Scholar
  63. 63.
    Rosenbrock CW, Bieniek B, Blum V (2014) Hands-On Tutorial on Cluster Expansion. IPAM Los Angeles, CaliforniaGoogle Scholar
  64. 64.
    Sacks J, Welch WJ, Mitchell T, Wynn HP (1989) Design and analysis of computer experiments. Statistical Science 4(4):409–423, URL
  65. 65.
    Sakiyama M, Tomaszewicz P, Wallwork G (1979) Oxidation of iron-nickel aluminum alloys in oxygen at 600–800  C. Oxidation of Metals 13(4):311–330CrossRefGoogle Scholar
  66. 66.
    Sanchez J, Ducastelle F, Gratias D (1984) Generalized cluster description of multicomponent systems. Physica A: Statistical Mechanics and its Applications 128(1–2):334–350, DOI 10.1016/0378-4371(84)90096-7, URL
  67. 67.
    Sanchez JM (2010) Cluster expansion and the configurational theory of alloys. Phys Rev B 81:224,202, DOI 10.1103/PhysRevB.81.224202, URL
  68. 68.
    Settles B (2009) Active Learning Literature Survey. Computer Sciences Technical Report 1648, University of Wisconsin–MadisonGoogle Scholar
  69. 69.
    Shanno DF (1970) Conditioning of quasi-Newton methods for function minimization. Mathematics of Computation 24(111):647–656MathSciNetCrossRefzbMATHGoogle Scholar
  70. 70.
    Stoloff NS, Sims CT, Hagel WC (1987) Superalloys II. WileyGoogle Scholar
  71. 71.
    Taylor A, Floyd R (1953) Constitution of nickel-rich alloys of nickel-chromium-aluminum system. Institute of Metals - Journal 81:451–464Google Scholar
  72. 72.
    Taylor RH, Curtarolo S, Hart GL (2010) Ordered magnesium-lithium alloys: First-principles predictions. Physical Review B 81(2):024,112CrossRefGoogle Scholar
  73. 73.
    Torn A, Zilinskas A (1987) Global Optimization. SpringerGoogle Scholar
  74. 74.
    Van Der Walt S, Colbert SC, Varoquaux G (2011) The NumPy array: a structure for efficient numerical computation. Computing in Science &amp; Engineering 13(2):22–30CrossRefGoogle Scholar
  75. 75.
    van de Walle A (2009) Multicomponent multisublattice alloys, nonconfigurational entropy and other additions to the Alloy Theoretic Automated Toolkit. Calphad 33(2):266–278, DOI 10.1016/j.calphad.2008.12.005, URL
  76. 76.
    Walle A, Ceder G (2002) Automating first-principles phase diagram calculations. Journal of Phase Equilibria 23:348–359, DOI 10.1361/105497102770331596, URL
  77. 77.
    van de Walle A, Asta M, Ceder G (2002) The alloy theoretic automated toolkit: A user guide. Calphad 26(4):539–553, DOI 10.1016/S0364-5916(02)80006-2, URL

Copyright information

© Springer Science+Business Media LLC 2017

Authors and Affiliations

  1. 1.School of Applied and Engineering PhysicsCornell UniversityIthacaUSA
  2. 2.School of Mechanical EngineeringPurdue UniversityWest LafayetteUSA
  3. 3.Department of Aerospace and Mechanical EngineeringUniversity of Notre DameNotre DameUSA

Personalised recommendations