Machine Learning of Atomic-Scale Properties Based on Physical Principles

Reference work entry


We briefly summarize the kernel regression approach, as used recently in materials modeling, to fitting functions, particularly potential energy surfaces, and highlight how the linear algebra framework can be used to both predict and train from linear functionals of the potential energy, such as the total energy and atomic forces. We then give a detailed account of the smooth overlap of atomic position (SOAP) descriptor and kernel, showing how it arises from an abstract representation of smooth atomic densities and how it is related to several popular density-based descriptors of atomic structure. We also discuss recent generalizations that allow fine control of correlations between different atomic species, prediction, and fitting of tensorial properties and also how to construct structural kernels – applicable to comparing entire molecules or periodic systems – that go beyond an additive combination of local environments.


  1. Bartók AP, Csányi G (2015) Gaussian approximation potentials: A brief tutorial introduction. Int J Quant Chem 116:1051CrossRefGoogle Scholar
  2. Bartók AP, Payne MC, Kondor R, Csányi G (2010) Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. Phys Rev Lett 104:136403CrossRefADSGoogle Scholar
  3. Bartók AP, Gillan MJ, Manby FR, Csányi G (2013a) Machine-learning approach for one- and two-body corrections to density functional theory: Applications to molecular and condensed water. Phys Rev B 88:054104CrossRefADSGoogle Scholar
  4. Bartók AP, Kondor R, Csányi G (2013b) On representing chemical environments. Phys Rev B 87:184115CrossRefADSGoogle Scholar
  5. Bartók AP, De S, Poelking C, Bernstein N, Kermode JR, Csányi G, Ceriotti M (2017) Machine learning unifies the modeling of materials and molecules. Sci Adv 3:e1701816CrossRefADSGoogle Scholar
  6. Behler J, Parrinello M (2007) Generalized Neural-Network Representation of High-Dimensional Potential-Energy Surfaces. Phys Rev Lett 98:146401CrossRefADSGoogle Scholar
  7. Bernstein N, Kermode JR, Csányi G (2009) Hybrid atomistic simulation methods for materials systems. Rep Prog Phys 72:026501CrossRefADSGoogle Scholar
  8. Bishop CM (2016) Pattern recognition and machine learning. Springer, New YorkzbMATHGoogle Scholar
  9. Braams BJ, Bowman JM (2009) Permutationally invariant potential energy surfaces in high dimensionality. Int Rev Phys Chem 28:577–606CrossRefGoogle Scholar
  10. Brenner DW (2000) The Art and Science of an Analytic Potential. Phys Status Solidi B 217:23CrossRefADSGoogle Scholar
  11. Caro MA, Deringer VL, Koskinen J, Laurila T, Csányi G (2018) Growth Mechanism and Origin of High sp3 Content in Tetrahedral Amorphous Carbon. Phys Rev Lett 120:166101CrossRefADSGoogle Scholar
  12. Ceriotti M, Tribello GA, Parrinello M (2013) Demonstrating the Transferability and the Descriptive Power of Sketch-Map. J Chem Theory Comput 9:1521CrossRefGoogle Scholar
  13. Cheng B, Behler J, Ceriotti M (2016) Nuclear Quantum Effects in Water at the Triple Point: Using Theory as a Link Between Experiments. J Phys Chem Lett 7:2210CrossRefGoogle Scholar
  14. Chmiela S, Tkatchenko A, Sauceda HE, Poltavsky I, Schütt KT, Müller K-R (2017) Machine learning of accurate energy-conserving molecular force fields. Sci Adv 3:e1603015CrossRefADSGoogle Scholar
  15. Cuturi M (2013) Sinkhorn distances: lightspeed computation of optical transport. In: Burges CJC, Bottou L, Welling M, Ghahramani Z, Weinberger KQ (eds) Advances in neural information processing systems 26. Curran Associates, Inc., pp 2292–2300Google Scholar
  16. De S, Bartók AP, Csányi G, Ceriotti M (2016) Comparing molecules and solids across structural and alchemical space. Phys Chem Chem Phys 18:13754CrossRefGoogle Scholar
  17. Deringer VL, Csányi G (2017) Machine learning based interatomic potential for amorphous carbon. Phys Rev B 95:094203CrossRefADSGoogle Scholar
  18. Deringer VL, Pickard CJ, Csányi G (2018) Data-Driven Learning of Total and Local Energies in Elemental Boron. Phys Rev Lett 120:156001CrossRefADSGoogle Scholar
  19. Dragoni D, Daff TD, Csányi G, Marzari N (2018) Achieving DFT accuracy with a machine-learning interatomic potential: Thermomechanics and defects in bcc ferromagnetic iron. Phys Rev Mater 2:013808CrossRefGoogle Scholar
  20. Eshet H, Khaliullin RZ, Kühne TD, Behler J, Parrinello M (2012) Microscopic origins of the anomalous melting behavior of sodium under high pressure. Phys Rev Lett 108:115701CrossRefADSGoogle Scholar
  21. Faber F, Lindmaa A, von Lilienfeld OA, Armiento R (2015) Crystal structure representations for machine learning models of formation energies. Int J Quant Chem 115:1094–1101CrossRefGoogle Scholar
  22. Faber FA, Hutchison L, Huang B, Gilmer J, Schoenholz SS, Dahl GE, Vinyals O, Kearnes S, Riley PF, von Lilienfeld OA (2017) Prediction Errors of Molecular Machine Learning Models Lower than Hybrid DFT Error. J Chem Theory Comput. 13:5255–5264. CrossRefGoogle Scholar
  23. Finnis MW (2004) Interatomic forces in condensed matter. Oxford University Press, OxfordGoogle Scholar
  24. Fujikake S, Deringer VL, Lee TH, Krynski M, Elliott SR, Csányi G (2018) Gaussian approximation potential modeling of lithium intercalation in carbon nanostructures. J Chem Phys 148:241714CrossRefADSGoogle Scholar
  25. Galli G, Parrinello M (1992) Large scale electronic structure calculations. Phys Rev Lett 69:3547CrossRefADSGoogle Scholar
  26. Glielmo A, Sollich P, De Vita A (2017) Accurate interatomic force fields via machine learning with covariant kernels. Phys Rev B 95:214302CrossRefADSGoogle Scholar
  27. Glielmo A, Zeni C, De Vita A (2018) Efficient nonparametric n-body force fields from machine learning. Phys Rev B 97:184307 CrossRefADSGoogle Scholar
  28. Goedecker S (1999) Linear scaling electronic structure methods. Rev Mod Phys 71:1085CrossRefADSGoogle Scholar
  29. Gonzalez TF (1985) Clustering to minimize the maximum intercluster distance. Theor Comput Sci 38:293CrossRefMathSciNetzbMATHGoogle Scholar
  30. Grisafi A, Wilkins DM, Csányi G, Ceriotti M (2018) Symmetry-Adapted Machine Learning for Tensorial Properties of Atomistic Systems. Phys Rev Lett 120:036002CrossRefADSGoogle Scholar
  31. Haar A (1933) Der Massbegriff in der Theorie der kontinuerlichen Gruppen. Ann Math 34:147CrossRefMathSciNetzbMATHGoogle Scholar
  32. Hartigan JA, Wong MA (1979) Algorithm AS 136: A K-Means Clustering Algorithm. J R Stat Soc Ser C (Appl Stat) 28:100zbMATHGoogle Scholar
  33. Imbalzano G, Anelli A, Giofré D, Klees S, Behler J, Ceriotti M (2018) Automatic selection of atomic fingerprints and reference configurations for machine-learning potentials. J Chem Phys 148:241730CrossRefADSGoogle Scholar
  34. John ST, Csányi G (2017) Many-Body Coarse-Grained Interactions Using Gaussian Approximation Potentials. J Phys Chem B 121:10934CrossRefGoogle Scholar
  35. Kajita S, Ohba N, Jinnouchi R, Asahi R (2017) A Universal 3D Voxel Descriptor for Solid-State Material Informatics with Deep Convolutional Neural Networks. Sci Rep 7:1CrossRefGoogle Scholar
  36. Kazhdan M, Funkhouser T, Rusinkiewicz S (2003) Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors. In: Proceedings of the 2003 Eurographics/ACM SIGGRAPH symposium on geometry processing, SGP’03. Eurographics Association, pp 156–164Google Scholar
  37. Mahoney MW, Drineas P (2009) CUR matrix decompositions for improved data analysis. Proc Natl Acad Sci USA 106:697CrossRefADSMathSciNetzbMATHGoogle Scholar
  38. Morawietz T, Singraber A, Dellago C, Behler J (2016) How van der Waals interactions determine the unique properties of water. Proc Natl Acad Sci USA 113:8368CrossRefADSGoogle Scholar
  39. Nguyen TT, Szekely E, Imbalzano G, Behler J, Csányi G, Ceriotti M, Götz AW, Paesani F (2018) Comparison of permutationally invariant polynomials, neural networks, and Gaussian approximation potentials in representing water interactions through many-body expansions. J Chem Phys 148:241725CrossRefADSGoogle Scholar
  40. Prabhakaran S, Raman S, Vogt JE, Roth V (2012) Automatic Model Selection in Archetype Analysis. In: Joint DAGM (German Association for pattern recognition) and OAGM symposium. Lecture Notes in Computer Science, vol 7476. Springer, Berlin Heidelberg, pp 458–467Google Scholar
  41. Prodan E, Kohn W (2005) Nearsightedness of electronic matter. Proc Natl Acad Sci USA 102:11635CrossRefADSGoogle Scholar
  42. Quinonero-Candela JQ, Rasmussen CE (2005) A Unifying View of Sparse Approximate Gaussian Process Regression. J Mach Learn Res 6:19391959MathSciNetzbMATHGoogle Scholar
  43. Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. MIT Press, CambridgezbMATHGoogle Scholar
  44. Rupp M, Tkatchenko A, Müller K-R, von Lilienfeld OA (2012) Fast and accurate modeling of molecular atomization energies with machine learning. Phys Rev Lett 108:058301CrossRefADSGoogle Scholar
  45. Rowe P, Csányi G, Alfè D, Michaelides A (2018) Development of a machine learning potential for graphene. Phys Rev B 97:054303CrossRefADSGoogle Scholar
  46. Schölkopf B, Smola AJ (2002) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, CambridgeGoogle Scholar
  47. Schütt KT, Glawe H, Brockherde F, Sanna A, Müller KR, Gross EKU (2014) How to represent crystal structures for machine learning: Towards fast prediction of electronic properties. Phys Rev B 89:205118CrossRefADSGoogle Scholar
  48. Smith JS, Isayev O, Roitberg AE (2017) ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost. Chem Sci 8:3192CrossRefGoogle Scholar
  49. Snelson E, Ghahramani Z (2006) Sparse Gaussian Processes using Pseudo-inputs. In: Weiss V, Schölkopf B, Platt JC (eds) Advances in neural information processing systems 18 (NIPS 2005) MIT Press, pp 1257–1264Google Scholar
  50. Solak E, Rasmussen CE, Leith DJ, Murray-Smith R, Leithead WE (2003) Derivative observations in Gaussian Process Models of Dynamic Systems. In: NIPS’02: Proceedings of the 15th International Conference on Neural Information Processing System 2002, pp 1057–1064Google Scholar
  51. Szlachta WJ, Bartók AP, Csányi G (2014) Accuracy and transferability of Gaussian approximation potential models for tungsten. Phys Rev B 90:104108CrossRefADSGoogle Scholar
  52. Thompson AP, Swiler LP, Trott CR, Foiles SM, Tucker GJ (2015) Spectral neighbor analysis method for automated generation of quantum-accurate interatomic potentials. J Comput Phys 285:316CrossRefADSMathSciNetzbMATHGoogle Scholar
  53. Tikhonov AN, Goncharsky A, Stepanov VV, Yagola AG (1995) Numerical methods for the solution of ill-posed problems. Kluwer Academic, DordrechtCrossRefzbMATHGoogle Scholar
  54. Varshalovich DA, Moskalev AN, Khersonskii VK (1988) Quantum theory of angular momentum. World Scientific, SingaporeCrossRefGoogle Scholar
  55. Yang W (1991) Direct calculation of electron density in density-functional theory. Phys Rev Lett 66:1438CrossRefADSGoogle Scholar
  56. Zhang L, Han J, Wang H, Car R, Weinan E (2018) Deep potential molecular dynamics: A scalable model with the accuracy of quantum mechanics. Phys Rev Lett 120:143001CrossRefADSGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Laboratory of Computational Science and Modelling, Institute of MaterialsÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
  2. 2.Engineering LaboratoryUniversity of CambridgeCambridgeUK

Personalised recommendations