High-Dimensional Neural Network Potentials for Atomistic Simulations

Abstract

High-dimensional neural network potentials, proposed by Behler and Parrinello in 2007, have become an established method to calculate potential energy surfaces with first-principles accuracy at a fraction of the computational costs. The method is general and can describe all types of chemical interactions (e.g., covalent, metallic, hydrogen bonding, and dispersion) for the entire periodic table, including chemical reactions, in which bonds break or form. Typically, many-body atom-centered symmetry functions, which incorporate the translational, rotational, and permutational invariances of the potential energy surface exactly, are used as descriptors for the atomic environments. This chapter describes how such symmetry functions and high-dimensional neural network potentials are constructed and validated.

Appendix: Calculating the Force Components on an Atom

The following example illustrates how the force components on the atom O1 in a three-atom system consisting of two O atoms, O1 and O2, and one H atom, H3, are calculated. The NN for O has the architecture 2-2-1 (one hidden layer containing two nodes; two input features described by two radial symmetry functions G ^O:H and G ^O:O). Similarly, the NN for H also has the architecture 2-2-1 with two radial symmetry functions G ^H:H and G ^H:O.

$$\displaystyle \begin{aligned} {\mathbf{G}}^{\mathrm{O}}(\mathrm{O}1) = \begin{pmatrix} G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}1) \\ G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}1) \end{pmatrix}, \,\, {\mathbf{G}}^{\mathrm{O}}(\mathrm{O}2) = \begin{pmatrix} G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}2) \\ G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}2) \end{pmatrix}, \,\, {\mathbf{G}}^{\mathrm{H}}(\mathrm{H}3) = \begin{pmatrix} G^{\mathrm{H}:\mathrm{H}}(\mathrm{H}3) \\ G^{\mathrm{H}:\mathrm{O}}(\mathrm{H}3) \end{pmatrix} \end{aligned}$$

None of the symmetry functions are scaled, and R _s = 0 Å. The activation function in the hidden layer for both NNs is the logistic function, $f(x) = \frac {1}{1+\exp (-x)}$, for which the derivative can easily be expressed in terms of the function value: f′(x) = f(x)(1 − f(x)).

The total energy E is obtained as

$$\displaystyle \begin{aligned} E = E_{\mathrm{O}1} + E_{\mathrm{O}2} + E_{\mathrm{H}3} \end{aligned}$$

and the force along the α-component of O1 (with α often being one of the Cartesian x, y, or z components) is calculated as

$$\displaystyle \begin{aligned} -\frac{\partial E}{\partial \alpha_{\mathrm{O}1}} &= -\frac{\partial E_{\mathrm{O}1}}{\partial \alpha_{\mathrm{O}1}} -\frac{\partial E_{\mathrm{O}2}}{\partial \alpha_{\mathrm{O}1}} -\frac{\partial E_{\mathrm{H}3}}{\partial \alpha_{\mathrm{O}1}} \\ &= -\frac{\partial E_{\mathrm{O}1}}{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}1)}\frac{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}1)}{\partial \alpha_{\mathrm{O}1}} -\frac{\partial E_{\mathrm{O}1}}{\partial G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}1)}\frac{\partial G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}1)}{\partial \alpha_{\mathrm{O}1}} \\ & \quad -\frac{\partial E_{\mathrm{O}2}}{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}2)}\frac{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}2)}{\partial \alpha_{\mathrm{O}1}} -\frac{\partial E_{\mathrm{O}2}}{\partial G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}2)}\frac{\partial G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}2)}{\partial \alpha_{\mathrm{O}1}} \\ & \quad -\frac{\partial E_{\mathrm{H}3}}{\partial G^{\mathrm{H}:\mathrm{H}}(\mathrm{H}3)}\frac{\partial G^{\mathrm{H}:\mathrm{H}}(\mathrm{H}3)}{\partial \alpha_{\mathrm{O}1}} -\frac{\partial E_{\mathrm{H}3}}{\partial G^{\mathrm{H}:\mathrm{O}}(\mathrm{H}3)}\frac{\partial G^{\mathrm{H}:\mathrm{O}}(\mathrm{H}3)}{\partial \alpha_{\mathrm{O}1}} \end{aligned} $$

The partial derivatives of the atomic energies with respect to the NN input features depend on the NN architecture (number of input features, number of hidden layers, number of nodes per hidden layer) as well as the activation function employed in the hidden units. Here, $y_m^{[1]}$ denotes the mth node in the first (and only) hidden layer. The NN weights are stored in matrices

$$\displaystyle \begin{aligned} {\mathbf{A}}^{[1],\mathrm{O}} = \begin{pmatrix} a_{11}^{[1],\mathrm{O}} & a_{12}^{[1],\mathrm{O}} \\[0.2cm] a_{21}^{[1],\mathrm{O}} & a_{22}^{[1],\mathrm{O}} \end{pmatrix}, \,\,\, {\mathbf{b}}^{[1], \mathrm{O}} = \begin{pmatrix} b_1^{[1],\mathrm{O}} \\[0.2cm] b_2^{[1],\mathrm{O}} \end{pmatrix}, \,\,\, {\mathbf{A}}^{[2], \mathrm{O}} = \begin{pmatrix} a_{11}^{[2],\mathrm{O}} \\[0.2cm] a_{21}^{[2],\mathrm{O}} \end{pmatrix}, \,\,\, b^{[2],\mathrm{O}} \end{aligned}$$

with a similar setup for the weights in the hydrogen NN.

$$\displaystyle \begin{aligned} y_1^{[1],\mathrm{O}1} &= f\left(a_{11}^{[1],\mathrm{O}} G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}1) + a_{21}^{[1],\mathrm{O}} G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}1) + b_1^{[1],\mathrm{O}}\right) \\ y_2^{[1],\mathrm{O}1} &= f\left(a_{12}^{[1],\mathrm{O}} G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}1) + a_{22}^{[1],\mathrm{O}} G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}1) + b_2^{[1],\mathrm{O}}\right) \\ E_{\mathrm{O}1} &= a_{11}^{[2],\mathrm{O}}y_1^{[1],\mathrm{O}1} + a_{21}^{[2],\mathrm{O}}y_2^{[1],\mathrm{O}1} + b^{[2],\mathrm{O}} \\ \frac{\partial E_{\mathrm{O}1}}{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}1)} &= a_{11}^{[2],\mathrm{O}}\frac{\partial y_1^{[1],\mathrm{O}1}}{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}1)} + a_{21}^{[2],\mathrm{O}}\frac{\partial y_2^{[1],\mathrm{O}1}}{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}1)} \\ &= a_{11}^{[2],\mathrm{O}}a_{11}^{[1],\mathrm{O}}y_1^{[1],\mathrm{O}1}\left(1-y_1^{[1],\mathrm{O}1}\right) + a_{21}^{[2],\mathrm{O}}a_{12}^{[1],\mathrm{O}} y_2^{[1],\mathrm{O}1}\left(1-y_2^{[1],\mathrm{O}1}\right) \\ \frac{\partial E_{\mathrm{O}1}}{\partial G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}1)} &= a_{11}^{[2],\mathrm{O}}a_{21}^{[1],\mathrm{O}}y_1^{[1],\mathrm{O}1}\left(1-y_1^{[1],\mathrm{O}1}\right) + a_{21}^{[2],\mathrm{O}}a_{22}^{[1],\mathrm{O}}y_2^{[1],\mathrm{O}1}\left(1-y_2^{[1],\mathrm{O}1}\right)\\ y_1^{[1],\mathrm{O}2} &= f\left(a_{11}^{[1],\mathrm{O}} G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}2) + a_{21}^{[1],\mathrm{O}} G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}2) + b_1^{[1],\mathrm{O}}\right) \\ y_2^{[1],\mathrm{O}2} &= f\left(a_{12}^{[1],\mathrm{O}} G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}2) + a_{22}^{[1],\mathrm{O}} G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}2) + b_2^{[1],\mathrm{O}}\right) \\ E_{\mathrm{O}2} &= a_{11}^{[2],\mathrm{O}}y_1^{[1],\mathrm{O}2} + a_{21}^{[2],\mathrm{O}}y_2^{[1],\mathrm{O}2} + b^{[2],\mathrm{O}} \\ \frac{\partial E_{\mathrm{O}2}}{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}2)} &= a_{11}^{[2],\mathrm{O}}a_{11}^{[1],\mathrm{O}}y_1^{[1],\mathrm{O}2}\left(1-y_1^{[1],\mathrm{O}2}\right) + a_{21}^{[2],\mathrm{O}}a_{12}^{[1],\mathrm{O}} y_2^{[1],\mathrm{O}2}\left(1-y_2^{[1],\mathrm{O}2}\right) \\ \frac{\partial E_{\mathrm{O}2}}{\partial G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}2)} &= a_{11}^{[2],\mathrm{O}}a_{21}^{[1],\mathrm{O}}y_1^{[1],\mathrm{O}2}\left(1-y_1^{[1],\mathrm{O}2}\right) + a_{21}^{[2],\mathrm{O}}a_{22}^{[1],\mathrm{O}}y_2^{[1],\mathrm{O}2}\left(1-y_2^{[1],\mathrm{O}2}\right)\\ y_1^{[1],\mathrm{H}3} &= f\left(a_{11}^{[1],\mathrm{H}} G^{\mathrm{H}:\mathrm{H}}(\mathrm{H}3) + a_{21}^{[1],\mathrm{H}} G^{\mathrm{H}:\mathrm{O}}(\mathrm{H}3) + b_1^{[1],\mathrm{H}}\right) \\ y_2^{[1],\mathrm{H}3} &= f\left(a_{12}^{[1],\mathrm{H}} G^{\mathrm{H}:\mathrm{H}}(\mathrm{H}3) + a_{22}^{[1],\mathrm{H}} G^{\mathrm{H}:\mathrm{O}}(\mathrm{H}3) + b_2^{[1],\mathrm{H}}\right) \\ E_{\mathrm{H}3} &= a_{11}^{[2],\mathrm{H}}y_1^{[1],\mathrm{H}3} + a_{21}^{[2],\mathrm{H}}y_2^{[1],\mathrm{H}3} + b^{[2],\mathrm{H}} \\ \frac{\partial E_{\mathrm{H}3}}{\partial G^{\mathrm{H}:\mathrm{H}}(\mathrm{H}3)} &= a_{11}^{[2],\mathrm{H}}a_{11}^{[1],\mathrm{H}}y_1^{[1],\mathrm{H}3}\left(1-y_1^{[1],\mathrm{H}3}\right) + a_{21}^{[2],\mathrm{H}}a_{12}^{[1],\mathrm{H}}y_2^{[1],\mathrm{H}3}\left(1-y_2^{[1],\mathrm{H}3}\right) \\ \frac{\partial E_{\mathrm{H}3}}{\partial G^{\mathrm{H}:\mathrm{O}}(\mathrm{H}3)} &= a_{11}^{[2],\mathrm{H}}a_{21}^{[1],\mathrm{H}}y_1^{[1],\mathrm{H}3}\left(1-y_1^{[1],\mathrm{H}3}\right) + a_{21}^{[2],\mathrm{H}}a_{22}^{[1],\mathrm{H}}y_2^{[1],\mathrm{H}3}\left(1-y_2^{[1],\mathrm{H}3}\right) \end{aligned} $$

The force component along α _O1 thus depends on, for example, $\frac {\partial E_{\mathrm{O}2}}{\partial G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}2)}$, which in turn depends on $y_1^{[1],\mathrm{O}2}$ and $y_2^{[1],\mathrm{O}2}$, which depend on G ^O:H(O2) and G ^O:O(O2). Thus all symmetry functions on the atom O2, and consequently the entire environment within the cutoff sphere around O2, contribute to the force acting on the atom O1.

The partial derivatives of the radial symmetry functions with respect to the coordinate α _O1 become

$$\displaystyle \begin{aligned} \frac{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}1)}{\partial \alpha_{\mathrm{O}1}} & = \mathrm{e}^{-\eta R_{\mathrm{O}1\mathrm{H}3}^2} \left( -2\eta R_{\mathrm{O}1\mathrm{H}3}{f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{H}3})\frac{\partial R_{\mathrm{O}1\mathrm{H}3}}{\partial \alpha_{\mathrm{O}1}} + \frac{\partial {f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{H}3})}{\partial \alpha_{\mathrm{O}1}} \right) \\ \frac{\partial G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}1)}{\partial \alpha_{\mathrm{O}1}} & = \mathrm{e}^{-\eta R_{\mathrm{O}1\mathrm{O}2}^2} \left( -2\eta R_{\mathrm{O}1\mathrm{O}2}{f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{O}2})\frac{\partial R_{\mathrm{O}1\mathrm{O}2}}{\partial \alpha_{\mathrm{O}1}} + \frac{\partial {f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{O}2})}{\partial \alpha_{\mathrm{O}1}} \right) \\ \frac{\partial G^{\mathrm{O}:\mathrm{H}}(\mathrm{O}2)}{\partial \alpha_{\mathrm{O}1}} & = \mathrm{e}^{-\eta R_{\mathrm{O}2\mathrm{H}3}^2} \left( -2\eta R_{\mathrm{O}2\mathrm{H}3}{f_{\mathrm{c}}}(R_{\mathrm{O}2\mathrm{H}3})\frac{\partial R_{\mathrm{O}2\mathrm{H}3}}{\partial \alpha_{\mathrm{O}1}} + \frac{\partial {f_{\mathrm{c}}}(R_{\mathrm{O}2\mathrm{H}3})}{\partial \alpha_{\mathrm{O}1}} \right) \\ & = 0 \\ \frac{\partial G^{\mathrm{O}:\mathrm{O}}(\mathrm{O}2)}{\partial \alpha_{\mathrm{O}1}} & = \mathrm{e}^{-\eta R_{\mathrm{O}1\mathrm{O}2}^2} \left( -2\eta R_{\mathrm{O}1\mathrm{O}2}{f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{O}2})\frac{\partial R_{\mathrm{O}1\mathrm{O}2}}{\partial \alpha_{\mathrm{O}1}} + \frac{\partial {f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{O}2})}{\partial \alpha_{\mathrm{O}1}} \right) \\ \frac{\partial G^{\mathrm{H}:\mathrm{H}}(\mathrm{H}3)}{\partial \alpha_{\mathrm{O}1}} & = 0 \\ \frac{\partial G^{\mathrm{H}:\mathrm{O}}(\mathrm{H}3)}{\partial \alpha_{\mathrm{O}1}} & = \mathrm{e}^{-\eta R_{\mathrm{O}1\mathrm{H}3}^2} \left( -2\eta R_{\mathrm{O}1\mathrm{H}3}{f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{H}3})\frac{\partial R_{\mathrm{O}1\mathrm{H}3}}{\partial \alpha_{\mathrm{O}1}} + \frac{\partial {f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{H}3})}{\partial \alpha_{\mathrm{O}1}} \right) \\ & \quad + \mathrm{e}^{-\eta R_{\mathrm{O}2\mathrm{H}3}^2} \left( -2\eta R_{\mathrm{O}2\mathrm{H}3}{f_{\mathrm{c}}}(R_{\mathrm{O}2\mathrm{H}3})\frac{\partial R_{\mathrm{O}2\mathrm{H}3}}{\partial \alpha_{\mathrm{O}1}} + \frac{\partial {f_{\mathrm{c}}}(R_{\mathrm{O}2\mathrm{H}3})}{\partial \alpha_{\mathrm{O}1}} \right) \\ & = \mathrm{e}^{-\eta R_{\mathrm{O}1\mathrm{H}3}^2} \left( -2\eta R_{\mathrm{O}1\mathrm{H}3}{f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{H}3})\frac{\partial R_{\mathrm{O}1\mathrm{H}3}}{\partial \alpha_{\mathrm{O}1}} + \frac{\partial {f_{\mathrm{c}}}(R_{\mathrm{O}1\mathrm{H}3})}{\partial \alpha_{\mathrm{O}1}} \right), \end{aligned} $$

where η is the η-value of the pertinent symmetry function. All the above partial derivatives are calculated as sums over neighbors, but in this example, there are only 1 H and 1 O neighbor around each O atom. There are two O neighbors around H3, but one of the terms in the sum defining $\frac {\partial G^{\mathrm{H}:\mathrm{O}}(\mathrm{H}3)}{\partial \alpha _{\mathrm{O}1}}$ becomes 0, since the position of O1 does not affect the distance between O2 and H3.