Liquid–Crystal Structure Inheritance in Machine Learning Potentials for Network-Forming Systems

It has been studied whether machine learning interatomic potentials parameterized with only disordered configurations corresponding to liquid can describe the properties of crystalline phases and predict their structure. The study has been performed for a network-forming system SiO₂, which has numerous polymorphic phases significantly different in structure and density. Using only high-temperature disordered configurations, a machine learning interatomic potential based on artificial neural networks (DeePMD model) has been parameterized. The potential reproduces well ab initio dependences of the energy on the volume and the vibrational density of states for all considered tetra- and octahedral crystalline phases of SiO₂. Furthermore, the combination of the evolutionary algorithm and the developed DeePMD potential has made it possible to reproduce the really observed crystalline structures of SiO₂. Such a good liquid–crystal portability of the machine learning interatomic potential opens prospects for the simulation of the structure and properties of new systems for which experimental information on crystalline phases is absent.

INTRODUCTION

The application of machine learning methods to construct classical interatomic potentials in recent years has provided significant progress in the atomistic simulation of materials [1–3]. The main idea of such an approach is to approximate the potential energy surface of a system by some general many-body function (e.g., a multilayer neural network) using reference values of the energies and interparticle forces obtained in ab initio calculations. This idea is similar to the force-matching method proposed in [4], which was successfully used to parameterize model potentials such as an embedded-atom potential [5–7]. The fundamental difference of machine learning interatomic potentials (MLIPs) is the use of much more complex many-body functions sometimes having hundreds of thousands of tunable parameters, which allows one to reach an accuracy comparable with the accuracy of ab initio methods using orders of magnitude less computational resources.

Although this approach is attractive, many important items concerning the construction of MLIPs and their application to solve particular problems are unsolved. The key problem is the development of effective methods of the formation of a training dataset (TD), i.e., a set of representative atomic configurations and the corresponding energies and forces used to parameterize MLIPs. In most cases, TD is based on ordered configurations corresponding to various crystalline phases of the system under study; disordered (liquid, amorphous) configurations are used only as supplements to crystalline configurations or in the case where the aim is to study the corresponding disordered phases [3]. Natural questions arise: What are the crystalline configurations that should be considered to parameterize potentials and what is an algorithm for their generation? One of the possible variants is to use crystallographic databases or repositories of computational projects such as Materials Project, OQMD, and AiiDA. Configurations from such databases form a basic TD, which is then expanded by introducing structural defects and/or in the process of molecular dynamics simulations [8]. The main problem of this approach is the necessity of a priori knowledge or at least some assumptions on crystalline structures that can appear in the studied system. However, this information for many (particularly multicomponent) systems can be very limited or unavailable alt-ogether. In these cases, methods that do not involve a priori information on crystalline structures are necessary for the construction of representative TDs.

One of the methods to solve this problem is the parameterization of MLIPs with the use of only disordered configurations corresponding to liquid. Since atoms in liquid are fairly mobile, the molecular dynamics simulation of liquids at various densities, temperatures, and pressures allows one to quite rapidly form the representative TD and to parameterize an MLIP that makes it possible to simulate the properties of melts with a high accuracy [9, 10]. How will the MLIP parameterized only with disordered liquid configuration describe the properties of crystalline phases?

This problem is still poorly studied; only a few examples of such an approach are reported. In particular, the MLIP parameterized with primarily disordered configurations was recently proposed for the simulation of rubidium [11]. It was shown that this potential reproduces a maximum on the pressure dependence of the melting temperature of the bcc phase of rubidium. However, rubidium is a quite simple alkali metal with a predominantly isotropic interatomic interaction. In addition, the authors of [11] used a small number of crystalline configurations for the parameterization. Will such a machine learning procedure be efficient for the description of more complex multicomponent systems with an anisotropic interaction? This question was partially answered in [12], where the MLIP parameterized on liquid water configurations was used to calculate the density, energies, and vibrational spectra of various ice phases in comparison with ab initio calculations. However, the accuracy of the reproduction of the energies of crystalline structures in [12] was low. Indeed, the correlation between the MLIP and ab initio results for the energies systematically deviates from the diagonal by about the standard deviation (cf. Fig. 4 in this work). A similar deviation was also observed in [13], where the MLIP parameterized on liquid configurations was used to seek stable crystalline structures with the evolutionary algorithm. This is a serious drawback because systematic errors in the description of the energies of crystals are unallowable when the MLIP is used to predict their structure.

A fundamental question arises: Can a self-consistent method of constructing MLIPs that does not require a priori information on crystalline structures be developed to describe both disordered and ordered phases with a high accuracy and to predict their structure? A universal answer to this question is obviously absent and an answer will be different for systems of different natures, different machine learning models, and different methods of the formation of the TD. In this work, we analyze this problem for network-forming systems considering SiO₂ as a representative of these systems. This is a rather complex system with numerous polymorphic phases significantly different in structure and density. The construction of MLIPs for SiO₂ that reproduce well the known experimental and calculated properties of both crystalline and disordered phases was reported in [10, 14, 15]. However, crystals were simulated in all cases with potentials parameterized using the corresponding crystalline configurations. The problems of liquid–crystal transferability in the construction of MLIPs and, in particular, the possibility of prediction of the structure of crystalline phases using information on the local structure of liquid were not considered in the cited works.

The discussed problems are not only important for computational materials science but are also of fundamental interest for the physics of disordered systems. Indeed, it is very interesting and poorly studied whether the local structure of liquids carries information on the structure of crystals and to what extent it can be used for the calculation and prediction of their properties. Recent studies of this problem [16–19] show that the analysis of the properties of a melt sometimes allows important (even predictive) conclusions on the structure and properties of solid phases. The genetic relation between the structures of liquid and crystal is called below structure inheritance.

DEVELOPMENT OF THE POTENTIAL

The key stages of the development of any MLIP are (i) the choice of a method for the transformation of the coordinates of the local environment of atoms to a set of so-called descriptors, i.e., numbers invariant under translations, rotations, and permutations, and (ii) the choice of a regression model used to approximate the potential energy surface. Experience shows that the choice of descriptors is more significant for the accuracy and efficiency of the model than the choice of the regression model [9]. In this work, we use the DeePMD model [20] involving two coupled artificial neural networks (ANNs): an embedded network for the transformation of the local environment of atoms to descriptors and an approximating network for the transformation of the descriptors of all atoms of the system to the potential energy. The architecture of both ANNs is chosen in the form of a multilayer fully connected forward propagation neural network (so‑called perceptron). The complete mathematical description of the model is rather lengthy and is given in [2]. Here, we briefly describe only the main ideas.

The local environment of the ith atom is described by the matrix ${{\mathcal{R}}^{i}} \in {{\mathbb{R}}^{{{{N}_{i}} \times 3}}}$, which includes the coordinates of ${{N}_{i}}$ nearest neighbors in a sphere with the radius r_cut, which are specified in a local coordinate system. This matrix is transformed into ${{\tilde {\mathcal{R}}}^{i}} \in {{\mathbb{R}}^{{{{N}_{i}} \times 4}}}$ as $\{ {\mathbf{r}_{{ij}}}\} \to s({{r}_{{ij}}})\{ 1,{\mathbf{r}_{{ij}}}{\text{/}}{{r}_{{ij}}}\}$, where $s({{r}_{{ij}}})$ is a certain continuously differentiable function monotonically vanishing at r → r_cut. A set of descriptors for the environment of the ith atom is formed as ${{\mathcal{D}}_{i}} = \frac{1}{{N_{i}^{2}}}{{\left( {\mathcal{G}_{i}^{{ < M}}} \right)}^{T}}{{\tilde {\mathcal{R}}}_{i}}\tilde {\mathcal{R}}_{i}^{T}{{\mathcal{G}}_{i}}$, where ${{\mathcal{G}}_{i}}$ is the vector with the length M returned by the embedded ANN to the input layer of which the vectors $s({{R}_{{ij}}})$ are supplied, and $\mathcal{G}_{i}^{{ < M}}$ is the vector consisting of the first ${{M}_{1}} < M$ components of the vector ${{\mathcal{G}}_{i}}$. It can be shown that the descriptors ${{\mathcal{D}}_{i}}$ ensure invariance under translations, rotations, and permutations. It is important that these descriptors are adaptive because the weights of the embedded ANN are also modified during the parameterization. The use of such descriptors requires minimum human control; only the architecture of the embedded ANN should be specified. Because of a large number of parameters compared to other descriptors, ANN descriptors are quite flexible to describe different phases of a material in wide temperature and pressure ranges.

The method of formation of the TD is of key importance for the parameterization of MLIPs. One of the standard methods is to build the TD from configurations obtained in an ab initio molecular dynamics simulation. This approach can give unsatisfactory results because the length of molecular dynamics trajectories is usually no longer than several picoseconds. Configurations thus generated are strongly correlated because the system at simulation times can be confined in the configuration space region corresponding to a certain local minimum of the potential energy (this problem likely arose in [13]). To solve this problem, various active learning algorithms are used to efficiently select configurations in a wide configuration space region [2, 21].

In this work, to form the TD, we used one of the active learning algorithms implemented in the DP-GEN software [22]. The algorithm is based on the iterative repetition of three stages: training, exploration, and labeling. In the training stage, ${{N}_{{{\text{models}}}}}$ DeePMD potentials with the same architecture of the ANN but with different random distributions of the initial parameters are parameterized on the existing TD. These potentials are used to perform the classical molecular dynamics simulation at different temperatures T, pressures P, and densities (exploration stage). The use of the classical molecular dynamics simulation allows one to start tens of parallel calculations with different thermodynamic parameters, to consider trajectories with a length of hundreds of picoseconds, and thereby to effectively cover wide configuration space regions. For each trajectory, the accuracy of the potential is controlled, as well as configurations on which the model gives a low accuracy. To this end, the following quantity is estimated for configurations ${{\mathcal{R}}_{t}}$ appearing in this trajectory in the tth step:

$${{\epsilon }_{t}} = \mathop {\max }\limits_i \sqrt {\langle {\text{|}}{{F}_{{w,i}}}({{\mathcal{R}}_{t}}) - \langle {{F}_{{w,i}}}({{\mathcal{R}}_{t}})\rangle {{{\text{|}}}^{2}}\rangle } ,$$

where ${{F}_{{w,i}}}({{\mathcal{R}}_{t}})$ is the force acting on the ith atom in the DeePMD model with the parameters w in the configuration ${{\mathcal{R}}_{t}}$ and averaging is performed over an ensemble of models. Configurations for which the inequalities ${{\epsilon }_{l}} < {{\epsilon }_{t}} < {{\epsilon }_{h}}$ are valid are selected as candidates for the supplement of the TD. Then, ab initio calculations of the energies, interatomic forces, and stress tensors are carried out for a certain given number of configurations from the list of candidates (labeling stage). The described procedure is repeated until the saturation of the TD; i.e., new configurations are no longer selected in the exploration stage. This algorithm allows the formation of a compact representative TD including statistically independent configurations.

In this work, the classical molecular dynamics simulation with the MLIP was performed using the LA-MMPS [23], and the ab initio DFT calculations were carried out with the VASP [24]. Supercells consisting of 342 particles (114 SiO₂ structural units) were considered for the parameterization.

Important parameters of the DeePMD model are the cutoff radius ${{r}_{{{\text{cut}}}}}$, the smoothing radius ${{r}_{{{\text{smth}}}}}$, the architecture of the embedded ANN, the architecture of the approximating ANN, the initial learning rate ${{l}_{{{\text{start}}}}}$, the final learning rate ${{l}_{{{\text{stop}}}}}$, the initial (final) weight of energies (forces, virials) $p_{{e(f,v)}}^{{{\text{start(limit)}}}}$, the number of learning epochs ${{N}_{{{\text{epoch}}}}}$, and the learning batch size ${{N}_{{{\text{batch}}}}}$.

In addition to the parameters listed above, the parameters of ab initio calculations affect the ANN potential. The most important parameters in our case are the type of approximation used for the exchange correlation energy, the type of pseudopotentials, the cutoff energy of the plane wave basis ${{E}_{{{\text{cut}}}}}$, the method of smoothing of the electron density of states (ISMEAR), the smoothing parameter σ, and the grid in the k space. The calculations in the VASP were performed with the PAW–PBE pseudopotentials [25].

Table 1 summarizes all key parameters of the active learning algorithm, DeePMD model, and ab initio calculations. These parameters were used to develop two potentials; one of them (LP-DeePMD) was parameterized only on melt configurations corresponding to trajectories at zero pressure, whereas the second potential (HP-DeePMD) was parameterized with the inclusion of high pressures up to 70 GPa.

Table 1. Parameters of the DeePMD potential and the VASP used to parameterize the potentials

TEST OF THE POTENTIAL

Figures 1a and 1b show correlations between the DFT and HP-DeePMD results for the energies and force components, respectively, calculated for the same configurations. It is seen that the DeePMD potential reproduces quite well the ab initio energies and forces for configurations corresponding to the liquid state. Experience shows that such an accuracy ensures good reproducibility of the structure and dynamic properties of melts [9, 10].

Fig. 1.

(Color online) Correlations between the VASP and DeePMD results for the (а) energies and (b) force components.

To analyze the transformability of the determined potential to crystalline phases of SiO₂, we calculated the volume dependences of the energies for the α and β modifications of cristobalite, quartz, and tridymite, as well as for keatite, coesite, moganite, saferite, and stishovite. These dependences calculated with the LP-DeePMD and HP-DeePMD potentials are presented in Fig. 2. It is seen that the LP-DeePMD potential reproduces very well the equations of state of tetrahedral phases (see Figs. 2a–2h and 2k), but it cannot describe octahedral phases (see Figs. 2i and 2k). This result is not surprising because the local structure of a melt at zero pressure has only tetrahedral configurations. However, the inclusion of high-pressure liquid configurations, which include octahedral configurations, makes it possible to reproduce the equations of states of saferite and stishovite along with the description of tetrahedral phases with almost the same accuracy. In addition to the equations of states, we calculated vibrational densities of states (Fourier transforms of the autocorrelation function of the velocity) at temperatures and pressures in the region of stability of a certain phase using the HP-DeePMD potential and DFT for all crystalline phases studied in the work (see the insets of Fig. 2). It is seen in Fig. 2 that the developed potential ensures a high accuracy of the calculation of vibrational spectra compared to ab initio calculations. Differences at low frequencies are due to a rather short length of molecular dynamics trajectories.

Fig. 2.

(Color online) Volume dependences of the energy obtained with the DeePMD potentials and in the DFT calculations with the VASP for (a) α-cristobalite, (b) α-quartz, (c) α-tridymite, (d) $\beta$-cristobalite, (e) $\beta$-quartz, (f) $\beta$-tridymite, (g) keatite, (h) moganite, (i) saferite, (j) coesite, and (k) stishovite. The insets show vibrational densities of states obtained with the HP-‑DeePMD potential and in ab initio molecular dynamics simulation with the VASP.

Thus, the DeePMD potential parameterized only on disordered configurations can correctly describe the energies and vibrational properties of crystalline phases, which is a nontrivial result.

EVOLUTIONARY SEARCH

To verify the possibility of predicting stable crystalline structures with the liquid MLIP, we used the HP-DeePMD potential to optimize structures and to calculate their enthalpies in the evolutionary search with the USPEX code [26–28]. Using this method, we determined both the most stable and metastable crystalline phases at pressures of 0 and 10 GPa. This problem for SiO₂ is very difficult because this system has not only phases present in the equilibrium phase diagram but also tens of metastable phases differing in energy by several meV per atom. For this reason, search for stable phases of SiO₂ is difficult even with ab initio calculations [29]. In particular, the experimentally most stable $\alpha$-quartz is in the 56th and 29th places in the energy-ordered lists of possible structures in the databases of known computational materials science projects materialsproject.org and oqmd.org, respectively. Therefore, the successful test of the potential in this search is not even the correct prediction of stable phase but good correlations between DeePMD and VASP predictions of the energies of the structures. Figure 3 shows such a correlation for structures found using the USPEX code with the HP-DeePMD potential. A pronounced linear correlation is seen between the ab initio and DeePMD energies (the Pearson correlation coefficient is 0.987). The standard deviation of the DeePMD energies from the DFT values is 59 meV/atom. However, the correlation curve in Fig. 3 includes a noticeable number of metastable structures with energies differing from the energies of experimentally observed phases by several electronvolts per atom. Such nonphysical structures are always generated in the evolutionary search. Obviously, their local structure can significantly differ from the local structure of the liquid and their energies can hardly be reproduced well. However, this does not affect the efficiency of searching for stable configurations because the correct description of the energies of phases in a certain quite narrow energy range is important in this case. Such a range in Fig. 3 is marked by a green ellipse covering the enthalpy range of (‒8, –7) eV/atom (cf. the energies of stable structures in Fig. 2). The standard deviation of the DeePMD energies from the DFT values in this range is noticeably smaller and is 29 meV/atom.

Fig. 3.

(Color online) Correlation between the DeePMD and DFT energies of crystalline structures predicted by the USPEX code with the DeePMD potential. The inset shows the E_DFT – E_DeePMD deviation distribution histogram. The green ellipse marks the energy range corresponding to the most stable structures (cf. the energy range of experimentally observed structures in Fig. 2).

The analysis of the found structure shows that Si atoms are in all cases in the tetrahedral and octahedral environments of O atoms at P = 0 and 10 GPa, respectively, which is observed for all known low- and high-pressure phases of SiO₂, respectively. The results also reproduce the existence of tens of structures with close energies (differing by several meV per atom). Among the most stable structures at zero pressure, we found β-cristobalite with space group no. 227. The evolutionary search at 10 GPa indicates that the most stable structure has space group no. 136 and corresponds to stishovite, which is the most stable high-pressure phase of SiO₂. Figure 4 shows the partial radial distribution functions of β-cristobalite and stishovite in comparison with the functions of the corresponding structures from crystallographic databases. Peaks of the calculated distribution function of stishovite are slightly shifted toward larger distances because the experimental sample was under a pressure of about 30 GPa, whereas the calculation was performed for 10 GPa.

Fig. 4.

(Color online) Partial radial distribution functions for crystalline phases of SiO₂ found by the evolutionary search with the USPEX code in comparison with the experimentally observed functions for (a) $\beta$-cristobalite at P = 0 and (b) stishovite at P = 10 GPa; these structures are visualized in the insets.

CONCLUSIONS

To summarize, liquid–crystal structure inheritance in machine-learning potentials for network-forming systems has been studied. It has been shown that MLIPs parameterized only on high-temperature disordered configurations can reproduce the properties of crystalline phases with an ab initio accuracy and, moreover, can predict their structure using evolutionary algorithms. More precisely, the volume dependences of the energies of known crystalline phases of SiO₂ have been calculated with an ab initio accuracy using the DeePMD potential parameterized on configurations of the SiO₂ melt in the temperature range of 3000–4500 K and the pressure range from 1 atm to 70 GPa. In addition, it has been shown that the evolutionary search implemented in the USPEX code allows one to predict the structure of β-cristobalite and stishovite and, even more important, to describe well the energies of all found structures compared to ab initio calculations.

The results obtained demonstrate a close relation between the structures of crystal and liquid for network-forming systems. This relation not only is of fundamental importance but also makes it possible to use MLIPs parameterized on liquids as initial potentials for the construction of MLIPs for crystals. Applying a genetic algorithm to a MLIP parameterized only on liquids, one can obtain a set of crystalline structures that should be subsequently included in the extended training dataset used to additionally train the MLIP. This self-consistent scheme will allow one to develop MLIPs for systems for which a priori information on crystalline structures is either absent or limited. We note that the successful implementation of this idea requires the application of active training algorithms to create a representative set of disordered configurations.

The reported results pose a number of new fundamental questions. How can the accuracy and computational efficiency of the proposed scheme be increased? Is this method applicable to the study of complex multicomponent systems? What are classes of systems where liquid–crystal structure inheritance is fairly pronounced and can be used to construct MLIPs? These questions require a separate study and stimulate the development of new lines of research in computational materials science and condensed matter physics.