Introduction

Hohenberg and Kohn [1] and Kohn and Sham [2], respectively, introduced density functional theory (DFT) in 1964 and the local density approximation (LDA) to it in 1965. “Understanding DFT and completing it in practice” [3] has elaborated on the inception of the theory and its foundational theorems, Reference [3] is openly available on line. The reader is urged to consult this publication for details. In particular, the article reiterated the first corollary to the first DFT theorem: it states that the energy functional (in addition to the external potential) is a unique functional of the ground state charge density, provided the number of particles is kept constant. Given that this energy functional is simply the sum of all the occupied eigenvalues of the Hamiltonian, we [3] introduced the second corollary stating that the spectrum of the Hamiltonian is a unique functional of the ground state charge density, provided the total number of particles is kept constant. The second DFT theorem simply states that the energy functional reaches its minimum (the ground state), if the charge density employed in calculating it is that of the ground state, provided the total number of particles is kept constant. As noted in the abstract, however, the authors of DFT [1, 2] provided no mechanism for finding the referenced ground state charge density. A priori, the three-dimensional ground state charge density is not available for atoms, molecules, clusters, nanostructures, and solids.

The completion of DFT in practice

Kohn and Sham, upon introducing the local density approximation [2], derived the four equations to be solved in order to get the ground state of a material. After selecting an exchange correlation energy and its functional derivative, the exchange correlation potential, the referenced four (4) reduce to two (2): The Kohn–Sham equation and the one giving the charge density in terms of the wave functions of the occupied states. Kohn and Sham [2] correctly stated that these equations are to be solved self-consistently in order to obtain the ground state properties. In the same paragraph, they proceeded to describe what amounts to solving the system of equation with a single basis set: With a judiciously selected basis set, one constructs the charge density using the wave functions of the occupied states; the resulting density and the basis set are employed to generate and to solve the Kohn–Sham equation; with the new wave functions for the occupied states, a new charge density is constructed the cycle continues until self-consistency is reached.

It is crucial to underscore here that the state of the material described by self-consistent results obtained with a single basis set is a stationary state among an infinite number of such states—with no proven relation to the ground state. The realization of the full potential of DFT requires the knowledge of this mathematical fact. Indeed, the Rayleigh theorem for eigenvalues [3,4,5] states the following: Let an eigenvalue equation be solved with N and N + 1 basis functions, respectively, where the N functions are entirely included in the N + 1, then, the ordered eigenvalues, from the lowest (λN1 and λN+11) to the highest (λNN and λN+1N+1) ones, obey the inequality λN+1i ≤ λNi for i ≤ N. Hence, eigenvalues obtained with the larger basis set are lower than or equal to their corresponding values obtained with the smaller basis set. This theorem points to an energy minimization process far beyond the one obtained with self-consistency with a single basis set; this generalized minimization of the energy requires successive, self-consistent calculations with progressively augmented basis sets.

In 1998, we realized that the Rayleigh theorem for eigenvalues provides a rigorous process for finding the ground state of BaTiO3 [6]. Starting with a relatively small basis set, which was large enough to accommodate all the electrons in the system under study, we performed successive, self-consistent calculations. Except for the first one, the basis set for a calculation was that of the one preceding it as augmented by one orbital. Of course, depending on the s, p, d, or f nature of the added orbital, the dimension of the Hamiltonian increased by 2, 6, 10, or 14, taking the spin into account. We compared graphically the occupied energies from consecutive calculations after setting the Fermi level to zero. As dictated by the Rayleigh theorem, the occupied energies were lowered or unchanged as the size of the basis set increased. This process continued until two consecutive calculations produced the same occupied energies; these occupied energies were taken to be those of the ground state of the material; we took the first of these last two calculations to be the one providing the DFT description of the material. We named this approach the Bagayoko, Zhao, and Williams (BZW) [7] method, where orbitals were added in the order of the excited energies they represent. As we employed the method to study several semiconductors, we found, for GaP [8], two consecutive calculations that led to the same, occupied energies while the following calculation lowers some of these occupied energies. Hence, our criterion for finding the ground state was changed to having three (3) consecutive calculations produce the same occupied energies. Calculated band gaps were equal to or generally within 0.2 eV from the corresponding, measured ones. Extensive calculations by Ekuma [9] and Franklin [10] led to a change in the augmentation of a basis set. Instead of adding orbitals in the order of the excited states they represent, as done in the BZW method, we changed to the following: For a given principal quantum number n, the p, d, and f orbitals, if applicable, are added before the spherically symmetric s orbital for that principal quantum number; an orbital is applicable if it is occupied in the neutral system under study. This change led to the Bagayoko, Zhao, and Williams method as enhanced by Ekuma and Franklin (BZW-EF). Interestingly enough, the EF enhancement drastically improved our description of materials in the vicinity of the Fermi energy [EF]. In particular, the BZW-EF method generally leads to the measured band gaps of materials. The counterintuitive enhancement recognizes the primacy of polarization orbitals over spherically symmetric s ones for description of valence states! Unlike single basis set calculations, our BZW and particularly the BZW-EF methods correctly described and predicted properties of semiconductors [3]. The annotated figure below shows the step by step implementation of the BZW-EF method for electronic structures calculations atoms, molecules, clusters, nanostructures, semiconductors, and insulators (Fig. 1).

Fig. 1
figure 1

Diagram of the implementation of BZW-EF method in electronic structure calculations. For each “column” the blue arrows show the flow of self-consistency iterations, with stationary eigenvalues at the bottom of the Column. Calculations follow each other, from the left to the right, with the stationary energies, at the bottom, compared for every pair of calculations

Full size image

Let us recall the statement of the second DFT theorem [1]: “It is well known that for a system of N particles, the energy functional of Ψ′, Ev[Ψ′] = (Ψ′,VΨ′) + (Ψ′, (T + U)Ψ′), has a minimum at the ‘correct’ ground state Ψ, relative to arbitrary variations of Ψ′ in which the total number of particles is kept constant.” While we do not know, a priori, the ground state charge density of a material, we can, with the BZW-EF method described above, search for and reach the absolute minima of the occupied energies, i.e., the ground state. As explained in several of our publications, when three (3) consecutive calculations produce the same occupied energies, it is the first of these calculations that provides the DFT description of the material. The basis set for this calculation is the optimal basis set. This optimal basis set, upon reaching self-consistency, produces the ground state charge density of the material. While the two other calculations yield the same occupied energies, they lead to some unoccupied energies that are lower than their corresponding values obtained with the optimal basis set. The second corollary to the first DFT theorem states that the spectrum of the Hamiltonian is a unique functional of the ground state charge density. Hence, these unoccupied energies lowered from their values obtained with the optimal basis set no longer belong to the spectrum of the Hamiltonian. We have therefore provided a method for rigorously searching for and verifiably reaching the ground state of a material, despite the fact that we did not know, a priori, the ground state charge density. At the end of the correct application of our method, we obtain the ground state charge density by performing self-consistent calculations with the optimal basis set. In so doing, our method completes DFT in practice.

Current, mainstream applications of DFT and related issues

From 1964 to present, most mainstream calculations of electronic and related properties of materials have employed a single basis set to perform self-consistency iterations to produce what they consider to be the ground states of the materials under study. As shown above, a stationary state obtained by such calculations cannot be taken to be the ground state of the material. The wide spread underestimation of the band gaps of semiconductors and of insulators by mainstream calculations is presently explained in the literature using one of the following: (a) the correct functional is not known; (b) non-local effects may not be taken into account, particularly in calculations using local density approximation potential; (c) a derivative discontinuity of the exchange–correlation energy is almost universally accepted as an explanation of the serious underestimation of the band gaps of real semiconductors by DFT calculations; and (d) self-interaction effects. Our accurate descriptions and predictions of properties of semiconductors [3], without invoking any of these explanations from (a) to (d), strongly suggest that the discrepancies between results from mainstream DFT calculations and corresponding experimental findings are simply the differences between arbitrarily selected stationary states and the true ground states of the materials under study.

While the totally correct functional may not be known, the many accurate descriptions and predictions [3] of properties of materials suggest that the functional is not the problem. Incidentally we studied several materials using the Ceperley and Alder [11] local density approximation (LDA) potential as parameterized by Vosko et al. [12]. Our studies of some of these materials with a generalized gradient approximation (GGA) potential led to no major difference from LDA findings. This result suggests that while non-locality cannot be totally dismissed, it seems not to contribute much to the explanation of failures erroneously attributed to DFT. Kohn and Sham obtained their equation with the condition that the total number of particles is kept constant. The derivative discontinuity was derived for semiconductors [13] and for insulators [14] by using systems whose total number of particles is not constant. So, while these discontinuities may apply to studies of some excited states of materials, they have no bearing on the ground state solution of the Kohn–Sham equation whose validity requires a constant number of particles. Further, Sham and Schlüter [14] stated that they do not know whether or not the derivative discontinuity is non-zero in real insulators. They also recalled the fact that the derivation of the Kohn–Sham equation requires the total number of particles to be constant. As for the derivative discontinuity in semiconductors, Perdew and Levy [13] claimed that the chances for it to be zero are extremely small; based on this claim, they asserted that it is non-zero in real semiconductors. Nowhere is it proven in the literature that the derivative discontinuity of the exchange correlation energy is non-zero in real materials. We refer the reader to Reference [3] for discussion of self-interaction which well-define only for a single particle. Applications of the self-interaction correction almost uniformly overestimate the band gaps of materials.

We have successfully described and predicted properties of numerous semiconductors without invoking the derivative discontinuity or self-interaction correction. The content of the table provided farther below amply illustrates the point.

Another dominant feature of current mainstream calculations consists of the extensive utilization of program packages for pseudopotential and plane wave calculations. The extensive saving in computing time and related efforts partly explains this trend. However, the above description of the BZW-EF method implies the use of full potentials and of exponential or Gaussian functions in the basis sets. For DFT calculations, a rigorous approach points to the use of the total charge density as a fundamental variable. In any event, we do not see a possibility for reproducing the richness of the basis functions, in terms of their radial and angular components, with plane waves.

In a 2017 Science article, Medvedev et al. [15] examined the trend in the search for DFT functionals. The review of 128 functionals revealed that while the ones before the year 2000 were better approximation to the exact functional, after 2000, “this trend was reversed by unconstrained functionals sacrificing physical rigor for the flexibility of empirical fitting.” [15]. The construction of ad hoc potentials, in particular, seems to be a growing trend. We submit here that the correct application of DFT, as discussed above, obviates the use of ad hoc potentials that do not possess any predictive capabilities—given that computational results depend on the parameters utilized in their construction. Bagayoko [3] underscored the fact that ad hoc potentials, despites their wide-spread use in the literature, are not entirely DFT potentials—given that they are not functional derivatives of an ab-initio exchange correlation functional.

Realizing the potential of DFT

Our work, using the correctly understood and completed DFT, has demonstrated that DFT eigenvalues do have physical meanings, contrary to some claims in the literature. As noted in several of our articles cited in the table below, our computational findings possess the full, physical content of DFT—given that we reached the ground state in a verifiable manner. Provided the stupendous financial and human resources presently invested in single basis set DFT calculations are redirected to true DFT studies, with successive, self-consistent calculations, nothing less than a revolution in theoretical materials research will ensue.

We should underscore that our emphasis on obtaining the measured band gaps of materials stems from the importance of these gaps in calculated other properties of materials, from optical transition energies, to dielectric functions, density of states, and electron effectives masses. The reader is urged to consult Bagayoko [3] for details on the outright predictions we have made and that have been verified by subsequent experimental articles. These predictions were for the following materials: cubic silicon nitride, wurtzite and cubic indium nitride, and rutile TiO2. Interestingly, for the latter, our DFT-BZW-EF calculated indirect band gap was qualitatively in disaccord with both previous calculated and measured direct band gaps for TiO2. Details of the experimental confirmation of our prediction are in Reference [3].

A recent article by Diakité et al. [16] shows the potential of DFT when it is correctly applied. Indeed, the BZW calculations of properties of zinc blend InAs, using an ab-initio LDA potential, not only obtained the exact, measured experimental band gap, but also the locations of the peaks in the density of states for the valence band, the measured effective mass of the electron and of both heavy and light holes, and the experimental bulk modulus. While one may think that a single number (the band gap) may be obtained by luck, as some referees claimed in the past, the reproductions of the peaks in the valence DOS denotes not only the correct description of each valence band, but also of the separation between them. The role of the effective masses in charge mobility studies underscores their importance. The agreement between theory and experiment, for each of the effective masses measured, points to a highly accurate description of the curvatures of the band in the vicinity of the minimum of the conduction band and of the maxima of the valence bands. Similarly, accurate descriptions of atoms, molecules, clusters, nanostructures, semiconductors, and insulators are now within the reach of DFT, provided the calculations search for and actually reach the ground states of the materials, as done with the BZW-EF method. The generalized minimization of the energy functional is graphically illustrated in the above article. A larger, graphical illustration of the search for the ground state is available from Malozovsky et al. [17], for hexagonal boron nitride (h-BN): the graphs of the bands from pairs of consecutive calculations, from 1 and 2, 2 and 3, to 5 and 6, show the decrease (minimization) of the occupied energies as the basis set is augmented and the attainment of the ground state with Calculation 4. An upcoming article by Dr. Dioum et al. [18] also illustrates this search with figures that plot the bands from a few consecutive calculations of the BZW-EF method.

In 1973, Rajagopal and Callaway [19] introduced the relativistic generalization of DFT. This work widened the range of applications of DFT, given that relativistic effects are unavoidable in the theoretical study of any material containing a heavy element. The prototypical heavy elements are those where an occupied f orbital. Of course, some relatively heaving elements, without an occupied f orbital, may require a relativistic treatment. We presented an understanding and a completion of this relativistic generalization of DFT in 2016 [20] and its French version in 2022 [21]. This understanding requires calculations to search for and to reach, verifiably, the ground state of the material. The relativistic application of the BZW-EF method straightforwardly leads to the correct results. Of course, in the relativistic application of the LCAO method, regular basis functions are replaced by basis spinors and the charge density is replaced by the current density. One of the four (4) components of this current density is the actual electronic charge density. A needed primer for understanding the content of Ref. [21] is Ref. [22].

We end our illustration of the highly accurate nature of correctly performed DFT calculations with the content of Table 1. For the material in Column 1, the table shows in Column 3 the number of LDA and GGA calculations prior to our work, and the ranges of values for the calculated gaps in Column 4. Column 5 contains the experimental findings, and Column 6 shows the DFT band gap obtained with the BZW-EF method. The reader is urged to consult Ref. [3] for a similar table for some nine (9) materials. While all 99 ab-initio DFT calculations underestimated the measured band gaps, the BZW or BZW-EF results in Column 6 were in agreement with experiment. Likewise, the 233 previous, mainstream ab-initio DFT calculations in Table 1 produced underestimates of the corresponding band gaps. Before our work, these 99 and 233 failures were unwittingly ascribed to DFT.

Table 1 Band gaps from ab-initio, self-consistent DFT calculations
Full size table

Realizing the potential of the materials genome initiative (MGI)

According to its Web site [36], “The Materials Genome Initiative (MGI) is a U.S. Federal government multi-stakeholder initiative to develop an infrastructure to accelerate and sustain domestic materials discovery and deployment in the United States.” MGI started during the US Presidency of his Excellency Barak H. Obama. The 2021 Strategic Plan of MGI is freely available on the Internet [37]. The reader is requested to consult these sources for ample details on MGI. Here, we deal with a crucial component of MGI, i.e., highly accurate descriptions and predictions of properties of materials that can inform and guide the design and fabrication of devices. The correct execution of DFT calculations, explicitly including a verifiable attainment of the ground state, can lead to quantum leaps in materials science, innovation, and engineering. A few of the specific areas of materials research and development that can immensely benefit from accurate, theoretical descriptions and predictions of properties of materials are listed below. Results to spelled out tomorrow.

  • Molecular & nanostructure engineering, including the prediction or design of novel molecules, small or large, inorganic or organic, can immensely succeed with accurate, theoretical description and prediction of properties of materials.

  • The same is true for semiconductor engineering, including the prediction or design of materials with desired properties for applications.

  • Materials simulations can be dramatically enhanced by using accurate inter-atomic potential or empirical pseudo potential parameters derived from accurate DFT results.

  • Our understanding of superconductivity, including the high temperature ones, can tremendously benefit from accurate knowledge of the electronic and related properties of affected materials. In particular, the understood and completed relativistic generalization of DFT [20, 21] lends itself to the correct study of materials containing very heaving elements.

  • The same is true for the search for metastable nuclear isotopes that may be used to produce gamma ray amplification by stimulated emission of radiation (graser). Indeed, the nuclear shell model has been studied for years, using a nuclear version of DFT known as NDFT. The correct application of the understood and completed DFT is expected to open new horizons in nuclear studies.

The above bullet points are far from being exhaustive. They merely illustrate some specific areas were the correct application of DFT is bound to result in significant progress.

Conclusion

We have presented highly disruptive and transformative findings relative to DFT and its application in the literature. Specifically, with the Rayleigh theorem for eigenvalues, we have described a method of performing electronic structure calculations with ab-initio DFT potentials, in such a way that the ground state of the material and its ground state charge density are both verifiably obtained. In doing so, we have completed DFT in practice, given that the theory required a minimization of the energy functional even though the articles that introduced it did not provide a mechanism for a generalized minimization of the energy to lead to the ground state energy and charge density. Our calculations produce results that possess the full, physical content of DFT and generally agree with corresponding experimental findings. We have illustrated this fact with a table whose content shows 233 previously calculated ab-initio DFT bang gaps that uniformly underestimate the corresponding experimental values; in contradistinction, for all the materials in the table, our calculated band gaps agree with corresponding experimental ones. Unquestionably, we have paved the way for the realization of the full potential of DFT and of the MGI.