Abstract
Ecology lacks a holistic approach that can model phenomena across temporal and spatial scales, largely because of the challenges in modelling systems with a large number of interacting constituents. This hampers our understanding of complex ecosystems and the impact that human interventions (e.g., deforestation, wildlife harvesting and climate change) have on them. Here we use density functional theory, a computational method for many-body problems in physics, to develop a computational framework for ecosystem modelling. Our methods accurately fit experimental and synthetic data of interacting multi-species communities across spatial scales and can project to unseen data. As the key concept we establish and validate a cost function that encodes the trade-offs between the various ecosystem components. We show how this single general modelling framework delivers predictions on par with established, but specialised, approaches for systems from predatory microbes to territorial flies to tropical tree communities. Our density functional framework thus provides a promising avenue for advancing our understanding of ecological systems.
Similar content being viewed by others
Introduction
Ecology has detailed mechanistic theories that capture the behaviour of a small number of interacting species at fine scales1,2,3. It also has broad ‘unified’ theories that ignore the details of individual species and their interactions, but can reproduce large-scale patterns, such as the relationship between number of species and area4,5,6,7. Nevertheless, theories bridging different scales are lacking. Ideally, models would be parameterisable at one scale of ecological organisation (e.g., the individual organism, or the species) and make accurate predictions at other scales (e.g., the local community or landscape). Meeting this challenge would allow knowledge from field data and limited experimental setups to be leveraged to predict the behaviour of more-complex, realistic ecosystems4,8,9,10,11,12. Such an enterprise calls for a diverse range of quantitative approaches.
Here, we draw on density functional theory (DFT)13,14,15,16, which has been successfully applied to a staggering number of many-body problems in physics, involving scales from atoms to molecules to planetary cores17,18,19,20. DFT is based on the mathematical proof that all properties of all quantum systems can, in principle, be obtained from the density distributions of its constituents. At the core of DFT lies an energy (cost) functional in terms of these density distributions. Of fundamental interest in physics are the density distributions with lowest energy (ground states and equilibria), obtained from minimising the energy over all permissible spatially varying densities. DFT predicts key properties of physical systems while automatically accounting for trade-offs between system components and constraints with efficiency and accuracy. The conceptual framework of DFT is the mathematically rigorous generalisation of the Thomas–Fermi (TF) model21,22 from the advent of quantum mechanics. The development and fruitful application of DFT has sky-rocketed since then, with nowadays tens of thousands of yearly publications23. In light of the success of DFT in simulating many-body systems within a general framework, it is worth considering whether complex systems in the life sciences can also be modelled with DFT techniques.
While there are numerous DFT applications in molecular biology24, only one DFT-inspired study has so far emerged in macrobiology: The authors of Ref. 25 inferred spatial density distributions of Drosophila melanogaster (fruit flies) in an experimental arena by pairing the grand-canonical ensemble of statistical physics with an energy functional. This statistical method accurately predicts the distributions of fruit flies in new environments. Here we develop a more general approach, which we term density functional theory for ecology (DFTe). The core of DFTe is an energy functional for ecosystems that merges ecological principles with insights from DFT. Though DFTe draws inspiration in particular from analogies with the TF model of multi-component quantum gases, it does not of course introduce any quantum effects into ecology itself. Our DFTe bridges ecological scales by allowing sub-components of an ecological model to be parameterised separately and then combined to predict the behaviour of more-complex systems. We first describe the general approach and then present applications to a variety of ecological systems with different properties.
Results
The cost function as the key concept of DFTe
Many key questions in ecology, such as abundance redistribution in response to environmental change, ask for constrained equilibria—a goal shared by DFT in physics. At the core of DFT lies a cost function, the density functional of the energy E. Once the building blocks of E are specified, its minimisation delivers the position-(r-)dependent equilibrium (viz., ground-state) densities n(r) = {n1(r), n2(r), …, nS(r)} for the abundances N = {N1, N2, …, NS} of any interacting many-body system of S species, such as a gas of several atomic species.
We have transferred the DFT narrative and methodology to ecology by building a cost function, the energy functional
whose minimisation balances species-specific costs due to dispersal, environment, and interactions, including resource consumption (the three terms on the right-hand side of Eq. (1); see Methods). When using square brackets in f[n], we declare that a function f is a functional of the function n. In referring to E as an energy for a given realisation (n, N) of an ecological system, we quantify a relation to other configurations \(({{{{{{{{\bf{n}}}}}}}}}^{{\prime} },\, {{{{{{{{\bf{N}}}}}}}}}^{{\prime} })\). The insight that a system aims at occupying the state of lowest (ground-state) energy among all possible system configurations is one of the fundamental principles of physics. Here, we construct the DFTe energy E(N) such that its global minimum realises the ecosystem equilibrium with abundances \({{{{{{{\bf{N}}}}}}}}=\hat{{{{{{{{\bf{N}}}}}}}}}\). While we focussed on establishing the equilibrium properties of DFTe as a necessary step towards the modelling of dynamics, we note that E represents an S-dimensional (hyper-)surface whose shape can determine (i) steady-state dynamics (see Methods)—in analogy to, for example, Kepler orbits in a gravitational field—and (ii) nonequilibrium dynamics, where also drivers such as stochastic environments can prevent equilibration.
The energy of a system is a measure of its ability to invoke change in another system. Twice the energy, relative to the other system, doubles this ability—energy is additive. But this does not imply that the energy of a system decomposes into a sum as in Eq. (1). We base the additive decomposition of E on our understanding of how constituents of physical systems come together to assemble the energy and determine the functional form of E by introducing and building on intuitive concepts in analogy to the energy components of physical systems. For example, for the dispersal energy Edis we adopt the TF kinetic energy expression for species s:
where τs is a species-specific dispersal pressure constant, and the integral is over all spatial positions r in a focal area A. Equation (2) encodes an intra-specific pressure of species s and implements one form of dispersal where, for example, individuals repel conspecifics from territory; note that positive (negative) energy density encodes repulsion (attraction). The functional form Edis in Eq. (2) implies a conspecific negative density dependence, relative to other energy components whose integrands scale less than quadratically with density. We emphasise that we neither aim at describing all forms of dispersal via Eq. (2) nor necessarily need to include Edis even if a specific ecosystem exhibits dispersal, which can potentially be modelled with other energy components. The functional in Eq. (2) is species-specific due to τs, and we assume that the quadratic density dependence is the result of an individual-level mechanism that is not necessarily known to us. The second term in Eq. (1) is the environmental energy, which for each species s we model via analogy to the external energy in physics:
where \({V}_{s}^{{{{{{{{\rm{env}}}}}}}}}({{{{{{{\bf{r}}}}}}}})\) governs the effect of the environment on the energy for species s. Equation (3) parallels the parameterisation in ref. 25 and models, for example, habitat preference and external influences, such as spatially varying climate and local deforestation stress. We may illustrate the effect of Eq. (3) for an amphibian species s by assigning values \({V}_{s}^{{{{{{{{\rm{env}}}}}}}}} < \,0\) in a lake, while \({V}_{s}^{{{{{{{{\rm{env}}}}}}}}} > \,0\) in the surrounding desert penalises high densities ns as we minimise the total energy that includes Eenv[ns]. Summing Eqs. (2) and (3) over s, we obtain the respective total energy components displayed in Eq. (1).
We emphasise that we do not pursue the hopeless goal of establishing DFTe based on actual chemical quantities and mechanisms. Rather, we use our DFTe and its energy functional E, derived by analogy with DFT and informed by ecological principles, as a tool for parameterising sub-components of ecological systems and then combining the components to generate higher-order predictions. Furthermore, in ecology our data of interest are species abundances, resource levels and related variables, rather than energy. To the extent that the hypothesised mathematical structure of E is correct in any given instance, the predictions of these ecological variables will be accurate.
Deriving species distributions from the cost function
Among the various implementations of DFT, we find that density-potential functional theory (DPFT)26,27,28,29,30,31 is uniquely qualified for achieving our objectives. When combined with our energy functional for dispersal (Eq. (2)), DPFT delivers the equilibrium density
of species s, where the operator [ ]+ ensures non-negative densities by replacing negative arguments with zero, μs is a species-specific Lagrange multiplier that enforces the density constraint ∫A(dr) n(r) = N in a focal area A (see Methods), and
is the potential energy as a function of all species (density vector n) at position r, which merges the environment \({V}_{s}^{{{{{{{{\rm{env}}}}}}}}}\) (as perceived by species s; see Eq. (3)) with the interaction potential \({V}_{s}^{{{{{{{{\rm{int}}}}}}}}}[{{{{{{{\bf{n}}}}}}}}]\) (the functional derivative of Eint[n]), coupling s to the other species. The simple relation between density ns and effective potential Vs in Eq. (4) arises directly from our choice of functional form for the dispersal energy (Eq. (2)). This functional form suffices for the proof-of-principle examples studied here, although other functional forms are also plausible (e.g., Eq. (6) in the Methods). Henceforth, we omit arguments of functions for brevity wherever the command of clarity permits.
The coupled Eqs. (4) and (5) are the essence of DPFT in general and of DFTe in particular. According to Eq. (4), the members of species s seek to reside where the cost Vs due to environment and interactions is small: a reduction of Vs increases the density, that is, we interpret small values of Vs as favourable to species s. The spatially varying values of Vs themselves are determined through the self-consistent solution of Eqs. (4) and (5). This automatically produces a trade-off between intra-specific dispersal pressure, which rises with larger density (see Methods), and a confining potential Vs that can be interpreted as an effective environment, where interactions modify the bare environment \({V}_{s}^{{{{{{{{\rm{env}}}}}}}}}\). Although this is only one of potentially many ways a computational framework for DFTe could be constructed, it provides substantial flexibility in terms of the forms of the species-interaction functionals Eint[n] (see Methods), and we will see that it describes ecological data very well.
Roadmap for establishing DFTe
In the following, we enlist six biological systems to build and verify the computational DFTe framework. We thereby (i) demonstrate the alignment of DFTe results with both empirical data and existing modelling approaches, (ii) discuss advantages as well as limitations of DFTe relative to existing modelling approaches in ecology, (iii) illuminate how to deploy the building blocks of DFTe in practice, and (iv) show how to extract problem-adjusted DFTe tools from the general DFTe energy for studying modifications and extensions of ecosystems to which our framework has been fit. Figure 1 summarises our agenda by listing the ingredients and main insights of the DFTe predictions for our six systems.
We find that the performance of DFTe is similar to that of existing system-targeted modelling approaches, including density-functional fluctuation theory, the maximum entropy theory of ecology, R*-theory, a climate envelope model, a Lotka–Volterra model, and statistical models. This range of alternative approaches required for modelling our case studies highlights generality as a major benefit of DFTe. We note that specific models constructed within the DFTe framework do not necessarily encode fine-scale causal relations, but may provide effective descriptions, where ecological phenomena cannot always be linked unambiguously to specific DFTe components. The corresponding links declared in Supplementary Table 1 are therefore prototypical rather than rigorous. For example, in our first case study, we may attribute the territoriality of fruit flies either to the local dispersal pressure or to the nonlocal repulsion or some combination of the two (Fig. 1b).
Fruit flies in heated chambers
To illustrate the fundamental computational procedure of DFTe, i.e., minimisation of the energy functional E for fixed abundances N, we used the fruit fly system of Méndez-Valderrama et al.25. We found that our DFTe is able to predict distributions of flies in novel situations with similar accuracy to state-of-the-art methods (Fig. 2). Briefly, the DFTe workflow (Fig. 1b) involves fitting a two-parameter energy functional E (Eq. (24) in Methods) to two experimental data sets on fly spatial distributions, one representing responses to high density (crowding) in a heterogeneously heated environment (see Supplementary Fig. 2) and the other to low density in heterogeneous geometry, and then predicting the response to all factors combined (Fig. 2a). DFTe yields similar accuracies to density-functional fluctuation theory (DFFT) (Fig. 2b), which is developed in ref. 25 as a statistical approach based on the same experimental data but, unlike DFTe, cannot identify mechanisms of the kind implied by Eq. (24). As expected, we found that DFTe performs more poorly if parameterised only on one of the data sets (Fig. 2b), demonstrating that meaningful information on the separate sub-components of the model is extracted from each of the data sets in the full DFTe fit. Although the full DFTe fit is superior to these, any DFTe approach is better than the naive approach of simply scaling up the low-density observations to high density (DFFT* in Fig. 2b), similar to what would be obtained from an approach based on the maximum entropy theory of ecology if solely constrained to the low-density data: Without system-specific information beyond the low-density distribution, a maximum entropy evaluation has to assume independent individuals. This case study illustrates the interplay between dispersal, environment, and interactions in DFTe.
Resource competition among four algae
Next, we demonstrated how DFTe can incorporate the constraints of resource availability on species’ coexistence. In seminal experiments testing predictions of R*-theory, D. Tilman showed that the outcomes of pairwise competition between four algal species can be predicted by fitting a dynamic model to data from single-species experiments1. The fitted models can also be used to predict outcomes of multi-species competition, although these have not yet been tested experimentally. We built an alternative tool for predicting the outcomes of multi-species competition by fitting DFTe to predicted pairwise outcomes of algal competition under R*-theory (not Tilman’s experimental pairwise outcomes, which in some cases represented transients). We found extremely close correspondence between the predictions of R*-theory and DFTe (Fig. 3; see also Supplementary Table 2). Our method assumed that the algal species’ competitive abilities depend on the inverses of the minimum resource requirements (R*-values) and the nutrient requirements per cell (see Methods). Parameterising the DFTe energy components (Eq. (28) in Methods) with species-specific resource requirements and fitness proxies based on Tilman’s monoculture data1, then left just two free parameters to be fit to all pairwise outcomes of resource competition under R*-theory.
This application illustrates the ability of DFTe, despite its very general nature, to scale up from simpler to more-complex systems with performance similar to more-specialised theories, in this case R*-theory but also a recently proposed community assembly rule that states that the species able to coexist in multi-species competition will be those that can all coexist pairwise32. One disadvantage of DFTe relative to R*-theory, however, is that it was parameterised at the two-species level rather than the single-species level, thus avoiding the difficult path of highly parameterised descriptions for multi-species systems at fine scales33. We also disregarded the parameters of resource flow rates F and maximal growth rates in R*-theory: While equilibria for different F are readily available through R*-theory, they cannot be obtained by the DFTe energy fitted to data for fixed F. This application highlights what in general may be a fruitful path for modelling complex ecological systems: first achieve a detailed understanding in systems with few interacting components (e.g., via R*-theory); then scale up to more-complex systems using DFTe (Fig. 3).
Competition between three grasses over salinity gradients
We next explored how DFTe can make predictions from sparse information, using an example of three grass species in a saline environment34. We fitted the DFTe energy to experimental data on monoculture and mixture densities of the grass species in environments of uniform salinity, and used the fitted model to predict spatially varying densities in a hypothetical environment with heterogeneous salinity (Fig. 4a). We achieved this by making the species’ unknown resource requirements parameters that can be fitted to the known data (see Methods). The DFTe predictions were substantially different from a generic envelope model for species distribution modelling (Fig. 4d,e and Supplementary Fig. 7), which simply maps the observed abundances in mixture in different homogeneous environments on to the heterogeneous environment and, unlike our DFTe model, fails to account for changes in species interactions in the heterogeneous environment. The envelope model’s predictions are unlikely to be accurate, because the mixture experiments of ref. 34 already demonstrate that the species do interact in homogeneous environments (mixture abundances are not just rescaled monoculture abundances), and it is likely that these interactions would change in heterogeneous environments. We emphasise, however, that no final verdict on the accuracy of either model, DFTe or the envelope model, can be reached without additional data that test the predictions.
All our examples thus far have focused on equilibria, but time-dependent dynamics are also of great interest to ecologists. We used our case study of competing grasses to illustrate the potential for DFTe to approximate transient dynamics by assuming that the trajectory between the initial and equilibrium states is linear in the total abundance of each species. Using this assumption, we ran a simulation in which two grasses invade the equilibrium monoculture distribution of the third grass (Fig. 4). One phenomenon arising from this simulation is zonation, i.e., the occurrence of sharp boundaries between species distributions despite a continuous environment35. We identify zonation as a transient, interaction-induced phenomenon, which in a real system may even persist as an itinerant steady state if environmental disturbances prohibit complete equilibration. This result demonstrates DFTe’s ability to produce non-trivial predictions that can be potentially tested with experimental data.
Dynamics of microbial predation
Our first three examples showed how DFTe can predict equilibria and non-trivial transient dynamics; a fourth example demonstrates that DFTe can also model non-equilibrium steady states, in this case occurring in predator–prey dynamics. Our case study here is of Didinium nasutum feeding on Paramecium aurelia, where experiments have indicated that the long-term steady state can be cycles rather than an equilibrium36,37 (Fig. 5). We found that the observed predator–prey cycles are accurately captured by the DFTe energy functional (Eq. (30) in Methods). This shows that the DFTe hypersurface \({{{{{{{\mathcal{H}}}}}}}}({{{{{{{\bf{N}}}}}}}})\), which is the space of DFTe equilibria spanned by all permissible N (see Eq. (14) in Methods) and is specified through system-specific instances of Eq. (1), has predictive power also away from the equilibrium abundances \(\hat{{{{{{{{\bf{N}}}}}}}}}\), although \(\hat{{{{{{{{\bf{N}}}}}}}}}\) may a priori follow from a myriad of energy functionals E[n, μ](N) that share the same global minimum. The examples of Figs. 4 and 5 are exploratory studies that demonstrate that the DFTe hypersurface can be an appropriate platform for dynamics and that DFTe can potentially be augmented with actual temporal dynamics, thereby opening the door to modelling time-dependent phenomena in ecology with DFTe (see Supplementary Notes).
A complex large-scale synthetic community
Next, we explored the full potential of DFTe with the aid of a synthetic community of seven interacting species (Fungus, Deer, Pig, Snail, Tree, Grass, Cat; Fig. 6) subjected to heterogeneous environments and resource distributions intended to represent regional-scale distributions. We selected the parameters of this food web solely for pedagogical reasons, namely to reveal the functioning of all ingredients of the DFTe energy in a controlled way. Using DFTe, we quantified the impact of removing the apex predator and thereby highlighted the capacity of DFTe for predicting the effects of perturbations on complex multi-species communities in complex spatially structured environments38. Despite the complexity of this multi-trophic system, we retain ample intuition of the causal relations between the model input and output by successively building the complete ecosystem. This understanding extends to effects that may appear counterintuitive when examining only the interactions in Fig. 6a (for example, the small decrease of the Pig abundance upon Cat extermination; see Methods). In a practical setting, this would allow us to promote the apex predator through informed and tailored actions, here for example by reducing the Tree’s deforestation stress (see Supplementary Fig. 11). This demonstrates the potential of DFTe for informing management strategies for biodiversity, for which purpose food web models are currently commonly consulted. Our modular and spatially explicit DFTe framework overcomes major challenges of traditional food web analyses, which often lack (i) the crucial incorporation of relevant processes beyond inter-species interactions and (ii) spatial heterogeneity39, with some exceptions that have been emerging recently (for example, ref. 40) or that are designed for specific ecosystems (for example, ref. 41).
Twenty tree species in a tropical forest
Finally, we explored the scalability of DFTe towards multi-species systems by modelling the twenty most abundant tree species found in the tropical forest plot on Barro Colorado Island (BCI), Panama42. The mechanisms underlying the diversity of tropical forests in general, and the BCI plot in particular, are still not yet fully understood, but have broad implications for ecological theory and policies aimed at maintaining biodiversity43,44.
Our BCI case study shows how DFTe can deal with an extreme case of missing information, since we use no data beyond the observed data on abundances, soil, and altitude (see Methods). This is not a situation most suited for DFTe models, which are preferably built by determining some parameters from simpler versions of the ecosystem. Here, for example, if the resource requirements were known from monoculture experiments, we could eliminate 220 out of up to 341 fit parameters. High variability arises in the fitted parameters across simulation runs because the large number of energy components of E permits an increase in some components and a decrease in others without significant impact on the density distributions. We can nonetheless draw some general conclusions: the DFTe simulations with and without bipartite interactions yield spatial basal area distributions for each species of similar quality (Fig. 7a–c) despite disparate species-specific fit parameters (Fig. 7d). These results support the hypothesis that inter-species interactions in many-species communities are weak relative to other mechanisms45. In the DFTe fit to BCI, the total interaction energy is relatively small and the extracted interaction kernels (Fig. 7e) are distributed across positive and negative values for all bipartite interactions. Similar conclusions have been reached in previous studies using a specialised multi-scale analysis44, a maximum entropy approach46, a stochastic birth–death model46, and spatial point pattern analysis47.
Discussion
Ecology has a long tradition of modelling simple systems with mechanistic equations, for example, of the Lotka–Volterra type. In parallel, large and complex ecological systems have been tackled with statistical approaches. Here we have presented a single framework that facilitates bridging simple and complex models across scales—a goal that has been recognised as both elusive and important for addressing fundamental ecological questions. Inspired by techniques from density functional theory (DFT) in physics, we have created a density functional theory for ecology (DFTe) and demonstrated its alignment with experimental and synthetic data across various ecological systems. No other single theoretical ecological framework has been shown to model such a broad range of systems. We have not yet found a case study of which DFTe does not provide an excellent description, although we encourage further work exploring more case studies.
The key advantage of DFTe comes from its modularity: the energy functional comprises multiple components, each of which has an intuitive ecological interpretation and each of whose parameters can be inferred from simpler subsystems. Another advantage is that the complexity of the DFTe energy functional, which mirrors that of the target ecosystem, is of little technical concern for retrieving the density distributions and the trade-offs between the involved processes. This resolves the ubiquitous problem of finding the proper trade-offs among given mechanisms and constraints for many species that interact within a complex environment. The ecological constituents and mechanisms are independently parameterised components of, and coupled through, the DFTe functional, whose composite structure permits hassle-free revision and introduction of causal mechanisms that can correct disparities between predictions and data. For example, higher-order interactions48,49 can simply be added to Eq. (9) in the Methods. Densities and abundances may refer to individuals or aggregated observables like biomass or land cover. Conversely, we may fork one species into many, for example, to account for heterogeneity among individuals9,50.
In the degree of mechanism incorporated, DFTe is intermediate among past approaches to theoretical ecology. It is more mechanistic than statistical or correlative approaches such as MaxEnt. It explicitly encodes processes like predation, competition, and consumption, and it uses individual-level parameters such as individual nutrient requirements and can produce predictions for systems at fine scales. (By analogy, in physics Newton’s law of gravity, a prototypical example of a mechanistic model, aggregations of elementary particles, that is, density distributions n(r), yield the external/environmental energy ∫(dr) Vgravity(r) n(r), the exact analogue of Eq. (3).) But DFTe is more phenomenological than, for example, R*-theory. Indeed, in our application to algal competition, DFTe takes outputs from R*-theory as input for addressing more complex situations (Fig. 3). In common with all other existing ecological theories, DFTe is not fully mechanistic in the sense of a yet-to-be-discovered fundamental (first-principles) framework at the level of individuals from which DFTe might emerge, since its components are not derived from individual-level interactions. This intermediate representation of mechanism in DFTe is a strength of our approach, because while existing mechanistic approaches become increasingly intractable with rising system complexity3 and while statistical approaches such as MaxEnt—by design—make no attempt at interpreting the data, DFTe can be scaled to complex systems without sacrificing a degree of mechanistic understanding.
One important consideration when applying DFTe is how to deal with data scarcity. Here again DFTe’s performance is intermediate between past mechanistic and statistical approaches—the latter of which perform particularly poorly in data-poor situations. Important practical challenges towards a universally useful DFTe in this regard are (i) the selection of the most important components of the energy functional, which is expedient for including as few fitted parameters as possible, and (ii) the required basic intuition on how to combine the ecologically relevant parameters into an energy functional expression. With our range of examples (summarised in Fig. 1), we have highlighted that rudimentary insight into ecosystem functioning can be sufficient for generating results on par with established approaches. Additional illustrations of the intuition behind constructing energy functionals for ecological phenomena are provided in the Supplementary Notes. We judge the main technical challenge in future applications of DFTe to be the parameterisation of models at landscape scales relevant to many ecological applications. Such applications will require not only more data and careful choice of parsimonious functional forms for the energy components, but also high-performance computing.
We have demonstrated the potential for DFTe to provide a unified modelling framework for ecology via applications to a variety of systems, from predatory microbes in test tubes to interacting grasses in saline landscapes to a synthetic food web. Our approach is inspired by DFT in physics, with the key difference that, whereas the DFT energy functional is derived from first principles, the DFTe energy functional must be established via ecological intuition and parsimony. Ultimately, as with any ecological model, the arbiter of the value of DFTe must be determined by its ability to fit and predict ecological data, and on this count our applications have shown DFTe to be comparable to existing specialised approaches. While we are optimistic that DFTe can become a universal tool for an intuitive approach to ecology as a whole, including dynamics on all relevant time scales (see Supplementary Notes), only in-depth future analyses can be the ultimate judge on that matter. History provides vivid examples of physics-inspired models that have led to powerful frameworks in other fields, such as the gravity model of human mobility, whose initial heuristic foundations have been refined over decades towards a parameter-free model51.
On a general note, we hope that our foundational work at the cross-roads of physics and ecology inspires researchers and practitioners across disciplines towards developing DFT-based tools that tackle their big questions, such as pandemics, traffic control, or the dynamics of financial assets. These are serious enterprises, and a concerted effort will be required. We hope to start an interdisciplinary dialogue that transfers one of the most powerful methods of physics to unexpected realms.
Methods
Modular components of the DFTe energy functional
The central ingredient of DFTe is an energy functional E, assembled according to Eq. (1). The methodology of DFTe can be understood by inspecting the dispersal and environmental energies in Eqs. (2) and (3) without interactions. In our first case study, illustrated in Fig. 2 and Supplementary Fig. 2, we demonstrate that equation (3), in conjunction with Eq. (2), can realistically describe the influence of the environment on species’ distributions. Mechanisms that alter the trade-off between dispersal and environment can be introduced as part of Eint. For instance, back reactions on the environment could be modelled with a bifunctional Ebr[Venv, n] that yields the equilibrated modified environment \({V}_{s}^{{{{{{{{\rm{env}}}}}}}}}+\delta {E}_{{{{{{{{\rm{br}}}}}}}}}[{{{{{{{{\bf{V}}}}}}}}}^{{{{{{{{\rm{env}}}}}}}}},{{{{{{{\bf{n}}}}}}}}]/\delta {n}_{s}({{{{{{{\bf{r}}}}}}}})\), cf. Eq. (5).
In the following we make explicit the interaction and resource energies that enter Eq. (1) and are used in our case studies of Figs. 2–7. We let Eint[n] include all possible bipartite interactions
which include amensalism, commensalism, mutualism, and so forth. Here, \({\alpha }_{s},\, {\beta }_{{s}^{{\prime} }}\ge 0\), and the interaction kernels \({\gamma }_{s{s}^{{\prime} }}\) are assembled from fitness proxies of species s and \({s}^{{\prime} }\) (Supplementary Table 1). Higher-order interactions can be introduced, for example, through (i) terms like \({n}_{s}\,{\gamma }_{s{s}^{{\prime} }}\,{n}_{{s}^{{\prime} }}\,{\gamma }_{{s}^{{\prime} }{s}^{{\prime\prime} }}^{{\prime} }\,{n}_{{s}^{{\prime\prime} }}\) that build on pairwise interactions or (ii) genuinely multipartite expressions like \({\gamma }_{s{s}^{{\prime} }{s}^{{\prime\prime} }}{n}_{s}\,\,{n}_{{s}^{{\prime} }}\,{n}_{{s}^{{\prime\prime} }}\). Multi-partite interactions based on bipartite interactions do not seem to be an uncommon scenario48. However, there may be systems where nonzero coefficients \({\gamma }_{s{s}^{{\prime} }{s}^{{\prime\prime} }}\) couple all species. This poses a challenge for mechanistic theories in general. Then, ‘simpler subsystems’ that have to be included in the DFTe workflow of Fig. 1a can only refer to situations where other energy components are absent, such as resource terms or complex environments. For example, the coefficients \({\gamma }_{s{s}^{{\prime} }{s}^{{\prime\prime} }}\) could be extracted in an experiment with a controlled simple environment and then used to model the interacting species in a real-world setting. For \(({\alpha }_{s},\, {\beta }_{{s}^{{\prime} }})=(1,1)\) we identify the contact interaction in physics as \({\gamma }_{s{s}^{{\prime} }}\propto \delta ({{{{{{{\bf{r}}}}}}}}-{{{{{{{{\bf{r}}}}}}}}}^{{\prime} })\) with the two-dimensional delta function δ( ), while the Coulomb interaction amounts to setting \({\gamma }_{s{s}^{{\prime} }}\propto 1/|{{{{{{{\bf{r}}}}}}}}-{{{{{{{{\bf{r}}}}}}}}}^{{\prime} }|\). The mechanistic effect of these interaction kernels on the density distributions is the same in ecology as it is in physics—a mathematical insight that inspired us to build ecological analogues to the phenomenology of quantum gases, which feature functionals of the kind in Eq. (6). Note that we do not introduce any quantum effects into ecology despite the fact that the mathematical structure of DFTe is borrowed in part from quantum physics. While the contact interaction is a suitable candidate for plants and especially microbes52, we expect long-range interactions (for example, repulsion of Coulomb type) to be more appropriate for species with long-range sensors, such as eyes. Both types of interactions feature in describing the ecosystems addressed in this work.
In a natural setting the equilibrium abundances are ultimately constrained by the accessible resources. It is within these limits of resource availability that environment as well as intra- and inter-specific interactions can shape the density distributions. An energy term for penalising over- and underconsumption of resources is thus of central importance. Each species consumes resources from some of the K provided resources, indexed by k. A subset of species consumes the locally available resource density ρk(r) according to the resource requirements νks, which represent the absolute amount of resource k consumed by one individual (or aggregated constituent) of species s. The simple quadratic functional
proves appropriate. Here, νksns is the portion of resource density ρk that is consumed by species s. That is, νks > 0 indicates that s requires resource k. If Eq. (7) is the total energy functional, then a single-species system with a single resource equilibrates with density n1(r) = ρ1(r)/ν11 at every position r, and additional DFTe energy components would modify this equilibrium. Predator–prey relationships are introduced by making species k a resource \({\rho }_{k}=\left]{n}_{k}\right[\), where \(\left]n\right[\) declares n a constant w.r.t. the functional differentiation of E, that is, the predator tends to align with the prey, not the prey with the predator. In view of the energy minimisation, the quadratic term in Eq. (7) entails that regions of low resource density ρk are less important than regions of high ρk. The different resources k have the same ability to limit the abundances, such that the limiting resource k = l at r has to come with the largest of weights wl(r), irrespective of the absolute amounts of resources at r. For example, the weights wk have to ensure that an essential but scarce mineral has (a priori) the same ability to limit the abundances as a resource like water, which might be abundant in absolute terms. To that end, we specify the weights
which are inspired by the smooth minimum function, where σ < 0, λks(r) = ρk(r)/νks, and the carrying capacity of the limiting resource is \({\lambda }_{ls}=\mathop{\min }\limits_{k}\,{\lambda }_{ks}\). The step function η( ) in Eq. (8) ensures that only resources k that are actually consumed by species s contribute to wk. We rescale wk using the average \({\bar{\rho }}_{k}=\frac{1}{A}{\int}_{A}({{{{{{{\rm{d}}}}}}}}{{{{{{{\bf{r}}}}}}}})\,{\rho }_{k}({{{{{{{\bf{r}}}}}}}})\), which puts all resources on equal footing in their ability to limit abundances and renders \({E}_{{{{{{{{\rm{Res}}}}}}}}}\) invariant under change of resource units. The ratio of carrying capacities in Eq. (8) implements the relative importance of all resources at r, with resources k ≠ l suppressed exponentially according to their deviation from their ability to limit the abundances. For example, with σ = − 4 and λks = 2λls, resource k ≠ l is largely irrelevant for species s as it weighs in at less than 2% compared to the limiting resource l. There can be multiple limiting resources for a species s, most conceivably, multiple resources that vanish locally. In the limit of vanishing resource density (ρl, λls → 0), the exponential in Eq. (8) reduces to the Kronecker-delta δkl, thereby rendering all resources with λks > λls irrelevant at r. Using \({E}_{{{{{{{{\rm{Res}}}}}}}}}\), we show that an analytically solvable minimal example of two amensalistically interacting species already exhibits a plethora of resource-dependent equilibrium states (see Supplementary Notes and Supplementary Fig. 1).
We specify the DFTe energy functional in Eq. (1) by summing Eqs. (2), (3), (6), and (7) and by (optionally) constraining the abundances to N via Lagrange multipliers μ:
Uniform situations are characterised by spatially constant ingredients ns = Ns/A, ρk = Rk/A, coefficients τs, etc. for the DFTe energy, such that Eq. (9) reduces to a function E(N) with building blocks
Ecosystem equilibria from the DFTe energy functional
The general form of Eq. (9) gives rise to two types of minimisers (viz., equilibria): First, we term
the ‘DFTe hypersurface’, with \(\tilde{{{{{{{{\bf{n}}}}}}}}}\) the energy-minimising spatial density profiles for given (fixed) N. Second, the ecosystem equilibrium is attained at the equilibrium abundances \(\hat{{{{{{{{\bf{N}}}}}}}}}={\int}_{A}({{{{{{{\rm{d}}}}}}}}{{{{{{{\bf{r}}}}}}}})\,\hat{{{{{{{{\bf{n}}}}}}}}}({{{{{{{\bf{r}}}}}}}})\), which yield the global energy minimum
where the minimisation samples all admissible abundances, that is, \({{{{{{{\bf{N}}}}}}}}\in {\left({{\mathbb{R}}}_{0}^{+}\right)}^{\times S}\) if no further constraints are imposed.
The direct minimisation of E[n] is most practical for uniform systems, which only require us to minimise E(N) over an S-dimensional space of abundances. For the general nonuniform case, we adopt a two-step strategy that reflects Eqs. (14) and (15). First, we obtain the equilibrated density distributions on \({{{{{{{\mathcal{H}}}}}}}}\) for fixed N from the computational DPFT framework26,27,28,29,30,31. Second, a conjugate gradient descent searches \({{{{{{{\mathcal{H}}}}}}}}({{{{{{{\bf{N}}}}}}}})\) for the global minimiser \(\hat{{{{{{{{\bf{N}}}}}}}}}\). Technically, we perform the computationally more efficient descent in μ-space. Local minima are frequently encountered, and we identify the best candidate for the global minimum from many individual runs that are initialised with random μ. Note that system realisations with energies close to the global minimum, especially local minima, are likely observable in reality, assuming that the system can equilibrate at all. There is always an equilibrium if the energy functional is bounded from below, together with the fact that the support (abundances/densities) of the energy functional is finite in any practical application. If some DFTe energy components are chosen (too) negative, the system can be unstable, in which case the energy functional has no minimum and is inappropriate for modelling the equilibrium in question. This means that another energy functional has to be considered, or, in the worst case, that DFTe is incapable of simulating this system. We also caution that no numerical optimisation algorithms for non-convex black-box functions can guarantee to find the global minimum, not even approximately. Without analytically available characteristics of the global minimum, all one may hope for are candidates of the minimiser, and those may not even be local minima—there is no way to be certain that an optimum proposed by a numerical optimisation algorithm is stable.
Density-potential functional theory (DPFT) in Thomas–Fermi (TF) approximation
Defining
for all s, we obtain the reversible Legendre transform
of the dispersal energy and thereby supplement the total energy with the additional variables V:
This density-potential functional is equivalent to (but more flexible than) the density-only functional E[n, μ](N). The minimisers of E[n] are thus among the stationary points of Eq. (18) and are obtained by solving
and
self-consistently for all ns while enforcing ∫A(dr) ns(r) = Ns. Specifically, starting from V(0) = Venv, such that \({n}_{s}^{(0)}={n}_{s}[{V}_{s}^{(0)}-{\mu }_{s}^{(0)}]\), we iterate
until all ns are converged sufficiently. This self-consistent loop establishes a trade-off between dispersal energy and effective environment V by forcing an initial out-of-equilibrium density distribution to equilibrate at fixed N. We adjust \({\mu }_{s}^{(i)}\) in each iteration i such that \({n}_{s}^{(i)}\) integrates to Ns. Small enough density admixtures, with 0 < θs < 1, are required for convergence. Each point of \({{{{{{{\mathcal{H}}}}}}}}({{{{{{{\bf{N}}}}}}}})\) in Eq. (14) represents the thus-equilibrated system for given N. The dispersal energy Edis[ns] in Eq. (2), where an increase of the dispersal pressure constants τs tends to dilute ns (see also Supplementary Fig. 2), is only one component of the total energy and has to be balanced against, for example, the environmental energy. This trade-off produces the aggregation of, for example the fruit flies in Fig. 2 and the grasses in Fig. 4. In the latter case the confining role is played by the interaction potential, not the environmental potential, which are combined anyway through Eq. (5). If Edis with positive τs were the only energy component in an unconstraining, i.e., flat and infinite environment, then \(\mathop{\min }\limits_{{n}_{s}}\,{E}_{{{{{{{{\rm{dis}}}}}}}}}[{n}_{s}]=0\) for a density that is maximally dispersed/diluted with ns (r) → 0 everywhere.
Assuming a given interaction energy, we require an explicit expression for the right-hand side of Eq. (19) to realise the self-consistent loop of Eq. (21). For two-dimensional fermion gases in TF approximation28, we have
which delivers (i) the density in Eq. (4) to be used for the right-hand side of Eq. (19) and (ii) Eq. (2) via Eq. (17) upon inverting the functional relationship n[V] of Eq. (4). For stabilising the numerics if necessary or if, for example, unambiguous derivatives of the density are sought, we replace Eq. (4) by its smooth version
where \({[x]}_{T}=T\,\log \left[1+\exp \left(x/T\right)\right]\) is a smooth version of [x]+. If τs = 0, we can add a dispersal energy with positive τs to E in order to execute the self-consistent loop with Eqs. (4) or (23), and compensate by subtracting the same dispersal energy from the interaction energy, which enters Eq. (20). The notation ns[Vs − μs](r) declares that ns is a functional (function of functions) of Vs − μs, and that ns is also a function of r. In the specific case of Eq. (4), the functional dependence of ns reduces to a trivial dependence on Vs − μs, which renders ns dependent on r alone—a consequence of Eq. (22) as the source of Eq. (4). Alternatives to Eq. (22) with nonlocal integrands will make ns a nontrivial functional of Vs − μs, for example, \({n}_{s}[{V}_{s}-{\mu }_{s}]({{{{{{{\bf{r}}}}}}}})=\frac{1}{{\tau }_{s}}\int({{{{{{{\rm{d}}}}}}}}{{{{{{{{\bf{r}}}}}}}}}^{{\prime} })\frac{{[{\mu }_{s}-{V}_{s}({{{{{{{{\bf{r}}}}}}}}}^{{\prime} })]}_{+}}{|{{{{{{{\bf{r}}}}}}}}-{{{{{{{{\bf{r}}}}}}}}}^{{\prime} }|}\), which reduces to Eq. (4) if \(1/|{{{{{{{\bf{r}}}}}}}}-{{{{{{{{\bf{r}}}}}}}}}^{{\prime} }|\) is replaced by \(\delta ({{{{{{{\bf{r}}}}}}}}-{{{{{{{{\bf{r}}}}}}}}}^{{\prime} })\).
Explicit parameterisations of the DFTe energy functional
We model ecological phenomena with the help of the potentials introduced in Eqs. (5) and (20) as a consequence of the energy functional. They are mathematical constructs that can be measured and validated only through a mathematical relation with observable quantities like densities and abundances. At best, we can claim that these mathematical constructs describe our observations appropriately. The situation is no different in physics, see Supplementary Notes. We also note that all DFTe parameters of the specific data-driven models pertinent to Figs. 2–5 and 7 are either fit parameters or are inferred from the data.
Parameterisations for the fruit flies in heated chambers
In view of Eq. (4), an educated guess informed by the measured reference density \({n}_{{{{{{{{\rm{ref}}}}}}}}}^{{{{{{{{\rm{1Dc}}}}}}}}}\) (see Supplementary Fig. 2) is a quadratic environment induced by the heat source at x0. We also include a repulsive long-range interaction, since (i) the data in Fig. 2g of ref. 25 suggests that the interaction is quadratic in the density and (ii) fruit flies have to sense their peers remotely, for example, in defending territory53. This leaves us with two parameters, ε and γ, for fitting the minimising density of
to \({n}_{{{{{{{{\rm{ref}}}}}}}}}^{{{{{{{{\rm{1Dc}}}}}}}}}\). In the Supplementary Notes we offer a more detailed account of the reasoning behind Eq. (24). In ref. 25 the interaction parameters from the quasi-one-dimensional (1D) chamber are combined with the environmental information from a low-density experiment (three flies) in a ‘staircase chamber’ to predict a high-density distribution of 220 flies (labelled ‘DFFT’ in Fig. 2b). This is a reasonable procedure, since the low-density three-flies experiment represents a situation with very small total repulsion and is therefore well suited for extracting the environmental influence on the density distribution. We thus follow the same strategy. Keeping γ = 9 cm from the density fit to the quasi-1D chamber setup, we get ε = 14.5 cm−2 for three flies in the staircase chamber and use both parameters for predicting the density distribution of 220 flies (labelled ‘DFTe’ in Fig. 2b). Since γ is (i) one of the system’s characteristic length scales, (ii) an estimator of the spatial reach of the long-range interaction, and (iii) close to the system size, we deem the fruit flies setup a small-scale system—in contrast to the large-scale contact-interacting systems of Figs. 4 and 6.
Parameterisations for the resource competition among four algae
We face a uniform environment with S species and two resources. An amensalistic interaction is an appropriate candidate for putting a subset of species at a disadvantage relative to their heterospecifics (see Supplementary Table 1). We rule out parasitism, understood in the sense of an asymmetric interaction, for which the DFTe equilibria do not produce the correct survivors in all reference resource cases \({{{{{{{{\mathcal{R}}}}}}}}}_{1-7}\) (see Supplementary Table 2). In contrast to the fruit-flies study above, the abundances are not fixed but are rather to be determined as minimisers of the N-dependent energy function \(E({{{{{{{\bf{N}}}}}}}})={E}_{{{{{{{{\rm{Res}}}}}}}}}({{{{{{{\bf{N}}}}}}}})+{E}_{\gamma }({{{{{{{\bf{N}}}}}}}})\), assembled from Eqs. (10)–(13). The spatially uniform interaction kernel
for Eγ(N), see Eq. (12), is assembled from the fitness proxies
and
which we assume to influence the species’ ability to consume the provided resources in the presence of heterospecifics. In the Supplementary Notes we provide a complementary step-by-step account of the intuition that leads us to Eqs. (25)–(28). The s-specific traits \({R}_{sk}^{*}\) (the densities of resource k, below which species s cannot survive in monoculture) and νks (see Eq. (7)) follow from monoculture experiments (see Supplementary Notes and Table 1 of ref. 1). The exponent κ in Eq. (25) introduces a hierarchy between the fitness proxies of Eqs. (26) and (27) and serves as a second parameter to be used in the fitting of our reference data, which are all 42 ‘reference’ abundances \({{{{{{{{\bf{N}}}}}}}}}_{{{{{{{{\mathcal{R}}}}}}}}}\) of two-species R*-equilibria that follow from \({{{{{{{{\mathcal{R}}}}}}}}}_{1-7}\). The parameter γ enables the trade-off between competition and resource energy. In the spirit of R*-theory, we choose σ → − ∞ in Eq. (8), such that only the limiting resource for each species enters the weights wk for \({E}_{{{{{{{{\rm{Res}}}}}}}}}\). Finally, the minimisers of
yield the best fit to \({{{{{{{{\bf{N}}}}}}}}}_{{{{{{{{\mathcal{R}}}}}}}}}\) for γ = 8 × 10−8 and κ = 8, used for modelling the three- and four-species communities in Fig. 3.
It would be surprising if Eq. (25) were the only possible choice for the interaction kernel, and it is quite likely that many different energy functionals that encode different mechanisms and input parameters are equally suited for reproducing the data targeted in this example. Only future DFTe studies that take into account additional data can reduce the space of possible functionals. Taking the cue from physics, we may hope that a universal functional will prevail and thereby provide deeper insights into the fundamental mechanisms and relations underlying ecological systems in general and microbial communities in particular.
Parameterisations for the competition between three grasses in salinity gradients
The slim characterisation of the experimental setting reported in Ref. 34 (see Supplementary Notes) contrasts with the number of parameters of the DFTe functional E[n, μ](N) in Eq. (9), for which all components require input—except the environmental potential energies, which are constant in this uniform situation and therefore irrelevant. In order to proceed, we assume the following: (i) Coexistence of the three competing grasses in mixture suggests that three different resources are exclusively limiting each species; (ii) all species exhibit equal dispersal τ as well as interaction strength γ factored into \({\gamma }_{s{s}^{{\prime} }}=\gamma \,{[{f}_{s}/{f}_{{s}^{{\prime} }}-1]}_{+}\) (mixture- and monoculture abundances are not proportional, implying some kind of salinity-dependent competition); (iii) asymmetric interactions, since amensalism and repulsion yield distributions inconsistent with field data, see Supplementary Notes and Supplementary Fig. 5; (iv) as suggested in Ref. 34, we fix the salinity-dependent fitness proxies fs as the fraction of above-ground biomass of s in mixture. This reflects increasing salt tolerance in the order Poa < Hord < Pucc. We then fit the nine parameters (τ, σ, νk≠s, γ) of
to the abundances for the mixture in uniform salinity (see Supplementary Figs. 5 and 6), such that the nonuniform version of Eq. (29) allows us to predict the density distributions in Fig. 4 implied by heterogeneous resource distributions (see Supplementary Notes).
Parameterisations for the dynamics of microbial predation
P. aurelia (P) feeds on Cerophyl (c) and serves as the single resource for the predator D. nasutum (D). From the general DFTe energy we thus select the two quadratic resource energies and a parasitic interaction with fit parameter γ, akin to an asymmetric repulsive contact interaction that favours D. For this spatially uniform microbial system in suspension, E[n] reduces to the function
We fit its minimiser \(\hat{{{{{{{{\bf{N}}}}}}}}}\) (red cross in Fig. 5) to the average abundances (cyan cross in Fig. 5) for the last cycle of the most stable time series (Fig. 14c in ref. 36); see also Supplementary Notes, Supplementary Fig. 9, and Supplementary Table 3. These average abundances anchor our comparison between the experimental data and the DFTe trajectory \({{{{{{{\mathcal{H}}}}}}}}(\hat{{{{{{{{\bf{N}}}}}}}}})+\Delta E\), with a suitably chosen ‘excitation’ energy ΔE. As expected, amensalistic interactions are not supported by the data and are therefore ruled out, see Supplementary Fig. 8 and Supplementary Fig. 10. By comparing, for example, the energies in Eqs. (29) and (30) for the grasses and the microbial predator–prey system, respectively, we see that DFTe is capable of determining what (very) different systems share—in this case the parasitic interaction, but not the dispersal pressure.
Parameterisations for the complex large-scale synthetic community
The community encompasses heterogeneous environments, competition over resources, all bipartite interactions of Supplementary Table 1, and predator–prey relations. From Eq. (9), we assemble the according DFTe energy functional
The species differ in their dispersal strengths τs, habitat preferences \({V}_{s}^{{{{{{{{\rm{env}}}}}}}}}\), the types \(({\alpha }_{s},\, {\beta }_{{s}^{{\prime} }})\) of heterospecific interactions, modulated by interaction strengths \({\gamma }_{s{s}^{{\prime} }}^{({\alpha }_{s},{\beta }_{{s}^{{\prime} }})}\), as well as the types and amounts of required resources, see Supplementary Table 4. The synthetic environments and non-prey resource distributions are depicted in Supplementary Fig. 11. The density ns of each species may, independently from the other densities \({n}_{{s}^{{\prime} }\ne s}\), refer to individuals per area, frequency, biomass density, fractional land cover, or any other expedient metric—units and absolute scales are absorbed in the parameters \({\tau }_{s},{V}_{s}^{{{{{{{{\rm{env}}}}}}}}}\), and so forth. By deploying only contact-type interactions (see Supplementary Table 4), we implicitly state that members of any species are not directly influenced by conspecifics or heterospecifics beyond their local pixel of our coarse-grained area A. For example, if the Cat habitat area is ~50 km2, then A exceeds ~50,000 km2 (see Supplementary Notes). This large-scale system contrasts with the fruit flies experiment in Fig. 2, where the long-range repulsive interaction between individuals turns out to couple all individuals explicitly across the entire area. Note that contact-type interactions on large spatial scales do not imply that the system is weakly interacting: Both the synthetic food web here and the fruit flies experiment are strongly interacting since the interaction energies comprise a substantial part of the total energy for both systems. Using the so specified energy functional along with the density expression of Eq. (4), we simulate 33 × 33 = 1089 parcels in the focal area A = 1. We obtain the equilibrium density profiles from a conjugate gradient descent towards the global energy minimum of Eq. (31) in the up to seven-dimensional space of abundances N. In view of the stark disparity between species distributions (i) in isolation and (ii) under the influence of interactions, see Supplementary Fig. 14, we build intuition on the entire community by modelling subsystems with fewer species (Supplementary Figs. 12 and 13).
We illustrate the connections between DFTe inputs and outputs by explaining the small decrease of the Pig abundance upon Cat extermination (Fig. 6b): We observed that the Pig is the limiting resource for the Cat only in regions of very low Pig density (see Supplementary Fig. 13), such that changes in the Pig distribution are not primarily connected to the removal of the Cat. Second, the heavily predated Deer as well as the mutualistically connected Tree generally benefit from the absence of the Cat. As a result, the Grass and, consequently, the Snail come under pressure, which helps the Fungus, whose mutualistic connection to the Tree closes the positive feedback loop between Deer, Tree, and Fungus. The increased Fungus density then attracts the Pig, except in two small enclaves (indicated with arrows in Fig. 6b), where the processes just described are reversed, such that the Pig is mainly redistributed globally. Of course, this narrative conveys only the broad strokes of the quantitative simulation and plays out under the constraints imposed by environments and resources, but it conveys the quantitative knowledge about the community functioning obtained from the DFTe simulation.
Parameterisations for the twenty tree species in a tropical forest
Since we lack additional ecological information beyond densities and soil composition for constraining the full DFTe energy functional, we chose the same energy functional as in Eq. (31), though with (i) S = 20 species, which constitute more than 50% of the total basal area of the 328 censused species, (ii) all K = 11 measured resources, and (iii) the ansatz \({V}_{s}^{{{{{{{{\rm{env}}}}}}}}}={\epsilon }_{s}^{{{{{{{{\rm{alt}}}}}}}}}\,{V}^{{{{{{{{\rm{alt}}}}}}}}}+{\epsilon }_{s}^{{{{{{{{\rm{pH}}}}}}}}}\,{V}^{{{{{{{{\rm{pH}}}}}}}}}\). Accordingly, there are 17 DFTe fit parameters per species—the two parameters \({\epsilon }_{s}^{{{{{{{{\rm{alt}}}}}}}}}\) and \({\epsilon }_{s}^{{{{{{{{\rm{pH}}}}}}}}}\) that encode the response to altitude and pH-level, eleven resource requirements νks for k = 1, …, 11, dispersal pressure τs, and three fitness proxies \({f}_{s}^{({{{{{{{\rm{a/c}}}}}}}})},\, {f}_{s}^{({{{{{{{\rm{r/m}}}}}}}})}\), and \({f}_{s}^{({{{{{{{\rm{asym}}}}}}}})}\) that encode all the bipartite interactions of Supplementary Table 1 through the kernels \({\gamma }_{s{s}^{{\prime} }}^{{{{{{{{\rm{(i)}}}}}}}}}={f}_{s}^{{{{{{{{\rm{(i)}}}}}}}}}-{f}_{{s}^{{\prime} }}^{{{{{{{{\rm{(i)}}}}}}}}}\) for \({s}^{{\prime} }\, > \,s\). We also leave σ for the resource weights as a free parameter. All these 341 (281) DFTe parameters of the energy functional with (without) bipartite interactions are unknown and have to be extracted from fitting E to the reference densities nref. This presents a high-dimensional constrained problem of maximising the least-squares overlap ξ between the densities \(\tilde{{{{{{{{\bf{n}}}}}}}}}\left({\tilde{{{{{{{{\bf{n}}}}}}}}}}^{{}_{0}}\right)\) and nref, which we solve by using stochastic evolutionary algorithms that are efficient in escaping local optima, see Supplementary Notes.
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.
Data availability
The experimental data used in this manuscript are extracted from refs. 1,25,34,42,54. The fruit-flies data from ref. 25 are available at https://doi.org/10.1038/s41467-018-05750-z. We digitised Fig. 6 of ref. 1, available at https://doi.org/10.2307/1937747, for displaying the experimental data on the competing algae in Supplementary Table 2. For our case study on three grasses, we obtained the above-ground biomass data in monoculture and mixture by digitising Fig. 3 of ref. 34, available at 10.1139/b91-310. For our predator–prey case study we used the digitised data of ref. 36, which is provided in the appendix of ref. 54, available at 10.1098/rspb.2000.1186. The data for Barro Colorado Island is accessible through Dryad at 10.15146/5xcp-0d46.
Code availability
All the codes used in this study are available as open access repositories. The DFTe formalism for the heterogeneous systems is part of the C++ software package ‘mpDPFT’, available at https://doi.org/10.5281/zenodo.6999439. Analyses for the case studies of Figs. 2–7 are available as Mathematica notebooks at https://doi.org/10.5281/zenodo.7002198.
References
Tilman, D. Tests of resource competition theory using four species of Lake Michigan Algae. Ecology 62, 802 (1981).
Pacala, S. W. et al. Forest models defined by field measurements: estimation, error analysis and dynamics. Ecol. Monogr. 66, 1 (1996).
Cabral, J. S., Valente, L. & Hartig, F. Mechanistic simulation models in macroecology and biogeography: state-of-art and prospects. Ecography 40, 267 (2017).
Levin, S. A. The problem of pattern and scale in ecology: the Robert H. MacArthur award lecture. Ecology 73, 1943 (1992).
Leibold, M. A. et al. The metacommunity concept: a framework for multi-scale community ecology. Ecol. Lett. 7, 601 (2004).
McGill, B. J. Towards a unification of unified theories of biodiversity. Ecol. Lett. 13, 627 (2010).
Udy, K. et al. Environmental heterogeneity predicts global species richness patterns better than area. Global Ecol. Biogeogr. 30, 842 (2021).
Benton, T. G., Solan, M., Travis, J. M. J. & Sait, S. M. Microcosm experiments can inform global ecological problems. Trends Ecol. Evol. 22, 516 (2007).
Evans, M. R. Modelling ecological systems in a changing world. Philos. Trans. R. Soc. Lond. B Biol. Sci. 367, 181 (2012).
Stewart, R. I. A. et al. Mesocosm Experiments as a Tool for Ecological Climate-Change Research, Ch. 2 in Global Change in Multispecies Systems: Part 3, Woodward, G. (Ed.) (Academic Press, 2013).
O’Dwyer, J. P. & Cornell, S. J. Cross-scale neutral ecology and the maintenance of biodiversity. Sci. Rep. 8, 1 (2018).
Kéfi, S. et al. Advancing our understanding of ecological stability. Ecol. Lett. 22, 1349 (2019).
Hohenberg, P. & Kohn, W. Inhomogeneous electron gas. Phys. Rev. 136, B864 (1964).
Kohn, W. & Sham, L. J. Self-consistent equations including exchange and correlation effects. Phys. Rev. 140, A1133 (1965).
Levy, M. Universal variational functionals of electron densities, first-order density matrices, and natural spin-orbitals and solution of the v-representability problem. Proc. Natl. Acad. Sci. USA 76, 6062 (1979).
Dreizler, R. M. & Gross, E. K. U. Density Functional Theory (Springer, 1990).
Sholl, D. S. & Steckel, J. A. Density Functional Theory: A Practical Introduction (Wiley, 2009).
Burke, K. Perspective on density functional theory. J. Chem. Phys. 136, 150901 (2012).
Becke, A. D. Perspective: fifty years of density-functional theory in chemical physics. J. Chem. Phys. 140, 18A301 (2014).
Hasnip, P. J. et al. Density functional theory in the solid state. Phil. Trans. R. Soc. A 372, 20130270 (2014).
Thomas, L. H. The calculation of atomic fields. Math. Proc. Cambridge Philos. Soc. 23, 542 (1927).
Fermi, E. Un metodo statistico per la determinazione di alcune proprieta dell’atomo. Rend. Lincei 6, 602 (1927).
Kalita, B., Li, L., McCarty, R. J. & Burke, K. Learning to approximate density functionals. Acc. Chem. Res. 54, 818 (2021).
Yang, W. Preface: special topic on advances in density functional theory. J. Chem. Phys. 140, 18A101 (2014).
Méndez-Valderrama, J. F., Kinkhabwala, Y. A., Silver, J., Cohen, I. & Arias, T. A. Density-functional fluctuation theory of crowds. Nat. Commun. 9, 3538 (2018).
Englert, B.-G. & Schwinger, J. Thomas–Fermi revisited: the outer regions of the atom. Phys. Rev. A 26, 2322 (1982).
Englert, B.-G. Lecture Notes in Physics: Semiclassical Theory of Atoms (Springer, 1988).
Trappe, M.-I., Len, Y. L., Ng, H. K., Müller, C. A. & Englert, B.-G. Leading gradient correction to the kinetic energy for two-dimensional fermion gases. Phys. Rev. A 93, 042510 (2016).
Trappe, M.-I., Len, Y. L., Ng, H. K. & Englert, B.-G. Airy-averaged gradient corrections for two-dimensional fermion gases. Ann. Phys. (N. Y.) 385, 136 (2017).
Trappe, M.-I., Ho, D. Y. H. & Adam, S. First-principles quantum corrections for carrier correlations in double-layer two-dimensional heterostructures. Phys. Rev. B 99, 235415 (2019).
Englert, B.-G. Julian Schwinger and the Semiclassical Atom, pp. 261–269 in Proceedings of the Julian Schwinger Centennial Conference, Englert, B.-G. (Ed.) (World Scientific, 2019).
Friedman, J. & Higgins, L. M. Community structure follows simple assembly rules in microbial microcosms. Nat. Ecol. Evol. 1, 0109 (2017).
Marquet, P. A. et al. On theory in ecology. BioScience 64, 701 (2014).
Kenkel, N., McIlraith, A., Burchill, C. & Jones, G. Competition and the response of three plant species to a salinity gradient. Can. J. Bot. 69, 2497 (1991).
Ungar, I. A. Inland Halophytes of the Unites States, pp. 235–305 in Ecology of Halophytes, Reimold, J. R & Queen, W. H. (Eds.) (Academic Press, 1974).
Veilleux, B. G. The Analysis of a Predatory Interaction Between Didinium and Paramecium (M.Sc. thesis, University of Alberta, Edmonton, Canada, 1976).
Veilleux, B. G. An analysis of the predatory interaction between Paramecium and Didinium. J. Anim. Ecol. 48, 787 (1979).
Estes, J. A. et al. Trophic downgrading of planet earth. Science 333, 301 (2011).
Thompson, R. M. et al. Food webs: reconciling the structure and function of biodiversity. Trends Ecol. Evol. 27, 689 (2012).
Hamm, M. & Drossel, B. The concerted emergence of well-known spatial and temporal ecological patterns in an evolutionary food web model in space. Sci. Rep. 11, 4632 (2021).
Christensen, V. et al. Representing variable habitat quality in a spatial food web model. Ecosystems 17, 1397 (2014).
Condit, R. et al. Complete data from the Barro Colorado 50-ha plot: 423617 trees, 35 years, Dryad, Dataset, (2019).
Davies, S. J. et al. ForestGEO: understanding forest diversity and dynamics through a global observatory network. Biol. Conserv. 253, 108907 (2021).
Wiegand, T. et al. Consequences of spatial patterns for coexistence in species-rich plant communities. Nat. Ecol. Evol. 5, 965 (2021).
Hubbell, S. P. & Foster, R. B. Biology, Chance and History and the Structure of Tropical Rain Forest Tree Communities, pp. 314–329 in Community Ecology, Diamond, J.M. & Case, T. J. (Eds.) (Harper and Row, New York, 1986).
Volkov, I. & Maritan, A. Inferring species interactions in tropical forests. Proc. Natl Acad. Sci. USA 106, 13854 (2009).
Wiegand, T. et al. Testing the independent species’ arrangement assertion made by theories of stochastic geometry of biodiversity. Proc. R. Soc. B Biol. Sci. 279, 3312 (2012).
Grilli, J., Barabás, G., Michalska-Smith, M. J. & Allesina, S. Higher-order interactions stabilize dynamics in competitive network models. Nature 548, 210 (2017).
Mayfield, M. M. & Stouffer, D. B. Higher-order interactions capture unexplained complexity in diverse communities. Nat. Ecol. Evol. 1, 0062 (2017).
Benton, T. G., Plaistow, S. J. & Coulson, T. N. Complex population dynamics and complex causation: devils, details and demography. Proc. Biol. Sci. 273, 1173 (2006).
Simini, F., González, M. C., Maritan, A. & Barabási, A. L. A universal model for mobility and migration patterns. Nature 484, 96 (2012).
Dal Co, A., van Vliet, S., Kiviet, D. J., Schlegel, S. & Ackermann, M. Short-range interactions govern the dynamics and functions of microbial communities. Nat. Ecol. Evol. 4, 366 (2020).
Hoffmann, A. A. The influence of age and experience with conspecifics on territorial behavior in Drosophila melanogaster. J. Insect Behav. 3, 1 (1990).
Jost, C. & Ellner, S. P. Testing for predator dependence in predator-prey dynamics: a non-parametric approach. Proc. R. Soc. Lond. B 267, 1611 (2000).
Acknowledgements
We are grateful to James Patrick O’Dwyer for many insightful discussions and his crucial review of this work. We also thank Berthold-Georg Englert, Daniel Heyen, and Rosita Mei Ling Koh for valuable feedback on this manuscript. This research was supported by the ISF-NRF Singapore joint research programme (grant number WBS R-154-000-B09-281; R.A.C.).
Author information
Authors and Affiliations
Contributions
Both authors M.-I.T. and R.A.C. conceived the study and wrote the manuscript. M.-I.T. designed and carried out the research and analyses. R.A.C. conducted the Lotka–Volterra simulation and contributed to the choice of case studies for the research as well as to the interpretation of the analyses.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Communications thanks Oscar Godoy and Sandro Azaele for their contribution to the peer review of this work. Reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Trappe, MI., Chisholm, R.A. A density functional theory for ecology across scales. Nat Commun 14, 1089 (2023). https://doi.org/10.1038/s41467-023-36628-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467-023-36628-4
- Springer Nature Limited