Dynamic metabolic resource allocation based on the maximum entropy principle

Abstract

Organisms have evolved a variety of mechanisms to cope with the unpredictability of environmental conditions, and yet mainstream models of metabolic regulation are typically based on strict optimality principles that do not account for uncertainty. This paper introduces a dynamic metabolic modelling framework that is a synthesis of recent ideas on resource allocation and the powerful optimal control formulation of Ramkrishna and colleagues. In particular, their work is extended based on the hypothesis that cellular resources are allocated among elementary flux modes according to the principle of maximum entropy. These concepts both generalise and unify prior approaches to dynamic metabolic modelling by establishing a smooth interpolation between dynamic flux balance analysis and dynamic metabolic models without regulation. The resulting theory is successful in describing ‘bet-hedging’ strategies employed by cell populations dealing with uncertainty in a fluctuating environment, including heterogenous resource investment, accumulation of reserves in growth-limiting conditions, and the observed behaviour of yeast growing in batch and continuous cultures. The maximum entropy principle is also shown to yield an optimal control law consistent with partitioning resources between elementary flux mode families, which has important practical implications for model reduction, selection, and simulation.

Appendices

A Derivation of maximum entropy control

To solve the optimal control problem (14) one introduces the Hamiltonian

$$\begin{aligned} \mathcal {H}(\Delta \mathbf {X},\mathbf {u},\varvec{\lambda }) = \varvec{\lambda }^T[ \mathbf {F}(\mathbf {X}(t),\mathbf {u}^0) + \mathbf {A} \Delta \mathbf {X} + \mathbf {B} \Delta \mathbf {u} ] + \sigma H(\mathbf {u}) , \end{aligned}$$

where $\varvec{\lambda }$ is a co-state vector the same dimension as $\mathbf {X}$. Applying Pontryagin’s maximum principle implies maximisation of $\mathcal {H}$, or equivalently the functional

$$\begin{aligned} \mathcal {F}(\mathbf {u}) = \varvec{\lambda }^T \mathbf {B} \mathbf {u} + \sigma H(\mathbf {u}) , \end{aligned}$$

with respect to $\mathbf {u}$ subject to the constraint

$$\begin{aligned} \sum _{k=1}^K u_k = 1 . \end{aligned}$$

This results in the K first-order conditions

$$\begin{aligned} \varvec{\lambda }^T \mathbf {B}^k -\sigma ( 1+ \log u_k) - \alpha = 0 \end{aligned}$$

where $\alpha \ge 0$ is a Lagrange multiplier and $\mathbf {B}^k$ is the kth column of $\mathbf {B}$. The general solution of these equations takes the form

$$\begin{aligned} u_k = \frac{1}{Q}\exp (\varvec{\lambda }^T \mathbf {B}^k/\sigma ) \end{aligned}$$

and Q is determined from the above constraint such that

$$\begin{aligned} Q = \sum _{k=1}^K \exp (\varvec{\lambda }^T \mathbf {B}^k/\sigma ) . \end{aligned}$$

The value of the co-state vector $\varvec{\lambda }$ is obtained as in Young and Ramkrishna (2007) by solving the boundary value problem

$$\begin{aligned} - \frac{d}{d \tau } \varvec{\lambda } = \frac{\partial \mathcal {H}}{\partial \Delta \mathbf {X}} = \mathbf {A}^T \varvec{\lambda } , \quad \varvec{\lambda }(t + \Delta t ) = \mathbf {q} \end{aligned}$$

whose solution is

$$\begin{aligned} \varvec{\lambda }(t + \tau ) = \mathbf {e}^{\mathbf {A}^T(\Delta t - \tau )} \mathbf {q} , \quad 0 \le \tau \le \Delta t. \end{aligned}$$

This expression for $\varvec{\lambda }$ is substituted into the above expression for $u_k$ and because, ultimately, only the optimal control input at the current time t is of interest, one can set $\tau = 0$ as in Young and Ramkrishna (2007), which yields the maximum entropy control (15).

B First-order correction to return-on-investment

Expressed in terms of the average zeroth-order return-on-investment

$$\begin{aligned} \bar{R}_0(\mathbf {m}_s) = \sum _{k=1}^K r_k(\mathbf {m}_s)\mathbf {c}^T \mathbf {Z}^k u^0_k , \end{aligned}$$

the vector $\mathbf {q}^T\mathbf {A}$ is

$$\begin{aligned} \mathbf {q}^T\mathbf {A} = (\mathbf {0},1) \frac{\partial }{\partial \mathbf {X}} \mathbf {F}(\mathbf {X}(t),\mathbf {u}^0)= \left( x \left( \frac{\partial \bar{R}_0}{\partial \mathbf {m}_{s}} (\mathbf {m}_{s}) \right) ^T, \bar{R}_0(\mathbf {m}_s) \right) . \end{aligned}$$

The first-order expansion of $\mathbf {e}^{\mathbf {A} \Delta t}$ takes the form $\mathbf {I} + \Delta t \mathbf {A}$, where $\mathbf {I}$ is the identity matrix, and therefore the effective return-on-investment $\mathcal {R}^k_{\Delta t}(\mathbf {m}_s)$ in (16) is approximated to first-order by

$$\begin{aligned} x r_k(\mathbf {m}_s)\mathbf {q}^T(\mathbf {I} + \Delta t \mathbf {A}) \begin{pmatrix} \mathbf {S}_s \\ \mathbf {c}^T \end{pmatrix} \mathbf {Z}^k . \end{aligned}$$

Taking the coefficient of $x \Delta t$ and substituting for $\mathbf {q}^T\mathbf {A}$ gives the first-order correction to return-on-investment

$$\begin{aligned} R^k_1(\mathbf {m}_s) = r_k(\mathbf {m}_s) \left[ \bar{R}_0(\mathbf {m}_s) \mathbf {c}^T + x \left( \frac{\partial \bar{R}_0}{\partial \mathbf {m}_{s}} (\mathbf {m}_{s}) \right) ^T \mathbf {S}_s \right] \mathbf {Z}^k , \end{aligned}$$

which can be written as

$$\begin{aligned} R^k_1(\mathbf {m}_s) = \bar{R}_0(\mathbf {m}_s) R^k_0(\mathbf {m}_s) + x Y_k(\mathbf {m}_s) \end{aligned}$$

using the definition of $Y_k(\mathbf {m}_s)$ provided in (20).

C Expressing $\mathcal {F}(\mathbf {u})$ in terms of EFM families

Using (17) to substitute for $\mathbf {B}^k$ in the effective return-on-investment (16) means that $\mathcal {F}(\mathbf {u})$ takes the form

$$\begin{aligned} \mathcal {F}(\mathbf {u}) = x \mathbf {q}^T \mathbf {e}^{\mathbf {A} \Delta t} \begin{pmatrix} \mathbf {S}_{ex} \\ \mathbf {c}^T \end{pmatrix} \sum _{k=1}^K r_k(\mathbf {m}_s) \mathbf {Z}^k u_k + \sigma H(\mathbf {u}) . \end{aligned}$$

The sum over k in the first term can be written as

$$\begin{aligned} \sum _{k=1}^K r_k(\mathbf {m}_s) \mathbf {Z}^k u_k = \sum _{J=1}^M \sum _{j \in F_J} r_j(\mathbf {m}_s) \mathbf {Z}^j u_j = \sum _{J=1}^M \left( \sum _{j \in F_J} r_j(\mathbf {m}_s) \mathbf {Z}^j \tilde{u}_j \right) U_J, \end{aligned}$$

where the second equality follows from the identity $\tilde{u}_j = u_j /U_J$ for $j \in F_J$, which implies

$$\begin{aligned} \mathcal {F}(\mathbf {u}) = \sum _{J=1}^M \left( \sum _{j \in F_J} \mathcal {R}^j_{\Delta t}(\mathbf {m}_s) \tilde{u}_j \right) U_J + \sigma H(\mathbf {u}) . \end{aligned}$$

Using the composition property (24) of entropy $H(\mathbf {u})$, one has

$$\begin{aligned} \mathcal {F}(\mathbf {u}) = \sum _{J=1}^M \left( \sum _{j \in F_J} \mathcal {R}^j_{\Delta t}(\mathbf {m}_s) \tilde{u}_j + \sigma H(\tilde{\mathbf {u}}_J)\right) U_J + \sigma H(\mathbf {U}) \end{aligned}$$

which is (25) with

$$\begin{aligned} \mathcal {F}_J(\tilde{\mathbf {u}}_J) = \sum _{j \in F_J} \mathcal {R}^j_{\Delta t}(\mathbf {m}_s) \tilde{u}_j + \sigma H(\tilde{\mathbf {u}}_J) \end{aligned}$$

the objective functional $\mathcal {F}$ restricted to the Jth family. Notice that the weighting (22) defines the effective return-on-investment for the Jth family derived directly from system (9) to be

$$\begin{aligned} \mathcal {\tilde{R}}^J_{\Delta t} (\mathbf {m}_s) = \sum _{j \in F_J} \mathcal {R}^j_{\Delta t}(\mathbf {m}_s) \tilde{u}_j \end{aligned}$$

and therefore $\mathcal {F}_J(\tilde{\mathbf {u}}_J)$ includes an entropic correction that is only zero when $\tilde{u}_J$ describes an FBA/Bang–Bang policy.