Structural identifiability and sensitivity

Abstract

Ordinary differential equation models often contain a large number of parameters that must be determined from measurements by estimation procedure. For an estimation to be successful there must be a unique set of parameters that can have produced the measured data. This is not the case if a model is not structurally identifiable with the given set of inputs and outputs. The local identifiability of linear and nonlinear models was investigated by an approach based on the rank of the sensitivity matrix of model output with respect to parameters. Associated with multiple random drawn of parameters used as nominal values, the approach reinforces conclusions regarding the local identifiability of models. The numerical implementation for obtaining the sensitivity matrix without any approximation, the extension of the approach to multi-output context and the detection of unidentifiable parameters were also discussed. Based on elementary examples, we showed that (1°) addition of nonlinear elements switches an unidentifiable model to identifiable; (2°) in the presence of nonlinear elements in the model, structural and parametric identifiability are connected issues; and (3°) addition of outputs or/and new inputs improve identifiability conditions. Since the model is the basic tool to obtain information from a set of measurements, its identifiability must be systematically checked.

Appendices

Appendix 1

Hypotheses

Observations on the system consist in measurements y_i from m samples drawn at t_i times, i = 1:m. Vectors y and t compile the above y_i, and t_i, respectively. The ultimate goal is to fit these data by a model involving p parameters x_j, j = 1:p, by maximizing the likelihood function L(x/y), where x is the vector collection of x_j. The fundamental assumption is m > p.

Three working hypotheses are commonly introduced:

• 1. Measurements y_i are considered random, obtained by adding the observation error e_i to the model output y(t_i, x), i.e., y_i = y(t_i, x) + e_i.

• 2. The error e_i follows the normal distribution e_i ∼ N(0, σ ² _i ) with zero mean and variance σ ² _i . Then, the random y_i is distributed according to y_i ∼ N[y(t_i, x), σ ² _i ]. Also, the reduced observation error

  $$\varepsilon_{i} \left( {t_{i} ,\underline{x} } \right) = \frac{{y_{i} - y\left( {t_{i} ,\underline{x} } \right)}}{{\sigma_{i} }}$$

  follows the standard normal distribution, i.e., ɛ_i(t_i, x) ∼ N(0, 1).

• 3. Observation errors are independent for different samples, i.e., E[ɛ_iɛ_j] = 0 for i ≠ j (whereas E[ɛ ² _i ] = 1).

Likelihood and sensitivity

Under the above conditions, the negative log likelihood to be minimized is

$$- \ln L\left( {{{\underline{x} } \mathord{\left/ {\vphantom {{\underline{x} } {\underline{y} }}} \right. \kern-0pt} {\underline{y} }}} \right) = \frac{1}{2} \sum\limits_{i = 1}^{m} {\ln \left( {2\pi \sigma_{i}^{2} } \right)} + \frac{1}{2} \sum\limits_{i = 1}^{m} {\varepsilon_{i}^{2} }$$

with derivative with respect to a given model parameter x_j

$$- \frac{{\partial \ln L\left( {{{\underline{x} } \mathord{\left/ {\vphantom {{\underline{x} } {\underline{y} }}} \right. \kern-0pt} {\underline{y} }}} \right)}}{{\partial x_{j} }} = \sum\limits_{i = 1}^{m} {\frac{1}{{\sigma_{i} }}\frac{{\partial \sigma_{i} }}{{\partial x_{j} }}} + \sum\limits_{i = 1}^{m} {\frac{1}{{\varepsilon_{i} }}\frac{{\partial \varepsilon_{i} }}{{\partial x_{j} }}} = \sum\limits_{i = 1}^{m} {\left( {1 - \varepsilon_{i}^{2} } \right)\frac{1}{{\sigma_{i} }}\frac{{\partial \sigma_{i} }}{{\partial x_{j} }} - \sum\limits_{i = 1}^{m} {\varepsilon_{i} \frac{1}{{\sigma_{i} }}\frac{{\partial y\left( {t_{i} ,\underline{x} } \right)}}{{\partial x_{j} }}} } .$$

(5)

The sensitivity of the variance model σ_i is

$$\rho_{ij} = \frac{{\partial \sigma_{i} }}{{\partial x_{j} }}$$

(6)

and the sensitivity of the model output y(t_i, x) at time t_i and with respect to the parameter x_j is

$$\varphi_{ij} = \frac{{\partial y\left( {t_{i} ,\underline{x} } \right)}}{{\partial x_{j} }}.$$

(7)

They are compiled in vector forms

$$\underline{\rho }_{j} = [\begin{array}{*{20}c} {\rho_{1j} } & \cdots & {\rho_{mj} } \\ \end{array} ]^{{ \, {\rm T}}} \,\,{\text{and}}\,\,\underline{\varphi }_{j} = [\begin{array}{*{20}c} {\varphi_{1j} } & \cdots & {\varphi_{mj} } \\ \end{array} ]^{{ \, {\rm T}}} .$$

The error ɛ components in relationship (5) are also presented in vector form

$$\underline{w} = \left[ {\begin{array}{*{20}c} {1 - \varepsilon_{1}^{2} } & \cdots & {1 - \varepsilon_{m}^{2} } \\ \end{array} } \right]^{{ \, {\rm T}}} \,\,{\text{and}}\,\,\underline{z} = \left[ {\begin{array}{*{20}c} {1 - \varepsilon_{1} } & \cdots & {1 - \varepsilon_{m} } \\ \end{array} } \right]^{{ \, {\rm T}}}$$

to obtain the final form of the above derivative (5)

$$- \frac{{\partial \ln L(\underline{x} /\underline{y} )}}{{\partial x_{j} }} = \underline{w}^{\rm T} \varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \underline{\rho }_{j} - \underline{z}^{\rm T} \varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \underline{\varphi }_{j} ,$$

where Σ is a m-order diagonal matrix with elements σ ² _i .

A similar derivative of ln L(x/y) with respect to another parameter x_k is considered and the expectation of the above derivatives product is$\begin{aligned} {\rm E}\left[ {\frac{{\partial \ln L(\underline{x} /\underline{y} )}}{{\partial x_{j} }}\frac{{\partial \ln L(\underline{x} /\underline{y} )}}{{\partial x_{k} }}} \right] = & \underline{\rho }_{j}^{\rm T} \varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \, {\rm E}\left[ {\underline{w} \underline{w}^{\rm T} } \right]\varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \underline{\rho }_{k} - \varphi_{j}^{\rm T} \varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \, {\rm E}\left[ {\underline{z} \underline{w}^{\rm T} } \right]\varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \underline{\rho }_{k} \\ & - \underline{\rho }_{j}^{\rm T} \varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \, {\rm E}\left[ {\underline{w} \underline{z}^{\rm T} } \right]\varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \underline{\varphi }_{k} + \varphi_{j}^{\rm T} \varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \, {\rm E}\left[ {\underline{z} \underline{z}^{\rm T} } \right]\varSigma^{{ - {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-0pt} 2}}} \underline{\varphi }_{k} . \\ \end{aligned}$

According to the rules of moments for a multivariate normal distribution [34]

$${\rm E}\left[ {\underline{w} \underline{w}^{\rm T} } \right] = 2I\quad {\rm E}\left[ {\underline{z} \underline{w}^{\rm T} } \right] = {\rm E}\left[ {\underline{w} \underline{z}^{\rm T} } \right] = 0\quad {\rm E}\left[ {\underline{z} \underline{z}^{\rm T} } \right] = I,$$

the previous expression becomes

$${\rm E}\left[ {\frac{{\partial \ln L(\underline{x} /\underline{y} )}}{{\partial x_{j} }}\frac{{\partial \ln L(\underline{x} /\underline{y} )}}{{\partial x_{k} }}} \right] = 2\underline{\rho }_{j}^{\rm T} \varSigma^{ - 1} \underline{\rho }_{k} + \underline{\varphi }_{j}^{\rm T} \varSigma^{ - 1} \underline{\varphi }_{k} .$$

(8)

Error variance model

To weight observations, the error variance model

$$\sigma = Ky^{a} \left( {t,\underline{x} } \right)$$

(9)

is commonly used. Because it involves model outputs y(t_i, x), σ depend on model parameters x and the presence of the sensitivities of the variance model ρ_ij is justified in relationship (8). Given the model (9) and definition (7), the sensitivity of the variance model (6) becomes

$$\rho_{ij} = Kay^{a - 1} \left( {t_{i} ,\underline{x} } \right)\frac{{\partial y\left( {t_{i} ,\underline{x} } \right)}}{{\partial x_{j} }} = a\frac{{\sigma_{i} }}{{y\left( {t_{i} ,\underline{x} } \right)}}\varphi_{ij}$$

or ρ_j = aΣ^1/2Y⁻¹φ_j and alternatively ρ_k = aΣ^1/2Y⁻¹φ_k with Y the diagonal m-order matrix with elements y(t_i, x).

Accordingly, the final form of relationship (8) is

$${\rm E}\left[ {\frac{{\partial \ln L\left( {{{\underline{x} } \mathord{\left/ {\vphantom {{\underline{x} } {\underline{y} }}} \right. \kern-0pt} {\underline{y} }}} \right)}}{{\partial x_{j} }}\frac{{\partial \ln L\left( {{{\underline{x} } \mathord{\left/ {\vphantom {{\underline{x} } {\underline{y} }}} \right. \kern-0pt} {\underline{y} }}} \right)}}{{\partial x_{k} }}} \right] = \varphi_{j}^{\rm T} \left[ {2a^{2} Y^{ - 2} + \varSigma^{ - 1} } \right]\underline{\varphi }_{k} = \varphi_{j}^{\rm T} D\underline{\varphi }_{k}$$

with

$$D \triangleq 2a^{2} Y^{ - 2} + \varSigma^{ - 1} .$$

(10)

By expanding the above relationship for j, k = 1:p,

$${\rm E}\left[ {\frac{{\partial \ln L\left( {{{\underline{x} } \mathord{\left/ {\vphantom {{\underline{x} } {\underline{y} }}} \right. \kern-0pt} {\underline{y} }}} \right)}}{{\partial \underline{x} }}\frac{{\partial \ln L\left( {{{\underline{x} } \mathord{\left/ {\vphantom {{\underline{x} } {\underline{y} }}} \right. \kern-0pt} {\underline{y} }}} \right)}}{{\partial \underline{x}^{\rm T} }}} \right] = \varPhi^{\rm T} \left( {\underline{t} ,\underline{x} } \right)D\left( {\underline{t} ,\underline{x} } \right) \, \varPhi \left( {\underline{t} ,\underline{x} } \right)$$

where elements φ_ij are compiled in the Φ(t, x) m × p sensitivity matrix of model outputs at t with respect to the model parameters x. Again, D(t, x) is a m-order positive definite matrix depending on model output matrix Y and on variance model of observation error matrix Σ.

Appendix 2

The benchmark pharmacokinetic model [35] is described by the set of ordinary differential equations

$$\begin{array}{*{20}c} {\frac{{dy_{1} (t)}}{dt} = \alpha_{1} \left[ {y_{2} (t) - y_{1} (t)} \right] - \frac{{k_{a} V_{m} y_{1} (t)}}{{k_{c} k_{a} + k_{c} y_{3} (t) + k_{a} y_{1} (t)}}} & {y_{1} (0) = C_{0} } \\ {\frac{{dy_{2} (t)}}{dt} = \alpha_{2} \left[ {y_{1} (t) - y_{2} (t)} \right]} & {y_{2} (0) = 0} \\ {\frac{{dy_{3} (t)}}{dt} = \beta_{1} \left[ {y_{4} (t) - y_{3} (t)} \right] - \frac{{k_{c} V_{m} y_{3} (t)}}{{k_{c} k_{a} + k_{c} y_{3} (t) + k_{a} y_{1} (t)}}} & {y_{3} (0) = \gamma C_{0} } \\ {\frac{{dy_{4} (t)}}{dt} = \beta_{2} \left[ {y_{3} (t) - y_{4} (t)} \right]} & {y_{2} (0) = 0} \\ \end{array}$$

involving nine parameters presented below with their associated domains of definition

$$\begin{array}{*{20}c} {10^{ - 2} \le \alpha_{1} \le 10} \\ {10^{ - 2} \le \alpha_{2} \le 10} \\ {10^{ - 2} \le \beta_{1} \le 10} \\ {10^{ - 2} \le \beta_{2} \le 10} \\ {10^{ - 2} \le k_{a} \le 10} \\ {10^{ - 2} \le k_{c} \le 10} \\ {10 \le V_{m} \le 10^{3} } \\ {100 \le C_{0} \le 10^{4} } \\ {0.1 \le \gamma \le 10} \\ \end{array}$$