Optimal phase control of biological oscillators using augmented phase reduction

Abstract

We develop a novel optimal control algorithm to change the phase of an oscillator using a minimum energy input, which also minimizes the oscillator’s transversal distance to the uncontrolled periodic orbit. Our algorithm uses a two-dimensional reduction technique based on both isochrons and isostables. We develop a novel method to eliminate cardiac alternans by connecting our control algorithm with the underlying physiological problem. We also describe how the devised algorithm can be used for spike timing control which can potentially help with motor symptoms of essential and parkinsonian tremor, and aid in treating jet lag. To demonstrate the advantages of this algorithm, we compare it with a previously proposed optimal control algorithm based on standard phase reduction for the Hopf bifurcation normal form, and models for cardiac pacemaker cells, thalamic neurons, and circadian gene regulation cycle in the suprachiasmatic nucleus. We show that our control algorithm is effective even when a large phase change is required or when the nontrivial Floquet multiplier is close to unity; in such cases, the previously proposed control algorithm fails.

Appendices

A Numerical Methods

In this appendix, we give details on the numerical methods we used to compute the Floquet multipliers, PRC, and IRC, and solve the Euler Lagrange equations and the full model equations.

1.1 A.1 Computation of PRC

For the normal form of the Hopf bifurcation, we can compute the PRC and its derivative w.r.t. $\theta$ analytically, see, e.g., Brown et al. (2004). For computing the PRCs (and their derivatives w.r.t. $\theta$) of the YNI, thalamic neuron, and the clock gene regulation model, we use the XPP package (Ermentrout 2002), which is widely used by the community working on nonlinear oscillators. This package solves the appropriate adjoint equation backward in time along the periodic orbit to compute the PRC as a function of time. We scale the PRC computed by this package by $\omega$, as we consider PRC as $\mathcal {Z}(\theta )=\frac{\partial \theta }{\partial \mathbf {x}}$, whereas the computed PRC from the XPP package is $\tilde{\mathcal {Z}}(t)=\frac{\partial t}{\partial \mathbf {x}}$. Note that the XPP computes the derivative of the PRC w.r.t. time $\left( \dot{\tilde{\mathcal {Z}}}(t)=\frac{\partial ^2 t}{\partial \mathbf {x}\partial t}\right)$, which is numerically equivalent to its derivative w.r.t. $\theta$$\left( \mathcal {Z}'(\theta )=\frac{\partial ^2 \theta }{ \partial \mathbf {x}\partial \theta }\right)$. The XPP package gives the PRC and its derivative as a time series. After appropriately scaling the time series, we write them as an analytical expression of $\theta$ by approximating them as a finite Fourier series, to be used in the numerical computation of the Euler–Lagrange equations.

1.2 A.2 Computation of Floquet multipliers

Once the PRC has been computed, we choose an arbitrary point on the periodic orbit as $\theta =0$ and approximate the isochron $\varGamma _0$ as an $n-1$ dimensional hyperplane orthogonal to the PRC at that point. To compute the Jacobian DF, we compute $\mathbf {x}_\varGamma ^j$ (as defined beneath Eq. 8 in the main text) for a large j, for a number of initial conditions $\mathbf {x}_0$ spread out on the isochron. Eigenvector decomposition of DF gives us the Floquet multipliers of the periodic orbit and the corresponding Floquet exponents $k_i$. Note that for planar systems, the nontrivial Floquet exponent can be directly computed from the divergence of the vector field as (Glendinning 1994)

$$\begin{aligned} k=\frac{\int _0^T{ \nabla \cdot F(\gamma (t))\mathrm{d}t}}{T}. \end{aligned}$$

(43)

1.3 A.3 Two point boundary value problem with Newton iteration

We calculate the IRC and solve the Euler–Lagrange equations as a two point boundary value problem using Newton iteration, which we briefly summarize. Consider a general two point boundary value problem

$$\begin{aligned} \dot{y} = f(t,y), \qquad y \in \mathbb {R}^n, \qquad 0\le t \le b, \end{aligned}$$

(44)

with the linear boundary condition

$$\begin{aligned} B_0y(0)+B_by(b)=a, \qquad B_0,\ B_b \in \mathbb {R}^{n\times n}. \end{aligned}$$

To solve such a boundary value problem, we integrate Eq. (44) with the initial guess $c=y(0)$ and calculate the function g(c):

$$\begin{aligned} g(c)=B_0c+B_by(b)-a, \end{aligned}$$

where y(b) is the solution at time b with the initial condition c. If we had chosen the correct initial condition c, g(c) would be 0. Based on the current guess $c^\nu$, and the $g(c^\nu )$ value, we choose the next initial condition by the Newton Iteration as

$$\begin{aligned} c^{\nu +1}=c^\nu -\left( \left. \frac{\partial g}{\partial c}\right| _{c^\nu }\right) ^{-1}g(c^\nu ). \end{aligned}$$

(45)

We compute the Jacobian $J=\left. \frac{\partial g}{\partial c}\right| _{c^\nu }$ numerically as

$$\begin{aligned} J_i=\frac{g^+-g^-}{2\epsilon }, \end{aligned}$$

where

$$\begin{aligned} g^+= & {} g\left( c^\nu +e_i \epsilon \right) ,\\ g^-= & {} g\left( c^\nu -e_i \epsilon \right) , \end{aligned}$$

$J_i$ is the $i\mathrm{th}$ column of J, $\epsilon$ is a small number, and $e_i$ is a column vector with 1 in the $i\mathrm{th}$ position and 0 elsewhere.

1.3.1 A.3.1 Computation of IRC

To calculate the IRC, we first compute and save the periodic solution $\gamma (t)$ using Matlab’s ODE solver ode45 with a relative error tolerance of $3e-12$, and an absolute error tolerance of $1e-15$. The next step is to solve the adjoint equation

$$\begin{aligned} \dot{\mathcal I} =\left( k_iI-DF(\gamma (t))^T\right) {\mathcal I}, \qquad 0\le t \le T, \end{aligned}$$

with periodic boundary conditions

$$\begin{aligned} {\mathcal I}(0)={\mathcal I}(T). \end{aligned}$$

We choose an initial guess ${\mathcal I}(0)$, and integrate the adjoint equation using Matlab’s ODE solver ode45 with a relative error tolerance of $3e-12$, and an absolute error tolerance of $1e-15$. For Newton iteration, we take

$$\begin{aligned} c^\nu= & {} {\mathcal I}(0), \end{aligned}$$

(46)

$$\begin{aligned} g(c^\nu )= & {} \underbrace{I}_{B_0}{\mathcal I}(0)-\underbrace{I}_{B_b}{\mathcal I}(T), \nonumber \\\Rightarrow & {} g(c^\nu )={\mathcal I}(0)-{\mathcal I}(T), \end{aligned}$$

(47)

$$\begin{aligned} \left. \frac{\partial g}{\partial c}\right| _{c^\nu }= & {} I-J, \end{aligned}$$

(48)

where I is the identity matrix, and J is the Jacobian matrix

$$\begin{aligned} J=\frac{\partial {\mathcal I}(T)}{\partial {\mathcal I}(0)}, \end{aligned}$$

which we compute numerically. We use Eqs. (46)–(48) together with Eq. (45) to compute the next initial condition. Once a periodic solution is obtained, the computed IRC is scaled by the normalization condition $\nabla _{\mathbf {x_0}}\psi _i \cdot v_i=1$ (Wilson and Moehlis 2016). Its derivative w.r.t. $\theta$ is obtained numerically by a central difference scheme

$$\begin{aligned} \mathcal {I}'(\theta _i)=\frac{\mathcal {I}(\theta _{i+1})-\mathcal {I}(\theta _{i-1})}{\theta _{i+1}-\theta _{i-1}}. \end{aligned}$$

The obtained IRC and its derivative w.r.t. $\theta$ are written as analytical expressions of $\theta$ by a finite Fourier series approximation, which is used in the computation of the Euler–Lagrange equations.

1.3.2 A.3.2 Solving Euler–Lagrange equations

For Euler Lagrange equations based on augmented phase reduction, we set the boundary conditions as $\theta (0)=0, \ \theta (T_1)=2\pi ,\ \psi (0)=0,\ \psi (T_1)=0$. We can write this as a two point boundary value problem with the function g as

$$\begin{aligned} g(c)= & {} \underbrace{\begin{bmatrix}1&0&0&0\\0&1&0&0\\0&0&0&0\\0&0&0&0\end{bmatrix}}_{B_0}\underbrace{\begin{bmatrix}\theta (0)\\ \psi (0) \\ \lambda _1 (0)\\ \lambda _2 (0)\end{bmatrix}}_{c}+\underbrace{\begin{bmatrix}0&0&0&0\\0&0&0&0\\1&0&0&0\\0&1&0&0\end{bmatrix}}_{B_b}\begin{bmatrix}\theta (T_1)\\ \psi (T_1) \\ \lambda _1 (T_1)\\ \lambda _2 (T_1)\end{bmatrix}-\begin{bmatrix}0\\0 \\ 2\pi \\ 0\end{bmatrix},\\\Rightarrow & {} g(c)=\begin{bmatrix}0\\0 \\ \theta (T_1)-2\pi \\ \psi (T_1)-0\end{bmatrix}. \end{aligned}$$

Since $\theta (0)$, and $\psi (0)$ are fixed by our problem, g can be influenced by changing $\lambda _1(0)$ and $\lambda _2(0)$ only. So we get the following matrices for Newton Iteration:

$$\begin{aligned} c^\nu= & {} \begin{bmatrix}\lambda _1 (0)\\ \lambda _2 (0)\end{bmatrix}, \end{aligned}$$

(49)

$$\begin{aligned} g(c^\nu )= & {} \begin{bmatrix}\theta (T_1)-2\pi \\ \psi (T_1)\end{bmatrix}, \end{aligned}$$

(50)

$$\begin{aligned} \left. \frac{\partial g}{\partial c}\right| _{c^\nu }= & {} \begin{bmatrix}\frac{\partial \theta (T_1)}{\partial \lambda _1 (0)}&\frac{\partial \theta (T_1)}{\partial \lambda _2 (0)}\\ \frac{\partial \psi (T_1)}{\partial \lambda _1 (0)}&\frac{\partial \psi (T_1)}{\partial \lambda _2 (0)}\end{bmatrix}. \end{aligned}$$

(51)

In a similar way, we get the following matrices for Euler–Lagrange equations based on standard phase reduction:

$$\begin{aligned} c^\nu= & {} \lambda _1 (0), \end{aligned}$$

(52)

$$\begin{aligned} g(c^\nu )= & {} \theta (T_1)-2\pi , \end{aligned}$$

(53)

$$\begin{aligned} \left. \frac{\partial g}{\partial c}\right| _{c^\nu }= & {} \frac{\partial \theta (T_1)}{\partial \lambda _1 (0)}. \end{aligned}$$

(54)

All the integrations are done with Matlab ODE solver ode45 with relative error tolerance $\le 1e-10$ and absolute error tolerance $\le 1e-10$.

B Models

In this appendix, we give details of the mathematical models used and also their augmented and standard phase reduction models, which are necessary to reproduce the results of this article.

1.1 B.1 YNI model

Here we list the both full and reduced model parameters of the YNI model (Yanagihara et al. 1980) introduced in Sect. 4.2.2.

1.1.1 B.1.1 Full model equations and parameters

The full YNI model is given as

$$\begin{aligned} \dot{V}= & {} \frac{I_{m}-I_{Na}-I_k-I_l-I_s-I_h}{C}+u(t),\\ \dot{d}= & {} \alpha _d(1-d)-\beta _d d,\\ \dot{f}= & {} \alpha _f(1-f)-\beta _f f,\\ \dot{m}= & {} \alpha _m(1-m)-\beta _m m,\\ \dot{h}= & {} \alpha _h(1-h)-\beta _h h,\\ \dot{q}= & {} \alpha _q(1-q)-\beta _q q,\\ \dot{p}= & {} \alpha _p(1-p)-\beta _p p, \end{aligned}$$

where

$$\begin{aligned} \alpha _d= & {} \frac{0.01045(V+35)}{(1-\exp (-(V+35)/2.5))+\frac{0.03125V}{(1-\exp (-V/4.8))}},\\ \beta _d= & {} 0.00421(V-5)/(-1+\exp ((V-5)/2.5)),\\ \alpha _f= & {} 0.000355(V+20)/(-1+\exp ((V+20)/5.633)),\\ \beta _f= & {} 0.000944(V+60)/(1+\exp (-(V+29.5)/4.16)),\\ \alpha _m= & {} (V+37)/(1-\exp (-(V+37)/10)),\\ \beta _m= & {} 40\exp (-0.056(V+62)),\\ \alpha _h= & {} 0.001209(\exp (-(V+20)/6.534)),\\ \beta _h= & {} 1/(1+\exp (-(V+30)/10)),\\ \alpha _q= & {} 0.0000495+\frac{0.00034(V+100)}{(-1+\exp ((V+100)/4.4))},\\ \beta _q= & {} 0.0000845+0.0005(V+40)/(1-\exp (-(V+40)/6)),\\ \alpha _p= & {} 0.0006+0.009/(1+\exp (-(V+3.8)/9.71)),\\ \beta _p= & {} 0.000225(V+40)/(-1+\exp ((V+40)/13.3)),\\ i_s= & {} 12.5(\exp ((V-30)/15)-1),\\ I_s= & {} (0.95d+0.05)(0.95f+0.05)i_s,\\ I_{Na}= & {} 0.5m^3h(V-30),\\ I_h= & {} 0.4q(V+25),\\ I_k= & {} 0.7p(\exp (0.0277(V+90))-1)/\exp (0.0277(V+40)), \end{aligned}$$

$$\begin{aligned} I_l= & {} 0.8(-\exp (-(V+60)/20)+1),\\ C= & {} 1,\\ I_m= & {} 1.0609. \end{aligned}$$

1.1.2 B.1.2 Reduced model equations and parameters

For the augmented and standard phase reduction of the YNI model, we get $\omega =0.03088$, $k=-0.00135$. Once the PRC, IRC, and their derivatives w.r.t. $\theta$ are numerically computed (see “Appendix A”), we approximate them as finite Fourier series to be used as an analytical function in the numerical computation of Euler–Lagrange equations. $\theta =0$ corresponds to the initial condition $V=-19.2803, \ d= 0.6817, \ f=0.0236, \ m=0.8540, \ h=0.0013, \ q=0.0038, \ p= 0.6592$.

1.2 B.2 Thalamic neuron model

Here we list both full and reduced model parameters of the thalamus model (Rubin and Terman 2004) used in Section 4.3.2

1.2.1 B.2.1 Full model equations and parameters

The full thalamic neuron model is given as

$$\begin{aligned} \dot{v}= & {} \frac{-I_L-I_{Na}-I_K-I_T+I_b}{C_m}+u(t),\\ \dot{h}= & {} \frac{h_{\infty }-h}{\tau _h},\\ \dot{r}= & {} \frac{r_{\infty }-r}{\tau _r}, \end{aligned}$$

where

$$\begin{aligned} h_\infty= & {} 1/(1+\exp ((v+41)/4)),\\ r_\infty= & {} 1/(1+\exp ((v+84)/4)),\\ \alpha _h= & {} 0.128\exp (-(v+46)/18),\\ \beta _h= & {} 4/(1+\exp (-(v+23)/5)),\\ \tau _h= & {} 1/(\alpha _h+\beta _h),\\ \tau _r= & {} (28+\exp (-(v+25)/10.5)),\\ m_\infty= & {} 1/(1+\exp (-(v+37)/7)),\\ p_\infty= & {} 1/(1+\exp (-(v+60)/6.2)),\\ I_L= & {} g_L(v-e_L),\\ I_{Na}= & {} g_{Na}({m_\infty }^3)h(v-e_{Na}),\\ I_K= & {} g_K((0.75(1-h))^4)(v-e_K),\\ I_T= & {} g_T(p_\infty ^2)r(v-e_T),\\ C_m= & {} 1,\ g_L = 0.05,\ e_L = -70,\ g_{Na} = 3,\ e_{Na} = 50,\\g_K= & {} 5,\ e_K = -90,\ g_T = 5,\ e_T = 0, \ I_b = 5. \end{aligned}$$

1.2.2 B.2.2 Reduced model equations and parameters

For the augmented and standard phase reduction of the thalamic neuron model, we get $\omega =0.7484$, $k=-0.0225$. Once the PRC, IRC, and their derivatives w.r.t. $\theta$ are numerically computed (see “Appendix A”), we approximate them as finite Fourier series to be used as an analytical function in the numerical computation of Euler–Lagrange equations. $\theta =0$ corresponds to the initial condition $v=-57.5298,\ h=0.1424, \ r= 0.0017$.

1.3 B.3 Clock gene regulation model

Here we list the both full and reduced model parameters of the clock gene regulation model (Gonze et al. 2005) used in Sect. 4.4.2.

1.3.1 Full model equations and parameters

The full thalamus model is given as

$$\begin{aligned} \dot{X}= & {} v_1\frac{K_1^4}{K_1^4+Z^4}-v_2\frac{X}{K_2+X} + L(t),\\ \dot{Y}= & {} k_3X-v_4\frac{Y}{K_4+Y},\\ \dot{Z}= & {} k_5Y-v_6\frac{Z}{K_6+Z},\\ v_1= & {} 0.7,\ v_2 =0.35,\ v_4 = 0.35,\ v_6 = 0.35,\\ K_1= & {} 1,\ K_2 = 1,\ K_6 = 1,\ k_3 = 0.7,\ k_5 = 0.7. \end{aligned}$$

1.3.2 Reduced model equations and parameters

For the augmented and standard phase reduction of the clock gene regulation model, we get $\omega =0.2669$, $k=-0.0021$. Once the PRC, IRC, and their derivatives w.r.t. $\theta$ are numerically computed (see “Appendix A”), we approximate them as finite Fourier series to be used as an analytical function in the numerical computation of Euler–Lagrange equations. $\theta =0$ corresponds to the initial condition $X=0.1948, \ Y=0.4154,\ Z=1.8530$.