The Bifurcation Study of 1:2 Resonance in a Delayed System of Two Coupled Neurons

Abstract

In this paper, we consider a delayed system of differential equations modeling two neurons: one is excitatory, the other is inhibitory. We study the stability and bifurcations of the trivial equilibrium. Using center manifold theory for delay differential equations, we develop the universal unfolding of the system when the trivial equilibrium point has a double zero eigenvalue. In particular, we show a universal unfolding may be obtained by perturbing any two of the parameters in the system. Our study shows that the dynamics on the center manifold are characterized by a planar system whose vector field has the property of 1:2 resonance, also frequently referred as the Bogdanov–Takens bifurcation with $Z_2$ symmetry. We show that the unfolding of the singularity exhibits Hopf bifurcation, pitchfork bifurcation, homoclinic bifurcation, and fold bifurcation of limit cycles. The symmetry gives rise to a “figure-eight” homoclinic orbit.

Appendix

1.1 Proof of Lemma 3.1

Proof

In order to study (3.18), we want to simplify it term by term. Before doing so, we show that the term $\Psi (0)(L(\epsilon )-L(0)) h(z(t),\epsilon )$ only contributes to higher order terms of $z$, which can be neglected. To use results of [21], we partially adopt its notation and rewrite

$$\begin{aligned} h(z,\epsilon )=\sum _{k=2}^m h_k(z,\epsilon )+\chi (z,\epsilon ) \end{aligned}$$

where $h_k$ ($2\leqslant k\leqslant m$) is the homogeneous part of degree $k$ and $\chi (z,\epsilon )=o(|(z,\epsilon )|^m)$. If $z=(z_1,z_2)\in \mathbb R ^2, q=(q_1,q_2), \epsilon =(\epsilon _1,\epsilon _2,\epsilon _3), \epsilon ^l=\epsilon _1^{l_1}\epsilon _2^{l_2}\epsilon _3^{l_3}$ for $l=(l_1,l_2,l_3)\in \mathbb N ^3$, and $|l|=\sum _{i=1}^3 l_i$, then we can rewrite $h_k(z,\epsilon )$ as

$$\begin{aligned} h_k(z,\epsilon )=\sum _{(q,l)\in D_k} h_{(q,l)}^k z_1^{q_1}z_2^{q_2}\epsilon ^l \quad \mathrm for \quad k\geqslant 2, \end{aligned}$$

where $h_{(q,l)}^k\in \mathbb Q$ and $D_k=\{(q,l)\in \mathbb N ^5:|(q,l)|=k\}$. Recall that $\mathbb Q$ is the subspace of $C=C([-h,0], \mathbb R ^n)$ defined in Sect. 3.1. Let $R_{(q,l)}^{k-1}\in \mathbb R ^2$ with

$$\begin{aligned} R^{k-1}(z,\epsilon )=\sum _{(q,l)\in D_k} R_{(q,l)}^{k-1} z_1^{q_1}z_2^{q_2}\epsilon ^l \end{aligned}$$

being the homogeneous part of degree $i$ of $R(z,\epsilon )=(L(\epsilon )-L(0))(\Phi z+h(z,\epsilon )) +F(\Phi z+h(z,\epsilon ),\epsilon )$. In particular, for $k=2, R^1$ is the homogeneous part of degree $2$ of $(L(\epsilon )-L(0))(\Phi z)+F(\Phi z,\epsilon )$ which is independent from terms on the center manifold and given by

$$\begin{aligned} \begin{aligned} R^{1}(z,\epsilon )&=\begin{bmatrix} \left((1+a)\tau \epsilon _2-b\tau \epsilon _3 +O(|\epsilon |^2)\right)z_1+\left(b^2(1+a)\epsilon _1 +(a+1)a\tau \epsilon _2+b\tau \epsilon _3+O(|\epsilon |^2)\right)z_2\\ \left(-b\tau \epsilon _2+(1+a)\tau \epsilon _3 +O(|\epsilon |^2)\right)z_1+\left(b^3\epsilon _1+\epsilon _3 +O(|\epsilon |^2)\right)z_2 \end{bmatrix}\\&\quad +F(\Phi z,\epsilon ). \end{aligned} \end{aligned}$$

By Theorem 3.3**** of [21], for $k=2$ and $(q,l)\in D_2$, we have

$$\begin{aligned} \begin{aligned} h^2_{(q_1,0,l)}&=h^2_{(q_1,0,l)}(0)+\begin{bmatrix}\theta&\quad \theta ^2\\ 0&\quad \theta \end{bmatrix}\Phi (0)\Psi (0)R^1_{(q_1,0,l)},\\ h^2_{(q_1,q_2,l)}&=h^2_{(q_1,q_2,l)}(0)+\int _0^{\theta } (q_1+1)h^2_{(q_1+1,q_2-1,l)}(s)\mathrm d s\\&\quad +\begin{bmatrix}\theta&\quad \theta ^2\\ 0&\quad \theta \end{bmatrix}\Phi (0)\Psi (0)R^1_{(q_1,q_2,l)},\qquad \qquad \mathrm for \qquad q_2\ne 0. \end{aligned} \end{aligned}$$

Since $R^1(z,0)=0$, we have that $h_{(q_1,0,0)}^2=0$. We can obtain that $h_2(z,\epsilon )=O(|\epsilon |)O(|z|)$. We have

$$\begin{aligned} \begin{aligned} \Psi (0)(L(\epsilon )-L(0))h(z(t),\epsilon )&=\Psi (0)(L(\epsilon )\\&\quad -L(0)) \left[h_2(z(t),\epsilon ) +h_3(z(t),\epsilon )+h_4(z(t),\epsilon )+\cdots \right]\\&=\Psi (0)(L(\epsilon )-L(0))\bigg [\sum _{k=3}^ {\infty }h^3_{(q_1,q_2,0)}z_1^{q_1}z_2^{q_2}+O(|\epsilon |)O(|z|)\bigg ]\\&=O(|\epsilon |)\bigg [\sum _{k=3}^{\infty } h^3_{(q_1,q_2,0)}z_1^{q_1}z_2^{q_2}+O(|\epsilon |)O(|z|)\bigg ]\\&=O(|\epsilon |)O(|z|^3)+O(|\epsilon |^2)O(|z|). \end{aligned} \end{aligned}$$

One can verify that for the linear terms of $\dot{z}(t)$, we have

$$\begin{aligned} \begin{aligned} (L(\epsilon )\!-\!L(0))\Phi (\theta )z(t)\!=\!\begin{bmatrix} \left[(1+a)(\tau \epsilon _2+\epsilon _1(a-1)) -b(\tau \epsilon _3+b\epsilon _1)+O(|\epsilon |^2)\right]z_1\\ +\left[(\tau \epsilon _2+\epsilon _1(a-1))(1+a) a+b(\tau \epsilon _3+b\epsilon _1)+O(|\epsilon |^2)\right]z_2\\ \left[(\tau \epsilon _3+b\epsilon _1)(1+a)-b(\tau \epsilon _2 +\epsilon _1(a+1))+O(|\epsilon |^2)\right]z_1\\ +\left[b^2(\tau \epsilon _3+b\epsilon _1) +O(|\epsilon |^2)\right]z_2\end{bmatrix}. \end{aligned} \end{aligned}$$

Let $\mathcal L (z(t),\epsilon )=\begin{bmatrix} \mathcal L _1\\\mathcal L _2\end{bmatrix}= \Psi (0)(L(\epsilon )-L(0))\Phi (\theta )z(t)$. If we denote

$$\begin{aligned} d_1=\frac{1}{3b(1+a^2)^2}\qquad \mathrm and \qquad d_2=\frac{1}{3b(1+a^2)}, \end{aligned}$$

then by using (2.6), we have

$$\begin{aligned} \begin{aligned} \mathcal L _1&=d_1\big \{ \big [ab\tau (5-3a-3a^2-3a^3)\epsilon _2-(2-3a-6a^2-3a^3)\epsilon _3 +O(|\epsilon |^2)\big ]z_1\\&\quad +\big [b^3(2-3a-3a^2-3a^3+3a^4)\epsilon _1 + b\tau (1-3a^2)(a+a^2)\epsilon _2\\&\quad +(2-4a-3a^2+3a^4)\epsilon _3+O(|\epsilon |^2)\big ]z_2\big \} \end{aligned} \end{aligned}$$

and

$$\begin{aligned} \mathcal L _2&= 3d_2\{ \big [2ab\tau \epsilon _2-2\epsilon _3+O(|\epsilon |^2)\big ]z_1\\&+\big [b(a+a^2)\tau \epsilon _2+2b^3\epsilon _1+(2-a)\epsilon _3 +O(|\epsilon |^2)\big ]z_2\}. \end{aligned}$$

For higher order terms, we have

$$\begin{aligned} \begin{aligned} F(\Phi z(t))&=\frac{\tau }{3}\begin{bmatrix}-a(1+a)^3(z_1+a z_2)^3+b^4(z_1-z_2)^3+O(|\epsilon |)O(|z|^3)\\ a b^3 z_1^3-b(1+a)^3(z_1+(a-1)z_2)^3+O(|\epsilon |)O(|z|^3)\end{bmatrix}. \end{aligned} \end{aligned}$$

Letting $\tilde{\mathcal{L }}(z(t),\epsilon )=\begin{bmatrix} \tilde{\mathcal{L }}_1\\\tilde{\mathcal{L }}_2\end{bmatrix}=\Psi (0)F(\Phi z(t))$, we have

$$\begin{aligned} \begin{aligned} \tilde{\mathcal{L }}_1&=\tau d_1(1-a)(1/3-a-a^2-a^3)[a b^3 z_1^3-b(1+a)^3(z_1+(a-1)z_2)^3]\\&+\tau d_1 b(1/3-a^2)[-a(1+a)^3(z_1+a z_2)^3+b^4(z_1-z_2)^3]+O(|\epsilon |)O(|z|^3) \end{aligned} \end{aligned}$$

and

$$\begin{aligned} \begin{aligned} \tilde{\mathcal{L }}_2=&\tau d_2 (1-a)\left[a b^3 z_1^3-b(1+a)^3(z_1+(a-1)z_2)^3\right]\\&+\tau d_2 b[-a(1+a)^3(z_1+a z_2)^3+b^4(z_1-z_2)^3]+O(|\epsilon |)O(|z|^3). \end{aligned} \end{aligned}$$

Then the equation of $z$ can be reduced to

$$\begin{aligned} \begin{aligned} \dot{z}&=Bz+\Psi (0)(L(\epsilon )-L(0))\Phi (\theta )z(t)+\Psi (0)(L(\epsilon )-L(0))h(z(t), \epsilon )\\&\quad +\Psi (0)F(\Phi (\theta )z(t)+h(z(t),\epsilon ))\\&=Bz+\mathcal L (z(t),\epsilon )+O(|\epsilon |)O(|z|^3)+O(|\epsilon |^2)O(|z|)\\&\quad +\Psi (0)F\left(\Phi (\theta )z(t)+O(|z|^3)+O(|\epsilon |)O(|z|)\right)\\&=Bz+\mathcal L (z(t),\epsilon )+\tilde{\mathcal{L }}(z(t),\epsilon ) +O(|\epsilon |^2)O(|z|)+O(|\epsilon |)O(|z|^3)+O\left(|z|^5\right), \end{aligned} \end{aligned}$$

or equivalently system (3.19) in an explicit form. $\square$

1.2 Proof of Lemma 3.2

Proof

By Lemma 3.1, we have obtained the explicit form of equations (3.19) on the center manifold. In order to get a universal unfolding of the perturbed system (3.9), we need to reduce the system further. To do that, let $u=z_1$ and $w=\dot{z}_1$.

$$\begin{aligned} \begin{aligned} w&=z_2+3 d_1\left\{ l_{110} z_1+ l_{101}z_2\right\} \\&\quad +\frac{\tau d_1}{3}\left\{ b(1-3a^2)\right.\left[-a(1+a)^3(z_1+a z_2)^3+b^4(z_1-z_2)^3\right]\\&\quad +\left.(1-4a+3a^4)\left[a b^3 z_1^3-b(1+a)^3(z_1+(a-1)z_2)^3\right] {+}O(|\epsilon |)O(|z|^3)\right\} +O(|z^5|). \end{aligned} \end{aligned}$$

Expanding $w$ and substituting $z_1=u$ gives

$$\begin{aligned} \begin{aligned} w&=z_2+3 d_1\big \{l_{110} u+ l_{101}z_2\big \}\\&\quad +\tau d_1 b(1/3-a^2)\big \{\big [-a(1+a)^3+b^4\big ]u^3 +\big [-a(1+a)^3 3a -3b^4\big ]u^2 z_2\\&\quad +\big [-a(1+a)^3 3a^2+3 b^4\big ]u z_2^2+\big [-a(1+a)^3 a^3-b^4\big ]z_2^3+O(|\epsilon |)O(|z|^3)\big \}\\&\quad +\tau d_1(1/3-4a/3+a^4)\big \{[a b^3 -b(1+a)^3] u^3-b(1+a)^3 3(a-1) u^2 z_2 \\&\quad -b(1+a)^3 3(a-1)^2 u z_2^2-b^7 z_2^3+O(\epsilon )O(|z|^3)\big \}+O(|z^5|). \end{aligned} \end{aligned}$$

Assume that $z_2=f_{10}u+f_{01}w+f_{30}u^3+f_{03}w^3+f_{21} u^2 w +f_{12}u w^2+O(|(u,w)|^4)$. Plugging in $w$ and equating the coefficients of the same term yields

$$\begin{aligned} \begin{aligned} f_{10}&=3d_1 \left[\left(2/3-a-2a^2-a^3\right)\epsilon _3-ab\tau \left(5/3-a-a^2-a^3\right)\epsilon _2\right]+O(|\epsilon |^2),\\ f_{01}&=1-3d_1\left[ b(1/3-a^2)\tau (a+a^2)\epsilon _2{+}b^3(2/3-a-a^2-a^3+a^4)\epsilon _1\right.\\&\qquad \left.+(2/3-4a/3-a^2+a^4)\epsilon _3\right] +O(|\epsilon |^2),\\ f_{30}&=-b\tau d_1\left[(1/3-a^2)(b^4{-}a(1+a)^3){+}(1/3-4a/3+a^4)(ab^2-(1{+}a)^3)\right] {+}O(|\epsilon |),\\ f_{03}&=\tau b d_1\left[(1/3-a^2)(b^4+a^4(1+a)^3)+b^6(1/3-4a/3+a^4))\right] +O(|\epsilon |),\\ f_{12}&=b\tau d_1\left[(1/3-a^2)(3a^3(1+a)^3-3b^4)+3b^4(1+a)(1/3-4a/3+a^4))\right] +O(|\epsilon |),\\ f_{21}&=b\tau d_1\left[(1/3-a^2)(3a^2(1+a)^3+3b^4)+3b^2(1+a)^2(1/3-4a/3+a^4))\right] \!+\!O(|\epsilon |). \end{aligned} \end{aligned}$$

Then we have $\dot{u}=\dot{z}_1=w$ and

$$\begin{aligned} \begin{aligned} \dot{w}&=3d_1 l_{110}\dot{z}_1+\left\{ 1+3d_1 l_{101}\right\} \dot{z}_2\\&\quad +\tau d_1(1-3a^2)\left[-a(1+a)^3 (z_1+a z_2)^2 (\dot{z}_1+a\dot{z}_2)+b^4 (z_1-z_2)^2(\dot{z}_1-\dot{z}_2)\right]\\&\quad +\tau d_1(1-4a+3a^4)\left[a b^3 z_1^2\dot{z}_1 -b(1+a)^3(z_1+(a-1)z_2)^2(\dot{z}_1+(a-1)\dot{z}_2)\right]\\&\quad +O(|\epsilon |)O(|z|^3)+O(|z|^5)\\&=3d_1 l_{110}w +\left[1+O(|\epsilon |)\right]\dot{z}_2\\&\quad +\tau d_1(1-3a^2)\left[-a(1+a)^3 (z_1^2+2 z_1 a z_2+a^2 z_2^2) (w+a\dot{z}_2)\right]\\&\quad \left.+ b^4 (z_1^2-2z_1 z_2 +z_2^2)(w-\dot{z}_2)\right]\\&\quad +\tau d_1(1-4a+3a^4)\left[a b^3 z_1^2 w -b(1+a)^3 (z_1^2+2 z_1 (a-1)z_2\right.\\&\quad \left.+(a-1)^2z_2^2)(w+(a-1)\dot{z}_2)\right]\\&\quad +O(|\epsilon |)O(|z|^3)+O(|z|^5). \end{aligned} \end{aligned}$$

Plug in $\dot{z}_2$,

$$\begin{aligned} \begin{aligned} \dot{w}&=3d_1 l_{110}w+3d_2\big \{l_{210}u +l_{201}(f_{10}u+f_{01}w+O(|(u,w)|^3))\big \}\\&\quad +\tau d_2\big \{b\big [-a(1+a)^3(z_1+a z_2)^3+b^4(z_1-z_2)^3\big ]\\&\quad +(1-a)\big [a b^3 z_1^3-b(1+a)^3(z_1+(a-1)z_2)^3\big ]\big \}\\&\quad +\tau b d_1(1-3a^2)\big [-a(1+a)^3 (z_1^2+2 z_1 a z_2+a^2 z_2^2) (w+a\dot{z}_2)\\&\quad + b^4 (z_1^2-2z_1 z_2 +z_2^2)(w-\dot{z}_2)\big ]\\&\quad +\tau d_1 (1-4a+3a^4)\big [a b^3 z_1^2 w -b(1+a)^3(z_1^2+2 z_1 (a-1)z_2\\&\quad +(a-1)^2z_2^2)(w+(a-1)\dot{z}_2)\big ]\\&\quad +O(|\epsilon |)O(|z|^3)+O(|z|^5). \end{aligned} \end{aligned}$$

Noting that $f_{10}=O(|\epsilon |)$ and $f_{01}=1+O(|\epsilon |)$, we collect the similar order terms

$$\begin{aligned} \dot{w}&= 3d_2l_{210}u +3d_1 \big [l_{110}+(1+a^2)l_{201}\big ]w\\&+\,\tau d_2\big \{b\big [-a(1{+}a)^3(u^3{+}3au^2z_2{+}3a^2uz_2^2{+}a^3 z_2^3)+b^4(u^3-3u^2z_2+3uz_2^2-z_2^3)\big ]\\&+\,(1-a)\big [a b^3 u^3-b(1+a)^3(u^3+3u^2(a-1)z_2+3u(a-1)^2z_2^2+(a-1)^3z_2^3)\big ]\big \}\\&+\,\tau b d_1(1-3a^2)\big [-a(1+a)^3(u^2+2u a z_2+a^2 z_2^2) (w+a\dot{z}_2)\\&+\, b^4 (u^2-2u z_2 +z_2^2)(w-\dot{z}_2)\big ]\\&+\,\tau (1-4a+3a^4)d_1\big [a b^3 u^2 w -b(1+a)^3(u^2+2 u (a-1)z_2\\&+\,(a-1)^2z_2^2)(w+(a-1)\dot{z}_2)\big ]\\&+\,O(|\epsilon |)O(|z|^3)+O(|z|^5). \end{aligned}$$

Note that $\dot{z}_2=O(|\epsilon |)u+O(|\epsilon |)w+O(|(u,w)|^3)$ and it only contributes $\epsilon$ small terms to the third order terms of $u$ and $v$. We can neglect it. Plugging in the near-identity transformation

$$\begin{aligned} z_2=f_{10}u+f_{01}w+O(|(u,w)|^3)=w+O(|\epsilon |)u +O(|\epsilon |)w+O(|(u,w)|^3), \end{aligned}$$

we have

$$\begin{aligned} \begin{aligned} \dot{w}&=3d_2 l_{210}u +3d_1\big [2b^3(1+a^2)\epsilon _1+8/3ab\tau \epsilon _2+(4a^2+4/3) \epsilon _3+O(|\epsilon |^2)\big ]w\\&\quad +\tau d_2\big \{-ab(1+a)^3(u^3+3a u^2w+3a^2uw^2+a^3 w^3)\\&\quad +b^5(u^3-3u^2w+3uw^2-w^3)\\&\quad +a(1-a) b^3 u^3-b(1-a)(1+a)^3\big [u^3+3(a-1)u^2w\\&\quad +3(a-1)^2uw^2+(a-1)^3w^3\big ]\big \}\\&\quad +\tau b d_1(1-3a^2)\big [-a(1+a)^3 (u^2w+2auw^2+a^2 w^3) + b^4 (u^2w-2u w^2 +w^3)\big ]\\&\quad +\tau (1-4a+3a^4)d_1\big [a b^3 u^2 w -b(1+a)^3 (u^2w+2 u (a-1)w^2+(a-1)^2w^3)\big ]\\&\quad +O(|\epsilon |)O(|(u,w)|^3)+O(|(u,w)|^5). \end{aligned} \end{aligned}$$

Collecting the coefficients of common terms gives

$$\begin{aligned} \begin{aligned} \dot{w}&=3d_2l_{110}u +3d_1\left[2b^3(1+a^2)\epsilon _1+8/3ab\tau \epsilon _2 +(4a^2+4/3)\epsilon _3+O(|\epsilon |^2)\right]w\\&+g_{30}u^3+g_{21}u^2w+g_{12}uw^2+g_{03}w^3, \end{aligned} \end{aligned}$$

with

$$\begin{aligned} \begin{aligned} g_{30}&=\tau d_2\big \{-ab(1+a)^3+b^5+a(1-a) b^3 -b(1-a)(1+a)^3\big \}+O(|\epsilon |)\\&=\tau b d_2\big \{b^2(a-1)-(1+a^3)\big \}+O(|\epsilon |)\\&=-\frac{4}{3}\frac{a}{(a-1)(a^2+1)}+O(|\epsilon |),\\ g_{21}&=\tau d_2\big \{-3a^2b(1+a)^3 -3b^5 + 3b(1+a)^3 (a-1)^2\big \}\\&\quad +\tau bd_1(1-3a^2)\big \{-a(1+a)^3+ b^4\big \}\\&\quad +\tau b d_1(1-4a+3a^4)\big \{ab^2-(1+a)^3\big \}+O(|\epsilon |)\\&=-\frac{2}{3}\frac{(9a^4-1)a}{(1+a^2)^2(a-1)}+O(|\epsilon |),\\ \end{aligned} \end{aligned}$$

and

$$\begin{aligned} \begin{aligned} g_{12}&=\tau d_2\left\{ -ab(1+a)^3 3a^2+3b^5 + 3b(1+a)^3 (a-1)^3\right\} \\&\qquad +\tau d_1\left\{ b(1/3-a^2)(-3a)(1+a)^3 2a-6 (1/3-a^2) b^5\right.\\&\left.\qquad -\,6b(a-1)(1/3-4a/3+a^4)(1+a)^3\right\} +O(|\epsilon |)\\&=-\frac{1}{3}\frac{a(1+a)(6a^5+3a^4-21a^3+15a^2-3a+4)}{(1+a^2)^2(a-1)} +O(|\epsilon |),\\ g_{03}&=\tau d_2\left\{ -ab(1+a)^3 a^3-b^5 + b(1+a)^3 (a-1)^4\right\} \\&\qquad +\tau d_1\left\{ b(1/3-a^2)(-3a^3)(1+a)^3+b(1/3-a^2)3b^4\right.\\&\left.\qquad -(1/3-4a/3+a^4)3b(1+a)^3(a-1)^2\right\} +O(|\epsilon |)\\&=\tau b d_2\left\{ -b^4+(a-1)b^6-a^4(1+a)^3\right\} \\&\qquad +\tau b d_1\left\{ (1-3a^2)b^4-(1-4a+a^3)(1+a)^3(a-1)^2\right.\\&\qquad \left.-\,a^3(1-3a^2)(1+a)^3\right\} +O(|\epsilon |)\\&=-\frac{1}{3}\frac{a(1+a)(3a^5+a^4-7a^3-a^2-2a+2)}{(1+a^2)^2} +O(|\epsilon |). \end{aligned} \end{aligned}$$

Therefore

$$\begin{aligned} \left\{ \begin{array}{ll} \dot{u}&=\ \ w,\\ \dot{w}&=\ \ 3d_2[2ab\tau \epsilon _2-2\epsilon _3+O(|\epsilon |^2)]u \\ &\ \ \ \ \ +3d_1\left[2b^3(1+a^2)\epsilon _1+8/3ab\tau \epsilon _2 +(4a^2+4/3)\epsilon _3+O(|\epsilon |^2)\right]w\\ &\ \ \ \ \ +g_{30}u^3+g_{21}u^2w+g_{12}uw^2+g_{03}w^3+O(|(u,w)|^5), \end{array}\right. \end{aligned}$$

which is system (3.20). $\square$

1.3 Proof of Lemma 3.3

Proof

In order to simplify the third order terms in the equation of $\omega$ in system (3.20), we use the method of Perko [27, p. 164–167] and take the near-identity transformation with undetermined coefficients $x$ and $\phi$ as

$$\begin{aligned} \begin{bmatrix}u\\ w\end{bmatrix}=\begin{bmatrix}y_1\\ y_2\end{bmatrix}+h_3(y_1,y_2), \end{aligned}$$

where

$$\begin{aligned} h_3(y_1,y_2)=\begin{bmatrix}h_{31}(y_1,y_2)\\ h_{32}(y_1,y_2)\end{bmatrix}=\begin{bmatrix} x_{30}y_1^3+x_{21}y_1^2y_2+x_{12}y_1y_2^2 +x_{30}y_2^3\\ \phi _{30}y_1^3+\phi _{21}y_1^2y_2+\phi _{12}y_1y_2^2+\phi _{30}y_2^3 \end{bmatrix}\!. \end{aligned}$$

Then system (3.20) is reduced to

$$\begin{aligned} \dot{y}=J y +\tilde{F}_3(y)+O(|(y_1,y_2)|^5), \end{aligned}$$

(5.1)

where

$$\begin{aligned} \begin{aligned} \tilde{F}_3(y)&=\begin{bmatrix}\tilde{F}_{31}\\ \tilde{F}_{32}\end{bmatrix}=J h_3(y)-D h_3(y) J y +F_3(y)\\&=\begin{bmatrix}0,&1\\3d_2[2ab\tau \epsilon _2-2\epsilon _3],&3d_2\left[2b^3(1+a^2)\epsilon _1+8/3ab\tau \epsilon _2 +(4a^2+4/3)\epsilon _3\right]\end{bmatrix}\begin{bmatrix}h_{31}(y_1,y_2)\\ h_{32}(y_1,y_2)\end{bmatrix}\\[1ex]&\qquad -\begin{bmatrix} x_{30}3y_1^2+x_{21}2y_1y_2+x_{12}y_2^2,&x_{21}y^2_1 +2x_{12}y_1y_2+3x_{03}y_2^2\\ \phi _{30}3y_1^2+\phi _{21}2y_1y_2+\phi _{12}y_2^2,&\phi _{21}y^2_1+2\phi _{12}y_1y_2+3\phi _{03}y_2^2 \end{bmatrix} J y\\&\qquad +\begin{bmatrix}0\\ g_{30}y_1^3+g_{21}y_1^2y_2+g_{12}y_1y_2^2+g_{03}y_2^3\end{bmatrix}\\ \end{aligned} \end{aligned}$$

After simplification, we have

$$\begin{aligned} \begin{aligned} \tilde{F}_{31}=&h_{32}(y_1,y_2)-(x_{30}3y_1^2y_2+x_{21}2y_1y_2^2 +x_{12}y_2^3)\\&-(J_{21}y_1+J_{22}y_2)(x_{21}y^2_1+2x_{12}y_1y_2+3x_{03}y_2^2)\\ \tilde{F}_{32}=&J_{21}h_{31}+J_{22}h_{32} -(\phi _{30}3y_1^2y_2+\phi _{21}2y_1y_2^2+\phi _{12}y_2^3)\\&+(J_{21}y_1+J_{22}y_2)(\phi _{21}y^2_1+2\phi _{12}y_1y_2+3\phi _{03}y_2^2 \\&+ g_{30}y_1^3+g_{21}y_1^2y_2+g_{12}y_1y_2^2+g_{03}y_2^3. \end{aligned} \end{aligned}$$

The near-identity transformation does not change the linear part of the system (3.3), but only has impact on the higher order terms. We want to select coefficients $x$ and $\phi$ so that the higher order terms of the $y_1$ equation disappear i.e. $\tilde{F}_{31}=0$ and the higher order terms of the $y_2$ equation is as simple as possible. To be specific, we force the coefficients of $y_1y_2^2$ and $y_2^3$ in $\tilde{F}_{32}$ and all coefficients of $y_1^3, y_1^2y_2, y_1y_2^2$, and $y_2^3$ in $\tilde{F}_{31}$ to be zero. We have six equations and $8$ undetermined coefficients $x$ and $\phi$ in total. There exists infinitely many solutions. For the sake of simplicity, we let $x_{03}=0$ and $\phi _{03}=0$ and obtain a set of coefficients given by:

$$\begin{aligned} \begin{aligned}&x_{30}=\frac{1}{6}g_{12}-\frac{1}{3}J_{22}g_{03},&x_{21}=\frac{1}{2}g_{03},&\qquad x_{12}=0,&x_{03}=0,\\&\phi _{30}=J_{21},&\phi _{21}=\frac{1}{2}g_{12}+\frac{3}{2}J_{22}g_{03},&\qquad \phi _{12}=g_{03},&\phi _{03}=0. \end{aligned} \end{aligned}$$

Therefore

$$\begin{aligned} \begin{bmatrix}h_{31}(y_1,y_2)\\ h_{32}(y_1,y_2)\end{bmatrix}=\begin{bmatrix}(\frac{1}{6}g_{12} -\frac{1}{3}J_{22}g_{03})y_1^3+\frac{1}{2}g_{03}y_1^2y_2\\ J_{21}y_1^3+(\frac{1}{2}g_{12}+\frac{3}{2}J_{22}g_{03})y_1^2y_2 +g_{03}y_1y_2^2\end{bmatrix}. \end{aligned}$$

The system (5.1) is transformed to

$$\begin{aligned} \begin{bmatrix} \dot{y}_1\\\dot{y}_2 \end{bmatrix} =J \begin{bmatrix} y_1\\ y_2\end{bmatrix}+\begin{bmatrix} 0\\ (g_{30}+O(|\epsilon |))y_1^3+(g_{21}+O(|\epsilon |))y_1^2y_2 +O(|(y_1,y_2)|^5)\end{bmatrix} \end{aligned}$$

(5.2)

Plugging in $d_1, d_2, g_{30}$, and $g_{21}$ gives system (3.21). $\square$