1 Introduction

Consider the autonomous Hamiltonian system with n degrees of freedom

$$\begin{aligned} \dot{\textbf{x}}=\mathcal {H}_{\textbf{y}}\left( \textbf{x},\textbf{y}\right) ,\qquad \dot{\textbf{y}}=-\mathcal {H}_{\textbf{x}}\left( \textbf{x},\textbf{y}\right) , \end{aligned}$$
(1)

where \(\mathcal {H}=\mathcal {H}\left( \textbf{x},\textbf{y}\right) \) is an analytic function of \(\left( \textbf{x},\textbf{y}\right) \in U\) an open set of \(\mathbb {R}^{n}\times \mathbb {R}^{n}\) such that \(({\textbf {0}},\textbf{0}) \in U\). We deal with elliptic equilibria in the system Eq. 1 (that is, all the eigenvalues of the linearized system are on the imaginary axis \(\pm i \omega _j\), \(j=1, \ldots , n\) and the linear part is semi-simple), assuming the existence of two resonance vectors, without interaction, that is, the vector of frequencies \(\varvec{\omega }=(\omega _1, \ldots , \omega _n)\) is resonant. We establish conditions to determine their formal stability, also called Lie stability (Lyapunov stability of the truncated Hamiltonian system in any arbitrary order) and we give sufficient conditions to prove instability in the sense of Lyapunov.

According to the literature, Lyapunov stability (or topological stability) of the equilibria of Hamiltonian systems is one of the oldest problems in mathematical physics. The important contributions to the understanding of this problem, dating back to the 18th century, form a fundamental part of the foundation and the evolution of the theory of dynamical systems and Celestial Mechanics up to our days.

To characterize the type of stability of the equilibrium solution associated with Eq. 1 is very difficult when the Hamiltonian system has more than two degrees of freedom (2-DOF). Arnold in [2] states that for n-degrees of freedom (n-DOF) with \( n \ge 3 \), the study of stability is an open problem and until today it remains so. If we have a Hamiltonian system with two degrees of freedom in action-angle variables, according to the KAM theory [3] and due to Arnold’s result we have the following theorem: In an isoenergetically non-degenerate system with 2-DOF, for all initial conditions, the action variables remain forever near their initial values. This theorem implies the KAM stability, in classical KAM theory, KAM stability is established when the Birkhoff Normal form has an isoenergetically non-degenerate Hessian (see [3] or [12]). Note that here, its phase space is four-dimensional and consequently the energy levels are three dimensional, therefore a 2-dimensional torus on an energy level separates the 3-dimensional phase space and this implies that solutions between two tori are confined by them, hence when we have 2-DOF, KAM stability coincides with the stability in the Lyapunov sense. However, for more degrees of freedom, the Lyapunov stability cannot be ensured under any non-degeneracy condition of the KAM theory. In particular, for 3-DOF, a 3-dimensional KAM invariant torus do not separate the 5-dimensional energy level as before, then there can exist irregular orbits traveling between the tori and these orbits could be unstable, which could lead to the Arnold diffusion [4].

Moser in [23] affirms that the instability can only occur if there are resonance relations. Rather, if there are resonance relations it is known that instabilities are more likely to happen and algebraic examples are known long ago (see [7, 17]). The study in the case of simple resonance for Hamiltonian systems with \(n-\)DOF (\(n\ge 3\)) has already been completed (see [8, 10, 11, 16]), but when there exist relations of multiple resonances there are still open problems that have not been addressed due to their difficulty.

Summarizing, the study of the stability of equilibrium solutions in the Lyapunov sense is a classic topic, but not easy, in fact, it is a complex problem if we consider, for example: the number of degrees of freedom, the type of resonance that occurs (see also [25]) and the possible phenomena such as Arnold diffusion that could be present. On the other hand, in some applied problems of Celestial Mechanics and others branches, the study of stability of some equilibrium solutions leads us to situations such as those mentioned above, reason, it is necessary to advance in obtaining analytical results for more than two degrees of freedom.

Precisely, in this work we intend to advance from an analytical point of view obtaining new results for the nonlinear stability in the Lyapunov sense of one equilibrium solution in an autonomous Hamiltonian system with n-degrees of freedom, assuming the existence of two resonance vectors, \(\textbf{k}^1\) and \(\textbf{k}^2\) (\(|\textbf{k}^1|<|\textbf{k}^2|\)). In fact, we provide the necessary conditions to get a type of formal stability, called Lie stability, these conditions depend on the components of the resonance vectors and the coefficients of the Hamiltonian function. On the other hand, considering another approach, we are able to guarantee sufficient conditions in order to obtain exponential stability in the sense of Nekhoroshev for Lie stable systems. Finally, our main results are associated with the proof of the instability of the equilibrium solution in the Lyapunov sense of the full system. For this purpose, we assume that the components of at least one of resonance vector change of sign, and we construct a suitable Chetaev function. Moreover, we are able to characterize a convenient connected component where the standard Chetaev’s Theorem applies. In particular, in the interior of this connected component we give a characterization of the action variables which permit us to determine the sign of the derivative of the Chetaev function along the solutions of the full Hamiltonian system.

Since the resonance vectors have not interaction, we may tend to think that the Hamiltonian system can be written as two uncoupled Hamiltonian systems, but this is not always the case, for example, if the resonance vectors have the same even order and the components of at least one resonance vector do not change of sign, then the Hamiltonian system will not be decoupled with respect to the action variables, and in this instance, the system can be Lie stable (stable in the Lyapunov sense for the system truncated in any arbitrary order) or unstable Lyapunov. In this sense, we emphasize that to determine the Lie stability or instability in the Lyapunov sense of an elliptic equilibrium point when there are two vectors of resonance is still an open and complicated problem, and we must have in consideration the following aspects: the type of interaction that occurs between the resonance vectors (this is reflected in the computation of the Lie normal form and the characterization of the certain formal first integrals); the change of sign that can occur between the components of the resonance vectors; the coefficients of the normalized Hamiltonian function.

Our approach is divided into several steps. First, in Section 2 we characterize the \(\mathbb {Z}\)-module of the integer vectors which are orthogonal to the vector of frequency \(\varvec{\omega }\). Second, the Lie normal form Hamiltonian is computed up to a suitable order in Section 3. Third, in order to study the stability we characterize a linear subspace of \(\mathbb {R}^{2n}\) that we call S and that is contained in the orthogonal space related to the frequency vector. Then, using the Lie normal form Hamiltonian up to a suitable order, we check whether the truncated Hamiltonian evaluated on S vanishes only at the origin of \(\mathbb {R}^{2 n}\). If it occurs, we obtain a type of formal stability called Lie stability. The set S will be introduced in Section 4 and Appendix. Fourth, in Section 5 we will study under what conditions the solutions of a Hamiltonian system with 4-DOF, that start sufficiently close of the equilibrium, remain bounded for exponentially long times, this type of stability is known as exponential stability in the sense of Nekhoroshev. Fifth, in Section 6 we will analyze the instability, by using the \(n-2\) formal integrals (associated with the process of normalization) we will be able to construct a convenient Chetaev’s function in an appropriate region. Sixth, in Section 7 we make some observations and comment on some work that could be carried out in the future.

2 Formulation of the Problem and Statement of the Main Results

We suppose that \((\textbf{x}_0,\textbf{y}_0)=({\textbf {0}},\textbf{0})\) is an equilibrium solution of (1), and the Hamiltonian function \(\mathcal {H}\) developed in Taylor series in a neighborhood of the origin has the form

$$\begin{aligned} \mathcal {H}(\textbf{x},\textbf{y})=\mathcal {H}_2+\mathcal {H}_3+\cdots +\mathcal {H}_s+\cdots , \end{aligned}$$
(2)

where each \(\mathcal {H}_s\) is an homogeneous polynomial of degree s in the variables \(\left( \textbf{x},\textbf{y}\right) \), that is,

$$\begin{aligned} \mathcal {H}_s=\sum _{|\textbf{k}|+|\textbf{l}|=s} h_{\textbf{k}\textbf{l}}\textbf{x}^\textbf{k}\textbf{y}^\textbf{l}\end{aligned}$$
(3)

where \(\textbf{k}=(k_1,\ldots ,k_n)\in \mathbb {Z}^n_+\cup \{\textbf{0}\}\), \(\textbf{l}=(l_1,\ldots ,l_n)\in \mathbb {Z}^n_+ \cup \{\textbf{0}\}\), \(|\textbf{k}|=k_1+\cdots +k_n\), \(|\textbf{l}|=l_1+\cdots +l_n\), \(h_{\textbf{k}\textbf{l}}=h_{k_1 \ldots k_n l_1 \ldots l_n}\in \mathbb {R}\), \(\textbf{x}^\textbf{k}=x_1^{k_1}\cdots x_n^{k_n}\) and \(\textbf{y}^\textbf{l}=y_1^{l_1}\cdots y_n^{l_n}\).

The linear part associated with Eq. 1 in a neighborhood of \(\left( \textbf{0},\textbf{0}\right) \) is given by

$$\begin{aligned} \dot{\textbf{z}}=A\textbf{z}, \end{aligned}$$
(4)

where \(A=J Hess( \mathcal {H}_2\left( \textbf{0},\textbf{0}\right) )\), \(J=\left( \begin{array}{cc} 0&{}I\\ -I &{} 0 \end{array}\right) \) is a \(2n\times 2n\) matrix, I the identity matrix of size \(n\times n\), \(\textbf{z}=\left( \textbf{x},\textbf{y}\right) \) and A is diagonalizable. Our first assumption is that the equilibrium \(\left( \textbf{0},\textbf{0}\right) \) is elliptic, that is, the eigenvalues of A are pure imaginary, which we denote by \(\pm i\omega _j\), for \(j=1,\ldots , n\). We can suppose, without loss of generality (see [20, 21] for more details), that there is a linear canonical transformation which has already been constructed and such that

$$\begin{aligned} H_2 = \frac{\omega _1}{2}( x_1^2 + y_1^2 ) + \cdots + \frac{\omega _n}{2}( x_n^2 + y_n^2 ). \end{aligned}$$
(5)

The normal form Hamiltonian of H defined in (2) up to a finite degree m is the function

$$\begin{aligned} H = H_2 + H_3 + \cdots + H_m + \cdots \end{aligned}$$
(6)

obtained from (2) through a symplectic change of coordinates whose series expansion in \((\textbf{x}, \textbf{y})\) starts at degree two, such that each term \(H_k\) is a homogeneous polynomial of degree k in \(\textbf{w}= (\varvec{\xi }, \varvec{\eta })\), and satisfies \(\{H_2,H_k\} = 0\), \(k = 2, \ldots , m\) (the operator \(\{ \, , \, \}\) being the classical Poisson bracket in Hamiltonian theory), see [21].

Along this paper \(H^m\) represents the truncation of the normalized Hamiltonian function at degree m, that is,

$$\begin{aligned} H^m = H_2 + H_3 + \cdots + H_m, \end{aligned}$$
(7)

associated with the system

$$\begin{aligned} \dot{\textbf{w}} = J \nabla H^m(\textbf{w}). \end{aligned}$$
(8)

We stress that the transformation to normal form can be accomplished to any finite degree.

Customarily, one can introduce action-angle variables \(\textbf{r} = (r_1, \ldots , r_n )\), \(\varphi = ( \varphi _1, \ldots , \varphi _n )\) such that

$$ r_j = \frac{1}{2} (\xi _j^2 + \eta _j^2 ), \quad \varphi _j = \tan ^{-1} \left( \frac{\eta _j}{\xi _j}\right) , $$

where \(H_2\) takes the form

$$\begin{aligned} H_2 = \omega _1 r_1 + \cdots + \omega _n r_n. \end{aligned}$$
(9)

A second assumption is that \(H_2\) is an indefinite quadratic form, in other words, the signs of the \(\omega _i\) are not all the same, because in the opposite case, the stability in the Lyapunov sense easily follows.

Using action-angle variables defined above, the Hamiltonian function H in (3) leads to a Poisson series (a finite Fourier series in \(\varphi \) whose coefficients are polynomials in \(r_i^{1/2}\)) with terms of the form

$$ c \, r_1^{\alpha _1/2} \cdots r_n^{\alpha _n/2} \cos (\beta _1 \varphi _1 + \cdots + \beta _n \varphi _n) $$

where c is a real constant, the \(\alpha _j\) are non-negative integers and the \(\beta _j\) integers; there is also a similar \(\sin \)-term. Since the Poisson series came from a real power series, the terms must have the d’Alembert character [21], i.e., the coefficients \(\alpha _k\), \(\beta _k\) satisfy

$$\begin{aligned} \hbox {for} \,\,\, j = 1, \ldots , n, \quad \alpha _j \ge | \beta _j | \quad \hbox {and} \quad \alpha _j \equiv \beta _j \mod 2. \end{aligned}$$
(10)

By virtue of d’Alembert character, we write down the normal-form Hamiltonian up to degree p (when thought in rectangular coordinates) as

$$\begin{aligned} H(\textbf{r},\varvec{ \varphi }) = H_2(\textbf{r}) + \cdots + H_{2 l - 2}(\textbf{r}) + H_m(\textbf{r}, \varphi ) + \cdots \end{aligned}$$
(11)

with \(l \ge 2\), \(m = 2 l - 1\) or \(m = 2 l\) and \(m \le p\) or \(m > p\). Thus, m represents the lowest degree in which \(\varphi \) arises explicitly when transforming to action-angle coordinates. Terms of degree higher than p are not in normal form with respect to \(H_2\). Besides we can write, without loss of generality \(H_p(\textbf{r}, \varphi )\), assuming that \(H_p\) is independent of \(\varphi \) when \(p < m\) (then p is even), but it can hinge on the angles when \(p \ge m\).

We realize that H in (11) is an analytic function of the variables \(r_j^{1/2}\), \(\varphi _j\) and is \(2 \pi \)-periodic in \(\varphi _j\), for \(j = 1, \ldots , n\) excepting at \(\textbf{r} = 0\). To circumvent the problem at the origin of \(\mathbb {R}^{2 n}\) one uses d’Alembert condition. Specifically, the plan is to keep track of the d’Alembert character of the Hamiltonian and related formulas in action-angle coordinates. If the d’Alembert property is maintained through the different manipulations, transforming these formulas back to rectangular coordinates, the resulting expressions are polynomials in \(\textbf{z}\), thus analytic everywhere. Throughout the text all Hamiltonian functions satisfy (10).

Next, we will recall the concept of resonance which plays an important role in the theory of stability.

Definition 1

We say that the system Eq. 1 presents resonance relations, if there exists an integer vector \(\textbf{k}=(k_1,\ldots ,k_n)\in \mathbb {Z}^n\setminus \{\textbf{0}\}\) such that

$$\begin{aligned} \textbf{k}\cdot \varvec{\omega }= k_1\omega _1+\cdots +k_n\omega _n=0. \end{aligned}$$
(12)

Here \(\textbf{k}=(k_1,\ldots ,k_n)\) is called resonance vector and \(\varvec{\omega }=(\omega _1,\ldots ,\omega _n)\) is called frequency vector. The number \(|\textbf{k}|=|k_1|+\cdots +|k_n|\) is called the order of the resonance.

We consider the \(\mathbb {Z}-\)module

$$\begin{aligned} M_\omega =\{\textbf{k}\in \mathbb {Z}^n :\ \textbf{k}\cdot \varvec{\omega }=0\}, \end{aligned}$$

associated with the frequencies \(\omega _1,\ldots ,\omega _n\).

It is clear that if \(M_\omega =\{\textbf{0}\}\), it is equivalent to say that \(\omega _1,\ldots ,\omega _n\) are linearly independent on \(\mathbb {Q}\), that is, \(M_\omega =\{0\}\), if and only if, the system Eq. 1 does not have resonance relations. Otherwise, the system has resonances.

According to [8], it is satisfied that \(M_\omega \) is a sub-module of the module \(\mathbb {Z}^n\), then it follows that \(M_\omega \) is finitely generated, that is, there are integer vectors \(\textbf{k}^1,\ldots ,\textbf{k}^s\in M_\omega \) such that

$$\begin{aligned} M_\omega =\textbf{k}^1 \mathbb {Z}+\cdots +\textbf{k}^s\mathbb {Z}=\{j_1 \textbf{k}^1+\cdots +j_s \textbf{k}^s: \ j_1,\ldots ,j_s\in \mathbb {Z},\;\textbf{k}^1,\ldots ,\textbf{k}^s\in M_\omega \}, \end{aligned}$$
(13)

with \(s<n\). We will assume that the set of generators \(\{\textbf{k}^1,\ldots ,\textbf{k}^s\}\) of \(M_\omega \) is minimal, therefore, linearly independent. For each \(\alpha \in \{1,\ldots ,s\}\), in this work we will use the notation \(\textbf{k}^\alpha =\left( k_1^\alpha ,\ldots ,k_n^\alpha \right) \).

Definition 2

Assuming that \(M_\omega \ne \{0\}\). If \(s=1\) we say that the system Eq. 1 presents simple resonance and otherwise, if \(s>1\) we say that the system presents multiple resonances.

Definition 3

Assuming that \(s>1\). We say that two resonance vectors \(\textbf{k}^{\alpha _1},\textbf{k}^{\alpha _2}\) with \(\alpha _1,\alpha _2\in \{1,\ldots ,s\}\), \(\alpha _1\ne \alpha _2\) have not interaction if \(k_1^{\alpha _1}\ k_1^{\alpha _2}=\dots =k_n^{\alpha _1}\ k_n^{\alpha _2}=0\). We say that the set of resonance vectors \(\textbf{k}^{\alpha _1},\ldots ,\textbf{k}^{\alpha _m}\) with \(\alpha _1,\ldots ,\alpha _m \in \{1,\ldots ,s\}\), has not interaction, if \(\textbf{k}^{\alpha _i}\) and \(\textbf{k}^{\alpha _j}\) has not interaction for all \(j\in \{1,\ldots ,m\},i\ne j\).

This work concerns the study the stability in the Lyapunov sense of the origin of Hamiltonian system Eq. 1 that has n-degrees of freedom, assuming the existence of 2-resonance vectors without interaction. Thus, during this work we will assume:

  1. 1)

      \(M_\omega =\textbf{k}^1 \mathbb {Z}+ \textbf{k}^2 \mathbb {Z}\);

  2. 2)

      \(3\le |\textbf{k}^1|\le |\textbf{k}^2|\);

  3. 3)

      \(\textbf{k}^1\) and \(\textbf{k}^2\) have not interaction.

Note that, for the existence of two multiple resonances with interaction, one must have a Hamiltonian system with at least 3-DOF, while in the case of two vector resonances without interaction the Hamiltonian system must have at least 4-DOF.

At this moment we deal with the different types of stability. We start by recalling the definition of Lyapunov stability in the setting of the Hamiltonian system Eq. 1.

Definition 4

We say that the origin of \(\mathbb {R}^{2 n}\) in Eq. 1 is stable in the Lyapunov sense (or topologically stable), if for every \(\epsilon > 0\) there is \(\delta > 0\) such that if \(\textbf{z}(t, \textbf{z}_0)\) is the general solution of Eq. 1, then \(| \textbf{z}(t, \textbf{z}_0) | < \epsilon \) for all \(t \ge 0\) whenever \(| \textbf{z}_0 | < \delta \).

Regarding formal stability, we provide the definition due to Moser [23].

Definition 5

We say that the equilibrium solution \(\textbf{z} = \textbf{0}\) in Eq. 1 is formally stable if there exists a real formal power series \(V( \textbf{z} )\), which is an integral of Eq. 1 in the formal sense, and is positive definite near the origin of \(\mathbb {R}^{2 n}\).

Before talking about the results that exist in the literature about multiple resonances, we must define the concepts of Lie stability and invariant ray solution. Khazin in [17] introduced the concept of Birkhoff stability as follows: The original Hamiltonian is taken to the Birkhoff normal form and this process stops in terms of finite order, that is, it stops in terms of degree m a fixed number, but arbitrary, and we define \(H^m\) as the truncated Hamiltonian in terms of order m. Khazin affirmed that the equilibrium solution is Birkhoff stable, if this is stable for the Hamiltonian function \(H^m\) in the Lyapunov sense, for each value of m arbitrary. In our approach, we will also use this and like other authors we call it Lie stability (see [8, 9]), since we are considering the Hamiltonian function H after doing a Lie transformation.

Definition 6

The equilibrium solution \(\textbf{z} = 0\) in Eq. 1 is Lie stable if there exists \(p \ge 2\) such that the truncated Hamiltonian system in normal form associated with \(H^m\) is stable in the sense of Lyapunov for any \(m \ge p\) (arbitrary).

To our knowledge, there are no examples of systems that have resonance relations and are Lie stable but not Lyapunov stable, which gives an idea of the strength of Lie stability in the setting of nonlinear stability of equilibria. However, when the system is non-resonant, then the system is Lie stable and very recently there have appeared examples of unstable elliptic equilibria in non-resonant analytic Hamiltonian systems with three and four degrees of freedom, see [14].

In [6, 28, 29] and [30], the authors studied the instability of the full system for the cases of multiple resonances of the same order and odd order. To prove the instability they used a variant of the Classical Chetaev’s Theorem for autonomous Hamiltonian systems and one of the hypotheses that they assumed is that the components of at least one resonance vector do not change of sign and then they must assume other hypotheses that make the theorem not very simple to apply, this can be clearly seen in [6]. In our case, to prove the instability of the full system when we have multiple resonances of the same order and odd order, we are only going to assume that the components of at least one resonance vector do not change of sign, and for this, we are going to propose a suitable Chetaev’s function.

In [19], the authors studied the cases of two resonance vectors of order four, they only worked with the model system, that is, the truncated Hamiltonian function up to order four. They proved Lyapunov stability results defining a positive definite first integral and for instability, they showed the existence of an invariant ray solution.

Dos Santos-Vidal in [9] assumed the existence of s-multiple resonances such that \(M_\omega \) is as in Eq. 13. The main generic result about Lie stability of [9] is given in Appendix. There, they showed that a necessary condition of the set \(S=\{\textbf{r}=0\}\) (the set S is defined in Eq. A.1), is that the components of all resonance vectors change of sign and then directly from Theorem A.1 the Lie stability follows. Therefore, in this work it will be important to assume that the components of at least one resonance vector change of sign, and in this instance the instability could occur.

In addition, in [9] it is considered the situation of s-resonance vectors without interaction as follows, \(\eta =|\textbf{k}^1|=|\textbf{k}^2|=\dots =|\textbf{k}^\mu |\), \(2\eta <|\textbf{k}^{\mu +1}|\le \cdots \le |\textbf{k}^{s}|\) with \(2\le \mu \le s\) and one hypothesis of its result about Lyapunov instability is to consider that the components of the vectors \(\textbf{k}^1, \ldots ,\textbf{k}^\mu \) do not change of sign, for this, they used a Chetaev’s function to proof the instability. In our case, for example, if \(\eta =|\textbf{k}^1|=|\textbf{k}^2|\), to prove the Lyapunov instability, it is enough that the components of at least one resonance vectors do not change of sign and to proof the instability, we will use a different Chetaev’s function.

In our work, in order to prove the Lyapunov instability of the whole system we are going to propose a Chetaev’s function that consists of the product of three functions, that is, \(V=V_1 V_2 V_3\). The function \(V_1\) will be built using some formal first integrals and the function \(V_2\) will be built using some action variables, and both functions will allow us to characterize the points inside the set \(\Omega \) (set of Chetaev’s Theorem where V is defined), managing to write all the action variables as a function of one of them. The function \(V_3\) will be linked to resonance vector \(\textbf{k}^j\) that produces the instability and will depend on some action variables and an angle \(\phi _j=\textbf{k}^j\cdot \varphi \).

In another recent article, see [5], to give force to the formal stability, the authors have managed to connect the Lie stability with the exponential stability in the sense of Nekhoroshev, this last type of stability has also been working with other authors in [13, 24, 26]. The exponential stability consists in guaranteeing that solutions which start out close enough of the equilibrium will remain bound for exponentially long times. Nevertheless, in [5] the authors have incorporated Diophantine conditions in the Lie stable systems. So, another of the objectives of this work, after proving the Lie stability, we will show under what explicit conditions it is possible to guarantee the exponential stability of the Nekhoroshev type of a Hamiltonian system with 4-DOF.

Our objective is to advance in the study of the stability of the equilibrium solutions in the case of two resonance vectors without interaction, mainly covering the cases that have not yet been treated. In other words, treat the cases in which the components of at least one resonance vector do not change of sign. Consequently, our results will be stated depending on the sign of the components of the resonance vectors and other characteristic associated with \(H_{|\textbf{k}^1|}\). So, this work will have the following structure: In Section 4 we present some results that guarantee stability in the Lyapunov sense for the equilibrium point associated with the Hamiltonian system truncated up to order m, with m fixed, but arbitrary, in other words, we will show Lie stability results. In Section 5 we show that the Lie stable systems with 4-DOF that have a pair of frequencies such that their quotient is an algebraic number of degrees greater than or equal to two, are also exponentially stable systems. In Section 6 concerns with the instability result of the equilibrium solution for the complete system. Finally, in Section 7, we make some observations about the truncated Hamiltonian systems that have an invariant ray solution and give some ideas for possible future work.

3 Lie Normal Form in the Case of Two Resonance Vectors without Interaction

Suppose that \(M_\omega =\textbf{k}^1\mathbb {Z}+\textbf{k}^2\mathbb {Z}\) and without loss of generality, the two vectors of resonance without interaction are of the form

$$\begin{aligned} \begin{array}{rcl} \textbf{k}^1&{}=&{}\displaystyle \left( k_1^1,\ldots ,k_{m_1}^1,0,\ldots ,0\right) ,\\ \textbf{k}^2&{}=&{}\displaystyle \left( 0,\ldots ,0,k_{{m_1}+1}^2,\ldots ,k_{m_2}^2,0,\ldots ,0\right) ,\\ \end{array} \end{aligned}$$
(14)

and it is satisfied that \(3 \le |\textbf{k}^1|\le |\textbf{k}^2|\).

We consider the new Hamiltonian in its Lie normal form given in Eq. 6 in coordinates \((\varvec{\xi }, \varvec{\eta })\). By the Lie equation, we know that

$$\begin{aligned} \begin{array}{rl} 0 =&{} \{H_2, H_s \} = \displaystyle \sum _{j=1}^n\left[ \dfrac{\partial H_2}{\partial \xi _j} \dfrac{\partial H_s}{\partial \eta _j}- \dfrac{\partial H_2}{\partial \eta _j} \dfrac{\partial H_s}{\partial \xi _j}\right] \\[1pc] =&{} \omega _1 \left[ \xi _1 \dfrac{\partial H_s}{\partial \eta _1}- \eta _1 \dfrac{\partial H_s}{\partial \xi _1}\right] +\cdots + \omega _n \left[ \xi _n \dfrac{\partial H_s}{\partial \eta _n}- \eta _n \dfrac{\partial H_s}{\partial \xi _n}\right] , \end{array} \end{aligned}$$
(15)

where

$$\begin{aligned} H_s=\sum _{|{\varvec{\mu }}|+|{\varvec{\nu }}|=s} h_{{\varvec{\mu }}{\varvec{\nu }}}\varvec{\xi }^{\varvec{\mu }}\varvec{\eta }^{\varvec{\nu }}, \end{aligned}$$
(16)

with \({\varvec{\mu }}=(\mu _1,\ldots ,\mu _n)\), \( {\varvec{\nu }}=(\nu _1,\ldots ,\nu _n)\in Z^n_+\cup \{\textbf{0}\}\), \(|{\varvec{\mu }}|=\mu _1+ \cdots +\mu _n\), \(|{\varvec{\nu }}|=\nu _1+\cdots +\nu _n\), \(h_{{\varvec{\mu }}{\varvec{\nu }}}=h_{{\mu _1}\ldots {\mu _n}{\nu _1}\ldots {\nu _n}}\), \(\varvec{\xi }^{{\varvec{\mu }}}=\xi _1^{\mu _1}\cdots \xi _n^{\mu _n} \), \(\varvec{\eta }^{{\varvec{\mu }}}=\eta _1^{\mu _1}\cdots \eta _n^{\mu _n} \) and \( s\ge 3\).

To obtain the necessary restrictions or conditions on the Hamiltonian coefficients in the Lie normal form, it is convenient to introduce the following 2i-symplectic coordinates:

$$\begin{aligned} \zeta _j= \xi _j+ i \eta _j, \quad \overline{\zeta }_j= \xi _j- i \eta _j, \quad \text {with }j=1,\ldots ,n. \end{aligned}$$
(17)

We denote by

$$ \mathcal {K}= \mathcal {K}_2+ \mathcal {K}_3+ \cdots + \mathcal {K}_s+ \cdots $$

to Hamiltonian H given in Eq. 6 in complex coordinates, that is,

$$\begin{aligned} \mathcal {K}= \mathcal {K}(\varvec{\zeta }, \overline{\varvec{\zeta }})= H\left( \frac{\zeta _1+ \overline{\zeta }_1}{2},\ldots , \frac{\zeta _n+ \overline{\zeta }_n}{2}, \frac{\zeta _1- \overline{\zeta }_1}{2i},\ldots , \frac{\zeta _n- \overline{\zeta }_n}{2i}\right) , \end{aligned}$$
(18)

where \( \mathcal {K}_s\) denote a homogeneous polynomial in the variables \((\varvec{\zeta }, \overline{\varvec{\zeta }})=(\zeta _1,\ldots ,\zeta _n,\overline{\zeta }_1,\) \(\ldots ,\overline{\zeta }_n)\). So,

$$\begin{aligned} \mathcal {K}_{s}=\sum _{|{\varvec{\mu }}|+|{\varvec{\nu }}|=s} g_{\varvec{\mu }\varvec{\nu }} \varvec{\zeta }^{\varvec{\mu }}\bar{\varvec{{\zeta }}}^{\varvec{\nu }}, \end{aligned}$$
(19)

where \(g_{{\varvec{\mu }}{\varvec{\nu }}}= \alpha _{{\varvec{\mu }}{\varvec{\nu }}}+ i \beta _{{\varvec{\mu }}{\varvec{\nu }}} \in \mathbb {C}\) depend of coefficients \(h_{{\varvec{\mu }}{\varvec{\nu }}}\) and \(g_{{\varvec{\mu }}{\varvec{\nu }}}=\bar{g}_{{\varvec{\mu }}{\varvec{\nu }}}\). By the chain rule we have

$$\begin{aligned} \frac{\partial }{\partial \xi _j}= \frac{\partial }{\partial \zeta _j}+ \frac{\partial }{\partial \overline{\zeta }_j},\quad \frac{\partial }{\partial \eta _j}= i \left[ \frac{\partial }{\partial \zeta _j}- \frac{\partial }{\partial \overline{\zeta }_j}\right] , \end{aligned}$$
(20)

then, using the definition of \( \mathcal {K}^s\) given in (19), we obtain that (15) assume the form

$$\begin{aligned} 0=\{\mathcal {K}_2,\mathcal {K}_s \}= i\ \displaystyle \sum _{|{\varvec{\mu }}|+|{\varvec{\nu }}|=s} [\omega _1 (\mu _1-\nu _1)+ \cdots +\omega _n (\mu _n-\nu _n)]\ g_{{\varvec{\mu }}{\varvec{\nu }}}\, \varvec{\zeta }^{\varvec{\mu }}\overline{\varvec{\zeta }}^{\varvec{\nu }}. \end{aligned}$$
(21)

Therefore, in the new variables \((\varvec{\zeta }, \overline{\varvec{\zeta }})\), from equation (21) we have

$$\begin{aligned}{}[\omega _1 (\mu _1-\nu _1)+\cdots + \omega _n (\mu _n-\nu _n)]\ g_{{\varvec{\mu }}{\varvec{\nu }}}= 0, \quad \text{ for } \text{ all }\quad |{\varvec{\mu }}|+|{\varvec{\nu }}|= s \ge 3, \end{aligned}$$

considering \(g_{{\varvec{\mu }}{\varvec{\nu }}}\ne 0\), then

$$\begin{aligned} \omega _1\left( \mu _1-\nu _1\right) +\cdots +\omega _n\left( \mu _n-\nu _n\right) =0. \end{aligned}$$
(22)

If \(|\textbf{k}^1|=|\textbf{k}^2|\) is odd and since \(\textbf{k}^1\cdot \varvec{ \omega } =\textbf{k}^2\cdot \varvec{\omega } =0\), it follows that Eq. 22 is satisfied if \({\varvec{\mu }}-{\varvec{\nu }}=\pm \textbf{k}^1\) and \({\varvec{\mu }}-{\varvec{\nu }}=\pm \textbf{k}^2\). Thus, the normal form of \(H_{|\textbf{k}^1|}\) is

$$\begin{aligned} H_{|\textbf{k}^1|}(\textbf{r},\phi _1,\phi _2)=H_{|\textbf{k}^1|}^1(\textbf{r},\phi _1)+H_{|\textbf{k}^1|}^2(\textbf{r},\phi _2), \end{aligned}$$
(23)

where

$$\begin{aligned} H_{|\textbf{k}^1|}^1 \left( \textbf{r}, \phi _1\right)= & {} A_1 R_1\cos \phi _1, \end{aligned}$$
(24)
$$\begin{aligned} H_{|\textbf{k}^1|}^2 \left( \textbf{r}, \phi _2\right)= & {} A_2 R_2\cos \phi _2, \end{aligned}$$
(25)

with \(\phi _j=\textbf{k}^j\cdot \varphi \) for \(j=1,2\), \(A_1 \in \mathbb {R}\setminus \{0\}\), \(A_2\in \mathbb {R}\) and

$$\begin{aligned} \begin{array}{r} R_1 =\displaystyle \prod _{j=1}^{m_1} r_j^{|k_j^1|/2},\quad R_2= \prod _{j=m_1+1}^{m_2}r_j^{|k_j^2|/2}. \end{array} \end{aligned}$$
(26)

If \(|\textbf{k}^1|=|\textbf{k}^2|\) is even, then Eq. 22 is satisfied if \({\varvec{\mu }}-{\varvec{\nu }}=0\), \({\varvec{\mu }}-{\varvec{\nu }}=\pm \textbf{k}^1\) and \({\varvec{\mu }}-{\varvec{\nu }}=\pm \textbf{k}^2\), so

$$\begin{aligned} H_{|\textbf{k}^1|}(\textbf{r},\phi _1,\phi _2)=H_{|\textbf{k}^1|}^0(\textbf{r})+H_{|\textbf{k}^1|}^1(\textbf{r},\phi _1)+H_{|\textbf{k}^1|}^2(\textbf{r},\phi _2), \end{aligned}$$
(27)

with \(H_{|\textbf{k}^1|}^1(\textbf{r},\phi _1)\) as in Eq. 24, \(H_{|\textbf{k}^1|}^2 \left( \textbf{r}, \phi _2\right) \) as in Eq. 25 and

$$\begin{aligned} H_{|\textbf{k}^1|}^0(\textbf{r})=\sum _{ j_1,j_2,\ldots , j_{\sigma _1}=1}^n A^{j_1 j_2 \cdots j_{\sigma _1} }r_{j_1}r_{j_2}\cdots r_{j_{\sigma _1}}. \end{aligned}$$
(28)

where \(\sigma _1=|\textbf{k}^1|/2\), \(A^{j_1 j_2 \cdots j_{\sigma _1} }\in \mathbb {R}\).

On the other hand, if \(|\textbf{k}^1|<|\textbf{k}^2|\), then \(H_{|\textbf{k}^1|}=H_{|\textbf{k}^1|}^1(\textbf{r},\phi _1)\) whenever \(|\textbf{k}^1|\) is odd and \(H_{|\textbf{k}^1|}=H_{|\textbf{k}^1|}^0(\textbf{r})+H_{|\textbf{k}^1|}^1(\textbf{r},\phi _1)\) as long as \(|\textbf{k}^1|\) is even. Similarly, if \(|\textbf{k}^2|\) odd, then \(H_{|\textbf{k}^2|}=H_{|\textbf{k}^2|}^2(\textbf{r},\phi _2)\) and if \(|\textbf{k}^2|\) is even, \(H_{|\textbf{k}^2|}=H_{|\textbf{k}^2|}^0(\textbf{r})+H_{|\textbf{k}^2|}^1(\textbf{r},\phi _2)\), with

$$\begin{aligned} H_{|\textbf{k}^2|}^0(\textbf{r})=\sum _{ j_1,j_2,\ldots , j_{\sigma _2}=1}^n A^{j_1 j_2 \cdots j_{\sigma _2} }r_{j_1}r_{j_2}\cdots r_{j_{\sigma _2}}, \end{aligned}$$

where \(\sigma _2=|\textbf{k}^2|/2\).

To study results about stability, we are going to consider the truncated normalized Hamiltonian function up to order m inclusively, with m a fixed natural number, but arbitrary. Then the normalized Hamiltonian function up to m order is given by

$$\begin{aligned} H^{m}:=H^{m}(\textbf{r},\phi _1,\phi _2)= H_2+H_\star +H_4+\cdots +H_{|\textbf{k}^1|-\nu }+H_{|\textbf{k}^1|}+\sum _{j=|\textbf{k}^1|+1}^{m}H_{j}, \end{aligned}$$
(29)

with

$$\begin{aligned} H_\star =\left\{ \begin{array}{ll} H_3, &{} \text {if } |\textbf{k}^1|=3,\\ 0, &{}\text {if } 4\le |\textbf{k}^1|, \end{array}\right. \qquad \nu =\left\{ \begin{array}{ll} 1, &{} \text {if } |\textbf{k}^1| \text { is odd}\\ 2, &{} \text {if } |\textbf{k}^1| \text { is even.}\, \end{array}\right. \end{aligned}$$
(30)

For every j, \(H_j\) is an homogeneous polynomial of degree j/2 in the action variable \(\textbf{r}\).

4 Results About Stability

In this section we present Lie stability results, considering that we have only two vectors of resonance as in Eq. 14 such that \(4\le |\textbf{k}^1|\le |\textbf{k}^2|\) and \(|\textbf{k}^1|\) is even.

We know that \(dim M_{\omega }= 2\), thus \(dim (M_{\omega })^{\perp }= n-2\). Next, we characterize explicitly the linearly generator of \((M_{\omega })^{\perp }\). For this, let \(\{\textbf{e}_j\}\) be the canonical vectors of \(\mathbb {R}^n\), and define the convenient linearly independent vectors \(\{ \textbf{a}_j\}\), \(j=1, \ldots , n-2\) by

$$\begin{aligned} \begin{array}{rl} \textbf{a}_1&{}=-k_2^1\textbf{e}_1+k_1^1\textbf{e}_2\\ \textbf{a}_2&{}=-k_3^1\textbf{e}_1+k_1^1\textbf{e}_3\\ \vdots \ \ &{}= \qquad \qquad \vdots \\ \textbf{a}_{m_1-1}&{}=-k_{m_1}^1\textbf{e}_1+k_1^1\textbf{e}_{m_1}\\ \hline \textbf{a}_{m_1}&{}=-k_{{m_1}+2}^2\textbf{e}_{m_1+1}+k_{{m_1}+1}^2 \textbf{e}_{m_1+2}\\ \textbf{a}_{m_1+1}&{}=-k_{{m_1}+3}^2\textbf{e}_{m_1+1}+k_{{m_1}+1}^2 \textbf{e}_{m_1+3}\\ \vdots \ \ &{}= \qquad \qquad \vdots \\ \textbf{a}_{m_2-2}&{}=-k_{m_2}^2\textbf{e}_{m_1+1}+k_{{m_1}+1}^2 \textbf{e}_{m_2}\\ \hline \textbf{a}_{m_2-1}&{}=\textbf{e}_{m_2+1} \\ \textbf{a}_{m_2}&{}=\textbf{e}_{m_2+2} \\ \vdots \ \ &{}= \ \ \vdots \\ \textbf{a}_{n-2}&{}=\textbf{e}_{n}. \\ \end{array} \end{aligned}$$
(31)

Clearly, it is satisfied that \(\textbf{a}_j\cdot \textbf{k}^1=\textbf{a}_j\cdot \textbf{k}^2=0\) for all \(j=1, \ldots ,n-2\).

By using the set of vectors \(\{\textbf{a}_j\}\), we define the following functions

$$\begin{aligned} \begin{array}{ll} I_j=I_j(\textbf{r})=\textbf{a}_j\cdot \textbf{r},&j=1,\ldots ,n-2. \end{array} \end{aligned}$$
(32)

Lemma 1

The functions \(I_j\) defined in Eq. 32 for \(j=1, \ldots ,n-2\) are first integrals of the Hamiltonian system associated with the truncated Hamiltonian function Eq. 29.

Proof

In fact, for \(j=1,\ldots ,n-2\) we have that

$$\begin{aligned} \dot{I}_j= & {} \textbf{a}_j\cdot \dot{\textbf{r}}\\= & {} \textbf{a}_j\cdot \dfrac{\partial H^{m}}{\partial \varphi }\\= & {} \textbf{a}_j\cdot \left( \dfrac{\partial H^{m}}{\partial \phi _1}\dfrac{\partial \phi _1}{\partial \varphi }+\dfrac{\partial H^{m}}{\partial \phi _2}\dfrac{\partial \phi _2}{\partial \varphi }\right) \\= & {} \textbf{a}_j\cdot \left( \textbf{k}^1\dfrac{\partial H^{m}}{\partial \phi _1}+\textbf{k}^2\dfrac{\partial H^{m}}{\partial \phi _2}\right) \\= & {} 0. \end{aligned}$$

\(\square \)

To demonstrate results concerning with Lie stability, it is necessary to define a positive definite first integral V with help of the first integrals given in Eq. 32 and the Hamiltonian function \(H^{m}\), as follows

$$\begin{aligned} V=\sum _{j=1}^{n-2}I_j^2+(H^{m})^2. \end{aligned}$$
(33)

If \(\textbf{r}=\tilde{\textbf{r}}\) is the solution of the system \(I_j(\textbf{r})=0\) for \(j=1,\ldots ,n-2\) and \(H^{m}(\tilde{\textbf{r}},\phi _1,\phi _2)\ne 0\) in a sufficiently small neighborhood of the origin, then V is positive definite in a small enough neighborhood of \(\textbf{r}=0\), this implies that the null solution is Lie stable. Therefore, as in [9] is important to define set S as

$$\begin{aligned} S=\{\textbf{r}\in \mathbb {R}^n_+\cup \{\textbf{0}\}:\ I_j(\textbf{r})=0 , \ j=1,\ldots ,n-2\}. \end{aligned}$$
(34)

After that, we define the function

$$F_{m}=\left. H^{m}(\textbf{r},\phi _1,\phi _2)\right| _{S\times \mathbb {T}^2}.$$

If \(F_{m}\ne 0\), then \(\textbf{r}=0\) is Lie stable. In other words, to prove Lie stability we will apply Theorem A.1 that can be found in the appendix. For this, the first step is to characterize the set S and its dimension. We do it in the following lemma.

Remark 1

If the components of the vector of resonance \(\textbf{k}^{j}\) do not change of sign, we will assume during all this work that each component is positive.

Lemma 2

If the components of the vector \(\textbf{k}^j\) do not change of sign for some \(j\in \{1,2\}\), then \(S\ne \{\textbf{r}=0\}\). More precisely,

  1. (a)

    If the components of \(\textbf{k}^1\) and \(\textbf{k}^2\) change of sign, then \(S=\left\{ \textbf{r}=0\right\} \).

  2. (b)

    If the components of \(\textbf{k}^1\) do not change of sign and the components of \(\textbf{k}^2\) change of sign, then \(S=\left\{ \frac{r_1}{k_1^1}{} \textbf{k}^1\right\} \).

  3. (c)

    If the components of \(\textbf{k}^1\) change of sign and the components of \(\textbf{k}^2\) do not change of sign, then \(S=\left\{ \frac{r_{{m_1}+1}}{k_{{m_1}+1}^2}{} \textbf{k}^2\right\} \).

  4. (d)

    If the components of \(\textbf{k}^1\) and \(\textbf{k}^1\) do not change of sign, then

    $$S=\left\{ \frac{r_1}{k_1^1}{} \textbf{k}^1+ \frac{r_{{m_1}+1}}{k_{{m_1}+1}^2}{} \textbf{k}^2\right\} .$$

Proof

We must solve the system

$$\begin{aligned} \begin{array}{rl} I_1=&{}-k_2^1 r_1+k_1^1 r_2=0\\ I_2=&{}-k_3^1 r_1+k_1^1 r_3=0\\ \vdots &{}\quad \vdots \quad \qquad \quad \vdots \qquad \quad \\ I_{{m_1}-1}=&{}-k_{m_1}^1 r_1+k_1^1 r_{m_1}=0\\ I_{m_1}=&{}-k_{{m_1}+2}^2 r_{{m_1}+1}+k_{{m_1}+1}^2r_{{m_1}+2}=0\\ I_{{m_1}+1}=&{}-k_{{m_1}+3}^2 r_{{m_1}+1}+k_{{m_1}+1}^2r_{{m_1}+3}=0\\ \vdots &{}\quad \vdots \quad \qquad \quad \vdots \qquad \quad \\ I_{m_2-2}=&{}-k_{m_2}^2 r_{{m_1}+1}+k_{{m_1}+1}^2 r_{m_2}^2=0\\ I_{{m_2}-1}=&{}r_{n+1}=0\\ \vdots &{}\quad \vdots \qquad \quad \\ I_{n-2}=&{}r_{n}=0.\\ \end{array} \end{aligned}$$
(35)

For item (a), since the components of the vector \(\textbf{k}^1\) and \(\textbf{k}^2\) change of sign, there exists \(j_\star \in \{2, \ldots ,m_1\}\) such that \(sg(k^1_1)=-sg(k_{j_\star }^1)\) and there exists \(j_*\in \{{m_1}+2,\ldots ,{m_2}\}\) such that \(sg(k_{{m_1}+1}^2)=-sg(k_{j_*}^2)\). Then, from Eq. 35, we obtain that

$$I_{{j_\star }-1}=-k_{j_\star }^1 r_{1}+k_{1}^1 r_{j_\star }=0 \quad \text {and}\quad I_{{j_*}-2}=-k_{j_*}^2 r_{{m_1}+1}+k_{{m_1}+1}^2 r_{j_*}=0,$$

then

$$r_{j_*}=\frac{k_{j_\star }^1}{k_1^1}r_{1}\le 0\quad \text {and}\quad r_{j_*}=\frac{k_{j_*}^2}{k_{{m_1}+1}^2}r_{{m_1}+1}\le 0,$$

this implies that

$$r_{j_\star }=0\Longleftrightarrow r_{1}=0 \quad \text {and}\quad r_{j_*}=0\Longleftrightarrow r_{{m_1}+1}=0,$$

respectively, so from Eq. 35 it follows that \(r_2=r_3=\cdots =r_{m_1}=r_{{m_1}+2}=r_{{m_1}+3}=\cdots =r_{m_2}=0\), in consequence, the solution of the system Eq. 35 is \(\textbf{r}=0\).

With the same idea, for item (b), the solution of the system Eq. 35 is \(r_j=\frac{k_j^1}{k_1^1} r_1\ge 0\) for \(j=2, \ldots ,m_1\), \(r_{j}=0\) for \(j=m_1+1,\ldots ,n\), then \(S=\left\{ \textbf{r}=\frac{r_1}{k_1^1}\textbf{k}^1\right\} \) and \(dim(S)=1\).

Similarly, for item (c), \(S=\left\{ \textbf{r}=\frac{r_{{m_1}+1}}{k_{{m_1}+1}^2}\textbf{k}^2\right\} \) and \(dim(S)=1\).

Now, if the components of \(\textbf{k}^1\) and \(\textbf{k}^2\) do not change of sign, then the solution of the system Eq. 35 is characterized by

$$\begin{aligned} r_j= & {} \frac{k_j^1}{k_1^1}r_1\ge 0, \ \ j=1,\ldots ,m_1,\\ r_j= & {} \frac{k_j^2}{k_{{m_1}+1}^2}r_{{m_1}+1}\ge 0, \ \ j=m_1+1,\ldots ,m_2,\\ r_{j}= & {} 0, \ \ j=m_2+1, \ldots ,n, \end{aligned}$$

therefore \(S=\left\{ \textbf{r}=\frac{r_1}{k_{1}}\textbf{k}^1+\frac{r_{{m_1}+1}}{k_{{m_1}+1}}\textbf{k}^2\right\} \) and \(dim(S)=2\). \(\square \)

Next, we present four theorems about Lie stability. The proofs of these theorems are consequence of Theorem A.1.

Theorem 4.1

Suppose that the components of the vectors \(\textbf{k}^1\) and \(\textbf{k}^2\) change of sign, then the null solution of Eq. 1 is Lie stable.

Proof

By Lemma 2, we have \(S=\{\textbf{r}=0\}\), then the result is immediate from Theorem A.1, in others words, the function V defined in Eq. 33 is a first integral positive definite in a small neighborhood of the origin.

Next, we exhibit an example in which this theorem can be applied.

Example 1

We consider the Hamiltonian as in Eq. 2 with six degrees of freedom and \(\mathcal {H}_2=3\sqrt{2} r_1+\sqrt{2}r_2+3\sqrt{3}r_3+\sqrt{3} r_4-\pi r_5-r_6\) . Then, \(M_\omega =\textbf{k}^1\mathbb {Z}+\textbf{k}^2\mathbb {Z}\), where

$$\textbf{k}^1=(1,-3,0,0,0,0), \quad \textbf{k}^2=(0,0, 1,-3,0,0).$$

So, by Theorem 4.1 the null solution of the system Eq. 1 is Lie stable.

For the following result we are going to consider the normalized Hamiltonian function up to an order m as in Eq. 29. If \(|\textbf{k}^j|\) is even, we need to define the auxiliary functions \(\Psi _j:[0,2\pi )\longrightarrow \mathbb {R}\) by

$$\begin{aligned} \begin{array}{rcl} \Psi _j(\phi _j)= & {} =H_{|\textbf{k}^j|}^0(\textbf{k}^j)+H_{|\textbf{k}^j|}^j(\textbf{k}^j, \phi _j), \quad \text {for }j=1,2. \end{array} \end{aligned}$$
(36)

Remember that \(\sigma _j=|\textbf{k}^j|/2\) for \(j=1,2\), so, explicitly,

$$\begin{aligned} \begin{array}{l} \vspace{0.2cm} \Psi _1(\phi _1)=\displaystyle \sum _{j_1,\ldots , j_{\sigma _1}=1}^{m_1} A^{j_1 \cdots j_{\sigma _1} } k_{j_1}^1\cdots k_{j_{\sigma _1}}^1+A_1 P_1 \cos \phi _1, \quad P_1=\prod _{j=1}^{m_1} (k_j^1)^{|k_j^1|/2},\\ \Psi _2(\phi _2)=\!\displaystyle \sum _{j_1,\ldots , j_{\sigma _2}=m_1+1}^{m_2} A^{j_1 \cdots j_{\sigma _2} } k_{j_1}^2\cdots k_{j_{\sigma _2}}^2+A_2 P_2 \cos \phi _2, \quad P_2=\!\prod _{j=m_1+1}^{m_2} (k_j^2)^{|k_j^2|/2}. \end{array} \end{aligned}$$
(37)

Remark 2

Note that the function \(\Psi _j\) is well defined, when \(\prod _{l=m_{j-1}+1}^{m_j}{k_l^j}^{|k_l^{j}|} >0\), for \(j=1,2\) (\(m_0=0\)).

Remark 3

Fixed \(j \in \{1, 2\}\) and the components of the vector of resonance \(\textbf{k}^{j}\) do not change of sign, then by Remark 1 the function \(\Psi _j\) is always well defined.

Theorem 4.2

Suppose that \(4=|\textbf{k}^1|\le |\textbf{k}^2|\), the components of the vector \(\textbf{k}^{1}\) do not change of sign, the components of the vector \(\textbf{k}^2\) change of sign and \(\Psi _1(\phi _1)\ne 0\) for all \(\phi _1\in [0,2\pi )\), with \(\Psi _1(\phi _1)\) as in Eq. 36, then the null solution of Eq. 1 is Lie stable.

Proof

If the components of the vector \(\textbf{k}^1\) do not change of sign, then by Lemma 2, we have that \(S=\left\{ \textbf{r}=\dfrac{r_1}{k_1^1} \textbf{k}^1\right\} \), so

$$\begin{aligned} \begin{array}{rcl} \vspace{0.2cm} F_{m}&{}=&{}H^{m}|_{S\times \mathbb {T}^2}= H^{m}\left( \dfrac{r_1}{k_1^1}{} {\textbf {k}}^1,\phi _1,\phi _2\right) \\ \vspace{0.2cm} &{}=&{}H_{|\textbf{k}^1|}\left( \dfrac{r_1}{k_1^1} \textbf{k}^1,\phi _1,\phi _2\right) +\cdots +H_{m}\left( \dfrac{r_1}{k_1^1} \textbf{k}^1,\phi _1,\phi _2\right) \\ \vspace{0.2cm} &{}=&{}\left( \dfrac{r_1}{k_1^1}\right) ^{|\textbf{k}^1|/2}\left( H_{|\textbf{k}^1|}^0\left( \textbf{k}^1\right) +H_{|\textbf{k}^1|}^1\left( \textbf{k}^1,\phi _1\right) \right) +\cdots +\left( \dfrac{r_1}{k_1^1}\right) ^{m/2}H_{m}\left( \textbf{k}^1,\phi _1,\phi _2\right) \\ \vspace{0.2cm} &{}=&{}\left( \dfrac{r_1}{k_1^1}\right) ^{|\textbf{k}^1|/2}\Psi _1(\phi _1)+\cdots \mathcal +\left( \dfrac{r_1}{k_1^1}\right) ^{m/2}{H}_{m}\left( {\textbf {k}}^1,\phi _1,\phi _2\right) . \end{array} \end{aligned}$$
(38)

Therefore, using the hypothesis \(\Psi _1(\phi _1)\ne 0\) for all \(\phi _1\in [0,2\pi )\), it follows that \(F_{m}\ne 0\) for all \((r_1,\phi _1, \phi _2)\) with \(r_1\) sufficiently small, then the result follows from Theorem A.1. \(\square \)

With the same idea of the previous theorem we can analyze the case in which \(|\textbf{k}^1|\) is even and \(6\le |\textbf{k}^1|\le |\textbf{k}^2|\), but we must incorporate an extra hypothesis.

Theorem 4.3

Suppose that \(|\textbf{k}^1|\) is even, \(6\le |\textbf{k}^1|\le |\textbf{k}^2|\), \(H_4=H_6=\dots =H_{|\textbf{k}^1|-2}\equiv 0\), the components of the vector \(\textbf{k}^{1}\) do not change of sign, the components of the vector \(\textbf{k}^2\) change of sign and \(\Psi _1(\phi _1)\ne 0\) for all \(\phi _1\in [0,2\pi )\), with \(\Psi _1(\phi _1)\) as in Eq. 36, then the null solution of Eq. 1 is Lie stable.

Proof

This proof is identical to that given for Theorem 4.2. \(\square \)

In the following two theorems we analyze the case where \(|\textbf{k}^1|<|\textbf{k}^2|\), but in this case, assuming that the components of \(\textbf{k}^1\) change of sign.

Theorem 4.4

Suppose that \(3=|\textbf{k}^1|<|\textbf{k}^2|=4\), the components of the vector \(\textbf{k}^{1}\) change of sign, the components of the vector \(\textbf{k}^2\) do not change of sign and \(\Psi _2(\phi _2)\ne 0\) for all \(\phi _2\in [0,2\pi )\), with \(\Psi _2(\phi _2)\) as in Eq. 36, then the null solution of Eq. 1 is Lie stable.

Proof

The proof follows the same idea as the one given for Theorem 4.2. In fact, by Lemma 2, \(S=\left\{ \textbf{r}=\dfrac{r_{m_1+1}}{k_{m_1+1}^2} \textbf{k}^2\right\} \), so

$$\begin{aligned} F_{m}=\left. H^m\right| _{S\times \mathbb {T}^2}\left( \dfrac{r_{m_1+1}}{k_{m_1+1}^1}\right) ^{|\textbf{k}^2|/2}\Psi _2(\phi _2)+\cdots \mathcal +\left( \dfrac{r_{m_1+1}}{k_{m_1+1}^2}\right) ^{m/2}{H}_{m}\left( {\textbf {k}}^2,\phi _1,\phi _2\right) , \end{aligned}$$
(39)

because, \(H^m=H_2+H_{|\textbf{k}^1|}+H_{|\textbf{k}^2|}+\cdots +H_m\) and considering that \(H_{|\textbf{k}^1|}\) is as in Eq. 24, it follows that \(H_{|\textbf{k}^1|}\left( \dfrac{r_{m_1+1}}{k_{m_1+1}^1} \textbf{k}^2,\phi _1 \right) =0\). Therefore, since \(\Psi _2(\phi _2)\ne 0\) for all \(\phi _2\in [0,2\pi )\), it follows that \(F_{m}\ne 0\) for all \((r_{m_1+1},\phi _1, \phi _2)\) with \(r_{m_1+1}\) sufficiently small, then by Theorem A.1 the proof is over. \(\square \)

The previous theorem can be generalize when \(3\le |\textbf{k}^1| <4\le |\textbf{k}^2|\) incorporating a hypothesis.

Theorem 4.5

Suppose that \(|\textbf{k}^1|\) is odd, \(|\textbf{k}^2|\) is even, \(3\le |\textbf{k}^1| <4\le |\textbf{k}^2|\), \(H_4=\dots =H_{|\textbf{k}^1|-1}=H_{|\textbf{k}^1|+1}=\dots =H_{|\textbf{k}^2|-2}\equiv 0\), the components of the vector \(\textbf{k}^{1}\) change of sign, the components of the vector \(\textbf{k}^2\) do not change of sign and \(\Psi _2(\phi _2)\ne 0\) for all \(\phi _2\in [0,2\pi )\), with \(\Psi _2(\phi _2)\) as in Eq. 36, then the null solution of Eq. 1 is Lie stable.

Proof

The proof is followed in a similar way to that given for Theorem 4.4, because \(F_m=\left. H^{m}\right| _{S\times \mathbb {T}^2}\) is as in Eq. 39.\(\square \)

In what follows we present an example where it is possible to apply Theorem Eq. 4.2.

Example 2

Assume that the Hamiltonian Eq. 2 has four degrees of freedom with \(\mathcal {H}_2=3\sqrt{2} r_1-\sqrt{2}r_2+3\sqrt{3}r_3+\sqrt{3} r_4\). Then, \(M_\omega =\textbf{k}^1\mathbb {Z}+\textbf{k}^2\mathbb {Z}\), with \(\textbf{k}^1=(1,3,0,0) \) and \(\textbf{k}^2=(0,0, 1,-3).\) Now, in cartesian coordinates we perturbed by considering \(\mathcal {H}_3=0\) and

$$\begin{aligned} \begin{array}{rcl} \vspace{0.1cm} \mathcal {H}_4&{}=&{}\frac{1}{4} f_{2000} \left( x_1^2+y_1^2\right) {}^2 +\frac{1}{2} A_2 x_3 x_4^3-\frac{3}{2} x_2^2 y_1 y_2-\frac{3}{2} x_1 x_2 y_2^2+x_1 x_3 x_2 y_2+\frac{1}{2} x_1 x_2^3\\ \vspace{0.1cm} &{}&{}+\frac{1}{2} y_1 y_2^3+\frac{1}{4} f_{0200} \left( x_2^2+y_2^2\right) {}^2 +\frac{1}{4}f_{1100} \left( x_1^2+y_1^2\right) \left( x_2^2+y_2^2\right) \\ \vspace{0.1cm} &{}&{}+\frac{1}{4} f_{0020} \left( x_3^2+y_3^2\right) {}^2 +\frac{1}{4}f_{0110} \left( x_2^2+y_2^2\right) \left( x_3^2+y_3^2\right) +\frac{3}{2} \delta x_4^2 y_4 y_3+ y_4 y_3^3\\ \vspace{0.1cm} &{}&{}+\frac{1}{4} f_{1001} \left( x_1^2+y_1^2\right) \left( x_4^2+y_4^2\right) +\frac{1}{4} f_{0101} \left( x_2^2+y_2^2\right) \left( x_4^2+y_4^2\right) -\frac{3}{2} x_3 x_4 y_4^2\\ \vspace{0.1cm} &{}&{}-\frac{1}{2} \delta y_3 y_4^3+\frac{1}{4}f_{0002} \left( x_4^2+y_4^2\right) {}^2 +\frac{1}{4} f_{0011} \left( x_3^2+y_3^2\right) \left( x_4^2+y_4^2\right) \\ &{}&{}+\frac{1}{4}f_{1010} \left( x_1^2+y_1^2\right) \left( x_3^2+y_3^2\right) , \end{array} \end{aligned}$$
(40)

with \(\delta =1\), \(f_{0200}= 1, \ f_{1100}= 3, \ f_{200 0}= 9\) and the resting parameters are any arbitrary constant. Using the Lie normal form process, we have that \(H_3=0\) and the normalized terms of order four are given by

$$\begin{aligned} \begin{array}{rl} H_4=&{} f_{0110} r_2 r_3 + f_{0101} r_2 r_4+ f_{0020} r_3^2 +f_{0011} r_3 r_4 +f_{0002} r_4^2 +f_{1010} r_1 r_3 \\ &{}+f_{1001} r_1 r_4 + 9r_1^2 +3 r_1 r_2 + r_2^2+2 \sqrt{ r_1} r_2^{3/2} \cos \phi _1+2 A_2\sqrt{ r_3} r_4^{3/2} \cos \phi _2, \end{array} \end{aligned}$$

where \(\phi _j=\textbf{k}^j\cdot \varphi \) for \(j=1, 2\). Then the auxiliary function \(\Psi _1(\phi _1)\) in Eq. 36 has the form

$$\Psi _1(\phi _1)=H_4^0(\textbf{k}^1)+H_4^1(\textbf{k}^1,\phi _1)=27+6\sqrt{3}\cos \phi _1>0,\quad \forall \,\phi _1\in [0,2\pi ).$$

Thus, by Theorem 4.2 the null solution for this case is Lie stable.

For the following three theorems we are going to assume that the components of \(\textbf{k}^1\) and \(\textbf{k}^2\) do not change of sign.

Theorem 4.6

Suppose that \(|\textbf{k}^1|\) is even, \(4\le |\textbf{k}^1|<|\textbf{k}^2|\), \(H_4=\dots =H_{|\textbf{k}^1|-2}\equiv 0\) if \(6\le |\textbf{k}^1|\), the components of \(\textbf{k}^{1}\) and \(\textbf{k}^{2}\) do not change of sign and \(\Psi _1(\phi _1)\ne 0\) for all \(\phi _1\in [0,2\pi )\), with \(\Psi _1(\phi _1)\) as in Eq. 36, then the null solution of Eq. 1 is Lie stable.

Proof

The proof given in Theorem 4.2 is also valid for this case, because, by Lemma 2, it follows that \(S=\left\{ \textbf{r}=\dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{m_1+1}}{k_{m_1+1}^2}{} \textbf{k}^2\right\} \) and \(F_{m}=H^{m}|_{S\times \mathbb {T}^2}\) is as in Eq. 38.

For the next two theorems we will assume that \(|\textbf{k}^1|=|\textbf{k}^2|=4\) and it is necessary to re-write the homogeneous polynomial \(H_4^0(\textbf{r})\) given in Eq. 28 as

$$\begin{aligned} \begin{array}{rcl} H_4^0(\textbf{r}) &{}=&{}\displaystyle \sum _{1\le i,j \le m_1} A^{ij}r_i r_j+2\sum _{\begin{array}{c} 1\le i \le {m_1}\\ {m_1}+1\le j\le m_2 \end{array}} A^{ij} r_i r_j+\sum _{{m_1}+1\le i,j \le m_2} A^{ij}r_i r_j\\ &{}&{}+\displaystyle \sum _{{m_2}+1\le i,j \le n} A^{ij}r_i r_j, \end{array} \end{aligned}$$

thus,

$$\begin{aligned} \begin{array}{rcl} H_4^0\left( \dfrac{r_1}{k_1^1}{} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}} \textbf{k}^2\right) &{}=&{}H_4^0\left( \dfrac{r_1}{k_1^1}k_1^1,\ldots ,\dfrac{r_1}{k_1^1}k_{m_1}^1,\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2}k_{{m_1}+1}^2,\ldots , \dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2}k_{m_2}^2,0,\ldots ,0\right) \\ &{}=&{} \displaystyle \sum _{1\le i,j \le m_1} A^{ij}\dfrac{r_1}{k_1^1}k_i^1 \dfrac{r_1}{k_1^1}k_j^1+ \displaystyle 2\sum _{\begin{array}{c} 1\le i \le {m_1}\\ {m_1}+1\le j\le m_2 \end{array}} A^{ij} \dfrac{r_1}{k_1^1}k_i^1 \dfrac{r_{m_1}+1}{k_{{m_1}+1}^2}k_j^2\\ &{}&{}+\displaystyle \sum _{{m_1}+1\le i,j \le m_2} A^{ij}\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} k_i^2 \dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} k_j^2\\ &{}=&{} \dfrac{H_4^0(\textbf{k}^1)}{(k_1^1)^2} \ r_1^2+\dfrac{H_4^0(\textbf{k}^2)}{(k_{{m_1}+1}^2)^2} \ r_{{m_1}+1}^2 + \dfrac{C}{k_1^1 k_{{m_1}+1}^2} \ r_1 r_{{m_1}+1}, \end{array} \end{aligned}$$

where

$$\begin{aligned} C= 2\displaystyle \sum _{\begin{array}{c} 1\le i \le {m_1}\\ {m_1}+1\le j\le m_2 \end{array}} A^{ij} k_i^1 k_j^2=H_4^0(\textbf{k}^1+\textbf{k}^2)-H_4^0(\textbf{k}^1)-H_4^0(\textbf{k}^2). \end{aligned}$$
(41)

Theorem 4.7

Assume that \(|\textbf{k}^1|=|\textbf{k}^2|=4\), the components of \(\textbf{k}^j\) for \(j=1,2\) do not change of sign, \(\Psi _j(\phi _j)\ne 0\) for all \(\phi _j\in [0,2\pi )\) for \(j=1,2\), \(sg(\Psi _1(\phi _1))=sg(\Psi _2(\phi _2))=sg(C)\) or \(sg(\Psi _1(\phi _1))=sg(\Psi _2(\phi _2))\) whether \(C=0\), then the null solution of Eq. 1 is Lie stable.

Proof

By Lemma 2, we have that \(S=\left\{ \textbf{r}=\dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2\right\} \), then taking \(\textbf{r}\in S\), it follows that

$$\begin{aligned} \begin{array}{rcl} \ F_{2m}&{}=&{}H^{2m}|_{S\times \mathbb {R}^2}\\ &{}=&{}H^{2m}\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2,\phi _1,\phi _2\right) \\ &{}=&{}H_4\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2,\phi _1,\phi _2\right) +\cdots +H_{2m}\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2,\phi _1,\phi _2\right) \\ &{}=&{}H_4^0\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2\right) +H_4^1\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2,\phi _1\right) \\ &{}&{}+H_4^2\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2,\phi _2\right) +\cdots +H_{2m}\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2,\phi _1,\phi _2\right) \\ &{}=&{}\dfrac{r_1^2}{(k_1^1)^2} H_4^0(\textbf{k}^1)+\dfrac{r_{{m_1}+1}^2}{(k_{{m_1}+1}^2)^2} H_4^0(\textbf{k}^2)+ \dfrac{r_1 r_{{m_1}+1}}{k_1^1 k_{{m_1}+1}^2} C+\dfrac{r_1^2}{(k_1^1)^2} H_4^1(\textbf{k}^1,\phi _1)\\ &{}&{}+\dfrac{r_{{m_1}+1}^2}{(k_{{m_1}+1}^2)^2} H_4^2(\textbf{k}^2,\phi _2)+\cdots +H_{2m}\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2,\phi _1,\phi _2\right) \\ &{}=&{}\dfrac{\Psi _1(\phi _1)}{(k_1^1)^2}\ r_1^2+\dfrac{ \Psi _2(\phi _2)}{(k_{{m_1}+1}^2)^2} \ r_{{m_1}+1}^2+\dfrac{C }{k_1^1 k_{{m_1}+1}^2}\ r_1 r_{{m_1}+1}+\cdots \\ &{}&{}+H_{2m}\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2,\phi _1,\phi _2\right) . \end{array} \end{aligned}$$
(42)

Therefore, using the hypothesis \(sg(\Psi _1(\phi _1))=sg(\Psi _2(\phi _2))=sg(C)\), the quadratic function

$$ \dfrac{\Psi _1(\phi _1)}{(k_1^1)^2}\ r_1^2+\dfrac{ \Psi _2(\phi _2)}{(k_{{m_1}+1}^2)^2} \ r_{{m_1}+1}^2+\dfrac{C }{k_1^1 k_{{m_1}+1}^2}\ r_1 r_{{m_1}+1}, $$

is of definite sign, so the function \(F_{2m}\ne 0\) for all \((r_1,r_{{m_1}+1},\phi _1, \phi _2)\) with \(r_1, r_{{m_1}+1}\) small enough, then the result follows from Theorem A.1. \(\square \)

Now, an example where we can apply the previous theorem is the following.

Example 3

We consider the Hamiltonian Eq. 2 with four degrees of freedom such that

$$\begin{aligned} \mathcal {H}_2=3\sqrt{2} r_1-\sqrt{2}r_2+3\sqrt{3}r_3-\sqrt{3} r_4. \end{aligned}$$
(43)

Then, \(M_\omega =\textbf{k}^1\mathbb {Z}+\textbf{k}^2\mathbb {Z}\) where \(\textbf{k}^1=(1,3,0,0) \) and \(\textbf{k}^2=(0,0, 1,3).\) Now, in cartesian coordinates we take the perturbed terms \(\mathcal {H}_3=0\) and \(\mathcal {H}_4\) as in Eq. 40 but considering

$$\begin{aligned} \begin{array}{llll} \delta =-1, &{} f_{0200}= 1,&{}f_{1100}= 3,&{}f_{2000}= 9, \\ f_{1010}= 1,&{}f_{0002}=1,&{}f_{0011}= 3, &{}f_{0020}=9,\\ f_{0101}= 1,&{}f_{0110}= 1,&{}f_{1001}= 1, &{} A_2=1. \end{array} \end{aligned}$$

Through the Lie normal form process, we verify that \(H_3=0\) and the normalized terms of order four are given by

$$\begin{aligned} \begin{array}{rcl} H_4&{}=&{} 9 {r_1}^2+3 {r_1} {r_2}+{r_1} {r_3}+{r_1} {r_4}+{r_2}^2+{r_2} {r_3}+{r_2} {r_4}+9 {r_3}^2+3 {r_3} {r_4}+{r_4}^2\\ &{}&{}+2 \sqrt{ r_1} r_2^{3/2} \cos \phi _1+2 \sqrt{ r_3} r_4^{3/2} \cos \phi _2, \end{array} \end{aligned}$$
(44)

where \(\phi _j=\textbf{k}^j\cdot \varphi \) for \(j=1, 2\). Then, for \(j=1, 2\) we have that

$$\Psi _j(\phi _j)=27+6\sqrt{3}\cos \phi _j>0,\quad \forall \,\phi _j\in \mathbb {R},$$

and the constant defined in Eq. 41 is \(C=16\). So, by Theorem 4.7 the null solution of this system is Lie stable.

Theorem 4.8

Assume that \(|\textbf{k}^1|=|\textbf{k}^2|=4\), the components of \(\textbf{k}^j\) do not change of sign, \(\Psi _j(\phi _j)\ne 0\) for all \(\phi _j\in [0,2 \pi )\), for \(j=1,2\) and \(C^2-4 \Psi _1(\phi _1) \Psi _2(\phi _2)<0\), then the null solution of Eq. 1 is Lie stable.

Proof

The function \(F_m\) given in Eq. 42, can be re-written as

$$\begin{aligned} F_{2m}= & {} \dfrac{r_1^2}{(k_1^1)^2}\left( \Psi _1(\phi _1)+\dfrac{(k_1^1)^2}{r_1^2}\dfrac{r_{{m_1}+1}^2}{(k_{{m_1}+1}^2)^2}\Psi _2(\phi _2)+\dfrac{k_1^1 r_{{m_1}+1}}{r_1 k_{{m_1}+1}^2}C\right) +\cdots \\{} & {} +H_{2m}\left( \dfrac{r_1}{k_1^1} \textbf{k}^1+\dfrac{r_{{m_1}+1}}{k_{{m_1}+1}^2} \textbf{k}^2,\phi _1,\phi _2\right) . \end{aligned}$$

Let \(x=\frac{k_1^1 }{ k_{{m_1}+1}^2} \frac{r_{{m_1}+1}}{r_1} \), then the quadratic function \(f(x)= \Psi _1(\phi _1)+C x+\Psi _2(\phi _2) x^2\) is of sign definite whether \(C^2-4 \Psi _1(\phi _1) \Psi _2(\phi _2)< 0\). So by hypothesis, we have that \(F_{2m}\ne 0\) for all \((r_1,r_{{m_1}+1},\phi _1, \phi _2)\) with \(r_1, r_{{m_1}+1}\) small enough, then the result follows from Theorem A.1. \(\square \)

Remark 4

Note that the previous theorem also is true if \(C=0\).

The above theorem can be applied in the following example.

Example 4

Assume we have a Hamiltonian as in Eq. 2 with four degrees of freedom, with \(\mathcal {H}_2\) as in Eq. 43, \(\mathcal {H}_3=0\) and \(\mathcal {H}_4\) is as in Eq. 40, but here

$$\begin{aligned} \begin{array}{llll} \delta =-1, &{} f_{0200}= 1,&{}f_{1100}= 3,&{}f_{2000}= 9, \\ f_{1010}= 1,&{}f_{0002}=1,&{}f_{0011}= 3, &{}f_{0020}=9,\\ f_{0101}= -1,&{}f_{0110}= -1,&{}f_{1001}= 1, &{} A_2=1. \end{array} \end{aligned}$$

Then \(H_3=0\) and the normalized terms of order four are given by

$$\begin{aligned} \begin{array}{rcl} H_4&{}=&{} 9 {r_1}^2+3 {r_1} {r_2}+{r_1} {r_3}+{r_1} {r_4}+{r_2}^2-{r_2} {r_3}-{r_2} {r_4}+9 {r_3}^2+3 {r_3} {r_4}+{r_4}^2\\ &{}&{}+2 \sqrt{ r_1} r_2^{3/2} \cos \phi _1+2 \sqrt{ r_3} r_4^{3/2} \cos \phi _2, \end{array} \end{aligned}$$

where \(\phi _j=\textbf{k}^j\cdot \varphi \) for \(j=1, 2\). Thus, for \(j=1, 2\) we have that

$$\Psi _j(\phi _j)=H_4(\textbf{k}^j,\phi _1,\phi _2)=27+6\sqrt{3}\cos \phi _j>0,\quad \forall \,\phi _j\in [0,2\pi ),$$

and the constant defined in Eq. 41 is \(C=-8\). Subsequently, we obtain that

$$C^2-4 \Psi _1(\phi _1)\Psi _2(\phi _2)<64-4(27-6\sqrt{3})^2\approx -1039.26<0, \quad \forall \, \phi _1,\phi _2\in \mathbb {R}.$$

In consequence, by Theorem 4.8 the null solution of this system is Lie stable.

5 Nekhoroshev Stability

In case of Lie stable systems we can bound the solutions of the full system Eq. 1 near the equilibrium over exponentially long times. In other words, we speak of exponential stability in the Nekhoroshev sense. First of all, in this section we are going to assume that we have four degrees of freedom and that the components of at least one of the resonance vectors do not change of sign. Furthermore, we are going to suppose that it is possible to guarantee the Lie stability, in other words, that the hypotheses given in Theorems 4.2 or 4.3 or 4.4 or 4.5 or 4.6 or 4.7 or 4.8 are satisfied. Finally, we will show under what conditions it is possible to apply Theorem A.2 given in the appendix to ensure the exponential stability of the equilibrium solution. Later on, we will generalize the ideas to n-DOF.

Before to enunciate the following theorem we recall the concept of algebraic number. We say that \(\alpha \) is an algebraic number if it is a root of a non-zero polynomial in one variable with integer (or, equivalently, rational) coefficients. If an algebraic number is a solution of a polynomial equation of degree \(d>0\), and it is not a solution of a polynomial equation of lesser degree \(l<d\), then it is said to be an algebraic number of degree d [1]. For example, the golden ratio, \( (1+{\sqrt{5}})/2\), is an algebraic number of the degree 2, because it is a root of the polynomial \(x^2-x-1\). According to Liouville’s Theorem [27] any algebraic number satisfies the Diophantine conditions. Now, we are in position to announce the following result, concerning to Nekhoroshev type-estimates.

Theorem 5.1

Suppose that the hypothesis of Theorem 4.2 or 4.3 or 4.4 or 4.5 or 4.6 or 4.7 or 4.8 are satisfied and \(\frac{\omega _2 k_3^2}{\omega _4 k_1^1} \) (or \(\frac{\omega _4 k_1^1}{\omega _2 k_3^2} \)) is an algebraic real number of degree \(d\ge 2\), then there exist \(C>0\), \(K>0\), \(a>1\) and \(\varepsilon _0>0\) such that for all \(\varepsilon \in (0,\varepsilon _0)\), and for all \(\textbf{z}_0\) with \(|\textbf{z}_0|<\varepsilon \), the solution \( \textbf{z}(t,\textbf{z}_0)\) of system Eq. 1 satisfies that

$$|\textbf{z}(t,\textbf{z}_0)|<a\varepsilon ^{1/|\textbf{k}^1|} \quad \text {for all } \ t \ \text { with } \ 0\le t\le C\exp \left( \frac{K}{\varepsilon ^{1/d}}\right) . $$

Proof

If the hypotheses of Theorem 4.2 or 4.3 or 4.4 or 4.5 or 4.6 or 4.7 or 4.8 are satisfied, then \(H^{|\textbf{k}^1|}(\textbf{r},\phi _1,\phi _2)\ne 0\) for \(\textbf{r}\in S\setminus \{\textbf{0}\}\), \(|\textbf{r}|\) small enough and for all \(\phi _1\), \(\phi _2\). On the other hand, the quadratic part has no definite sign and is given by

$$\begin{aligned} H_2=\omega _1r_1+\omega _2 r_2+\omega _3 r_3+\omega _4 r_4, \end{aligned}$$

and since we have 4-DOF its resonances vectors associated (without loss of generality) are \(\textbf{k}^1=(k_1^1,k_2^1,0,0)\) and \(\textbf{k}^2=(0,0,k_3^2,k_4^2)\). This implies that \(\omega _1=-\frac{k_2^1}{k_1^1}\omega _2\) and \(\omega _3=-\frac{k_4^2}{k_3^2}\omega _4\), then substituting in \(H_2\) we have that

$$\begin{aligned} H_2=\dfrac{\omega _2}{k_1^1}(-k_2^1 r_1+k_1^1 r_2)+\dfrac{\omega _4}{k_3^2}(-k_4^2 r_3+k_3^2 r_4)=\dfrac{\omega _2}{k_1^1} I_1+\dfrac{\omega _4}{k_3^2}I_2. \end{aligned}$$

where \(I_1\) and \(I_2\) are formal first integrals defined in Eq. 32.

We know that \(\frac{\omega _2 k_3^2}{\omega _4 k_1^1}\) (or \(\frac{\omega _4 k_1^1}{\omega _2 k_3^2}\)) is an algebraic real number of degree \(d\ge 2\), then by Liouville’s Theorems for Diophantine approximations, see [27], there exists A such that for any \(p/q\ne \dfrac{\omega _2 k_3^2}{\omega _4 k_1^1}\) rational (with \(q\ne 0\) and p an integer number), we have

$$\left| \dfrac{\omega _2 k_3^2}{\omega _4 k_1^1}-\frac{p}{q}\right| \ge \frac{A}{|q|^d},$$

or what is the same

$$\left| (\omega _2 k_3^2,\omega _4 k_1^1)\cdot (q,-p)\right| \ge \frac{A|\omega _4 k_1^1|}{|q|^{d-1}}\ge \frac{A|\omega _4 k_1^1|}{|(q,-p)|^{d-1}}\quad \text {for all}\quad p\in \mathbb {Z}.$$

In consequence, for all \(\textbf{k}\in \mathbb {Z}^2\setminus \{\textbf{0}\}\) there are fixed constant \(c=A|\omega _4 k_1^1|\) and \(\nu =d-1\ge 1\) such that

$$|\textbf{k}\cdot {\varvec{\sigma }}|\ge c|\textbf{k}|^{-\nu },$$

with \(\varvec{\sigma }=(\omega _2 k_3^2,\omega _4 k_1^1)\). In other words, the Diophantine conditions given in Eq. A.3 is satisfied. So by Theorem A.2 there exist \(C>0\), \(K>0\), \(a>1\) and \(\varepsilon _0>0\) such that for all \(\varepsilon \in (0,\varepsilon _0)\), and for all \(\textbf{z}_0\) with \(|\textbf{z}_0|<\varepsilon \) we have

$$|\textbf{z}(t,\textbf{z}_0)|<a\varepsilon ^{1/|\textbf{k}^1|} \quad \text {for all} \ t \ \text {with} \ 0\le T\le C\exp \left( \frac{K}{\varepsilon ^{1/d}}\right) .$$

\(\square \)

Within the hypotheses of Theorem 5.1, saying that \(\frac{\omega _2 k_3^2}{\omega _4 k_1^1} \) is an algebraic number is simply to guarantee the Diophantine condition and the theorem has been written in that way because here in this work we will present a examples where we apply the Liouville’s Theorem.

Example 5

Firstly, suppose that \(H_2\) and \(H_4\) is as in Example 2 or Example 3 or Example 4, then the hypothesis of Theorem 4.2 or Theorem 4.7 or Theorem 4.8 are satisfied, respectively. Secondly, \(\frac{\omega _2 k_3^2}{\omega _4 k_1^1}=\sqrt{2/3}\) is an algebraic number of degree \(\rho =2\). Then, there exist \(C>0\), \(K>0\), \(a>1\) and \(\varepsilon _0>0\) such that for all \(\varepsilon \in (0,\varepsilon _0)\), and for all \(|\textbf{z}_0|<\varepsilon \), the solution \(\textbf{z}(t,\textbf{z}_0)\) of the system Eq. 1 satisfies that

$$|\textbf{z}(t,\textbf{z}_0)|<a\varepsilon ^{1/2} \quad \text {for all} \ t \ \text {with} \ 0\le t\le C\exp \left( \frac{K}{\varepsilon ^{1/2}}\right) .$$

6 Results About Instability

In this section, our objective is to obtain results about the instability of the null equilibrium solution for the full system Eq. 1. The arguments of our proof will make use of the classic Chetaev’s Theorem for autonomous Hamiltonian systems. According with Theorem 4.1, if the components of \(\textbf{k}^1\) and \(\textbf{k}^2\) change of sign, then the null solution of system Eq. 1 is Lie stable. In consequence, in order to get results concerning with the instability of the null equilibrium solution we are going to assume that the components of at least one of the resonance vectors \(\textbf{k}^j\) do not change of sign. In addition, if \(|\textbf{k}^j|\) is even, it will be necessary to incorporate other hypothesis, which is related to the auxiliary function \(\Psi _j\) defined in Eq. 36.

When using a Chetaev’s function, one of the main ideas is to manage to write all the action variables in a function of only one of them, in order to show that the derivative of the Chetaev’s function along the solutions is of sign definite. Furthermore, we must consider that the resonance vector that produces the instability should be involved in some way in the Chetaev’s function. In consequence, we are going to propose a Chetaev’s function that consists of the product of three functions, as follows, \(V=V_1 V_2 V_3\). The function \(V_1\) will be built using some formal first integrals and the function \(V_2\) will be built using some action variables, and both functions will allow us to characterize the points inside the set \(\Omega \) (set of Chetaev’ Theorem where V is defined), managing to write all the action variables as a function of one of them. The function \(V_3\) will be linked to resonance vector \(\textbf{k}^j\) that produces the instability and will depend on some action variables and an angle \(\phi _j=\textbf{k}^j\cdot \varphi \).

This section will be divided in two subsections, in Section 6.1 we study the case when \(3\le |\textbf{k}^1|\le |\textbf{k}^2|\) and the components of \(\textbf{k}^1\) do not change of sign. On the other hand, in Section 6.2, we analyze the case when \(3\le |\textbf{k}^1|<|\textbf{k}^2|\), the components of \(\textbf{k}^1\) change of sign and the components of \(\textbf{k}^2\) do not change of sign.

6.1 The Components of \(\textbf{k}^1\) do not Change of Sign and \(3\le |\textbf{k}^1|\le |\textbf{k}^2|\)

In this section, to prove that the null solution of the system Eq. 1 is unstable, it will be necessary to normalize the first \(|\textbf{k}^1|+2\) terms of the Hamiltonian function, i.e., we are going to assume

$$\begin{aligned} H=H(\textbf{r},\phi _1,\phi _2)=H^{|\textbf{k}^1|+2}(\textbf{r},\phi _1,\phi _2)+R(\textbf{r},\varphi ), \end{aligned}$$
(45)

where \(R(\textbf{r},\varphi )=\mathcal {O}(||\textbf{r}||^{|\textbf{k}^1|/2+3/2})\) and

$$H^{|\textbf{k}^1|+2}=H_2 +H_\star +H_4+\cdots +H_{|\textbf{k}^1|-\nu }+H_{|\textbf{k}^1|} +H_{|\textbf{k}^1|+1}+H_{|\textbf{k}^1|+2},$$

with \(H_\star \) and \(\nu \) as in Eq. 30. For every \(j=4, \ldots , |\textbf{k}^1|-\nu \), \(H_j\) is an homogeneous polynomial of degree j/2 in the action variable \(\textbf{r}\).

The hypotheses associated with the following theorems are related with the normalize Hamiltonian function up to \(|\textbf{k}^1|+2\) order and some equations of the system will be necessary for the proof. So, if \(|\textbf{k}^1|\) is odd, we must have clear that

$$\begin{aligned} \left. \begin{array}{rcl} \displaystyle \dot{r}_j=\{r_j,H\}&{}=&{}-2k_j^1A_1R_1\sin \phi _1+\dfrac{\partial H_{|\textbf{k}^1|+1}}{\partial \varphi _j }+\dfrac{\partial H_{|\textbf{k}^1|+2}}{\partial \varphi _j }+\dfrac{\partial R}{\partial \varphi _j },\quad j=1,\ldots ,m_1,\\ \dot{\phi }_1=\{\phi _1,H\}&{}=&{}- \textbf{k}^1\cdot \left( \dfrac{\partial H_\star }{\partial \textbf{r} }+\dfrac{\partial H_4}{\partial \textbf{r} }+\cdots +\dfrac{\partial H_{|\textbf{k}^1|-1}}{\partial \textbf{r} }\right) - A_1 \cos \phi _1 \left( \displaystyle \sum _{j=1}^{m_1} k_j^1 \dfrac{|k_j^1|}{2} \dfrac{R_1}{r_j} \right) \\ &{}&{}- \textbf{k}^1\cdot \left( \dfrac{\partial H_{|\textbf{k}^1|+1}}{\partial \textbf{r} }+\dfrac{\partial H_{|\textbf{k}^1|+2}}{\partial \textbf{r} }+\dfrac{\partial R}{\partial \textbf{r} }\right) . \end{array} \right. \end{aligned}$$
(46)

And if \(|\textbf{k}^1|\) is even, \(\dot{r}_j=\{r_j,H\}\) is as in Eq. 46 for \(j=1,\ldots , m_1\) and

$$\begin{aligned} \left. \begin{array}{ll} \dot{\phi }_1=\{\phi _1,H\}=&{}- \textbf{k}^1\cdot \left( \dfrac{\partial H_\star }{\partial \textbf{r} }+\dfrac{\partial H_4}{\partial \textbf{r} }+\cdots +\dfrac{\partial H_{|\textbf{k}^1|-2}}{\partial \textbf{r} }\right) \\ &{}-\displaystyle \dfrac{|\textbf{k}^1|}{2} \sum _{\begin{array}{c} {1\le j_1\le m_1}\\ {1\le j_2,\ldots ,j_{{\sigma _1}}\le n} \end{array}} A^{j_1 j_2 \cdots j_{{\sigma _1}}} k_{j_1}^1 r_{j_2} \cdots r_{j_{\sigma _1}} - A_1 \cos \phi _1 \left( \sum _{j=1}^{m_1} k_j^1 \dfrac{|k_j^1|}{2} \dfrac{R_1}{r_j} \right) \\ &{}- \textbf{k}^1\cdot \left( \dfrac{\partial H_{|\textbf{k}^1|+1}}{\partial \textbf{r} }+\dfrac{\partial H_{|\textbf{k}^1|+2}}{\partial \textbf{r} }+\dfrac{\partial R}{\partial \textbf{r} }\right) . \end{array} \right. \end{aligned}$$

Next, we state our first result when \(3= |\textbf{k}^1|\le |\textbf{k}^2|\).

Theorem 6.1

Assume that \(3= |\textbf{k}^1|\le |\textbf{k}^2|\), the components of \(\textbf{k}^1\) do not change of sign. Then the null solution of system Eq. 1 is unstable in the Lyapunov sense.

Proof

To prove the instability in the Lyapunov sense of the origin, we consider the Hamiltonian function in Eq. 45, that is, we need to normalize the Hamiltonian Eq. 2 up to order \(|\textbf{k}^1|+2\), inclusively. Second, Chetaev’s Theorem for autonomous Hamiltonian systems will be used.

For our proof, we propose the following Chetaev’s function \(V=V_1 V_2 V_3\), where

$$\begin{aligned} V_1=\sum _{j=1}^{m_1-1}I_j^2+\sum _{j=m_2-1}^{n-2}I_j^2- r_1^{3}, \quad V_2= r_1^{3/2}-\sum _{j=m_1+1}^{m_2}r_j, \quad V_3= R_1\sin \phi _1, \end{aligned}$$
(47)

with \(R_1\) as in Eq. 26. If \(A_1>0\), we take the region \(V>0\) by the following connected component \(V_1<0\), \(V_2>0\) and \(V_3<0\), that is, the region given by

$$\begin{aligned} \Omega _1=\{(\textbf{r},\phi _1): \ V_1<0, \ V_2> 0 , \ \phi _1\in (\pi ,2\pi ), \ 0< \Vert \textbf{r}\Vert <a \}, \end{aligned}$$
(48)

with \(a>0\) is sufficiently small. While, if \(A_1<0\), we consider the region \(V<0\), assuming \(V_1<0\), \(V_2>0\), \(V_3>0\), explicitly we take the region

$$\begin{aligned} \Omega _2=\{(\textbf{r},\phi _1): \ V_1<0, \ V_2> 0 , \ \phi _1\in (0,\pi ), \ 0< \Vert \textbf{r}\Vert <a\}. \end{aligned}$$
(49)

Of course, the boundary of the set \(V>0\) (respectively, \(V<0\)) will be \(V_1=0\) or \(V_2=0\) or \(V_3=0\).

Our objective from now on that is to prove that the pair \((V, \Omega _j)\) satisfies all the conditions of the classic Chetaev’s Theorem for autonomous Hamiltonian systems.

Note that, the set \(\Omega _j\) is non-empty, in fact, since the components of \(\textbf{k}^1\) do not change of sign, it is enough to consider the point \(\left( \frac{r_1}{k_1^1}{} \textbf{k}^1,\phi _1\right) \), with \(0<r_1<a\) and \(\phi _1\in \Omega _j\).

In the interior of the region \(\Omega _j\), since \(V_1<0\), we have that \(I_j<r_1^{3/2}\) for every \(j=1,\ldots ,m_1-1,m_2-1,\ldots ,n-2\), then there are real functions \(f_j\) such that \(I_j=f_j r_1^{3/2}\) with \(|f_j|<1\) for \(j=1,\ldots ,m_1-1\) and \(h_j\) such that \(I_j=h_j r_1^{3/2}\) with \(0<h_j<1\) for \(j=m_2-1,\ldots ,n-2\). In addition, from \(V_2>0\), we get \(r_{j}<r_1^{3/2}\) for every \(j=m_1+1,\ldots ,m_2\), then there exit real functions \(g_j\) such that \(r_{j}=g_j r_1^{3/2} \) with \(0<g_j<1\) for \(j=m_1+1,\ldots ,m_2\). From here, we obtain the following important relations which are valid inside \(\Omega _j\),

$$\begin{aligned} \begin{aligned} r_j&=\frac{k_{j}^1}{k_1^1}r_1+ \dfrac{f_{j-1}}{k_1^1} r_1^{3/2},\quad j=2,\ldots ,m_1;\\ r_j&= g_j r_1^{3/2}, \quad j=m_1+1,\ldots ,m_2;\\ r_j&= h_{j-2} r_1^{3/2}, \quad j=m_1+1,\ldots ,n. \end{aligned} \end{aligned}$$
(50)

Note that all the action variables are well defined in a sufficiently small neighborhood of the origin.

As, \(V_1<0\) and \(V_2>0\) on \(\Omega _j\), substituting the relations \(I_j^2=f_j^2 r_1^{3}\) on \(V_1\) and \(r_j=g_j r_1^{3/2}\) on \(V_2\), we arrive to \(V_1=(M_1-1)r_1^{3}<0\) and \(V_2=(1-M_2)r_1^{3/2}>0\) on \(\Omega _j\) with

$$\begin{aligned} 0<M_1=\sum _{j=1}^{m_1-1} f_j^2+\sum _{j=m_2-1}^{n-2} f_j^2<1\quad \text {and}\quad 0<M_2=\sum _{j=m_1+1}^{m_2} g_j<1. \end{aligned}$$

Next, we are going to compute the derivative of V through the solutions of the Hamiltonian system associated with H given in Eq. 45, that is,

$$\begin{aligned} \begin{array}{l} \dot{V}=\{V,H\}= \{ V_1,H\}V_2 V_3+V_1 \{V_2,H\} V_3+V_1 V_2\{V_3,H\}, \end{array} \end{aligned}$$
(51)

with

$$\begin{aligned}{} & {} \{V_1,H\}=2\sum _{j=1}^{m_1-1}I_j \{I_j,H\}+2\sum _{j=m_2-1}^{n-2}I_j \{I_j,H\}-3 r_1^{2} \{r_1,H\},\\{} & {} \{V_2,H\}=\frac{3}{2} r_1^{1/2} \{r_1,H\}-\sum _{j=m_1+1}^{m_2}\{r_j,H\},\\{} & {} \{V_3,H\}=\left( \sum _{j=1}^{m_1}\dfrac{\partial R_1}{\partial r_j}\{r_j,H\}\right) \sin \phi _1+\{\phi _1,H\} R_1 \cos \phi _1. \end{aligned}$$

Using the relations Eq. 50 and considering \(R_j\) as in Eq. 26 for \(j=1,2\), then inside the region \(\Omega _j\), we arrive to

$$\begin{aligned} \begin{array}{l} R_1=P_1\left( \frac{r_1}{k_1^1}\right) ^{|\textbf{k}^1|/2}+\mathcal {O}\left( r_1^{|\textbf{k}^1|/2+1/2}\right) , \\ \frac{R_1}{r_j}=\frac{P_1}{k_j^1}\left( \frac{r_1}{k_1^1}\right) ^{|\textbf{k}^1|/2-1}+\mathcal {O}\left( r_1^{|\textbf{k}^1|/2-1/2}\right) \quad \text {for} \ \ j=1,\ldots ,m_1,\\ R_2=\mathcal {O}\left( r_1^{( 3|\textbf{k}^2|)/4}\right) , \end{array} \end{aligned}$$
(52)

with \(P_1\) as in Eq. 37.

Additionally, by the relations Eq. 50 and considering \(R(\textbf{r},\varphi )\) as in Eq. 45, it follows that \(\dfrac{\partial R}{\partial \varphi _j}=\mathcal {O}\left( r_1^{|\textbf{k}^1|/2+3/2}\right) \) for \(j=1\ldots ,n\), \(I_j=\mathcal {O}(r_1)\) and \(\dot{I}_j=\mathcal {O}\left( r_1^{|\textbf{k}^1|/2+3/2}\right) \) for \(j=1,\ldots ,m_1-1,m_2-1,\ldots ,n-2\).

From the first equation in Eq. 52, it is clear that \(V_3=P_1\sin \phi _1 \left( \frac{r_1}{k_1^1}\right) ^{|\textbf{k}^1|/2}+\mathcal {O}\left( r_1^{|\textbf{k}^1|/2+1/2}\right) \). So, after some straightforward manipulations, we obtain the following for the region \(\Omega _j\),

$$\begin{aligned} \{V_1,H\} V_2 V_3= & {} \dfrac{3 A_1 P_1^2}{(k_1^1)^{|\textbf{k}^1|-1}} (1-M_2)\sin ^2\phi _1 r_1^{|\textbf{k}^1|+7/2}+\mathcal {O}(r_1^{|\textbf{k}^1|+4}),\\ \{V_2,H\} V_1V_3= & {} \dfrac{3 A_1 P_1^2}{2(k_1^1)^{|\textbf{k}^1|-1}}(1-M_1) \sin ^2\phi _1 r_1^{|\textbf{k}^1|+7/2}+\mathcal {O}(r_1^{|\textbf{k}^1|/2+3+\beta }),\\ \{V_3,H\}V_1 V_2= & {} \dfrac{|\textbf{k}^1| A_1 P_1^2}{2(k_1^1)^{|\textbf{k}^1|-1}}(1-M_1)(1-M_2) r_1^{|\textbf{k}^1|+7/2}+\mathcal {O}(r_1^{|\textbf{k}^1|+4}), \end{aligned}$$

where

$$\begin{aligned} \beta =\left\{ \begin{array}{ll} 9/4, &{}if \ |\textbf{k}^2|=|\textbf{k}^1|=3,\\ |\textbf{k}^1|/2+1, &{} if \ |\textbf{k}^2|>|\textbf{k}^1|=3. \end{array}\right. \end{aligned}$$

Note that,

$$\begin{aligned} \dfrac{3}{2} r_1^{1/2} \{r_1,H\}=- \dfrac{3A_1 P_1}{2(k_1^1)^{|\textbf{k}^1|/2-1}}\sin \phi _1 r_1^{|\textbf{k}^1|/2+1/2}+\mathcal {O}(r_1^{|\textbf{k}^1|/2+1}), \end{aligned}$$
(53)

and in the case \(|\textbf{k}^1|=|\textbf{k}^2|=3\), then from the third equation in Eq. 52, for each \(j=m_1+1,\ldots , m_2\), we get

$$\begin{aligned} \{r_j,H\}=-A_2 R_2 \sin \phi _2+\dfrac{\partial H_{|\textbf{k}^1|+1}}{\partial \varphi _j}+\dfrac{\partial H_{|\textbf{k}^1|+2}}{\partial \varphi _j}+\dfrac{\partial R}{\partial \varphi _j}=\mathcal {O}\left( r_1^{(3 |\textbf{k}^2|)/4}\right) =\mathcal {O}(r_1^{9/4}), \end{aligned}$$
(54)

then, \(\{V_2, H\}=- \dfrac{3A_1 P_1}{2(k_1^1)^{1/2}}\sin \phi _1 r_1^{2}+\mathcal {O}(r_1^{9/4})\). While, in the case \(|\textbf{k}^2|>|\textbf{k}^1|=3\), it is clear that \(\{V_2, H\}\) is exactly equal to the right hand side of Eq. 53.

Therefore, using all the previous information on \(\Omega _j\), we get

$$\begin{aligned} \begin{array}{rcl} \dot{V}&{}=&{}\dfrac{A_1 P_1^2}{(k_1^1)^{|\textbf{k}^1|-1}}\left[ \left( 3(1-M_2)+\dfrac{3}{2}(1-M_1)\right) \sin ^2\phi _1\right. \\[1pc] &{}&{}\left. +\dfrac{|\textbf{k}^1|}{2}(1-M_1)(1-M_2) \right] r_2^{|\textbf{k}^1|+7/2}+\mathcal {O}\left( r_1^{|\textbf{k}^1|/2+3+\beta }\right) . \end{array} \end{aligned}$$

Since, \(0<M_1<1\), \(0<M_2<1\) on \(\Omega _j\), moreover, if \(A_1>0\), it follows that \(\dot{V}>0\) on \(\Omega _1\) and if \(A_1<0\), it is clear that \(\dot{V}<0\) on \(\Omega _2\), so the proof is completed. \(\square \)

Next, we present a more general result, assuming that \(|\textbf{k}^1|\) is odd, \(5\le |\textbf{k}^1|\le |\textbf{k}^2|\) and for this we must incorporate other hypothesis.

Theorem 6.2

Assume that \(|\textbf{k}^1|\) is odd, \(5\le |\textbf{k}^1|\le |\textbf{k}^2|\), the components of the vector \(\textbf{k}^1\) do not change of sign and \(H_4=\dots = H_{|\textbf{k}^1|-1}\equiv 0\). Then the null solution of system Eq. 1 is unstable in the Lyapunov sense.

Proof

The proof is analogous to that of Theorem 6.1, the unique difference is that in this case \(\beta =|\textbf{k}^1|/2+1\), because if for example, \(|\textbf{k}^1|=|\textbf{k}^2|\), from Eq. 54, it follows that \(\{r_j,H\}=\mathcal {O}\left( r_1^{(3 |\textbf{k}^2|)/4}\right) \) for \(j=m_1+1,\ldots , m_2\) and as \(5\le |\textbf{k}^1|= |\textbf{k}^2|\), from Eq. 53, we get \(\{V_2, H\}\) is exactly equal to the right hand side of Eq. 53. Otherwise, if \(5\le |\textbf{k}^1|\le |\textbf{k}^2|\) it is clear that we obtain the same expression for \(\{V_2, H\}\) in \(\Omega _j\). \(\square \)

Next, we state our first result assuming that \(4=|\textbf{k}^1|\le |\textbf{k}^2|\) and it will be necessary to incorporate one extra hypothesis associated with the auxiliary function \(\Psi _1\).

Theorem 6.3

Assume that \(4= |\textbf{k}^1|\le |\textbf{k}^2|\), the components of the vector \(\textbf{k}^1\) do not change of sign and there exists \(\phi _1^*\in (0,2\pi )\) such that \(\Psi _1(\phi _1^*)=0\) and \(\Psi '_1(\phi _1^*)\ne 0\). Then the null solution of system Eq. 1 is unstable in the Lyapunov sense.

Proof

Again, to prove the instability it will be necessary to normalize the Hamiltonian function Eq. 2 up to order \(|\textbf{k}^1|+2\), inclusively. Next, we propose the following Chetaev’s function \(V=V_1 V_2 V_3\) with \(V_1\), \(V_2\) and \(V_3\) as in Eq. 47 and we follow the same ideas of the proof of Theorem 6.1.

Note that, by the periodicity of the function \(\Psi _1(\phi _1)\) it is possible to take the simple zero \(\phi _1^*\) satisfying \(\pi<\phi _1^*<2\pi \), or satisfying \(0<\phi _1^*<\pi \), then we can work with the sets \(\Omega _1\) and \(\Omega _2\), defined in Eqs. 48 and 49, respectively.

In this case, using the relations Eq. 50 inside the region \(\Omega _j\), we get \(R_1\), \(R_2\) and \(\frac{R_1}{r_j}\) for \(j=1,\ldots ,m_1\) is as in Eq. 52.

After some calculations, it is verified that

$$\begin{aligned} \{V_1,H\} V_2 V_3= & {} -\dfrac{3 P_1}{(k_1^1)^{|\textbf{k}^1|-1}} (1-M_2)\Psi '_1(\phi _1)\sin \phi _1 r_1^{|\textbf{k}^1|+7/2}+\mathcal {O}\left( r_1^{|\textbf{k}^1|+4}\right) ,\\ \{V_2,H\} V_1V_3= & {} -\dfrac{3 P_1}{2(k_1^1)^{|\textbf{k}^1|-1}}(1-M_1) \Psi '_1(\phi _1) \sin \phi _1 r_1^{|\textbf{k}^1|+7/2}+\mathcal {O}\left( r_1^{|\textbf{k}^1|+4}\right) ,\\ \{V_3,H\}V_1 V_2= & {} -\dfrac{|\textbf{k}^1| P_1}{2(k_1^1)^{|\textbf{k}^1|-1}}\left[ \Psi '_1(\phi _1) \sin \phi _1-\Psi _1(\phi _1)\cos \phi _1\right] (1-M_1)(1-M_2) r_1^{|\textbf{k}^1|+7/2}\\{} & {} +\mathcal {O}\left( r_1^{|\textbf{k}^1|+4}\right) . \end{aligned}$$

Taking into account the above on \(\Omega _j\), we arrive to

$$\begin{aligned} \begin{array}{rcl} \dot{V}&{}=&{}-\dfrac{P_1}{(k_1^1)^{|\textbf{k}^1|-1}}\left[ \left( 3(1-M_2)+\dfrac{3}{2}(1-M_1)\right) \Psi '_1(\phi _1)\sin \phi _1\right. \\[1pc] &{}&{}\left. +\dfrac{|\textbf{k}^1|}{2}\left[ \Psi '_1(\phi _1) \sin \phi _1-\Psi _1(\phi _1)\cos \phi _1\right] (1-M_1)(1-M_2) \right] r_2^{|\textbf{k}^1|+7/2}\\ &{}&{}+\mathcal {O}\left( r_1^{|\textbf{k}^1|+4}\right) . \end{array} \end{aligned}$$

Without loss of generality, let us assume that \(A_1<0\), the proof of the other case is similar. Since, \(0<M_1<1\), \(0<M_2<1\), \(\Psi '_1(\phi _1)=-2A_1 P_1 \sin \phi _1>0\) on \(\Omega _2\) and as

$$\begin{aligned} \begin{array}{rcl} \Psi '_1(\phi _1)\sin \phi _1-\Psi _1(\phi _1)\cos \phi _1=-A_1P_1 -\left( \displaystyle \sum _{ j_1,j_2=1}^{m_1} A^{j_1 j_2}k^1_{j_1}k^1_{j_2}\right) \cos \phi _1, \end{array} \end{aligned}$$

which is positive due to the fact that \(A_1<0\) and because there is a simple zero of \(\Psi _1(\phi _1)\). Thus, \(\dot{V}<0\) on \(\Omega _2\) and the proof is finished.

In the following result we present a similar and generic result assuming that \(\textbf{k}^1\) is even and \(6\le |\textbf{k}^1|\le |\textbf{k}^2|\), for this, we must add a hypothesis.

Theorem 6.4

Assume that \(|\textbf{k}^1|\) is even, \(6\le |\textbf{k}^1|\le |\textbf{k}^2|\), the components of the vector \(\textbf{k}^1\) do not change of sign, \(H_4=\dots = H_{|\textbf{k}^1|-2}\equiv 0\) and there exists \(\phi _1^*\in (0,2\pi )\) such that \(\Psi _1(\phi _1^*)=0\) and \(\Psi '_1(\phi _1^*)\ne 0\). Then the null solution of system Eq. 1 is unstable in the Lyapunov sense.

Proof

The proof is exactly the same as the one given for Theorem 6.3. \(\square \)

6.2 The Components of \(\textbf{k}^1\) Change of Sign, the Components of \(\textbf{k}^2\) do not Change of Sign and \(3=|\textbf{k}^1|<|\textbf{k}^2|\)

In this section, we are going to assume that the components of \(\textbf{k}^1\) change of sign and the components of \(\textbf{k}^2\) do not change of sign, and we will distinguish two cases. Firstly, when \(3=|\textbf{k}^1|<|\textbf{k}^2|=5\) and secondly, \(3=|\textbf{k}^1|<|\textbf{k}^2|=4\). So, to prove that the null solution of the system Eq. 1 is unstable, it will be necessary to normalize the first \(|\textbf{k}^2|+2\) terms of the Hamiltonian function, i.e.,

$$\begin{aligned} H=H(\textbf{r},\phi _1,\phi _2)=H^{|\textbf{k}^2|+2}(\textbf{r},\phi _1,\phi _2)+R(\textbf{r},\varphi ), \end{aligned}$$
(55)

where \(R(\textbf{r},\varphi )=\mathcal {O}\left( ||\textbf{r}||^{|\textbf{k}^2|/2+3/2}\right) \).

The hypotheses associated with the following theorems are related with some equations of the system. So, if \(|\textbf{k}^2|=5\), we must have clear that

$$\begin{aligned} \left. \begin{array}{rcl} \displaystyle \dot{r}_j=\{r_j,H\}&{}=&{}-2k_j^2A_2R_2\sin \phi _2+\dfrac{\partial H_{|\textbf{k}^2|+1}}{\partial \varphi _j }+\dfrac{\partial H_{|\textbf{k}^2|+2}}{\partial \varphi _j }+\dfrac{\partial R}{\partial \varphi _j },\\ \dot{\phi }_2=\{\phi _2,H\}&{}=&{} -\textbf{k}^2\cdot \dfrac{\partial H_4}{\partial \textbf{r}}- A_2 \cos \phi _2 \left( \displaystyle \sum _{j=m_1+1}^{m_2} k_j^2 \dfrac{|k_j^2|}{2} \dfrac{R_2}{r_j} \right) \\ &{}&{}- \textbf{k}^2\cdot \left( \dfrac{\partial H_{|\textbf{k}^2|+1}}{\partial \textbf{r} }+\dfrac{\partial H_{|\textbf{k}^2|+2}}{\partial \textbf{r} }+\dfrac{\partial R}{\partial \textbf{r} }\right) , \end{array} \right. \end{aligned}$$
(56)

for \(j=m_1+1,\ldots ,m_2\) and and if \(|\textbf{k}^2|=4\), \(\dot{r}_j=\{r_j,H\}\) is as in Eq. 56 for \(j=m_1+1,\ldots , m_2\) and this case

$$\begin{aligned} \left. \begin{array}{ll} \dot{\phi }_2=\{\phi _2,H\}=&{}-\displaystyle \dfrac{|\textbf{k}^2|}{2} \sum _{\begin{array}{c} {m_1+1\le j_1\le m_2}\\ {1\le j_2,\ldots ,j_{{\sigma _2}}\le n} \end{array}} A^{j_1 j_2 \cdots j_{{\sigma _2}}} k_{j_1}^2 r_{j_2} \cdots r_{j_{\sigma _2}}\\ &{} - A_2 \cos \phi _2 \left( \sum _{j=m_1+1}^{m_2} k_j^2 \dfrac{|k_j^2|}{2} \dfrac{R_1}{r_j} \right) \\ &{}- \textbf{k}^2\cdot \left( \dfrac{\partial H_{|\textbf{k}^2|+1}}{\partial \textbf{r} }+\dfrac{\partial H_{|\textbf{k}^2|+2}}{\partial \textbf{r} }+\dfrac{\partial R}{\partial \textbf{r} }\right) . \end{array} \right. \end{aligned}$$

Next, we state our first result assuming that \(3= |\textbf{k}^1|<|\textbf{k}^2|=5\).

Theorem 6.5

Assume that \(3=|\textbf{k}^1|<|\textbf{k}^2|=5\), the components of \(\textbf{k}^1\) change of sign, the components of \(\textbf{k}^2\) do not change of sign and \(H_4\equiv 0\). Then, the null solution of system Eq. 1 is unstable in the Lyapunov sense.

Proof

To prove the instability, we consider the Hamiltonian function \(H=H^{|\textbf{k}^2|+2}+R\), that is, the normalize Hamiltonian Eq. 2 up to order \(|\textbf{k}^2|+2\), inclusively.

We propose the following Chetaev’s function \(V=V_1 V_2 V_3\), where

$$\begin{aligned} V_1=\sum _{j=m_1}^{n-2}I_j^2- r_{m_1+1}^{3}, \quad V_2= r_{m_1+1}^{4}-\sum _{j=1}^{m_1}r_j, \quad V_3= R_2\sin \phi _2, \end{aligned}$$
(57)

with \(R_2\) as in Eq. 26. If \(A_2>0\), we take the region \(V>0\) by the region,

$$\begin{aligned} \Omega _1=\{(\textbf{r},\phi _1): \ V_1<0, \ V_2> 0 , \ \phi _2\in (\pi ,2\pi ), \ 0< \Vert \textbf{r}\Vert <a \}, \end{aligned}$$
(58)

with \(a>0\) is sufficiently small. While, if \(A_2<0\), we consider the region \(V<0\), by

$$\begin{aligned} \Omega _2=\{(\textbf{r},\phi _2): \ V_1<0, \ V_2> 0 , \ \phi _2\in (0,\pi ), \ 0< \Vert \textbf{r}\Vert <a\}. \end{aligned}$$
(59)

The set \(\Omega _j\) is not empty, for this, since the components of \(\textbf{k}^2\) do not change of sign, we can to take the point \((\textbf{r},\phi _2)=\left( \frac{r_{m_1+1}}{k_{m_1+1}^2}{} \textbf{k}^2,\phi _2\right) \) with \(0<\frac{r_{m_1+1}}{k_{m_1+1}^2}||\textbf{k}^2||<a\) and \(\phi _2\in \Omega _j\).

Inside the region \(\Omega _j\), from \(V_2>0\), there exit real functions \(g_j\) such that \(r_{j}=g_j r_{m_1+1}^{4} \) with \(0<g_j<1\) for \(j=1,\ldots ,m_1\). In addition, since \(V_1<0\), there exist real functions: \(f_j\) such that \(I_j=f_j r_{m_1+1}^{3/2}\) for \(j=m_1,\ldots ,m_2-2\) with \(|f_j|<1\); \(h_j\) such that \(I_j=h_j r_{m_1+1}^{3/2}\) for \(j=m_2-1,\ldots ,n-2\) with \(0<h_j<1\). In consequence, we obtain the following relations which are valid inside \(\Omega _j\),

$$\begin{aligned} \begin{aligned} r_j&= g_j r_{m_1+1}^{4}, \quad j=1,\ldots ,m_1;\\ r_j&=\frac{k_j^2}{k_{m_1+1}^2}r_{m_1+1}+ \dfrac{f_{j-2}}{k_{m_1+1}^2} r_{m_1+1}^{3/2},\quad j=m_1+2,\ldots ,m_2;\\ r_j&= h_{j-2} r_{m_1+1}^{3/2}, \quad j=m_2+1,\ldots ,n. \end{aligned} \end{aligned}$$
(60)

Since, \(V_1<0\) and \(V_2>0\) on \(\Omega _j\), substituting the relations \(I_j^2=f_j^2 r_{m_1+1}^{3}\) on \(V_1\) and \(r_j=g_j r_{m_1+1}^{4}\) on \(V_2\), we get \(V_1=(M_1-1)r_{m_1+1}^{3}<0\) and \(V_2=(1-M_2)r_{m_1+1}^{4}>0\) on \(\Omega _j\) with

$$\begin{aligned} 0<M_1=\sum _{j=m_1}^{n-2} f_j^2<1\quad \text {and}\quad 0<M_2=\sum _{j=1}^{m_1} g_j<1. \end{aligned}$$

Next, the derivative of V through the solutions of the Hamiltonian system associated with H given in Eq. 45 is as in Eq. 51 and in this case

$$\begin{aligned}{} & {} \{V_1,H\}=2\sum _{j=m_1}^{n-2}I_j \{I_j,H\}-3 r_{m_1+1}^{2} \{r_{m_1+1},H\},\\{} & {} \{V_2,H\}=4 r_{m_1+1}^{3} \{r_{m_1+1},H\}-\sum _{j=1}^{m_1}\{r_j,H\},\\{} & {} \{V_3,H\}=\left( \sum _{j=m_1+1}^{m_2}\dfrac{\partial R_2}{\partial r_j}\{r_j,H\}\right) \sin \phi _2+\{\phi _2,H\} R_2 \cos \phi _2. \end{aligned}$$

Using the relations Eq. 60 and considering \(R_j\) given in Eq. 26 for \(j=1,2\), inside the region \(\Omega _j\), we get that

$$\begin{aligned} \begin{array}{l} R_1=\mathcal {O}\left( r_{m_1+1}^{6}\right) \\ R_2=P_2\left( \frac{r_{m_1+1}}{k_{m_1+1}^2}\right) ^{5/2}+\mathcal {O}\left( r_{m_1+1}^{3}\right) , \\ \frac{R_2}{r_j}=\frac{P_2}{k_j^2}\left( \frac{r_{m_1+1}}{k_{m_1+1}^2}\right) ^{3/2}+\mathcal {O}\left( r_{m_1+1}^{2}\right) \quad \text {for} \ \ j=m_1+1,\ldots ,m_2. \end{array} \end{aligned}$$
(61)

Additionally, by the relations Eq. 60, it follows that \(\dfrac{\partial R}{\partial \varphi _j}=\mathcal {O}(r_{m_1+1}^4)\) for \(j=1\ldots ,n\), \(I_j=\mathcal {O}(r_{m_1+1})\) and \(\dot{I}_j=\mathcal {O}(r_{m_1+1}^4)\) for \(j=m_1,\ldots ,n-2\).

From the first equation in Eq. 61, it follows that \(V_3=P_2\sin \phi _2 \left( \frac{r_{m_1+1}}{k_{m_1+1}^2}\right) ^{5/2}+\mathcal {O}\left( r_{m_1+1}^{3}\right) \). So, after some calculations, we obtain

$$\begin{aligned} \begin{array}{l} \{V_1,H\} V_2 V_3=\dfrac{3A_2 P_2^2}{(k_{m_1+1}^2)^{4}} (1-M_2)\sin ^2\phi _2 r_{m_1+1}^{23/2}+\mathcal {O}(r_{m_1+1}^{12}),\\[1pc] \{V_2,H\} V_1V_3=\dfrac{4 A_2 P_2^2}{(k_{m_1+1}^2)^{4}}(1-M_1) \sin ^2\phi _2 r_{m_1+1}^{23/2}+\mathcal {O}\left( r_{m_1+1}^{12}\right) ,\\[1pc] \{V_3,H\}V_1 V_2=\dfrac{3 A_2 P_2^2}{2(k_{m_1+1}^2)^{4}}(1-M_1)(1-M_2) r_{m_1+1}^{23/2}+\mathcal {O}\left( r_{m_1+1}^{12}\right) . \end{array} \end{aligned}$$

Summarizing, on \(\Omega _j\), it follows that

$$\begin{aligned} \begin{array}{l} \dot{V}=\dfrac{A_2 P_2^2}{(k_{m_1+1}^2)^{4}}\left[ \left( 3(1-M_2)+4(1-M_1)\right) \sin ^2\phi _2+\dfrac{3}{2}(1-M_1)(1-M_2) \right] r_2^{23/2}+\mathcal {O}(r_{m_1+1}^{12}). \end{array} \end{aligned}$$

Since, \(0<M_1<1\), \(0<M_2<1\) on \(\Omega _j\), if \(A_2>0\), then \(\dot{V}>0\) on \(\Omega _1\) and if \(A_2<0\), it is clear that \(\dot{V}<0\) on \(\Omega _2\), so the proof is completed. \(\square \)

For the last result about instability, we are going to assume that \(|\textbf{k}^1|=3\), \(|\textbf{k}^2|=4\) and \(\Psi _2(\phi _2)\) has a simple zero.

Theorem 6.6

Assume that \(3=|\textbf{k}^1|<|\textbf{k}^2|=4\), the components of \(\textbf{k}^1\) change of sign, the components of \(\textbf{k}^2\) do not change of sign and there exists \(\phi _2^*\in (0,2\pi )\) such that \(\Psi _2(\phi _2^*)=0\) and \(\Psi '_2(\phi _2^*)\ne 0\). Then, the null solution of system Eq. 1 is unstable in the Lyapunov sense.

Proof

The arguments of the proof are very similar to the proof of Theorem 6.5. In this case, we consider the Hamiltonian function Eq. 2 normalized up to order \(|\textbf{k}^2|+2\), inclusively, and we take the Chetaev’s function is \(V=V_1 V_2 V_3\) where \(V_1\), \(V_2\) and \(V_3\) are as in Eq. 57.

Using the relations Eq. 60 and considering \(R_j\) given in Eq. 26 for \(j=1,2\), inside the region \(\Omega _j\), we arrive that

$$\begin{aligned} \begin{array}{l} R_1=\mathcal {O}\left( r_{m_1+1}^{6}\right) ,\\ R_2=P_2\left( \frac{r_{m_1+1}}{k_{m_1+1}^2}\right) ^{2}+\mathcal {O}\left( r_{m_1+1}^{5/2}\right) , \\ \frac{R_2}{r_j}=\frac{P_2}{k_j^2}\left( \frac{r_{m_1+1}}{k_{m_1+1}^2}\right) +\mathcal {O}\left( r_{m_1+1}^{3/2}\right) \quad \text {for} \ \ j=m_1+1,\ldots ,m_2.\\ \end{array} \end{aligned}$$
(62)

In addition, by the relations Eq. 50, it follows that \(\dfrac{\partial R}{\partial \varphi _j}=\mathcal {O}\left( r_{m_1+1}^{7/2}\right) \) for \(j=1\ldots ,n\), \(I_j=\mathcal {O}(r_{m_1+1})\) and \(\dot{I}_j=\mathcal {O}\left( r_{m_1+1}^{7/2}\right) \) for \(j=m_1,\ldots ,n-2\). From the first equation in Eq. 62, it follows that \(V_3=P_2\sin \phi _2 \left( \frac{r_{m_1+1}}{k_{m_1+1}^2}\right) ^{2}+\mathcal {O}\left( r_{m_1+1}^{5/2}\right) \). So, in this case, we get

$$\begin{aligned} \begin{array}{l} \{V_1,H\} V_2 V_3=-\dfrac{3 P_2}{(k_{m_1+1}^2)^{3}} (1-M_2)\Psi '_2(\phi _2)\sin \phi _2 r_1^{11}+\mathcal {O}\left( r_1^{23/2}\right) ,\\[1pc] \{V_2,H\} V_1V_3=-\dfrac{4 P_2}{(k_{m_1+1}^2)^{3}}(1-M_1) \Psi '_2(\phi _2) \sin \phi _2 r_1^{11}+\mathcal {O}\left( r_1^{23/2}\right) ,\\[1pc] \{V_3,H\}V_1 V_2=-\dfrac{2 P_2}{(k_{m_1+1}^2)^{3}}\left[ \Psi '_2(\phi _2) \sin \phi _2-\Psi _2(\phi _2)\cos \phi _2\right] (1\!-\!M_1)(1\!-\!M_2) r_1^{11}\!+\!\mathcal {O}\left( r_1^{23/2}\right) . \end{array} \end{aligned}$$

Thus, using all the previous information on \(\Omega _j\), we obtain

$$\begin{aligned} \begin{array}{rcl} \dot{V}&{}=&{}-\dfrac{P_2}{(k_{m_1+1}^2)^{3}}\left[ \left( 3(1-M_2)+4(1-M_1)\right) \Psi '_2(\phi _2)\sin \phi _2\right. \\[1pc] &{}&{}\left. +2\left( \Psi '_2(\phi _2) \sin \phi _2-\Psi _2(\phi _2)\cos \phi _2\right) (1-M_1)(1-M_2) \right] r_2^{11}+\mathcal {O}(r_1^{23/2}). \end{array} \end{aligned}$$

Now, since \(\Psi _2(\phi _2)\) has a simple zero and following the same ideas of Theorem 6.3, the proof is finished. \(\square \)

7 Concluding Remarks

We recall the concept of invariant ray solution associated with the truncated Hamiltonian systems with Hamiltonian function \(H^m\) as in Eq. 29.

Definition 7

  An invariant ray solution of the model Hamiltonian system associated with (29), is a particular solution of the form \(r_j(t)= a_j b(t)\), \(\varphi _j(t)= \varphi _j^*\), where \(a_j, \varphi _j^*\) are constants, \(a_j > 0\) for \(j=1, \ldots , n\) and b(t) is a positive, increasing and unbounded function.

This concept can also be found in [15] and [22].

Some authors have related the study of instability proving the existence of an invariant ray solution. In fact, Khazin in [16,17,18] had proved the instability of the equilibrium solution of one adequate truncated Hamiltonian using the presence of an invariant ray solution. Precisely, he proved the instability for the truncated system in the order of the vector resonance. Moreover, Molčhanov [22] asked the following question (conjecture) raised for Ordinary Differential Equations (ODE) which were formulated in his doctoral thesis (according Khazin in [16, 18]): a necessary and sufficient condition for the instability of the equilibrium point in an EDO is the presence of an invariant ray solution in the model system (truncated system in the order of the resonance vector in the case of single resonance). At this point, we must be very careful, because there is not an analytical proof for the previous affirmation. In other words, for now, it is not possible to ensure the instability of the complete system guarantying the existence of a ray invariant solution of the truncated system.

Next, we are going to show two examples where a truncated Hamiltonian system with 4-DOF has an invariant ray solution. Let \(\textbf{k}^1=(k_1^1,k_2^1,0,0)\) and \(\textbf{k}^2=(0,0,k_3^2,k_4^2)\) the resonance vectors. For the first example, we assume that \(4=|\textbf{k}^1|\le |\textbf{k}^2|\), the components of \(\textbf{k}^1\) do not change of sign and there exist \(\phi _1=\phi _1^*\in (0,2\pi )\) such that \(\Psi _1(\phi _1^*)=0\) and \(\Psi '_1(\phi _1^*)\ne 0\). Then, the normalized terms of order \(|\textbf{k}^1|\) are

$$\begin{aligned} H_{|\textbf{k}^1|}=\left\{ \begin{array}{ll} H_{|\textbf{k}^1|}^0(\textbf{r})+H_{|\textbf{k}^1|}^1({\textbf{r}},\phi _1), &{} \text {if } |\textbf{k}^1|<|\textbf{k}^2|,\\ H_{|\textbf{k}^1|}^0(\textbf{r})+H_{|\textbf{k}^1|}^1({\textbf{r}},\phi _1)+H_{|\textbf{k}^2|}^2({\textbf{r}},\phi _2), &{} \text {if }|\textbf{k}^1|=|\textbf{k}^2|, \end{array}\right. \end{aligned}$$

where

$$\begin{aligned} \begin{array}{rcl} H_{|\textbf{k}^1|}^0(\textbf{r})&{}=&{}a_{11}r_1^2 +a_{12}r_1 r_2 +a_{13} r_1 r_3+a_{14} r_1 r_4+ a_{22} r_2^2 +a_{23}r_2 r_3+a_{24}r_2 r_4\\ &{}&{}+a_{33} r_3^2+a_{34} r_3 r_4+a_{44} r_4^2,\\ H_{|\textbf{k}^1|}^1({\textbf{r}},\phi _1)&{}=&{}A_1 R_1 \cos \phi _1,\quad R_1=r_1^{|k_1^1|/2} r_2^{|k_2^1|/2},\quad \phi _1= \textbf{k}^1\cdot \varphi ,\\ H_{|\textbf{k}^2|}^2({\textbf{r}},\phi _2)&{}=&{}A_2 R_2 \cos \phi _2,\quad R_2=r_3^{|k_3^2|/2} r_4^{|k_4^2|/2} ,\quad \phi _2= \textbf{k}^2\cdot \varphi . \end{array} \end{aligned}$$

By the periodicity of \(\Psi _1(\phi _1)\) we can choose \(\phi _1=\phi _1^* \in (0,2\pi )\) such that \(\Psi _1(\phi _1^*)=\) and \(\Psi '_1(\phi _1^*)>0\).

We propose the invariant ray solution of the form

$$\begin{aligned} r_1(t)=b(t), \ r_2(t)=\dfrac{k_2^1}{k_1^1} b(t) , \ r_3(t)=0, \ r_4(t)=0, \ \phi _1(t)=\phi _1^*, \end{aligned}$$
(63)

then replacing in the Hamiltonian system associated with \(H^{|\textbf{k}^1|}=H_2(r)+H_{|\textbf{k}^1|}\), obtaining

$$\dot{b}(t)=\frac{k_2^1}{(k_1^1)^2} b^2 \Psi _1'\left( \phi _1^*\right) >0,\quad \dot{\phi }_1(t)=-\frac{2}{ k_1^1} b \Psi _1\left( \phi _1^*\right) =0.$$

Solving the first ODE on the left hand side, it follows that

$$b(t)=\dfrac{1}{-\left( k_2^1/(k_1^1)^2\right) ^{-1}\Psi '_1(\phi _1^*)t+C} >0, \quad \forall t\in \left[ 0,\frac{C}{\left( k_2^1/(k_1^1)^2\right) ^{-1}\Psi '_1(\phi _1^*)}\right) .$$

In consequence, \(b(t)\longrightarrow \infty \) when \(t\longrightarrow \frac{C}{\left( k_2^1/(k_1^1)^2\right) ^{-1}\Psi '_1(\phi _1^*)}\), so the truncated Hamiltonian system associated with \(H^{|\textbf{k}^1|}\) has an invariant ray solution of the form given in Eq. 63, which implies that the null solution is unstable in the Lyapunov sense.

For the second example, we suppose that \(3=|\textbf{k}^1|<|\textbf{k}^2|=4\), the components of \(\textbf{k}^1\) change of sign, the components of \(\textbf{k}^2\) do not change of sign and the auxiliary function \(\Psi _2(\phi _2)\) has a simple zero, then the Hamiltonian system associated with the Hamiltonian function \(H^{|\textbf{k}^2|}=H_2+H_{|\textbf{k}^1|}+H_{|\textbf{k}^2|}\) has an ray invariant solution of the form

$$\begin{aligned} r_1(t)=0, \ r_2(t)=0, \ r_3(t)=b(t), \ r_4(t)=\dfrac{k_4^2}{k_3^2} b(t) , \ \phi _2(t)=\phi _2^*. \end{aligned}$$

Note that the first example satisfies all the hypotheses of Theorem 6.4 and the second example satisfies all the hypotheses of Theorem 6.6, then it is also possible to ensure the instability of the complete system in both cases, which agrees with the Molčhanov conjecture.

Finally, we comment that some results in this work could be extended in the in cases where we have more than two resonance vectors. For example, if we assume that there are \(\textbf{k}^1, \textbf{k}^2, \ldots , \textbf{k}^s\) resonance vectors, such that \(\textbf{k}^1\) and \(\textbf{k}^2\) have not interaction and \(\textbf{k}^1\le \textbf{k}^2<\textbf{k}^3\le \ldots \le \textbf{k}^s\). Another case that we intend study in the future, is to suppose that \(\textbf{k}^1\) and \(\textbf{k}^2\) have interaction, this would be a very different work because the normal form and the first formal integrals are different.