1 Introduction

Long-short (LS) wave interaction equations have been proposed for many physical problems, such as internal, Rossby, and plasma waves. Kuznetsov et al. [1] proposed some generalized LS-type coupled equations. In this paper, we investigate one type of the generalized LS wave resonance equations with cubic-quintic strong nonlinear term

$$ \textstyle\begin{cases} iu_{t}+\lambda u_{{xx}}=\alpha {uv}+\gamma \vert u \vert ^{2}u+\delta \vert u \vert ^{4}u, \quad x \in R, \\ v_{t}+\beta \vert u \vert ^{2}_{x}=0, \quad x\in R. \end{cases} $$
(1.1)

When \(v=0\), Eqs. (1.1) reduce to the nonlinear Scrödinger equation describing electromagnetic wave propagation in nonlinear isotropic dielectrics, for example, in an isotropic plasma. In this case, u denotes the complex amplitude of the electric field, and \(\gamma |u|^{2}u+\delta |u|^{4}u\) is the nonlinear addition to the refraction index. However, for many problems, accounting for a finite time of medium relaxation is critical. Thus, for electromagnetic radiation propagation in an isotropic plasma, the nonlinear frequency shift is caused by density modulation under the action of a powerful wave, and the coupled equation was proposed.

In 2005, Shang [2] studied the explicit and exact special solutions of Eqs. (1.1), where α, β, γ, λ, and δ are all real constants with \(\lambda \alpha \beta \neq 0\). The quintic term \(\delta |u|^{4}u\) in the first equation of (1.1) describes the strong nonlinear self-interaction in the high-frequency subsystem, which corresponds to a self-focusing effect in plasma physics. Obviously, if \(\gamma =\delta =0\), then Eqs. (1.1) reduce to the classical LS wave equations

$$ \textstyle\begin{cases} iu_{t}+u_{{xx}}=\alpha {uv}, \quad x\in R, \\ v_{t}+\beta \vert u \vert ^{2}_{x}=0, \quad x\in R. \end{cases} $$
(1.2)

Equations (1.2) were first derived by Djordjevic and Redekopp [3] to describe the resonance interaction between long and short waves. In Eqs. (1.2), u is a complex-valued function and denotes the envelope of the short wave, and v is a real-valued function and denotes the amplitude of the long wave. As highlighted in [3], the physical significance of Eqs. (1.2) is that the dispersion of the short wave is balanced by the nonlinear interaction of the long and short waves, whereas the evolution of the long wave is driven by the self-interaction of the short wave. These equations also appear in an analysis of internal waves [4] and Rossby waves. In plasma physics, similar equations can be used to describe the resonance between high-frequency electron plasma oscillations and associated low-frequency ion density perturbations [5]. Ma [6] found that Eqs. (1.2) can be rewritten in Lax’s formulation, and the Cauchy problem of Eqs. (1.1) can be solved by the inverse scattering method. Adapting the method developed by Bona and Weinstein, Laurencot [7] confirmed that the solitary wave solution of (1.2) was stable.

Moreover, if \(\delta =0\), then Eqs. (1.1) reduce to the LS wave resonance equations

$$ \textstyle\begin{cases} iu_{t}+\lambda u_{{xx}}=\alpha {uv}+\gamma \vert u \vert ^{2}u, \quad x\in R, \\ v_{t}+\beta \vert u \vert ^{2}_{x}=0, \quad x\in R, \end{cases} $$
(1.3)

where \(\alpha,\beta,\gamma,\lambda \in R \) with \(\lambda \alpha \beta \neq 0\). Equations (1.3) were a particular case of the equations proposed by Benney [8]. In that study, Benney provided a general theory for deriving nonlinear partial differential equations that allow both long and short wave solutions. By an appropriate change of both independent and dependent variables, we can take \(\lambda =1,\alpha =1\) and \(\beta =-1\) to obtain

$$ \textstyle\begin{cases} iu_{t}+u_{{xx}}={uv}+\gamma \vert u \vert ^{2}u, \quad x\in R, \\ v_{t}= \vert u \vert ^{2}_{x},\quad x\in R. \end{cases} $$
(1.4)

System (1.4) arises in the study of surface waves with both gravity and capillary modes being present [9] and in plasma physics [10]. We can say that Eqs. (1.2), (1.3), (1.4) are all particular forms of Eqs. (1.1). The well-posedness of the local solution or/and global solution for the initial value problem and periodic initial value problem of system (1.4) and its extensions have been investigated by several authors. Among these, we refer the reader to [1115]. The existence of global attractors and approximation inertial manifolds have been studied by many researchers [1624]. Guo and Chen [25] studied the orbital stability of solitary waves of (1.4) by applying the abstract results of Grillakis et al. [26, 27]. Unfortunately, the conditions that ensure the orbital stability of solitary waves were incorrect because of incorrectness of \(d_{{cc}}(\omega,c) \) and consequently of \(\det (d^{\prime \prime })\) (see p. 893 of [25]).

Based on the qualitative theory and bifurcation theory of planar dynamical systems, a series of explicit and exact solutions of solitary waves for Eqs. (1.1) were obtained by seeking the homoclinic and heteroclinic orbits for a class of Liénard equations [2]. An interesting problem is whether the solitary waves of the generalized LS wave equations with a cubic-quintic strong nonlinear term (1.1) are orbitally stable or instable. However, till date, to the best of our knowledge, no research has been conducted on the orbital stability of the solitary waves of the generalized LS wave equations having a cubic-quintic strong nonlinear term (1.1).

In this paper, we consider the existence and orbital stability of solitary waves for the generalized LS wave equations (1.1). We focus on solutions for (1.1) of the form

$$\begin{aligned} u(x,t)=e^{-i\omega t}e^{i\frac{c}{2}(x-{ct})}\phi _{\omega,c}(x-{ct}) \quad\text{and}\quad v(x,t)=\psi _{\omega,c}(x-{ct}), \end{aligned}$$
(1.5)

where \(\omega, c\in R\), \(\xi =x-{ct}\), \(\phi _{\omega,c}\), \(\psi _{\omega,c}: R\rightarrow R\) are smooth functions, and \(\phi _{\omega,c}(\xi )\), \(\psi _{\omega,c}(\xi )\rightarrow 0\) as \(|\xi |\rightarrow \infty \). It is worth pointing out that Eqs. (1.1) contain two nonlinear terms. Our results contain the orbital stability of solitary waves for the classical LS wave equation with \(\gamma =\delta =0\), the orbital instability results for the nonlinear Schrödinger equation with \(\beta =0\), and the orbital stability of solitary wave for LS wave equation with one nonlinear term.

Because here the stability refers to perturbations of the solitary wave profile itself, a study for the initial value problem of (1.1) is necessary. Similarly to Theorem 1.2 in [14], by using Banach’s fixed point theorem and employing some smoothing-effect estimates, after slightly modifying the proof of [14], we obtain the well-posedness of the initial value problem of (1.1).

Theorem 1

For any \((u_{0},v_{0})\in H^{\frac{5}{2}}(R)\times H^{2}(R)\), there exists a unique function \((u,v)\in C(R^{+};H^{\frac{5}{2}(R)})\times C(R^{+};H^{2}(R))\) such that \(u\in X_{2}([0,T])\) for any \(T>0\), where

$$\begin{aligned} X_{2}(I)={}&\bigl\{ u\in C\bigl(I;H^{\frac{5}{2}}(R)\bigr):\partial _{x}(1-\triangle )u \in L^{2}\bigl(I;L^{\infty }(R) \bigr),\\ &(1-\triangle )u\in L^{4}\bigl(I;L^{\infty }(R)\bigr) \cap L^{\infty }\bigl(I;L^{4}(R)\bigr)\bigr\} ,\\ & I=[0,T]. \end{aligned}$$

The orbital stability of solitary waves is defined as follows.

Definition

([26])

The solitary waves \(T_{1}({ct})T_{2}(\omega t)\Phi _{\omega,c}(x)\) are orbitally stable if for any \(\varepsilon >0\), there exists \(\delta >0\) with the following property: If \(\|U_{0}-\Phi _{\omega,c}(x)\|_{X}<\delta \) and \(U(t)\) is a solution of (1.1) in some interval \([0,t_{0})\) with \(U(0)=U_{0}\), then \(U(t)\) can be continued to a solution in \(0\leq t< +\infty \), and

$$ \sup_{0\leq t< +\infty }\inf_{s_{1}\in R}\inf _{s_{2}\in R} \bigl\Vert U(t)-T_{1}(s_{1})T_{2}(s_{2}) \Phi _{\omega,c} \bigr\Vert _{X}< \varepsilon, $$
(1.6)

where \(\Phi _{\omega,c}(x)=(\phi _{\omega,c}(x),\psi _{\omega,c}(x))\). Otherwise, \(T_{1}({ct})T_{2}(\omega t)\Phi _{\omega,c}(x)\) is called orbitally unstable.

By applying the extended version of the general theory of orbital stability presented by Grillakis et al. [26], the lines of the stability theorem in the introduction of [27] or Theorem 4.1 in [27], the approach in [28], and detailed spectral analysis, we obtain the following abstract stability results of solitary waves (1.5) for Eqs. (1.1).

Theorem 2

Assume that (1.1) has a family of solitary waves that belong to \(H^{3}(R)\times H^{2}(R)\) as c ranges in \(R_{1}=(c_{1},c_{2})\), ω ranges in \(R_{2}\) satisfying \(4\omega +c^{2}<0\) with \(C^{1}\times C^{1}\) mapping \((\omega,c)\rightarrow T_{1}({ct})T_{2}(\omega t)\Phi _{\omega,c}(x)\) of the interval \((R_{1},R_{2})\) into \(H^{1}(R)\times L^{2}(R)\). Moreover, suppose that \(\phi _{\omega,c}^{\prime }\) has one simple zero and decays rapidly to zero at ±∞. Then the solitary wave \(T_{1}({ct})T_{2}(\omega t)\Phi _{\omega,c}(x)\) of (1.1) is stable in \(H^{1}(R)\times L^{2}(R)\) if the condition \(p(d^{\prime \prime })=n(H_{\omega,c})\) holds, where

$$\begin{aligned} &d(\omega,c)=E(\Phi _{\omega,c})-c Q_{1}(\Phi _{\omega,c})- \omega Q_{2}( \Phi _{\omega,c}), \\ &H_{\omega,c}=E^{\prime \prime }(\Phi )-c Q_{1}^{\prime \prime }(\Phi )- \omega Q_{2}^{\prime \prime }(\Phi ), \end{aligned}$$

\(n(H_{\omega,c})\) is the number of negative eigenvalues of \(H_{\omega,c}\), and \(p(d^{\prime \prime })\) is the number of positive eigenvalues of the Hessian \(d^{\prime \prime }\) at \((\omega,c)\). The results obtained improve and extend the previous studies.

The remainder of this paper is structured as follows: For convenience, we first introduce the existence of solitary waves for the generalized LS wave equations (1.1). Then, in Theorem 4, we present the spectral analysis of some certain self-adjoint operators necessary to obtain our stability result and state the stability results. Finally, we prove the stability results under three conditions.

2 Exact solitary waves of the generalized LS wave equations with cubic-quintic nonlinearity term

For convenience, in this section, we consider the solitary wave solutions of the following generalized LS wave resonance equations with a cubic-quintic strong nonlinear self-interaction term:

$$ \textstyle\begin{cases} iu_{t}+u_{{xx}}=\alpha {uv}+\gamma \vert u \vert ^{2}u+\delta \vert u \vert ^{4}u,\quad x\in R, \\ v_{t}+\beta \vert u \vert ^{2}_{x}=0,\quad x\in R, \end{cases} $$
(2.1)

with real α, β, γ, δ. Assume that Eqs. (2.1) have solutions of the form

$$ u(x,t)=e^{-i\omega t}\widehat{\phi }_{\omega,c}(x-{ct})=e^{-i\omega t}e^{{ia}(x-{ct})} \phi _{\omega,c}(x-{ct}),\qquad v(x,t)=\psi _{\omega,c}(x- {ct}), $$
(2.2)

where ω, c are real numbers, and a, \(\phi _{\omega,c}\), and \(\psi _{\omega,c}\) are real functions. We set \(\xi =x-{ct}\) and assume \(\phi _{\omega,c}(\xi ), \psi _{\omega,c}(\xi )\rightarrow 0\) as \(|\xi |\rightarrow \infty \). Substituting \(u(x,t)=e^{-i\omega t}\widehat{\phi }_{\omega,c}(x-{ct})\) and \(v(x,t)=\psi _{\omega,c}(x-{ct})\) into Eqs. (2.1), we have that \(\widehat{\phi }_{\omega,c}(\xi )\) and \(\psi _{\omega,c}(\xi )\) satisfy

$$ \textstyle\begin{cases} -\widehat{\phi }^{\prime \prime }_{\omega,c}+\alpha \widehat{\phi }_{ \omega,c}\psi _{\omega,c}+\gamma \vert \widehat{\phi }_{\omega,c} \vert ^{2} \widehat{\phi }_{\omega,c}+\delta \vert \widehat{\phi }_{\omega,c} \vert ^{4} \widehat{\phi }_{\omega,c}+{ic}\widehat{\phi }^{\prime }_{\omega,c}- \omega \widehat{\phi }_{\omega,c}=0, \\ c\psi _{\omega,c}-\beta \vert \widehat{\phi }_{\omega,c} \vert ^{2}=0. \end{cases} $$
(2.3)

Substituting \(\widehat{\phi }_{\omega,c}(\xi )=e^{{ia}(\xi )}\phi _{\omega,c}(\xi )\) into Eqs. (2.3) and taking the real and imaginary parts of Eqs. (2.3), we have

$$\begin{aligned} &{-}\phi ^{\prime \prime }_{\omega,c}+\bigl(a^{\prime }\bigr)^{2} \phi _{\omega,c}+ \alpha \phi _{\omega,c}\psi _{\omega,c}+\gamma \phi _{\omega,c}^{3}+ \delta \phi _{\omega,c}^{5}- {ca}^{\prime }\phi _{\omega,c}-\omega \phi _{\omega,c}=0, \end{aligned}$$
(2.4)
$$\begin{aligned} &2a^{\prime }\phi ^{\prime }_{\omega,c}+a^{\prime \prime }\phi _{ \omega,c}-c\phi _{\omega,c}^{\prime }=0, \end{aligned}$$
(2.5)
$$\begin{aligned} &\psi _{\omega,c}=\frac{\beta }{c}\phi _{\omega,c}^{2}. \end{aligned}$$
(2.6)

From (2.5) we have \(a^{\prime }(\xi )=\frac{c}{2}\) and \(a^{\prime \prime }(\xi )=0\), that is, \(a(\xi )=\frac{c}{2}\xi +D\). Without loss of generality, we assume that \(D=0\), and then \(a(\xi )=\frac{c}{2}\xi \). By collecting (2.6) and \(a^{\prime }(\xi )=\frac{c}{2}\), Eq. (2.4) becomes

$$\begin{aligned} \phi ^{\prime \prime }_{\omega,c}+\biggl(\omega +\frac{c^{2}}{4}\biggr)\phi _{ \omega,c}+\biggl(-\frac{\alpha \beta }{c}-\gamma \biggr)\phi _{\omega,c}^{3}- \delta \phi _{\omega,c}^{5}=0. \end{aligned}$$
(2.7)

Multiplying by \(2\phi ^{\prime }_{\omega,c}\) both sides of Eq. (2.7) and integrating from −∞ to ξ, it follows that

$$\begin{aligned} \biggl(\frac{\phi ^{\prime }_{\omega,c}}{\phi _{\omega,c}}\biggr)^{2}=d_{1}+d_{2} \phi ^{2}_{\omega,c}+d_{4}\phi ^{4}_{\omega,c}, \end{aligned}$$
(2.8)

where \(d_{1}=-\omega -\frac{c^{2}}{4}\), \(d_{2}=\frac{1}{2}(\frac{\alpha \beta }{c}+\gamma )\), and \(d_{4}=\frac{1}{3}\delta \). Equation (2.8) is the form of (3.25b) in [29]. Then, according to [29] (also see [30]), there exists a solitary wave of the form

$$\begin{aligned} \phi ^{2}_{\omega,c}(\xi )=\frac{1}{d_{3}+d_{5}\cosh d_{6}\xi }, \end{aligned}$$
(2.9)

where

$$ d_{3}=-\frac{d_{2}}{2d_{1}},\qquad d_{5}^{2}= \frac{d_{2}^{2}-4d_{1}d_{4}}{4d_{1}^{2}},\qquad d_{6}^{2}=4d_{1}. $$

Therefore we have the following lemma.

Lemma 1

Let \(d_{1}>0\) and \(d_{2}<0\). If \(d_{4}\leq 0\) or if \(d_{4}>0\) and \(d_{2}^{2}-4d_{1}d_{4}>0\), then Eq. (2.8) has bounded positive analytic solutions of the form

$$\begin{aligned} \phi _{\omega,c}(\xi )=\biggl[\frac{1}{d_{3}+d_{5}\cosh d_{6}\xi }\biggr]^{ \frac{1}{2}}. \end{aligned}$$
(2.10)

Especially, when \(d_{4}=0\), we obtain the following solution of Eq. (2.8):

$$\begin{aligned} \phi _{\omega,c}(\xi )=\sqrt{-\frac{d_{1}}{d_{2}}} {\mathrm{sech}} ( \sqrt{d_{1}} \xi ). \end{aligned}$$
(2.11)

Furthermore, from (2.2), (2.6), and (2.10) we can obtain solitary wave solutions \(u(x,t)\) and \(v(x,t)\) of Eqs. (1.1). We have the following existence results.

Theorem 3

Suppose that \(4\omega +c^{2}<0\) and \(\frac{\alpha \beta }{c}+\gamma <0\). If \(\delta \leq 0\) or if \(\delta >0\) and \((\frac{\alpha \beta }{c}+\gamma )^{2}+\frac{4\delta }{3}(4\omega +c^{2})>0\), then Eqs. (1.1) admit solitary waves \(u(x,t)=e^{-i\omega t}e^{i\frac{c}{2}(x-{ct})}\phi _{\omega,c}(x-{ct})\) and \(v(x,t)=\psi _{\omega,c}(x-{ct})\), where \(\phi _{\omega,c}(\xi )\) and \(\psi _{\omega,c}(\xi )\) are given by (2.10) and (2.6), respectively.

Corollary 1

For any real constants \(\omega,c,\alpha,\beta,\gamma \) satisfying \(4\omega +c^{2}<0\) and \((\alpha \beta +\gamma c)c<0\). Equations (1.3) (\(\lambda =1\)) admit solitary waves \(u(x,t)=e^{-i\omega t}e^{i\frac{c}{2}(x-ct)}\phi _{\omega,c}(x-{ct})\) and \(v(x,t)=\psi _{\omega,c}(x-{ct})\), where \(\phi _{\omega,c}(\xi )=\sqrt{ \frac{(4\omega +c^{2})c}{2(\alpha \beta +\gamma c)}}{\mathrm{sech}} ( \frac{\sqrt{-4\omega -c^{2}}}{2}\xi )\), and \(\psi _{\omega,c}(\xi )\) is given by (2.6).

Corollary 2

For any real constants \(\omega,c,\alpha,\beta,\gamma \) satisfying \(4\omega +c^{2}<0\), \(\beta =0\), and \(\gamma <0\), we obtain the solitary wave \(u(x,t)=e^{-i\omega t}e^{i\frac{c}{2}(x-ct)}\phi _{\omega,c}(x-ct)\) for the nonlinear Schrödinger equation, where \(\phi _{\omega,c}(\xi )=\sqrt{\frac{4\omega +c^{2}}{2\gamma }}{\mathrm{sech}} (\frac{\sqrt{-4\omega -c^{2}}}{2}\xi )\).

Remark 1

In particular, if \(\alpha =1\) and \(\beta =-1\), then Eqs. (1.4) have solitary waves of \(u(x,t)=e^{-i\omega t}e^{i\frac{c}{2}(x-{ct})}\) \(\phi _{\omega,c}(x-{ct})\) and \(v(x,t)=\psi _{\omega,c}(x-{ct})\), where \(\phi _{\omega,c}(\xi )=\sqrt{ \frac{-(4\omega +c^{2})c}{2(1-\gamma c)}}{\mathrm{sech}} ( \frac{\sqrt{-4\omega -c^{2}}}{2}\xi )\), and \(\psi _{\omega,c}(\xi )\) is given by (2.6). These solitary wave solutions are the same as those obtained by Guo and Chen in Theorem 1 of [25].

3 Verification of conditions that enable Eqs. (1.1) and its solitary waves to satisfy the abstract stability theory

In this section, we prove that Eqs. (1.1) are a Hamiltonian system and satisfy the conditions of the general orbital stability theory proposed by Grillakis et al. [26, 27] for some parameters.

In [25] the authors rewrite (1.4) in terms of real and imaginary parts and reduce Eqs. (1.4) to Eqs. (3.1). Then they define the function space wherein they work on and develop their analysis. In this paper, we define the function space wherein we work on and develop our analysis directly starting from Eqs. (1.1).

Let \(U=(u,v)^{T}\). The function space we will work on is defined by \(X=H^{1}_{\mathrm{complex}}(R)\times L^{2}(R)\). Let the inner product of X be

$$\begin{aligned} (f,g)= \int _{R}\bigl(\mathrm{Re}(f_{1} \overline{g_{1}})+\mathrm{Re}(f_{1x}\overline{g_{1x}})+f_{2}g_{2} \bigr)\,dx \end{aligned}$$
(3.1)

for \(f=(f_{1},f_{2}),g=(g_{1},g_{2})\in X\). The dual space of X is \(X^{*}=H^{-1}_{\mathrm{complex}}(R)\times L^{2}(R)\); there exists a natural isomorphism \(I: X\rightarrow X^{*}\) defined by

$$\begin{aligned} \langle {If},g\rangle =(f,g), \end{aligned}$$
(3.2)

where \(\langle \cdot,\cdot \rangle \) denotes the dual product between X and \(X^{*}\),

$$\begin{aligned} \langle f,g\rangle = \int _{R}\bigl(\mathrm{Re}(f_{1} \overline{g_{1}})+f_{2}g_{2}\bigr)\,{dx}. \end{aligned}$$
(3.3)

From (3.1)–(3.3) it is clear that

$$\begin{aligned} I= \begin{pmatrix} 1-\partial ^{2}_{ x}& 0 \\ 0 & 1 \end{pmatrix}. \end{aligned}$$

Let \(T_{1}\), \(T_{2}\) be one-parameter groups of the unitary operator on X given by

$$\begin{aligned} &T_{1}(s_{1})U(\cdot )=U(\cdot -s_{1}) \quad\text{for } U(\cdot ) \in X, s_{1}\in R, \end{aligned}$$
(3.4)
$$\begin{aligned} &T_{2}(s_{2})U(\cdot )= \begin{pmatrix} e^{-{is}_{2}}u(\cdot ) \\ v(\cdot ) \end{pmatrix} \quad\text{for } U(\cdot )\in X, s_{2}\in R. \end{aligned}$$
(3.5)

Differentiating (3.4) and (3.5) with respect to \(s_{1}\) and \(s_{2}\) at \(s_{1}=0\) and \(s_{2}=0\), respectively, we obtain

$$\begin{aligned} T^{\prime }_{1}(0)= \begin{pmatrix} -\frac{\partial }{\partial x} & 0 \\ 0 & -\frac{\partial }{\partial x} \end{pmatrix} \quad\text{and}\quad T^{\prime }_{2}(0)= \begin{pmatrix} -i & 0 \\ 0 & 0 \end{pmatrix}. \end{aligned}$$
(3.6)

Then we define the functional on X

$$\begin{aligned} E(U)= \int _{R} {\biggl(}\frac{1}{2} \vert u_{x} \vert ^{2}+\frac{\alpha }{2}v \vert u \vert ^{2}+ \frac{\gamma }{4} \vert u \vert ^{4}+\frac{\delta }{6} \vert u \vert ^{6} \biggr)\,{dx}. \end{aligned}$$
(3.7)

By (3.4), (3.5), and (3.7) we can verify that \(E(U)\) is invariant under \(T_{1}\) and \(T_{2}\), namely,

$$\begin{aligned} E\bigl(T_{1}(s_{1})T_{2}(s_{2})U \bigr)=E(U) \quad\text{for any } s_{1},s_{2} \in R. \end{aligned}$$
(3.8)

Moreover, for any \(t\in R\), \(U(t)\) is a flow of Eqs. (1.1):

$$\begin{aligned} E\bigl(U(t)\bigr)=E\bigl(U(0)\bigr). \end{aligned}$$
(3.9)

Note that system (1.1) can be written as the Hamiltonian system

$$\begin{aligned} \frac{dU}{dt}=JE^{\prime }(U),\qquad U=(u,v)\in X, \end{aligned}$$
(3.10)

where J is a skew-symmetrical linear operator defined by

$$\begin{aligned} J= \begin{pmatrix} -i & 0 \\ 0 & -\frac{2\beta }{\alpha }\partial _{x} \end{pmatrix}, \end{aligned}$$
(3.11)

and

$$\begin{aligned} E^{\prime }(U)= \begin{pmatrix} -u_{{xx}}+\alpha {uv}+\gamma \vert u \vert ^{2}u+\delta \vert u \vert ^{4}u \\ \frac{\alpha }{2} \vert u \vert ^{2} \end{pmatrix} \end{aligned}$$
(3.12)

is the Frechét derivative of E.

As in [26, 27], we define the operators

$$\begin{aligned} B_{1}= \begin{pmatrix} -i\frac{\partial }{\partial x} & 0 \\ 0 & \frac{\alpha }{2\beta } \end{pmatrix} \quad\text{and}\quad B_{2}= \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix} \end{aligned}$$

such that \(T^{\prime }_{1}(0)={JB}_{1}\) and \(T^{\prime }_{2}(0)={JB}_{2}\). Furthermore, we define the conserved functionals \(Q_{1}(U)\) and \(Q_{2}(U)\) as follows:

$$\begin{aligned} Q_{1}(U)=\frac{1}{2}\langle B_{1}U,U\rangle =\frac{1}{2} \int _{R}\biggl(\mathrm{Im}(u_{x} \overline{u})+ \frac{\alpha }{2\beta }v^{2}\biggr)\,{dx} \end{aligned}$$
(3.13)

and

$$\begin{aligned} Q_{2}(U)=\frac{1}{2}\langle B_{2}U,U\rangle =\frac{1}{2} \int _{R} \vert u \vert ^{2}\,{dx}. \end{aligned}$$
(3.14)

Differentiating (3.13) and (3.14) with respect to U, respectively, we have

$$\begin{aligned} Q_{1}^{\prime }(U)=B_{1}U= \begin{pmatrix} -i u_{x} \\ \frac{\alpha }{2\beta }v \end{pmatrix} \quad\text{and}\quad Q_{2}^{\prime }(U)=B_{2}U= \begin{pmatrix} u \\ 0 \end{pmatrix}. \end{aligned}$$
(3.15)

Moreover, combining (3.4), (3.5), (3.13), and (3.14), we can prove that \(Q_{1}(U)\) and \(Q_{2}(U)\) are also invariant under \(T_{1}\) and \(T_{2}\), that is,

$$\begin{aligned} &Q_{1}\bigl(T_{1}(s_{1})T_{2}(s_{2})U \bigr)=Q_{1}(U) \quad\text{for any } s_{1},s_{2} \in R, \end{aligned}$$
(3.16)
$$\begin{aligned} &Q_{2}\bigl(T_{1}(s_{1})T_{2}(s_{2})U \bigr)=Q_{2}(U)\quad \text{for any } s_{1},s_{2} \in R. \end{aligned}$$
(3.17)

We also have that \(U(t)\) is the flow of Eqs. (1.1): for any \(t\in R\),

$$\begin{aligned} Q_{1}\bigl(U(t)\bigr)=Q_{1}\bigl(U(0)\bigr) \quad\text{and}\quad Q_{2}\bigl(U(t)\bigr)=Q_{2}\bigl(U(0)\bigr). \end{aligned}$$
(3.18)

From Theorem 3, (3.4), and (3.5) we know that Eqs. (1.1) admit solitary waves \(T_{1}(ct)\times T_{2}(\omega t)\Phi _{\omega,c}(x)\) with \(\Phi _{\omega,c}\) defined by

$$\begin{aligned} \Phi _{\omega,c}(x)=\bigl(\widehat{\phi }_{\omega,c}(x),\psi _{\omega,c}(x)\bigr)=\bigl(e^{i \frac{c}{2}x}\phi _{\omega,c}(x),\psi _{\omega,c}(x)\bigr), \end{aligned}$$
(3.19)

where \(\phi _{\omega,c}(x)\) and \(\psi _{\omega,c}(x)\) are respectively defined by (2.10) and (2.6). For convenience, we write \(\Phi _{\omega,c}(x)\) as \(\Phi (x)\).

Furthermore, combining the first equations of (2.3), (3.12), and (3.15), we get

$$\begin{aligned} &E^{\prime }(\Phi )-c Q_{1}^{\prime }(\Phi )-\omega Q_{2}^{\prime }(\Phi ) \\ &\quad = \begin{pmatrix} -\widehat{\phi }_{{xx}}+\alpha \widehat{\phi }\psi +\gamma \vert \widehat{\phi } \vert ^{2}\widehat{\phi }+\delta \vert \widehat{\phi } \vert ^{4} \widehat{\phi } +{ic}\widehat{\phi }_{x}-\omega \widehat{\phi } \\ \frac{\alpha }{2} \vert \widehat{\phi } \vert ^{2}-\frac{\alpha c}{2\beta }\psi \end{pmatrix}=0. \end{aligned}$$
(3.20)

Now we define the operator from X to \(X^{*}\)

$$\begin{aligned} H_{\omega,c}=E^{\prime \prime }(\Phi )-c Q_{1}^{\prime \prime }(\Phi )- \omega Q_{2}^{\prime \prime }(\Phi ), \end{aligned}$$
(3.21)

where \(E^{\prime \prime }(U)\) is the Frechét derivative of \(E^{\prime }(U)\) defined as

$$\begin{aligned} &E^{\prime \prime }(U) (\eta ) \\ &\quad = \begin{pmatrix} -\partial ^{2}_{x}\eta _{1}+\alpha v\eta _{1}+\gamma \vert u \vert ^{2}\eta _{1}+2 \gamma u \mathrm{Re}(u\overline{\eta _{1}})+\delta \vert u \vert ^{4}\eta _{1}+4 \vert u \vert ^{2}u \mathrm{Re}(u \overline{\eta _{1}})+\alpha u\eta _{2} \\ \alpha \mathrm{Re}(u\overline{\eta _{1}}) \end{pmatrix}, \end{aligned}$$
(3.22)

and \(Q_{1}^{\prime \prime }(U)\) and \(Q_{2}^{\prime \prime }(U)\) are the Frechét derivatives of \(Q_{1}^{\prime }(U)\) and \(Q_{2}^{\prime }(U)\), respectively:

$$\begin{aligned} Q_{1}^{\prime \prime }(U)=B_{1}= \begin{pmatrix} -i\frac{\partial }{\partial x} & 0 \\ 0 & \frac{\alpha }{2\beta } \end{pmatrix} \quad\text{and}\quad Q_{2}^{\prime \prime }(U)=B_{2}= \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}. \end{aligned}$$
(3.23)

In what follows, we consider the orbital stability of solitary waves \(T_{1}({ct})T_{2}(\omega t)\Phi _{\omega,c}(x)\) of (1.1). To prove the orbital stability of solitary waves, we need to prove that they satisfy three hypotheses proposed by Grillakis et al. [26]. From Theorem 1, Theorem 3, (3.20), \(\phi _{\omega,c}^{\prime }\neq 0\), and \(\phi _{\omega,c}\in D(\frac{\partial ^{2}}{\partial x^{2}})\) we get that Assumptions 1 and 2 in [26] are satisfied. Then we need to compute the Hessian operator \(H_{\omega,c}\) and verify Assumption 3 in [26]. Combining (3.21), (3.22), and (3.23), we obtain that

$$\begin{aligned} &H_{\omega,c}\eta \\ &\quad =\bigl(E^{\prime \prime }(\Phi )-c Q_{1}^{ \prime \prime }(\Phi )-\omega Q_{2}^{\prime \prime }(\Phi )\bigr)\eta \\ &\quad = \begin{pmatrix} -\partial ^{2}_{x}\eta _{1}+\alpha \psi \eta _{1}+\gamma \vert \widehat{\phi } \vert ^{2}\eta _{1}+2\gamma \widehat{\phi } \mathrm{Re}(\widehat{\phi } \overline{\eta _{1}})+\delta \vert \widehat{\phi } \vert ^{4}\eta _{1}+4 \vert \widehat{\phi } \vert ^{2} \widehat{\phi } \mathrm{Re}(\widehat{\phi } \overline{\eta _{1}})+\alpha \widehat{\phi }\eta _{2}+{ic}\eta _{1x}- \omega \eta _{1} \\ \alpha \mathrm{Re}(\widehat{\phi }\overline{\eta _{1}})-\frac{\alpha }{2\beta }c \eta _{2} \end{pmatrix}. \end{aligned}$$
(3.24)

Observe that \(H_{\omega,c}\) is self-adjoint in the sense that \(H_{\omega,c}^{*}=H_{\omega,c}\). This means that \(I^{-1}H_{\omega,c}\) is a bounded self-adjoint operator on X. The spectrum of \(H_{\omega,c}\) comprises the real numbers λ such that \(H_{\omega,c}-\lambda I\) is not invertible.

For any \(\eta =(\eta _{1},\eta _{2})\in X\), \(\eta _{1}=e^{\frac{{ic}}{2}(x-{ct})}z\), \(z=z_{1}+{iz}_{2}\), from (3.24) we have that

$$\begin{aligned} H_{\omega,c}\eta& = \begin{pmatrix} (-\partial ^{2}_{x}z-{ic} z_{x}+\frac{z^{2}}{4}+\alpha \psi z+\gamma \phi ^{2}z+2\gamma \phi \mathrm{Re}(\phi \overline{z})+\delta \phi ^{4}z \\ +4\phi ^{3}\mathrm{Re}(\phi \overline{z})+\alpha \phi \eta _{2}- \frac{c^{2}}{2}z+{icz}_{x}-\omega z)e^{\frac{{ic}}{2}(x-{ct})} \\ \alpha \mathrm{Re}(\phi \overline{z})-\frac{\alpha c}{2\beta }\eta _{2} \end{pmatrix} \\ & = \begin{pmatrix} ([-\partial ^{2}_{x}+3\alpha \psi +3\gamma \phi ^{2}+5\delta \phi ^{4}-(\omega +\frac{c^{2}}{4})]z_{1}+\alpha \phi \eta _{2}-2 \alpha \psi z_{1} \\ +i[-\partial ^{2}_{x}+\alpha \psi +\gamma \phi ^{2}+\delta \phi ^{4}-( \omega +\frac{c^{2}}{4})]z_{2} )e^{\frac{{ic}}{2}(x-{ct})} \\ \alpha \phi z_{1}-\frac{\alpha c}{2\beta }\eta _{2} \end{pmatrix}. \end{aligned}$$
(3.25)

Let

$$\begin{aligned} L_{1}=-\partial ^{2}_{x}+3\alpha \psi +3\gamma \phi ^{2}+5\delta \phi ^{4}-\biggl(\omega +\frac{c^{2}}{4} \biggr) \end{aligned}$$
(3.26)

and

$$\begin{aligned} L_{2}=-\partial ^{2}_{x}+\alpha \psi +\gamma \phi ^{2}+\delta \phi ^{4}-\biggl( \omega +\frac{c^{2}}{4} \biggr), \end{aligned}$$
(3.27)

where \(\psi =\frac{\beta }{c}\phi ^{2}\). Then

$$\begin{aligned} H_{\omega,c}\eta = \begin{pmatrix} (L_{1}z_{1}+\alpha \phi \eta _{2}-2\alpha \psi z_{1} +i L_{2}z_{2} )e^{\frac{{ic}}{2}(x-{ct})} \\ \alpha \phi z_{1}-\frac{\alpha c}{2\beta }\eta _{2} \end{pmatrix}. \end{aligned}$$
(3.28)

Furthermore, we have

$$\begin{aligned} \langle H_{\omega,c}\eta,\eta \rangle &=\langle L_{1} z_{1},z_{1} \rangle +\langle L_{2} z_{2},z_{2}\rangle + \int _{R}\biggl(\alpha \phi \eta _{2}z_{1}-2 \alpha \psi z_{1}^{2}+\alpha \phi z_{1}\eta _{2}- \frac{\alpha c}{2\beta }\eta _{2}^{2}\biggr)\, {dx} \\ &=\langle L_{1} z_{1},z_{1}\rangle +\langle L_{2} z_{2},z_{2} \rangle -\frac{\alpha c}{2\beta } \int _{R}\biggl(\frac{2\beta }{c}\phi z_{1}- \eta _{2}\biggr)^{2}\,{dx}. \end{aligned}$$
(3.29)

Next, let us study the spectrum structure of the linear operators \(L_{1}\) and \(L_{2}\). For \(L_{1}\) and \(L_{2}\), we have \(\varphi \rightarrow 0\), \(\psi \rightarrow 0\), \(3\alpha \psi +3\gamma \phi ^{2}+5\delta \phi ^{4}\rightarrow 0\), and \(\alpha \psi +\gamma \phi ^{2}+\delta \phi ^{4}\rightarrow 0\) as \(x\rightarrow \infty \). Therefore by Weyl′s essential spectral theorem the essential spectra of \(L_{1}\) and \(L_{2}\) are \(\sigma _{\mathrm{ess}} L_{1}=[-\omega -\frac{c^{2}}{4},+\infty )\) and \(\sigma _{\mathrm{ess}} L_{2}=[-\omega -\frac{c^{2}}{4},+\infty )\), respectively. Differentiating (2.7) with respect to x and combining with (2.6), we have

$$\begin{aligned} \biggl(\partial ^{2}_{x}-3\alpha \psi -3\gamma \phi ^{2}-5\delta \phi ^{4}+\biggl( \omega +\frac{c^{2}}{4} \biggr)\biggr)\phi _{x}=0. \end{aligned}$$
(3.30)

Then we obtain \(L_{1}\phi _{x}=0\). From (2.6) and (2.7) we have

$$\begin{aligned} \biggl(\partial ^{2}_{x}-\alpha \psi -\gamma \phi ^{2}-\delta \phi ^{4}+\biggl( \omega +\frac{c^{2}}{4} \biggr)\biggr)\phi =0, \end{aligned}$$
(3.31)

that is, \(L_{2}\phi =0\).

As \(\phi _{x}\) has a unique zero at \(x=0\), we know that zero is the second eigenvalue of \(L_{1}\) using the Sturm–Liouville theorem. Thus \(L_{1}\) has exactly one strictly negative eigenvalue \(-\sigma ^{2}\) with an eigenfunction χ, that is,

$$\begin{aligned} L_{1}\chi =-\sigma ^{2}\chi. \end{aligned}$$
(3.32)

Moreover, since ϕ has no zero, we know that zero is the first eigenvalue of \(L_{2}\) using the Sturm–Liouville theorem.

For any real functions \(y\in H^{1}(R)\) satisfying \(\langle y, \chi \rangle =\langle y, \phi _{x}\rangle =0\), along the lines of proof in Appendix of [31], there exists a positive number \(\delta _{1}>0\) such that \(\langle L_{1} y, y\rangle \geq \delta _{1}\|y\|^{2}_{H^{1}(R)}\).

Moreover, for any real functions \(y\in H^{1}(R)\) satisfying \(\langle y, \phi \rangle =0\), there exists a positive number \(\delta _{2}>0\) such that \(\langle L_{2} y, y\rangle \geq \delta _{2}\|y\|^{2}_{H^{1}(R)}\).

From (3.29), (3.30), and (3.31) we have

$$\begin{aligned} H_{\omega,c}T_{1}^{\prime }(0)\Phi = \begin{pmatrix} (\partial ^{2}_{x}\widehat{\phi }-\alpha \psi \widehat{\phi }-\gamma \vert \widehat{\phi } \vert ^{2}\widehat{\phi }-\delta \vert \widehat{\phi } \vert ^{4} \widehat{\phi } -{ic}\widehat{\phi }_{x}+\omega \widehat{\phi })_{x} \\ -\frac{\alpha }{2\beta }( \beta \vert \widehat{\phi } \vert ^{2}-c\psi )_{x} \end{pmatrix}=0 \end{aligned}$$
(3.33)

and

$$\begin{aligned} H_{\omega,c}T_{2}^{\prime }(0)\Phi = \begin{pmatrix} i(\partial ^{2}_{x}\widehat{\phi }-\alpha \psi \widehat{\phi }-\gamma \vert \widehat{\phi } \vert ^{2}\widehat{\phi }-\delta \vert \widehat{\phi } \vert ^{4} \widehat{\phi } -{ic}\widehat{\phi }_{x}+\omega \widehat{\phi }) \\ 0 \end{pmatrix}=0. \end{aligned}$$
(3.34)

Moreover, for any \(\Psi =(y^{-}_{1},y^{-}_{2})\in X\), we choose \(y^{-}_{1}=e^{\frac{{ic}}{2}(x-{ct})}(\chi +i\phi )\), \(y^{-}_{2}=\frac{2\beta }{c}\phi \chi \), and \(\Psi ^{-}=(e^{\frac{{ic}}{2}(x-{ct})}(\chi +i\phi ),\frac{2\beta }{c} \phi \chi )\). Then

$$\begin{aligned} \bigl\langle H_{\omega,c} \Psi ^{-}, \Psi ^{-}\bigr\rangle =\langle L_{1}\chi, \chi \rangle =-\sigma ^{2}\langle \chi,\chi \rangle < 0. \end{aligned}$$
(3.35)

Let

$$\begin{aligned} Z={}&\bigl\{ k_{1}T_{1}^{\prime }(0)\Phi +k_{2}T_{2}^{\prime }(0)\Phi |k_{1},k_{2} \in R\bigr\} , \end{aligned}$$
(3.36)
$$\begin{aligned} P={}&\bigl\{ p\in X| p=(p_{1},p_{2}), p_{1}=e^{\frac{{ic}}{2}(x-{ct})}(p_{11}+{ip}_{12}), \\ &{}\times \langle p_{11},\chi \rangle =\langle p_{11},\phi _{x} \rangle =\langle p_{12},\phi \rangle =0\bigr\} , \end{aligned}$$
(3.37)
$$\begin{aligned} N={}&\bigl\{ k_{3}\Psi ^{-}|k_{3}\in R\bigr\} . \end{aligned}$$
(3.38)

Then for any \(U\in Z\), \(\langle H_{\omega,c}U,U\rangle =0\) by (3.33) and (3.34). For any \(0\neq V\in N\), \(\langle H_{\omega,c}V,V\rangle =k_{3}^{2}\langle H_{\omega,c}\Psi ^{-}, \Psi ^{-}\rangle =-\sigma ^{2}\langle \chi,\chi \rangle <0\). For the subspace P, we have the following:

Lemma 2

Suppose \(\alpha \beta c<0\). For any \(\zeta \in P\), defined by (3.37), there exists a constant \(\delta _{5}>0\) such that

$$\begin{aligned} \langle H_{\omega,c}\zeta,\zeta \rangle \geq \delta _{5} \Vert \zeta \Vert _{X}^{2}, \end{aligned}$$
(3.39)

where \(\delta _{5}\) is independent of ζ.

Proof

For any \(\zeta =(e^{\frac{{ic}}{2}(x-{ct})}(\zeta _{11}+i\zeta _{12}),\zeta _{2}) \in P\), by (3.37) and the spectrum analysis of operator \(L_{1}\) and \(L_{2}\) we have

$$\begin{aligned} \langle H_{\omega,c}\zeta,\zeta \rangle \geq \delta _{1} \Vert \zeta _{11} \Vert _{H^{1}}^{2}+\delta _{2} \Vert \zeta _{12} \Vert _{H^{1}}^{2}- \frac{\alpha c}{2\beta } \int _{R}\biggl(\frac{2\beta }{c}\phi \zeta _{11}- \zeta _{2}\biggr)^{2}\,{dx}. \end{aligned}$$
(3.40)

(1) If \(\|\zeta _{2}\|^{2}_{L^{2}}\geq \frac{8\beta ^{2}}{c^{2}}M^{2}\| \zeta _{11}\|^{2}_{L^{2}}\), \(M=\sup |\phi |\), then

$$\begin{aligned} \int _{R}\biggl(\frac{2\beta }{c}\phi \zeta _{11}- \zeta _{2}\biggr)^{2}\,{dx}&\geq \int _{R}\zeta _{2}^{2}\,{dx}- \int _{R}\frac{4\beta ^{2}}{c^{2}}\phi ^{2} \zeta _{11}^{2}\,{dx}\geq \int _{R}\zeta _{2}^{2}\,{dx}- \int _{R} \frac{4M^{2}\beta ^{2}}{c^{2}}\zeta _{11}^{2} \,{dx} \\ &\geq \frac{1}{2} \Vert \zeta _{2} \Vert _{L_{2}}^{2}. \end{aligned}$$
(3.41)

(2) If \(\|\zeta _{2}\|^{2}_{L^{2}}\leq \frac{8\beta ^{2}}{c^{2}}M^{2}\| \zeta _{11}\|^{2}_{L^{2}}\), then

$$\begin{aligned} \delta _{1} \Vert \zeta _{11} \Vert _{H^{1}}^{2}\geq \frac{\delta _{1}}{2} \Vert \zeta _{11} \Vert _{H_{1}}^{2} +\frac{c^{2}\delta _{1}}{16\beta _{2}M^{2}} \Vert \zeta _{2} \Vert _{L_{2}}^{2}. \end{aligned}$$
(3.42)

Thus for any \(\zeta =(e^{\frac{{ic}}{2}(x-{ct})}(\zeta _{11}+i\zeta _{12}),\zeta _{2}) \in P\), from (3.40)–(3.42) it follows that

$$\begin{aligned} \langle H_{\omega,c}\zeta,\zeta \rangle \geq \delta _{3} \Vert \zeta _{1} \Vert _{H^{1}}^{2}+\delta _{4} \Vert \zeta _{2} \Vert ^{2}_{L^{2}}, \end{aligned}$$
(3.43)

where \(\delta _{3}=\min \{\frac{\delta _{1}}{2},\delta _{2}\}>0\), \(\delta _{4}=\min \{-\frac{\alpha c}{4\beta }, \frac{c^{2}\delta _{1}}{16\beta _{2}M^{2}}\}>0\). Finally, from (3.43) we have

$$\begin{aligned} \langle H_{\omega,c}\zeta,\zeta \rangle \geq \delta _{5} \Vert \zeta \Vert _{X}^{2}, \end{aligned}$$
(3.44)

where \(\delta _{5}=\min \{\delta _{3},\delta _{4}\}\).

By Lemma 2, for any \(p\in P\), \(\langle H_{\omega,c}p,p\rangle \geq \delta \|p\|_{X}^{2}\), where \(\delta >0\). From the previous analysis it follows that \(n(H_{\omega,c})=1\), where \(n(H_{\omega,c})\) denotes the number of negative eigenvalues of \(H_{\omega,c}\).

Next, we define \(d(\omega,c): R\times R\rightarrow R\) by

$$\begin{aligned} d(\omega,c)=E(\Phi _{\omega,c})-c Q_{1}(\Phi _{\omega,c})- \omega Q_{2}( \Phi _{\omega,c}) \end{aligned}$$
(3.45)

and denote by \(d^{\prime \prime }(\omega,c)\) the Hessian matrix of the function \(d(\omega,c)\). This is a symmetric bilinear form. Then we denote by \(p(d^{\prime \prime })\) the number of positive eigenvalues of the Hessian \(d^{\prime \prime }\) at \((\omega,c)\).

From Theorem 2 we obtain the following main results regarding the orbital stability of solitary waves for Eqs. (1.1). □

Theorem 4

Let α, β, γ and δ be any real constants. Suppose that α, β, c, and ω satisfy \(\alpha \beta c<0\), \(4\omega +c^{2}<0\), and one of the following conditions:

(a) \(\delta <0, \frac{\alpha \beta }{c}+\gamma <0, \frac{4}{3} \delta c^{4}+2\alpha \beta c\gamma +3\alpha ^{2}\beta ^{2}>0\),

(b) \(\delta =0, \frac{\alpha \beta }{c}+\gamma <0\),

(c) \(\delta >0, \frac{\alpha \beta }{c}+\gamma <0, ( \frac{\alpha \beta }{c}+\gamma )^{2}>-\frac{4\delta }{3}(4\omega +c^{2})\),

Then the solitary waves \(T_{1}({ct})T_{2}(\omega t)\Phi _{\omega,c}(x)\) of Eqs. (1.1) are orbitally stable.

4 Orbital stability of solitary waves for Eqs. (1.1) in three cases

In this section, we verify that \(p(d^{\prime \prime })=1\) under the conditions of Theorem 2 and provide a detailed proof of Theorem 4.

Combining (3.13), (3.14), and (3.19) with (3.20) and differentiating (3.45) with respect to ω and c, respectively, it follows that

$$\begin{aligned} &d_{\omega }=-Q_{2}(\Phi _{\omega,c})=-\frac{1}{2} \int _{R}\phi ^{2}\,{dx},\qquad d_{c}=-Q_{1}(\Phi _{\omega,c})=-\frac{c}{4} \int _{R}\phi ^{2}\,{dx}- \frac{\alpha \beta }{4c^{2}} \int _{R}\phi ^{4}\,{dx}, \\ &d_{\omega \omega }=-\frac{1}{2}\frac{\partial }{\partial \omega } \int _{R} \phi ^{2}\,{dx},\qquad d_{\omega c}=-\frac{1}{2} \frac{\partial }{\partial c} \int _{R}\phi ^{2}\,{dx},\\ & d_{c\omega }=- \frac{c}{4}\frac{\partial }{\partial \omega } \int _{R}\phi ^{2}\,{dx}- \frac{\alpha \beta }{4c^{2}} \frac{\partial }{\partial \omega } \int _{R} \phi ^{4}\,{dx}, \\ &d_{{cc}}=-\frac{c}{4}\frac{\partial }{\partial c} \int _{R}\phi ^{2}\,{dx}- \frac{1}{4} \int _{R}\phi ^{2}\,{dx} +\frac{\alpha \beta }{2c^{3}} \int _{R} \phi ^{4}\,{dx}-\frac{\alpha \beta }{4c^{2}} \frac{\partial }{\partial c} \int _{R}\phi ^{4}\,{dx}. \end{aligned}$$

Therefore we obtain

$$\begin{aligned} d^{\prime \prime }= \begin{pmatrix} d_{\omega \omega } & d_{\omega c} \\ d_{c\omega } & d_{{cc}} \end{pmatrix} \end{aligned}$$

and

$$\begin{aligned} \det \bigl(d^{\prime \prime }\bigr)={}&\frac{1}{8} \int _{R}\phi ^{2}\,{dx} \frac{\partial }{\partial \omega } \int _{R}\phi ^{2}\,{dx}- \frac{\alpha \beta }{4c^{3}} \int _{R}\phi ^{4}\,{dx} \frac{\partial }{\partial \omega } \int _{R}\phi ^{2}\,{dx} \\ &{} +\frac{\alpha \beta }{8c^{2}} \frac{\partial }{\partial \omega } \int _{R}\phi ^{2}\,{dx} \frac{\partial }{\partial c} \int _{R}\phi ^{4}\,{dx} - \frac{\alpha \beta }{8c^{2}}\frac{\partial }{\partial c} \int _{R}\phi ^{2}\,{dx} \frac{\partial }{\partial \omega } \int _{R}\phi ^{4}\,{dx}. \end{aligned}$$
(4.1)

According to the stability theory developed by Grillakis et al. [26, 27], we only need to observe the sign of \(\det (d^{\prime \prime })\), from which we obtain the orbital stability of solitary waves \(T_{1}({ct})T_{2}(\omega t)\Phi _{\omega,c}(x)\) for Eqs. (1.1). In what follows, we will provide a detailed proof of \(\det (d^{\prime \prime })<0\) in three cases.

Case (a) \(4\omega +c^{2}<0,\delta <0,\frac{\alpha \beta }{c}+\gamma <0,\alpha \beta c<0,\frac{4}{3}\delta c^{4}+2\alpha \beta c\gamma +3\alpha ^{2} \beta ^{2}>0\).

In this case, we have \(d_{1}>0,d_{2}<0, d_{4}<0, \alpha \beta c<0\), and \(4\alpha \beta c d_{2}+4c^{4}d_{4}+\alpha ^{2}\beta ^{2}>0\). Moreover,

$$\begin{aligned} &d_{5}^{2}-d_{3}^{2}=- \frac{d_{4}}{d_{1}},\qquad \frac{d_{3}}{\sqrt{d_{5}^{2}-d_{3}^{2}}}=- \frac{d_{2}}{2\sqrt{-d_{1}d_{4}}}, \\ &\int _{R}\phi ^{2}\,dx= \int _{R}\frac{1}{d_{3}+d_{5}\cosh d_{6}x}\,{dx}= \frac{2}{d_{6}\sqrt{d_{5}^{2}-d_{3}^{2}}} \biggl(\frac{\pi }{2}-\arctan \frac{d_{3}}{\sqrt{d_{5}^{2}-d_{3}^{2}}}\biggr) \\ &\phantom{\int _{R}\phi ^{2}\,dx}=\frac{1}{\sqrt{-d_{4}}}\biggl(\frac{\pi }{2}-\arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}} \biggr), \\ &\frac{\partial }{\partial \omega } \int _{R}\phi ^{2}\,{dx}= \frac{d_{2}}{\sqrt{d_{1}}(d_{2}^{2}-4d_{1}d_{4})}, \\ &\frac{\partial }{\partial c} \int _{R}\phi ^{2}\,{dx}= \frac{\sqrt{d_{1}}}{2c^{2}(d_{2}^{2}-4d_{1}d_{4})} \biggl( \frac{d_{2}}{d_{1}}c^{3}-2\alpha \beta \biggr), \\ &\int _{R}\phi ^{4}\,{dx}= \int _{R}\biggl(\frac{1}{d_{3}+d_{5}\cosh d_{6}x}\biggr)^{2}\, {dx}\\ &\phantom{\int _{R}\phi ^{4}\,{dx}} = \frac{2}{d_{6}(d_{5}^{2}-d_{3}^{2})}+ \frac{d_{2}d_{1}^{\frac{3}{2}}}{2d_{1}^{\frac{3}{2}}(-d_{4})^{\frac{3}{2}}}\biggl(\frac{\pi }{2}- \arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}}\biggr) \\ &\phantom{\int _{R}\phi ^{4}\,{dx}} =-\frac{\sqrt{d_{1}}}{d_{4}}+\frac{d_{2}}{2(-d_{4})^{\frac{3}{2}}}\biggl( \frac{\pi }{2}-\arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}}\biggr), \\ &\frac{\partial }{\partial \omega } \int _{R}\phi ^{4}\,{dx}= \frac{-2\sqrt{d_{1}}}{d_{2}^{2}-4d_{1}d_{4}}, \\ &\frac{\partial }{\partial c} \int _{R}\phi ^{4}\,{dx}= \frac{c}{4\sqrt{d_{1}}d_{4}}- \frac{\alpha \beta }{4c^{2}(-d_{4})^{\frac{3}{2}}} \biggl(\frac{\pi }{2}- \arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}}\biggr)\\ &\phantom{\frac{\partial }{\partial c} \int _{R}\phi ^{4}\,{dx}=}{}- \frac{\sqrt{d_{1}}d_{2}}{4d_{4}(d_{2}^{2}-4d_{1}d_{4})c^{2}} \biggl( \frac{d_{2}}{d_{1}}c^{3}-2 \alpha \beta \biggr). \end{aligned}$$

By (4.1) we have

$$\begin{aligned} \det \bigl(d^{\prime \prime }\bigr)={}&\frac{1}{8}\frac{1}{\sqrt{-d_{4}}}\biggl( \frac{\pi }{2}-\arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}}\biggr) \frac{d_{2}}{d_{1}(d_{2}^{2}-4d_{1}d_{4})}\\ &{}- \frac{\alpha \beta }{4c^{3}} \frac{d_{2}}{(d_{2}^{2}-4d_{1}d_{4})\sqrt{d_{1}}} \biggl[- \frac{\sqrt{d_{1}}}{d_{4}} +\frac{d_{2}}{2(-d_{4})^{\frac{3}{2}}}\biggl(\frac{\pi }{2}-\arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}}\biggr) \biggr] \\ &{}+\frac{\alpha \beta }{8c^{2}} \frac{d_{2}}{\sqrt{d_{1}}(d_{2}^{2}-4d_{1}d_{4})}\biggl[ \frac{c}{4\sqrt{d_{1}}d_{4}} - \frac{\alpha \beta }{4c^{2}(-d_{4})^{\frac{3}{2}}} \biggl(\frac{\pi }{2}-\arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}}\biggr)\\ &{}- \frac{\sqrt{d_{1}}}{4c^{2}d_{4}(d_{2}^{2}-4d_{1}d_{4})} \biggl( \frac{d_{2}}{d_{1}}c^{3}-2\alpha \beta \biggr)\biggr]\\ &{}-\frac{\alpha \beta }{8c^{2}} \frac{\sqrt{d_{1}}}{2c^{2}(d_{2}^{2} -4d_{1}d_{4})} \biggl(\frac{d_{2}}{d_{1}}c^{3}-2\alpha \beta \biggr) \frac{-2\sqrt{d_{1}}}{d_{2}^{2}-4d_{1}d_{4}} \\ ={}&\biggl[\frac{1}{4(d_{2}^{2}-4d_{1}d_{4})}\biggl( \frac{\alpha \beta d_{2}}{8cd_{1}d_{4}}-1- \frac{\alpha ^{2}\beta ^{2}}{4c^{4}d_{4}} \biggr)\\ &{}- \frac{\alpha \beta c}{32c^{4}d_{4}(d_{2}^{2}-4d_{1}d_{4})}\biggl( \frac{d_{2}}{d_{1}}c^{2}- \frac{2\alpha \beta }{c}\biggr)\biggr] \\ &{}+\frac{1}{4(d_{2}^{2}-4d_{1}d_{4})}\biggl(1+ \frac{\alpha \beta d_{2}}{c^{3}d_{4}}+ \frac{\alpha ^{2}\beta ^{2}}{4c^{4}d_{4}}\biggr) \\ &{}\times \biggl[1+ \frac{d_{2}}{2\sqrt{-d_{1}d_{4}}} \biggl(\frac{\pi }{2}-\arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}} \biggr)\biggr] \\ ={}& \frac{-1}{4(d_{2}^{2}-4d_{1}d_{4})}+ \frac{1}{4(d_{2}^{2}-4d_{1}d_{4})}\biggl(1+ \frac{\alpha \beta d_{2}}{c^{3}d_{4}}+ \frac{\alpha ^{2}\beta ^{2}}{4c^{4}d_{4}}\biggr)\biggl[1- \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}} \\ & {}\times\biggl(\frac{\pi }{2}-\arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}}\biggr)\biggr] \\ ={}& I+\mathit{II}, \end{aligned}$$

where

$$\begin{aligned} I=\frac{-1}{4(d_{2}^{2}-4d_{1}d_{4})} \end{aligned}$$

and

$$\begin{aligned} \mathit{II}=\frac{1}{4(d_{2}^{2}-4d_{1}d_{4})}\biggl(1+ \frac{\alpha \beta d_{2}}{c^{3}d_{4}}+ \frac{\alpha ^{2}\beta ^{2}}{4c^{4}d_{4}} \biggr)\biggl[1- \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}} \biggl(\frac{\pi }{2}-\arctan \frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}}\biggr)\biggr]. \end{aligned}$$

Next, we estimate I and II. As \(d_{4}<0\), \(d_{2}<0\), \(d_{1}>0\), and \(\alpha \beta c<0\), we get \(I<0\). Let \(y=\frac{-d_{2}}{2\sqrt{-d_{1}d_{4}}}\). Then we have \(y>0\) and

$$\begin{aligned} &\mathit{II}=\frac{1}{4(d_{2}^{2}-4d_{1}d_{4})}\biggl(1+ \frac{\alpha \beta d_{2}}{c^{3}d_{4}}+ \frac{\alpha ^{2}\beta ^{2}}{4c^{4}d_{4}} \biggr)Y(y), \\ &Y(y)=1-y\biggl(\frac{\pi }{2}-\arctan y\biggr). \end{aligned}$$

It is clear that

$$\begin{aligned} &Y(0)=\lim_{y\rightarrow 0^{+}}\biggl[1-y\biggl(\frac{\pi }{2}-\arctan y \biggr)\biggr]=1, \end{aligned}$$
(4.2)
$$\begin{aligned} &Y(+\infty )=\lim_{y\rightarrow +\infty }\biggl[1-y\biggl( \frac{\pi }{2}- \arctan y\biggr)\biggr]= 1-\lim_{y\rightarrow +\infty } \frac{\frac{\pi }{2}-\arctan y}{1/y} \\ &\phantom{Y(+\infty )} =1-\lim_{y\rightarrow +\infty } \frac{-1/(1+y^{2})}{-1/y^{2}}=0. \end{aligned}$$
(4.3)

To estimate II, we first show that \(Y(y)\) is a decreasing function and \(Y(y)>0\). Differentiating \(Y(y)\) with respect to y, we have

$$\begin{aligned} &Y^{\prime }(y)=-\frac{\pi }{2}+\arctan y+\frac{y}{1+y^{2}}, \end{aligned}$$
(4.4)
$$\begin{aligned} &Y^{\prime \prime }(y)=\frac{2}{1+y^{2}}-\frac{2y^{2}}{(1+y^{2})^{2}}= \frac{2}{(1+y^{2})^{2}}>0, \end{aligned}$$
(4.5)

and

$$\begin{aligned} &Y^{\prime }(0)=\lim_{y\rightarrow 0^{+}}\biggl[-\frac{\pi }{2}+ \arctan y+\frac{y}{1+y^{2}}\biggr]=-\frac{\pi }{2}, \end{aligned}$$
(4.6)
$$\begin{aligned} &Y^{\prime }(+\infty )=\lim_{y\rightarrow +\infty }\biggl[- \frac{\pi }{2}+\arctan y+\frac{y}{1+y^{2}}\biggr]=0. \end{aligned}$$
(4.7)

Then (4.5), (4.6), and (4.7) imply that \(Y^{\prime }(y)<0\) for any \(y\in R^{+}\). Thus from (4.2) and (4.3) we have

$$\begin{aligned} Y(y)>0 \quad\text{for any } y\in R^{+}. \end{aligned}$$
(4.8)

As \(d_{4}<0\), \(d_{2}<0\), \(d_{1}>0\), \(\alpha \beta c<0\), and \(4c^{4}d_{4}+4\alpha \beta c d_{2}+\alpha ^{2}\beta ^{2}>0\), we have

$$\begin{aligned} 1+\frac{\alpha \beta d_{2}}{c^{3}d_{4}}+ \frac{\alpha ^{2}\beta ^{2}}{4c^{4}d_{4}}< 0. \end{aligned}$$
(4.9)

Then (4.8) and (4.9) ensure that \(\mathit{II}<0\). Thus we have \(\det (d^{\prime \prime })<0\), which implies that \(d^{\prime \prime }\) has exactly one negative eigenvalue and one positive eigenvalue, namely, \(p(d^{\prime \prime })=1\).

Case (b) \(\delta =0, \frac{\alpha \beta }{c}+\gamma <0, \alpha \beta c<0,4 \omega +c^{2}<0\).

In this case, we have

$$\begin{aligned} &\phi ^{2}=-\frac{2d_{1}}{d_{2}}\frac{1}{1+\cosh d_{6}x}, \\ &\int _{R}\phi ^{2}\,{dx}=- \frac{2d_{1}}{d_{2}} \int _{R} \frac{1}{1+\cosh d_{6}x}\,{dx}=- \frac{2\sqrt{d_{1}}}{d_{2}}, \\ &\frac{\partial }{\partial \omega } \int _{R}\phi ^{2}\,{dx}= \frac{1}{\sqrt{d_{1}}d_{2}}, \\ &\frac{\partial }{\partial c} \int _{R}\phi ^{2}\,{dx}= \frac{\sqrt{d_{1}}}{2d_{2}^{2}c^{2}}\biggl(\frac{d_{2}}{d_{1}}c^{3}-2 \alpha \beta \biggr), \\ &\int _{R}\phi ^{4}\,{dx}=\frac{4d_{1}^{2}}{d_{2}^{2}} \int _{R}\biggl( \frac{1}{1+\cosh d_{6}x}\biggr)^{2}\, {dx}=\frac{4}{3} \frac{d_{1}^{\frac{3}{2}}}{d_{2}^{2}}, \\ &\frac{\partial }{\partial \omega } \int _{R}\phi ^{4}\,{dx}=- \frac{2\sqrt{d_{1}}}{d_{2}^{2}}, \\ &\frac{\partial }{\partial c} \int _{R}\phi ^{4}\,{dx}=- \frac{\sqrt{d_{1}}c}{d_{2}^{2}}+ \frac{4d_{1}^{\frac{3}{2}}\alpha \beta }{3d_{2}^{3}c^{2}} =- \frac{d_{1}^{\frac{3}{2}}}{d_{2}^{3}c^{2}}\biggl( \frac{d_{2}}{d_{1}}c^{3}-2 \alpha \beta \biggr)- \frac{2d_{1}^{\frac{3}{2}}\alpha \beta }{3d_{2}^{3}c^{2}}. \end{aligned}$$

By (4.1) we have

$$\begin{aligned} \det \bigl(d^{\prime \prime }\bigr)={}&\frac{1}{8}\biggl(-\frac{2\sqrt{d_{1}}}{d_{2}} \biggr) \frac{1}{\sqrt{d_{1}}d_{2}} -\frac{\alpha \beta }{4c^{3}}\frac{4}{3} \frac{d_{1}^{\frac{3}{2}}}{d_{2}^{2}}\frac{1}{\sqrt{d_{1}}d_{2}}\\ &{} + \frac{\alpha \beta }{8c^{2}}\frac{1}{\sqrt{d_{1}}d_{2}} \biggl(- \frac{d_{1}^{\frac{3}{2}}}{d_{2}^{3}c^{2}}\biggr) \biggl(\frac{d_{2}}{d_{1}}c^{3}-2 \alpha \beta \biggr) \\ &{}-\frac{\alpha \beta }{8c^{2}\sqrt{d_{1}}d_{2}} \frac{2d_{1}^{\frac{3}{2}}\alpha \beta }{3d_{2}^{3}c^{2}} - \frac{\alpha \beta }{8c^{2}} \frac{\sqrt{d_{1}}}{2d_{2}^{2}c^{2}}\biggl( \frac{d_{2}}{d_{1}}c^{3}-2\alpha \beta \biggr) \biggl(- \frac{2\sqrt{d_{1}}}{d_{2}^{2}}\biggr) \\ ={}&{-}\frac{1}{12c^{4}d_{2}^{2}}\bigl[3c^{4}d_{2}^{2}+4 \alpha \beta {cd}_{1}d_{2}+ \alpha ^{2} \beta ^{2}d_{1}\bigr]. \end{aligned}$$

As \(d_{2}<0\), \(d_{1}>0\), and \(\alpha \beta c<0\), we have

$$\begin{aligned} 3c^{4}d_{2}^{2}+4\alpha \beta {cd}_{1}d_{2}+\alpha ^{2}\beta ^{2}d_{1}>0. \end{aligned}$$

Then we have \(\det (d^{\prime \prime })<0\). Thus \(d^{\prime \prime }\) has exactly one negative eigenvalue and one positive eigenvalue, that is, \(p(d^{\prime \prime })=1\).

Case (c) \(\delta >0, \frac{\alpha \beta }{c}+\gamma <0,4\omega +c^{2}<0, ( \frac{\alpha \beta }{c}+\gamma )^{2}>-\frac{4\delta }{3}(4\omega +c^{2}), \alpha \beta c<0\).

In this case, we have \(d_{1}>0,d_{2}<0,d_{4}>0,d_{2}^{2}-4d_{1}d_{4}>0\), \(\alpha \beta c<0\). Then

$$\begin{aligned} &d_{5}^{2}-d_{3}^{2}=- \frac{d_{4}}{d_{1}}< 0,\qquad d_{3}>d_{5}, \\ &\int _{R}\phi ^{2}\,{dx}= \int _{R}\frac{1}{d_{3}+d_{5}\cosh d_{6}x}\,{dx}= \frac{1}{2\sqrt{d_{4}}} \ln \frac{-d_{2}+2\sqrt{d_{4}d_{1}}}{-d_{2}-2\sqrt{d_{4}d_{1}}}, \\ &\frac{\partial }{\partial \omega } \int _{R}\phi ^{2}\,{dx}= \frac{d_{2}}{\sqrt{d_{1}}(d_{2}^{2}-4d_{1}d_{4})}, \\ &\frac{\partial }{\partial c} \int _{R}\phi ^{2}\,{dx}= \frac{\sqrt{d_{1}}}{2c^{2}(d_{2}^{2}-4d_{1}d_{4})} \biggl( \frac{d_{2}}{d_{1}}c^{3}-2\alpha \beta \biggr), \\ &\int _{R}\phi ^{4}\,{dx}= \int _{R}\biggl(\frac{1}{d_{3}+d_{5}\cosh d_{6}x}\biggr)^{2}\, {dx}=- \frac{\sqrt{d_{1}}}{d_{4}}-\frac{d_{2}}{4d_{4}^{\frac{3}{2}}} \ln \frac{-d_{2}+2\sqrt{d_{4}d_{1}}}{-d_{2}-2\sqrt{d_{4}d_{1}}}, \\ &\frac{\partial }{\partial \omega } \int _{R}\phi ^{4}\,{dx}= \frac{-2\sqrt{d_{1}}}{d_{2}^{2}-4d_{1}d_{4}}, \\ &\frac{\partial }{\partial c} \int _{R}\phi ^{4}\,{dx}= \frac{c}{4d_{4}\sqrt{d_{1}}}+ \frac{\alpha \beta }{8c^{2}d_{4}^{\frac{3}{2}}}. \ln \frac{-d_{2}+2\sqrt{d_{1}d_{4}}}{-d_{2}-2\sqrt{d_{1}d_{4}}}\\ &\phantom{\frac{\partial }{\partial c} \int _{R}\phi ^{4}\,{dx}=}{}- \frac{d_{2}\sqrt{d_{1}}}{4d_{4}(d_{2}^{2} -4d_{1}d_{4})c^{2}}\biggl(\frac{d_{2}}{d_{1}}c^{3}-2\alpha \beta \biggr). \end{aligned}$$

By (4.1) we have

$$\begin{aligned} &\det \bigl(d^{\prime \prime }\bigr)\\ &\quad =\frac{1}{8}\frac{1}{2\sqrt{d_{4}}}\ln \frac{-d_{2}+2\sqrt{d_{1}d_{4}}}{-d_{2}-2\sqrt{d_{1}d_{4}}} \frac{d_{2}}{\sqrt{d_{1}}(d_{2}^{2}-4d_{1}d_{4})}- \frac{\alpha \beta }{4c^{3}} \frac{d_{2}}{\sqrt{d_{1}}(d_{2}^{2}-4d_{1}d_{4})} \biggl(- \frac{\sqrt{d_{1}}}{d_{4}}\biggr) \\ &\qquad{}+\frac{\alpha \beta }{4c^{3}} \frac{d_{2}}{\sqrt{d_{1}}(d_{2}^{2}-4d_{1}d_{4})} \frac{d_{2}}{4d_{4}^{\frac{3}{2}}} \ln \frac{-d_{2}+2\sqrt{d_{1}d_{4}}}{-d_{2}-2\sqrt{d_{1}d_{4}}}+ \frac{\alpha \beta }{8c^{2}} \frac{d_{2}}{\sqrt{d_{1}}(d_{2}^{2}-4d_{1}d_{4})} \frac{c}{4d_{4}\sqrt{d_{1}}} \\ &\qquad{}- \frac{\alpha \beta d_{2}^{2}}{32c^{4}d_{4}(d_{2}^{2}-4d_{1}d_{4})^{2}}\biggl( \frac{d_{2}}{d_{1}}c^{3}-2\alpha \beta \biggr)\\ &\qquad{} + \frac{\alpha ^{2}\beta ^{2}d_{2}}{64c^{4}\sqrt{d_{1}d_{4}^{3}}(d_{2}^{2}-4d_{1}d_{4})} \ln \frac{-d_{2}+2\sqrt{d_{1}d_{4}}}{-d_{2}-2\sqrt{d_{1}d_{4}}} \\ &\qquad{}-\frac{\alpha \beta \sqrt{d_{1}}}{16c^{4}(d_{2}^{2}-4d_{1}d_{4})}\biggl(- \frac{2\sqrt{d_{1}}}{d_{2}^{2}-4d_{1}d_{4}}\biggr) \biggl( \frac{d_{2}}{d_{1}}c^{3}-2 \alpha \beta \biggr) \\ &\quad=\frac{4c^{4}d_{4}+4\alpha \beta {cd}_{2}+\alpha ^{2}\beta ^{2}}{16c^{4}d_{4}(d_{2}^{2}-4d_{1}d_{4})}\biggl(1+ \frac{d_{2}}{4\sqrt{d_{1}d_{4}}} \ln \frac{-\frac{d_{2}}{2\sqrt{d_{1}d_{4}}}+1}{-\frac{d_{2}}{2\sqrt{d_{1}d_{4}}}-1} \biggr)\\ &\qquad{} +\biggl[-\frac{\alpha ^{2}\beta ^{2}}{32c^{4}d_{4}(d_{2}^{2}-4d_{1}d_{4})}\biggl( \frac{d_{2}}{d_{1}} \frac{c^{3}}{\alpha \beta }-2\biggr) \\ &\qquad{}- \frac{8c^{4}d_{1}d_{4}+2\alpha ^{2}\beta ^{2}d_{1}-\alpha \beta c^{3}d_{2}}{32c^{4}d_{1}d_{4}(d_{2}^{2}-4d_{1}d_{4})}\biggr] \\ &\quad=\frac{4c^{4}d_{4}+4\alpha \beta {cd}_{2}+\alpha ^{2}\beta ^{2}}{16c^{4}d_{4}(d_{2}^{2}-4d_{1}d_{4})}\biggl(1+ \frac{d_{2}}{4\sqrt{d_{1}d_{4}}} \ln \frac{-\frac{d_{2}}{2\sqrt{d_{1}d_{4}}}+1}{-\frac{d_{2}}{2\sqrt{d_{1}d_{4}}}-1} \biggr)- \frac{1}{4(d_{2}^{2}-4d_{1}d_{4})} \\ &\quad = I+\mathit{II}, \end{aligned}$$

where

$$\begin{aligned} I= \frac{4c^{4}d_{4}+4\alpha \beta {cd}_{2}+\alpha ^{2}\beta ^{2}}{16c^{4}d_{4}(d_{2}^{2}-4d_{1}d_{4})}\biggl(1+ \frac{d_{2}}{4\sqrt{d_{1}d_{4}}} \ln \frac{-\frac{d_{2}}{2\sqrt{d_{1}d_{4}}}+1}{-\frac{d_{2}}{2\sqrt{d_{1}d_{4}}}-1} \biggr) \end{aligned}$$

and

$$\begin{aligned} \mathit{II}=-\frac{1}{4(d_{2}^{2}-4d_{1}d_{4})}. \end{aligned}$$

Next, we estimate I and II. As \(d_{2}^{2}-4d_{1}d_{4}>0\), we have \(\mathit{II}<0\). Let \(y=-\frac{d_{2}}{2\sqrt{d_{1}d_{4}}}\). Then we have \(y>1\) and

$$\begin{aligned} &I= \frac{4c^{4}d_{4}+4\alpha \beta {cd}_{2}+\alpha ^{2}\beta ^{2}}{16c^{4}d_{4}(d_{2}^{2}-4d_{1}d_{4})}Z(y), \\ &Z(y)=1-\frac{y}{2}\ln \frac{y+1}{y-1}. \end{aligned}$$

It is clear that

$$\begin{aligned} &Z(1)=\lim_{y\rightarrow 1^{+}}\biggl[1-\frac{y}{2}\ln \frac{y+1}{y-1}\biggr]=-\infty, \end{aligned}$$
(4.10)
$$\begin{aligned} &Z(+\infty )=\lim_{y\rightarrow +\infty }\biggl[1-\frac{y}{2} \ln \frac{y+1}{y-1}\biggr]= 1+\lim_{y\rightarrow +\infty } \frac{\frac{1}{y+1}-\frac{1}{y-1}}{2/y^{2}} \\ &\phantom{Z(+\infty )} =1+\lim_{y\rightarrow +\infty } \frac{-y^{2}}{y^{2}-1}=0. \end{aligned}$$
(4.11)

To estimate I, we first show that \(Z(y)\) is an increasing function and \(Z(y)<0\). Differentiating \(Z(y)\) with respect to y, we have

$$\begin{aligned} &Z^{\prime }(y)=-\frac{1}{2}\ln \frac{y+1}{y-1}+ \frac{y}{y^{2}-1}, \end{aligned}$$
(4.12)
$$\begin{aligned} &Z^{\prime \prime }(y)=-\frac{2}{(y^{2}-1)^{2}}< 0 \quad\text{for any } 1< y< +\infty, \end{aligned}$$
(4.13)

and

$$\begin{aligned} Z^{\prime }(+\infty )=\lim_{y\rightarrow +\infty }\biggl[- \frac{1}{2}\ln \frac{y+1}{y-1}+\frac{y}{y^{2}-1}\biggr]=0. \end{aligned}$$
(4.14)

Then (4.13) and (4.14) imply that \(Z^{\prime }(y)>0\) for any \(y\in (1,+\infty )\). Thus from (4.10) and (4.11) we have

$$\begin{aligned} Z(y)< 0 \quad\text{for any } y\in (1,+\infty ). \end{aligned}$$
(4.15)

As \(d_{4}>0\), \(d_{2}<0\), and \(\alpha \beta c<0\), we have

$$\begin{aligned} \frac{4c^{4}d_{4}+4\alpha \beta {cd}_{2}+\alpha ^{2}\beta ^{2}}{16c^{4}d_{4}(d_{2}^{2}-4d_{1}d_{4})}>0. \end{aligned}$$
(4.16)

Then, combining (4.15) with (4.16), we have \(I<0\). Thus we obtain \(\det (d^{\prime \prime })<0\), which implies that \(d^{\prime \prime }\) has exactly one negative eigenvalue and one positive eigenvalue, that is, \(p(d^{\prime \prime })=1\).

Therefore by Theorem 2 we prove that the solitary waves \(e^{-i\omega t}\Phi (x-ct)\) of Eqs. (1.1) are orbitally stable under the conditions of Theorem 4.

Corollary 3

For any real constants \(\omega,c,\alpha,\beta,\gamma \) satisfying \(4\omega +c^{2}<0,\frac{\alpha \beta }{c}+\gamma <0\), and \(\alpha \beta c<0\), the solitary waves \(T_{1}({ct})T_{2}(\omega t)\Phi _{\omega,c}(x)\) of Eqs. (1.2) (\(\lambda =1\)) with expression (3.20) are orbitally stable.

Remark 2

In particular, when the parameters \(\alpha =1\), \(\beta =-1\), and \(\delta =0\), Eqs. (1.1) reduce to the LS wave resonance equations (1.3) studied by Guo and Chen [25]. Because \(d_{{cc}}(\omega,c) \) is incorrect and consequently \(\det (d^{\prime \prime })\) as well, condition (3.28) of orbital stability for solitary waves in [25] is incorrect. The results obtained in Corollary 3 herein are correct and extend the results of [25]. The orbitally stable results obtained in Theorem 4 can be regarded as an extension of the results of Chen and Guo [25].

Furthermore, when \(\beta =0\), according to the instability theory [27] (\(n(H_{\omega,c})-p(d^{\prime \prime })\) is odd), we can obtain the following result by the same process as that detailed in Sect. 3 and Case (b).

Corollary 4

For any real constants \(\omega,c,\gamma \) satisfying \(4\omega +c^{2}<0\) and \(\gamma <0\), the solitary wave \(u(x,t)=e^{-i\omega t}\sqrt{\frac{4\omega +c^{2}}{2\gamma }}{\mathrm{sech}} \frac{\sqrt{-4\omega -c^{2}}}{2}(x-{ct})\) of the nonlinear Schrödinger equation is orbitally instable.