1 Introduction

A large number of studies have shown that fractional differential equations have many applications in science and engineering [1,2,3,4,5]. Compared with integer differential equations, fractional differential equations can more effectively and accurately simulate various natural phenomena, such as in neurons, economics, image processing, viscoelasticity, biochemical processes, bioengineering, electromagnetic phenomena, etc. Because of the importance of fractional equations in theory and application, in the past decade, researchers dealing with fractional differential equations have achieved significant development. By applying the fixed point theory, the topology theory, and the monotonically iterative techniques, a lot of results on the existence and multiplicity of solutions have been established [6,7,8,9]. In recent years, the study of fractional differential equations by variational methods has attracted researchers’ great attention [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. This is an interesting and new research trend. Especially, for the fractional differential equations involving both the left and right fractional derivatives, in general, the fixed point theory is not suitable for studying the existence to this kind of equations because it is not easy to convert this kind of fractional equation into an equivalent integral equation and then transform it into some fixed point problem of an operator. For the first time, Jiao and Zhou in [10] show that the variational method is a very effective way to study such problems. By establishing some appropriate spaces and variational structure, the authors successfully apply the critical point theory to the following problems of fractional differential equations:

$$ \textstyle\begin{cases} _{t} D_{T}^{\alpha }(_{0} D_{t}^{\alpha }u(t))=\nabla F(t,u(t)),\quad \mbox{a.e. } t\in [0,T], \\ u(0)=u(T)=0. \end{cases} $$
(1.1)

Under some suitable conditions, the existence results have been obtained.

Recently, in [24], motivated by the work in [10] mentioned above, Torres investigates a kind of fractional Hamiltonian systems with the left and right fractional derivatives on \(\mathbb{R}\) as follows:

$$ { }_{t}D_{\infty }^{\alpha }\bigl({ }_{-\infty }D_{t}^{\alpha }u(t)\bigr)+L(t)u(t)= \nabla W \bigl(t,u(t)\bigr), $$
(1.2)

where \(\alpha \in (1/2,1)\), \(t\in \mathbb{R}\), \(u\in \mathbb{R}^{N}\), \(L\in C( \mathbb{R},\mathbb{R}^{N\times N})\) is a symmetry matrix-valued function satisfying the following condition:

(L1):

\(L(t)\) is a positive definite symmetric matrix for all \(t\in \mathbb{R}\), and there is \(l\in C(\mathbb{R},(0,\infty ))\) such that \(l(t)\to \infty \) as \(t\to \infty \) as well as

$$\bigl(L(t)x,x\bigr)\ge l(t) \vert x \vert ^{2} \quad \mbox{for any } t\in \mathbb{R}, x \in \mathbb{R}^{N}. $$

Under condition (L1) and some other conditions, applying the mountain pass theorem, the author obtained the existence result to problem (1.2).

For the above system (1.2), if \(\alpha =1\), then it will reduce to the following second order Hamiltonian system:

$$ {u}''-L(t)u+\nabla W(t,u)=0. $$
(1.3)

Poincare [32] was first to recognize the existence of homoclinic solutions of Hamiltonian systems (1.3) and its importance for studying the behaving on dynamical systems. Following this existence, one can infer the existence of chaos nearby or the bifurcation behavior of periodic orbits. In the past two decades, variational methods and critical point theory have been successfully applied to research the existence and multiplicity of homoclinic solutions [33, 34].

Continuing the study in [24], Amado and Torres further investigate problem (1.2). Under some weaker condition than (L1) as well as some conditions, the authors have obtained infinitely many solutions to problems via the genus properties in the critical point theory [25]. For more research on problem (1.2), the readers can refer to [26,27,28,29, 35, 36] and the references therein.

Motivated by the works mentioned above, in this paper the authors consider the following fractional Hamiltonian system:

$$ \textstyle\begin{cases} { }_{t}D_{\infty }^{\alpha } ( {{ }_{-\infty }D_{t}^{\alpha }u(t)} )+V(t)u(t)=\lambda u(t)+b(t) \vert u(t) \vert ^{q-2}u(t)+\mu h(t),\quad t \in \mathbb{R}, \\ u\in H^{\alpha }(\mathbb{R},\mathbb{R}^{N}), \end{cases} $$
(1.4)

where \(\alpha \in (1/2,1)\), \(q>2\), \(N\ge 1\), \(b\in C(\mathbb{R},(0,\infty ))\), \(h \in C(\mathbb{R},\mathbb{R}^{N}), \lambda , \mu \) are parameters, \(V\in C(\mathbb{R},\mathbb{R}^{N\times N}) \) is a positive definite symmetry matrix for all \(t\in \mathbb{R}\). Indeed, system (1.4) is a class of quite important fractional Hamiltonian systems with the perturbed terms λu and μh. If \(\alpha =1\), (1.4) reduces to the following classical second order Hamiltonian system:

$$ \textstyle\begin{cases} {u}''(t)+V(t)u(t)=\lambda u(t)+b(t) \vert u(t) \vert ^{q-2}u(t)+\mu h(t),\quad t \in \mathbb{R}, \\ u\in H^{1}(\mathbb{R},\mathbb{R}^{N}), \end{cases} $$
(1.5)

which is a prototype of many problems. For more information, readers can refer to [37].

Because the energy functional corresponding to system (1.4) is unbounded from below, it is invalid to try finding a critical point by applying the direct method of minimization to the energy functional. To overcome this difficulty, we introduce the Nehari manifold \(N_{\lambda ,\mu }\) and split it into three parts \(N_{\lambda ,\mu }^{-}\), \(N_{\lambda , \mu }^{+}\), and \(N_{\lambda ,\mu }^{0}\). By looking for minimizer of the energy functional on \(N_{\lambda , \mu }^{-} \) and \(N_{\lambda ,\mu }^{+}\) respectively and with the help of more analysis techniques, the authors establish some results on the existence and multiplicity of solutions to problem (1.4).

The remaining part of this paper is organized as follows. In Sect. 2, we give some necessary definitions and suitable variational work frame for (1.4). Finally, we will establish the results on the existence of solutions to (1.4) in Sect. 3.

2 Preliminaries

In this section, we introduce some basic conceptions of fractional calculus and fractional space.

Definition 2.1

([3])

The left and right Liouville–Weyl fractional integrals of order \(0<\alpha <1\) for function u are defined as

$${ }_{-\infty }I_{t}^{\alpha }u(t)=\frac{1}{\varGamma (\alpha )} \int _{-\infty }^{t} {(t-s)^{\alpha -1}u(s)\,ds } $$

and

$${ }_{t}I_{\infty }^{\alpha }u(t)=\frac{1}{\varGamma (\alpha )} \int _{t} ^{\infty }{(s-t)^{\alpha -1}u(s)\,ds ,} $$

respectively.

Definition 2.2

([3])

The left and right Liouville–Weyl fractional derivatives of order \(0<\alpha <1\) for function u are defined as

$${ }_{-\infty }D_{t}^{\alpha }u(t)=\frac{d}{dt }{ }_{-\infty }I_{t}^{1- \alpha } u(t) $$

and

$${ }_{t}D_{\infty }^{\alpha }u(t)=-\frac{d}{dt }{ }_{t}I_{\infty }^{1- \alpha } u(t), $$

respectively.

On the Liouville–Weyl fractional derivative, we have the following relations:

$$\begin{aligned}& { }_{-\infty }D_{t}^{\alpha }u(t)=\frac{\alpha }{\varGamma (1-\alpha )} \int _{0}^{\infty }{\frac{u(t)-u(t-s)}{s^{\alpha +1}}\,ds } , \\& { } _{t}D_{\infty }^{\alpha }u(t)=\frac{\alpha }{\varGamma (1-\alpha )} \int _{0}^{\infty }{\frac{u(t)-u(t+s)}{s^{\alpha +1}}\,ds } . \end{aligned}$$

Moreover, we introduce the Fourier transform \(\hat{u}(w)\) of \(u(t)\) as follows:

$$\hat{u}(w)= \int _{-\infty }^{\infty }{e^{-iwt}u(t)\,dt } ,\quad w \in \mathbb{R}, $$

which satisfies

$$\begin{aligned}& { }_{-\infty }\widehat{I_{t}^{\alpha }u}(t) (w)=(iw)^{-\alpha }\hat{u}(w),\qquad { }_{t}\widehat{I_{\infty }^{\alpha }u}(t) (w)=(-iw)^{-\alpha }\hat{u}(w), \\& { }_{-\infty }\widehat{D_{t}^{\alpha }u}(t) (w)=(iw)^{\alpha }\hat{u}(w),\qquad { }_{t}\widehat{D_{\infty }^{\alpha }u}(t) (w)=(-iw)^{\alpha }\hat{u}(w). \end{aligned}$$

Now, we introduce some fractional derivative space; for more details, refer to [38].

As usual, for \(1\le p<\infty \), denote by \(L^{P}(\mathbb{R}, \mathbb{R}^{N})\) the Banach spaces of functions under the norm \(\Vert u\Vert _{L^{P}}= ( {\int _{\mathbb{R}} {\vert u(t) \vert ^{P} \,dt }} )^{\frac{1}{P}}\). \(L^{\infty }(\mathbb{R}, \mathbb{R}^{N})\) is the Banach spaces of essentially bounded functions on the norm \(\Vert u\Vert _{\infty }=\operatorname{ess}\sup \{{\vert u(t) \vert :t\in \mathbb{R}} \}\).

For \(\alpha >0\), define the semi-norm \(\vert u \vert _{I_{-\infty }^{\alpha }} =\Vert { }_{-\infty }D_{t}^{ \alpha }u\Vert _{L^{2}} \) and the norm

$$\Vert u \Vert _{I_{-\infty }^{\alpha }} = \bigl( { \vert u \vert _{I_{-\infty }^{\alpha }}^{2} + \Vert u \Vert _{L^{2}} ^{2} } \bigr)^{\frac{1}{2}}. $$

Let \(I_{-\infty }^{\alpha }\) be the closure of space \(C_{0}^{\infty }( \mathbb{R},\mathbb{R}^{N})\) on the norm \(\Vert \cdot \Vert _{I_{-\infty }^{\alpha }} \). Moreover, for \(0<\alpha <1\), define the semi-norm \(\vert u\vert _{\alpha }=\Vert \vert w\vert ^{ \alpha }\hat{u}\Vert _{L^{2}} \) and the norm

$$\Vert u \Vert _{\alpha }= \bigl( { \vert u \vert _{\alpha } ^{2} + \Vert u \Vert _{L^{2}}^{2} } \bigr)^{\frac{1}{2}}. $$

Denote by \(H^{\alpha }\) the closure of space \(C_{0}^{\infty }( \mathbb{R},\mathbb{R}^{N})\) on the norm \(\Vert \cdot \Vert _{\alpha }\).

Noting that a function \(u\in L^{2}(\mathbb{R},\mathbb{R}^{N})\) belongs to \(I_{-\infty }^{\alpha }\) if and only if \(\vert w\vert ^{\alpha } \hat{u}\in L^{2}(\mathbb{R},\mathbb{R}^{N})\). Moreover, \(\vert u \vert _{I_{-\infty }^{\alpha }} =\Vert \vert w\vert ^{\alpha } \hat{u}\Vert _{L^{2}} \), the spaces \(H^{\alpha }\) and \(I_{- \infty }^{\alpha }\) are equivalent on the corresponding semi-norms as well as norms.

Denote by \(C(\mathbb{R},\mathbb{R}^{N})\) the space of continuous functions with norm \(\Vert u\Vert _{\infty }:= \sup_{t\in \mathbb{R}} \vert u(t)\vert \). We have the following lemma.

Lemma 2.1

([24])

If \(\alpha >1 /2\), then \(H^{\alpha }\subset C( \mathbb{R},\mathbb{R}^{N})\) and there exists a constant \(C_{\alpha }\) such that \(\Vert u\Vert _{\infty }\le C_{\alpha }\Vert u\Vert _{\alpha }\).

It follows from Lemma 2.1 that if \(\alpha \in (1/2, 1)\), then \(H^{\alpha }\subset L^{p}(\mathbb{R},\mathbb{R}^{N})\) for any \(p\in [2,\infty ]\), noting that \(\int _{\mathbb{R}} \vert u(t)\vert ^{p}\,dt \le \Vert u\Vert _{\infty }^{p-2} \Vert u \Vert _{L^{2}}^{2} \).

We need to introduce some variational framework to investigate the existence of solutions to problem (1.4).

First, we give a condition as follows.

\((V_{0})\) :

For each \(t\in \mathbb{R}, V(t)\) is a positive definite symmetric matrix. Moreover, there is a function \(v_{0} \in C(\mathbb{R},(0,\infty ))\) satisfying that \(v_{0} (t)\to \infty \) as \(\vert t\vert \to \infty \) as well as

$$\bigl(V(t)u,u\bigr)\ge v_{0} (t) \vert u \vert ^{2}, \quad \mbox{for any } t \in \mathbb{R},u\in \mathbb{R}^{N}. $$

We also need another fractional space. Let

$$X^{\alpha }= \biggl\{ {u\in H^{\alpha }: \int _{\mathbb{R}} { \bigl[ { \bigl\vert {{ }_{-\infty }D_{t}^{\alpha }u(t)} \bigr\vert ^{2}+ \bigl( {V(t)u(t),u(t)} \bigr)} \bigr]\,dt < \infty } } \biggr\} , $$

then by [24], \(X^{\alpha }\) is a reflexive and separable Hilbert space with the inner product

$$\langle {u,v} \rangle _{X^{\alpha }} = \int _{\mathbb{R}} { \bigl[ { \bigl( {{ }_{-\infty }D_{t}^{\alpha }u(t),{ }_{-\infty }D _{t}^{\alpha }v(t)} \bigr)+\bigl(V(t)u(t),u(t) \bigr)} \bigr]\,dt } $$

and a Banach space with respect to the norm \(\Vert u\Vert _{X^{\alpha }} =\sqrt{ \langle {u,v} \rangle _{X ^{\alpha }}} \).

The following embedding theorem is important to seek some minimum point of a function on the Nehari manifold.

Lemma 2.2

([24])

Under condition \((V_{0})\), \(X ^{\alpha }\) is continuously embedded in \(H^{\alpha }\).

Remark 2.1

By Lemmas 2.1–2.2, there exists a constant \(C_{\infty }>0\) such that \(\Vert u\Vert _{\infty }\le C _{\infty }\Vert u\Vert _{X^{\alpha }} \) for all \(u\in X ^{\alpha }\).

Lemma 2.3

([24])

Under condition \((V_{0})\), the embedding of \(X^{\alpha }\) in \(L^{p}(\mathbb{R},\mathbb{R}^{N})\) is continuous for \(p\in [2,\infty ]\) and is compact for \(p\in [2,\infty )\).

By Lemma 2.3, there exists \(C_{p} >0\) such that \(\Vert u\Vert _{L^{p}} \le C_{p} \Vert u\Vert _{X^{\alpha }} \), \(\forall u\in X^{\alpha }\) for \(p\in [2,\infty ]\).

The following hypothesis will be used in the sequel.

(\(H_{1}\)):

\(b\in C(\mathbb{R},(0,\infty ))\cap L^{\infty }\), \(h \in C(\mathbb{R},\mathbb{R}^{N})\cap L^{1}\), \(1/2<\alpha <1 \), and \(q>2\).

In the following, we always assume that \(1/2 <\alpha <1\) and \(q>2\).

Now, we are going to establish a variational framework to investigate the existence of solutions for (1.4) with the help of the Nehari manifold. To this end, we define the functional \(I_{\lambda ,\mu } :X ^{\alpha }\to \mathbb{R}\) by

$$ I_{\lambda ,\mu } (u)=\frac{1}{2} \Vert u \Vert _{X^{\alpha }}^{2} -\frac{\lambda }{2} \int _{\mathbb{R}} { \bigl\vert u(t) \bigr\vert ^{2}\,dt } - \frac{1}{q} \int _{\mathbb{R}} {b(t) \bigl\vert u(t) \bigr\vert ^{q} \,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u(t)\,dt }. $$
(2.1)

It is easy to verify that \(I_{\lambda ,u} \in C^{1}(X^{\alpha }, \mathbb{R})\) under conditions \((V_{0} )\) and \((H_{1} )\); moreover,

$$\begin{aligned} \bigl\langle {I}'_{\lambda ,\mu } (u), v \bigr\rangle =& \int _{\mathbb{R}} { \bigl[ { \bigl( {{ }_{-\infty }D_{t}^{\alpha }u(t),{ }_{-\infty }D _{t}^{\alpha }v(t)} \bigr)+ \bigl({V(t)u(t),v(t)} \bigr)} \bigr]\,dt }- \lambda \int _{\mathbb{R}} {u(t)\cdot v(t)\,dt } \\ &{}- \int _{\mathbb{R}} {b(t) \bigl\vert u(t) \bigr\vert ^{q-2}u(t)\cdot v(t)\,dt } - \mu \int _{\mathbb{R}} {h(t)\cdot v(t)\,dt } \end{aligned}$$
(2.2)

for \(u,v\in X^{\alpha }\).

Owing to the fact that \(q>2\), the energy functional \(I_{\lambda , \mu } \) is not bounded below on \(X^{\alpha }\). We turn to considering the functional on the Nehari manifold

$$N_{\lambda ,\mu } := \bigl\{ {u\in X^{\alpha }\backslash \{ 0 \} \vert \bigl\langle {I}'_{\lambda ,\mu } (u),u\bigr\rangle =0} \bigr\} . $$

Obviously, \(u\in N_{\lambda ,\mu } \) if and only if \(u\in X^{\alpha }\) satisfies

$$ \Vert u \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } - \int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } - \mu \int _{\mathbb{R}} {h(t)\cdot u(t)\,dt } =0. $$
(2.3)

In order to seek a minimum point for \(I_{\lambda ,\mu } \), the Nehari manifold \(N_{\lambda ,\mu } \) needs to be split into three parts:

$$\begin{aligned}& N_{\lambda ,\mu }^{+} := \bigl\{ {u\in N_{\lambda ,\mu } \vert \bigl\langle {J}'_{\lambda ,\mu } (u),u\bigr\rangle >0} \bigr\} , \\& N_{\lambda ,\mu }^{0} := \bigl\{ {u\in N_{\lambda ,\mu } \vert \bigl\langle {J}'_{\lambda ,\mu } (u),u\bigr\rangle =0} \bigr\} , \\& N_{\lambda ,\mu }^{-} := \bigl\{ {u\in N_{\lambda ,\mu } \vert \bigl\langle {J}'_{\lambda ,\mu } (u),u\bigr\rangle < 0} \bigr\} , \end{aligned}$$

where \(J_{\lambda ,\mu } (u)=\langle {I}'_{\lambda ,\mu } (u),u\rangle \). Precisely, we have

$$ \bigl\langle {J}'_{\lambda ,\mu } (u),u\bigr\rangle =2 \Vert u \Vert _{X^{\alpha }}^{2} -2\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } -q \int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u(t)\,dt } . $$
(2.4)

In view of Lemma 2.3, for \(p\in [2,\infty ]\), we write

$$ S_{p} =\inf \bigl\{ {C>0 : \Vert u \Vert _{L^{p}} \le C \Vert u \Vert _{X^{\alpha }} \text{ for all } u\in X^{\alpha }} \bigr\} . $$
(2.5)

Let \(\lambda _{0} =1/ {S_{2}^{2} }\), \(b_{\infty }=\sup_{t\in \mathbb{R}} b(t)\). For \(-\infty <\lambda <\lambda _{0} \), denote \(\mu _{\lambda }\) as

$$ \mu _{\lambda }= \bigl( {b_{\infty }S_{q}^{q} } \bigr)^{ \frac{1}{2-q}}\frac{q-2}{ \Vert h \Vert _{L^{1}} S_{\infty }} \biggl(\frac{1- \lambda S_{2}^{2}}{q-1} \biggr)^{\frac{q-1}{q-2}}>0. $$
(2.6)

We have the following result.

Lemma 2.4

Assume that hypotheses (\(V_{0} \)) and (\(H _{1} \)) hold. If \(-\infty <\lambda <\lambda _{0} \), then \(I_{\lambda , \mu } \) is coercive and bounded below on \(N_{\lambda ,\mu } \).

Proof

For any \(u\in N_{\lambda ,\mu } \), it follows from (2.3) that

$$\int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } = \Vert u \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u(t)\,dt } . $$

Therefore, by (2.1) and (2.5), we have

$$\begin{aligned} I_{\lambda ,u} (u) =& \biggl( {\frac{1}{2}-\frac{1}{q}} \biggr) \biggl( { \Vert u \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } } \biggr)- \mu \biggl( {1-\frac{1}{q}} \biggr) \int _{\mathbb{R}} {h(t)u(t)\,dt } \\ \ge & \biggl( {\frac{1}{2}-\frac{1}{q}} \biggr) \bigl( {1-\lambda S _{2}^{2} } \bigr) \Vert u \Vert _{X^{\alpha }}^{2} -\mu \biggl( {1-\frac{1}{q}} \biggr) \int _{\mathbb{R}} { \bigl\vert h(t) \bigr\vert \bigl\vert u(t) \bigr\vert \,dt } \\ \ge & \biggl( {\frac{1}{2}-\frac{1}{q}} \biggr) \bigl( {1-\lambda S _{2}^{2} } \bigr) \Vert u \Vert _{X^{\alpha }}^{2} -\mu \biggl( {1-\frac{1}{q}} \biggr) \Vert h \Vert _{L^{1}} \Vert u \Vert _{\infty } \\ \ge & \biggl( {\frac{1}{2}-\frac{1}{q}} \biggr) \bigl( {1-\lambda S _{2}^{2} } \bigr) \Vert u \Vert _{X^{\alpha }}^{2} -\mu \biggl( {1-\frac{1}{q}} \biggr) \Vert h \Vert _{L^{1}} S_{\infty } \Vert u \Vert _{X^{\alpha }} . \end{aligned}$$

Hence, owing to the fact that \(\lambda <\lambda _{0} \) and \(q>2\), \(I_{\lambda ,\mu } \) is coercive and therefore bounded below on \(N_{\lambda ,\mu } \). □

Lemma 2.5

Assume that hypotheses \((V_{0})\) and \((H_{1})\) hold. If \(u_{0} \in N_{\lambda ,\mu } \) is a local minimizer of \(I_{\lambda ,\mu } \) on \(N_{\lambda ,\mu } \) with \(u_{0} \notin N_{\lambda , \mu }^{0} \), then \(u_{0} \) is a nontrivial solution to problem (1.4).

Proof

Because \(u_{0} \in N_{\lambda ,\mu } \) is a local minimizer of \(I_{\lambda ,\mu } \) on \(N_{\lambda ,\mu } \), by Theorem 4.1.1 in [39], there exists a number η such that

$$ {I}'_{\lambda ,\mu } (u_{0} )=\eta {J}'_{\lambda ,\mu } (u_{0} ), $$
(2.7)

where \(J_{\lambda ,\mu } (u )={I}'_{\lambda ,\mu } (u )u\). Thus, from \(u_{0} \in N_{\lambda ,\mu } \) it follows that

$$0= {{I}'_{\lambda ,\mu } (u_{0} )} u_{0} =\eta {{J}'_{\lambda ,\mu } (u _{0} )} u_{0} . $$

In view of \(u_{0} \notin N_{\lambda ,\mu }^{0} \), we obtain \({J}'_{\lambda ,\mu } (u_{0} )u_{0} \ne 0\), and therefore \(\eta =0\). Thus, \({I}'_{\lambda ,\mu } (u_{0} )=0\) follows from (2.7). It means that \(u_{0} \) is a nontrivial solution to problem (1.4). □

Lemma 2.6

Assume that hypotheses \((V_{0})\) and \((H_{1})\) hold. If \(h=0\), then \(N_{\lambda ,\mu }^{0} =\emptyset \); If \(h\ne 0\), then \(N_{\lambda ,\mu }^{0} =\emptyset \) for \(-\infty < \lambda <\lambda _{0} \), \(0<\mu <\mu _{\lambda }\), where \(\mu _{\lambda }\) as in (2.6).

Proof

In fact, if not, then there exists \(u_{0} \in N_{ \lambda ,\mu }^{0} \) with \(u_{0} \ne 0\). Then, by (2.3)–(2.4), we have

$$\begin{aligned}& \Vert u_{0} \Vert _{X^{\alpha }}^{2} - \lambda \int _{ \mathbb{R}} { \vert u_{0} \vert ^{2} \,dt } - \int _{\mathbb{R}} {b(t) \vert u _{0} \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u_{0} (t)\,dt =0} , \end{aligned}$$
(2.8)
$$\begin{aligned}& 2 \Vert u_{0} \Vert _{X^{\alpha }}^{2} -2\lambda \int _{\mathbb{R}} { \vert u_{0} \vert ^{2} \,dt } -q \int _{\mathbb{R}} {b(t) \vert u_{0} \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u_{0} (t)\,dt =0} . \end{aligned}$$
(2.9)
  1. (i)

    If \(h=0\), then by (2.8)–(2.9), we obtain \(\int _{\mathbb{R}} {b(t)\vert u_{0}\vert ^{q}\,dt } =0\), which is a contradiction noting that \(u_{0} \ne 0\) and \(b\in C(\mathbb{R},(0, \infty ))\).

  2. (ii)

    If \(h\ne 0\), then by subtracting (2.8) from (2.9), we have

    $$ \Vert u_{0} \Vert _{X^{\alpha }}^{2} - \lambda \int _{ \mathbb{R}} { \vert u_{0} \vert ^{2} \,dt } -(q-1) \int _{\mathbb{R}} {b(t) \vert u_{0} \vert ^{q}\,dt } =0. $$
    (2.10)

Multiplying (2.8) by q and subtracting (2.9), we get

$$ (q-2) \biggl( { \Vert u_{0} \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u_{0} \vert ^{2} \,dt } } \biggr)-\mu (q-1) \int _{\mathbb{R}} {h(t)\cdot u_{0}(t)\,dt =0}. $$
(2.11)

Thus, by (2.10)–(2.11) together with (2.5), we have respectively

$$ \bigl( {1-\lambda S_{2}^{2} } \bigr) \Vert u_{0} \Vert _{X^{\alpha }}^{2} \le (q-1)b_{\infty } \int _{\mathbb{R}} { \vert u_{0} \vert ^{q} \,dt } \le (q-1)b_{\infty }S_{q}^{q} \Vert u_{0} \Vert _{X^{\alpha }}^{q} $$
(2.12)

and

$$ (q-2) \bigl( {1-\lambda S_{2}^{2} } \bigr) \Vert u_{0} \Vert _{X^{\alpha }}^{2} \le \mu (q-1) \Vert h \Vert _{L^{1}} \Vert u_{0} \Vert _{\infty } \le \mu (q-1) \Vert h \Vert _{L^{1}} S_{\infty } \Vert u_{0} \Vert _{X^{ \alpha }}. $$
(2.13)

Because \(u_{0} \ne 0\) and \(\lambda <\lambda _{0} \), from (2.12)–(2.13) it follows that \(\mu \ge \mu _{\lambda }\), which contradicts the hypothesis \(0<\mu <\mu _{\lambda }\). Thus, \(N_{\lambda ,\mu }^{0} = \emptyset \). □

3 Main result

In this section, we establish some existence results on solutions to problem (1.4). To this end, we first establish several lemmas.

Lemma 3.1

Assume that hypotheses \((V_{0})\) and \((H_{1})\) hold. If \(h=0\), then \(N_{\lambda ,\mu }^{-} \ne \emptyset \) for \(-\infty <\lambda <\lambda _{0} \). If \(h\ne 0\), then \(N_{\lambda , \mu }^{+} \ne \emptyset \) and \(N_{\lambda ,\mu }^{-} \ne \emptyset \) for \(-\infty <\lambda <\lambda _{0} \), \(0<\mu <\mu _{\lambda }\), where \(\mu _{\lambda }\) as in (2.6).

Proof

(i) If \(h=0\), then for any \(u_{0} \in X^{\alpha } \backslash \{ 0 \}\),

$$I_{\lambda ,\mu }(u_{0} )=\frac{1}{2} \Vert u_{0} \Vert _{X^{\alpha }}^{2} -\frac{\lambda }{2} \int _{\mathbb{R}} { \vert u _{0} \vert ^{2} \,dt } -\frac{1}{q} \int _{\mathbb{R}} {b(t) \vert u_{0} \vert ^{q}\,dt } . $$

Let \(\phi _{u_{0}} (s)=I_{\lambda ,\mu }(su_{0} )=\frac{s^{2}}{2} \Vert u_{0} \Vert _{X^{\alpha }}^{2} - \frac{\lambda s^{2}}{2}\int _{\mathbb{R}} {\vert u_{0} \vert ^{2}\,dt } -\frac{s ^{q}}{q}\int _{\mathbb{R}} {b(t)\vert u_{0} \vert ^{q}\,dt } \), \(s\in [0, \infty )\). Then

$$ {\phi }'_{u_{0}} (s)=s \biggl( { \Vert u_{0} \Vert _{X^{ \alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u_{0} \vert ^{2} \,dt } } \biggr)-s ^{q-1} \int _{\mathbb{R}} {b(t) \vert u_{0} \vert ^{q}\,dt } ,\quad s\in [0,\infty ). $$
(3.1)

Write \(\psi _{u_{0}} (s)={\phi }'_{u_{0}} (s)\), \(s\in [0,\infty )\). Due to that \(u_{0} \ne 0, b(t)>0\), \(t\in \mathbb{R}\), \(q>2\), \(\lambda <\lambda _{0}\) and observing that

$$\Vert u_{0} \Vert _{X^{\alpha }}^{2} -\lambda \int _{ \mathbb{R}} { \vert u \vert ^{2}\,dt } \ge \bigl( {1-\lambda S_{2}^{2} } \bigr) \Vert u_{0} \Vert _{X^{\alpha }}^{2} >0, $$

we have that \(\psi _{u_{0}} (0)=0\), \(\psi _{u_{0}} (s)>0\) for small \(s>0\) and \(\psi _{u_{0}} (s)\to -\infty \) as \(s\to \infty \). Thus there exists unique \(\bar{s}>0\) such that

$$\begin{aligned}& {\psi }'_{u_{0}} (\bar{s})=0,\qquad \psi _{u_{0}} ( \bar{s})=\max_{s\ge 0} \psi _{u_{0}} (s)>0,\qquad {\psi }'_{u_{0}} (s)>0 \quad \mbox{for } s\in (0,\bar{s}); \\& {\psi }'_{u_{0}}(s)< 0 \quad \mbox{for } s\in (\bar{s},\infty ). \end{aligned}$$

In addition, it follows from (3.1) that

$$\bar{s}= \biggl( {\frac{ \Vert u_{0} \Vert _{X^{\alpha }} ^{2} -\lambda \int _{\mathbb{R}} { \vert u_{0} \vert ^{2}\,dt } }{(q-1) \int _{\mathbb{R}} {b(t) \vert u_{0} \vert ^{q}\,dt } }} \biggr)^{ \frac{1}{q-2}}. $$

On the other hand, since \(\psi _{u_{0}} (\bar{s})>0\), \({\psi }'_{u_{0}} (s)<0\) for \(s\in (\bar{s},\infty )\) and \(\psi _{u_{0}} (s)\to -\infty \), there exists unique \(s^{-}>\bar{s}\) such that \(\psi _{u} (s^{-})=0\). Of course, \({\psi }'_{u} (s^{-})<0\). Thus, from

$$\bigl\langle J'_{\lambda ,\mu } (su_{0} ),su_{0} \bigr\rangle = \bigl(\psi _{u_{0} } (s)s \bigr)'s = \psi '_{u_{0} } (s)s^{2} + \psi _{u_{0} } (s)s,\quad s\in [0, \infty ), $$

we immediately have

$$\bigl\langle J'_{\lambda ,\mu } \bigl(s^{-}u_{0} \bigr),s^{-}u_{0} \bigr\rangle = \psi '_{u _{0} } \bigl(s^{-}\bigr){\bigl(s^{-}\bigr)}^{2} + \psi _{u_{0} }\bigl(s^{-}\bigr)s^{-}=\psi '_{u_{0} } \bigl(s^{-}\bigr){\bigl(s^{-} \bigr)}^{2} < 0. $$

Hence,

$$ s^{-}u_{0} \in N_{\lambda ,\mu }^{-}, \quad {\phi }'_{u_{0}} \bigl(s^{-}\bigr)=0,\qquad { \phi }''_{u_{0}}\bigl(s^{-}\bigr)< 0 \quad \mbox{and} \quad \phi _{u_{0}} \bigl(s^{-}\bigr)= \sup _{s\ge 0} I_{\lambda ,\mu } (su_{0} ) $$
(3.2)

noting that \(\phi ''_{u_{0}}= \psi '_{u_{0}}\). Thus, \(N_{\lambda , \mu }^{-} \ne \emptyset \).

(ii) If \(h\ne 0\), then, without loss of generality, we can say that \(h(t)=(h_{1}, h_{2} ,\ldots ,h_{N} )\) satisfies \(h_{1} (t)>0 \) for all \(t\in (t_{0}-r, t_{0}+r)\) for some \(t_{0}\in \mathbb{R}\) and some \(r>0\). Take \(\bar{u}=(u_{1} ,u_{2} ,\ldots ,u_{N} )\) with \(u_{k} =0\) for \(2\le k\le N\), and \(u_{1} (t)\) is given by

$$u_{1} (t)= \textstyle\begin{cases} e^{\frac{(t-t_{0})^{2}}{(t-t_{0})^{2}-r^{2}}}& \text{for } \vert t-t _{0} \vert < r, \\ 0 & \text{for } \vert t-t_{0} \vert \ge r. \end{cases} $$

Then \(\bar{u}\in C_{0}^{\infty }(\mathbb{R},\mathbb{R}^{N})\), and therefore, \(\bar{u}\in X^{\alpha }\) with \(\bar{u}\ne 0\). Furthermore, \(\int _{\mathbb{R}} {h(t)\cdot \bar{u}(t)\,dt } >0\).

In the following, we make an argument similar to that in (i) above.

Let

$$\begin{aligned} \phi _{\bar{u}} (s) =&I_{\lambda ,\mu }(s\bar{u}) \\ =&\frac{s^{2}}{2} \Vert \bar{u} \Vert _{X^{\alpha }}^{2} - \frac{ \lambda s^{2}}{2} \int _{\mathbb{R}} { \vert \bar{u} \vert ^{2}\,dt } - \frac{s ^{q}}{q} \int _{\mathbb{R}} {b(t) \vert \bar{u} \vert ^{q}\,dt } \\ & {}-s\mu \int _{\mathbb{R}} {h(t)\cdot \bar{u}(t)\,dt =0} ,\quad s\in [0, \infty ). \end{aligned}$$
(3.3)

Then

$$\begin{aligned}& {\phi }'_{\bar{u}} (s)=s \biggl( { \Vert \bar{u} \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert \bar{u} \vert ^{2}\,dt } } \biggr)-s^{q-1} \int _{\mathbb{R}} {b(t) \vert \bar{u} \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot \bar{u}(t)\,dt } , \\& \quad s\in [0,\infty ). \end{aligned}$$
(3.4)

Let \(\psi _{\bar{u}} (s)=s ( {\Vert \bar{u}\Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} {\vert \bar{u}\vert ^{2}\,dt } } )-s^{q-1}\int _{\mathbb{R}} {b(t)\vert \bar{u}\vert ^{q}\,dt } \). Then

$$ {\psi }'_{\bar{u}} (s)= \Vert \bar{u} \Vert _{X^{\alpha }} ^{2} -\lambda \int _{\mathbb{R}} { \vert \bar{u} \vert ^{2}\,dt } -(q-1)s ^{q-2} \int _{\mathbb{R}} {b(t) \vert \bar{u} \vert ^{q}\,dt } ,\quad s\in [0, \infty ). $$
(3.5)

Obviously, \(\psi _{\bar{u}} (0)=0\), \(\psi _{\bar{u}} (s)>0\) as \(s>0\) small enough, and \(\psi _{\bar{u}} (s)\to -\infty \) as \(s\to \infty \). Hence, there exists unique \(\bar{s}>0\) such that

$$\begin{aligned}& \psi _{\bar{u}} (\bar{s})=\max_{s\ge 0} \psi _{\bar{u}} (s)>0,\qquad {\psi }'_{\bar{u}} (\bar{s})=0, \end{aligned}$$
(3.6)
$$\begin{aligned}& {\psi }'_{\bar{u}} (s)>0 \quad \mbox{for } s\in (0,\bar{s}); \qquad {\psi }'_{ \bar{u}}(s)< 0\quad \mbox{for } s \in (\bar{s},\infty ). \end{aligned}$$
(3.7)

Moreover, by calculation, we get

$$\begin{aligned} \psi _{\bar{u}} (\bar{s}) =&\frac{q-2}{q-1} \biggl( { \frac{ ( { \Vert \bar{u} \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert \bar{u} \vert ^{2}\,dt } } )^{q-1}}{(q-1) \int _{\mathbb{R}} {b(t) \vert \bar{u} \vert ^{q}\,dt } }} \biggr)^{ \frac{1}{q-2}} \\ \ge & \frac{q-2}{q-1} \biggl( {\frac{ ( {(1-\lambda S_{2}^{2} ) \Vert \bar{u} \Vert _{X^{\alpha }}^{2} } )^{q-1}}{(q-1)b _{\infty }S_{q}^{q} \Vert \bar{u} \Vert _{X^{\alpha }}^{q} }} \biggr)^{\frac{1}{q-2}} \\ =&\frac{q-2}{q-1} \biggl( {\frac{ ( {1-\lambda S_{2}^{2} } ) ^{q-1}}{(q-1)b_{\infty }S_{q}^{q} }} \biggr)^{\frac{1}{q-2}} \Vert \bar{u} \Vert _{X^{\alpha }}. \end{aligned}$$

Because \(0<\mu \int _{\mathbb{R}} {h(t)\cdot \bar{u}(t)\,dt } \le \mu \Vert h\Vert _{L^{1}} \Vert \bar{u}\Vert _{ \infty }\le \mu \Vert h\Vert _{L^{1}} S_{\infty }\Vert \bar{u}\Vert _{X^{\alpha }} \), it is easy to verify that \(0<\mu \int _{\mathbb{R}} {h(t)\cdot \bar{u}(t)\,dt } <\psi _{\bar{u}} ( \bar{s})\) as long as \(0<\mu <\mu _{\lambda }\), where \(\mu _{\lambda }\) is given by (2.6), and therefore, \(\phi '_{\bar{u}}(\bar{s})>0\) by (3.4).

Now, in terms of (3.4) together with the fact that \(\phi '_{ \bar{u}}(0)<0\), \(\phi '_{\bar{u}}(\bar{s})>0\), it follows from (3.6)–(3.7) that there exist unique \(0< s^{+}<\bar{s}\) and unique \(s_{2}^{-} >\bar{s}\) such that

$$ \begin{aligned} &{\phi }'_{\bar{u}} \bigl(s_{1}^{+} \bigr)={\phi }'_{\bar{u}} \bigl(s_{2}^{-} \bigr)=0,\qquad {\phi }''_{\bar{u}} \bigl(s_{1}^{+} \bigr)>0,\qquad {\phi }''_{\bar{u}} \bigl(s_{2}^{-} \bigr)< 0;\qquad \phi ''_{\bar{u}}(s)>0, \quad s\in (0,\bar{s}); \\ & \phi ''_{\bar{u}}(s)< 0,\quad s \in ( \bar{s}, \infty ) \end{aligned} $$
(3.8)

observing that (3.7) together with the fact that \(\phi '_{\bar{u}}(s)= \psi _{\bar{u}}(s)\), \(\phi ''_{\bar{u}}(s)=\psi '_{\bar{u}}(s)\), \(s\in [0, \infty )\).

Hence, from

$$\bigl\langle J'_{\lambda ,\mu } (s\bar{u}),s\bar{u}\bigr\rangle = \bigl( {\phi '_{ \bar{u}} (s)s} \bigr)^{\prime }s = \phi ''_{\bar{u}} (s)s^{2} + \phi '_{\bar{u}} (s)s, $$

together with (3.8), we have respectively

$$\bigl\langle J'_{\lambda ,\mu } \bigl(s_{1}^{+} \bar{u}\bigr),s_{1}^{+} \bar{u} \bigr\rangle = \phi ''_{\bar{u}} \bigl(s_{1}^{+} \bigr) \bigl(s_{1}^{+} \bigr)^{2} + \phi '_{ \bar{u}} \bigl(s_{1}^{+} \bigr)s_{1}^{+} =\phi ''_{\bar{u}} \bigl(s_{1}^{+} \bigr) \bigl(s_{1} ^{+} \bigr)^{2} > 0 $$

and

$$\bigl\langle J'_{\lambda ,\mu } \bigl(s_{2}^{-} \bar{u}\bigr),s_{2}^{-} \bar{u} \bigr\rangle = \phi ''_{\bar{u}} \bigl(s_{2}^{-} \bigr) \bigl(s_{2}^{-} \bigr)^{2} + \phi '_{ \bar{u}} \bigl(s_{2}^{-} \bigr)s_{2}^{-} =\phi ''_{\bar{u}} \bigl(s_{2}^{-} \bigr) \bigl(s_{2} ^{-} \bigr)^{2} < 0, $$

which means that

$$ s_{1}^{+} \bar{u}\in N_{\lambda ,\mu }^{+} , s_{2}^{-} \bar{u}\in N _{\lambda ,\mu }^{-} , \quad \phi _{\bar{u}} \bigl(s_{1}^{+} \bigr)=\min _{0\le s\le \bar{s}} \phi _{\bar{u}} (s),\qquad \phi _{\bar{u}} \bigl(s_{2} ^{-} \bigr)=\max_{s\ge 0} \phi _{u} (s) $$
(3.9)

noting that (3.8). □

Lemma 3.2

Assume that hypotheses \((V_{0} )\) and \((H_{1} )\) hold. If \(h\ne 0\), then \(-\infty < C_{\lambda ,\mu }^{+} := \inf_{u\in N_{\lambda ,\mu }^{+} } I_{\lambda , \mu } <0\) for \(-\infty <\lambda <\lambda _{0} \), \(0<\mu <\mu _{\lambda }\), where \(\mu _{\lambda }\) as in (2.6).

Proof

First, by Lemma 3.1, \(N_{\lambda ,\mu }^{+} \ne \emptyset \). For any \(u\in N_{\lambda ,\mu }^{+} \), we have

$$ 0={I}'_{\lambda ,\mu } (u)u= \Vert u \Vert _{X^{\alpha }} ^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } - \int _{ \mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t) \cdot u(t)\,dt } $$
(3.10)

and

$$\begin{aligned} 0 < & \bigl( {{I}'_{\lambda ,\mu } (u)u} \bigr)'u \\ =&2 \Vert u \Vert _{X^{\alpha }}^{2} -2\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } -q \int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u(t)\,dt }. \end{aligned}$$
(3.11)

By (3.10)–(3.11), we have

$$ (q-1) \int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } < \Vert u \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } . $$
(3.12)

On the other hand, by (2.1) together with (3.10) and (3.12), we have

$$\begin{aligned} I_{\lambda ,\mu } (u) =&-\frac{1}{2} \biggl( { \Vert u \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } } \biggr)+ \frac{q-1}{q} \int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } \\ < &-\frac{1}{2} \biggl( { \Vert u \Vert _{X^{\alpha }}^{2} - \lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } } \biggr)+ \frac{1}{q} \biggl( { \Vert u \Vert _{X^{\alpha }}^{2} - \lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } } \biggr) \\ =& \biggl( {\frac{1}{q}-\frac{1}{2}} \biggr) \biggl( { \Vert u \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } } \biggr) < 0 \end{aligned}$$

noting that \(q>2\) and \(u\ne 0\). Hence, \(C_{\lambda ,\mu }^{+} = \inf_{u\in N_{\lambda ,\mu }^{+} } I_{\lambda , \mu } (u)<0\).

On the other hand, in terms of Lemma 2.4 and the fact that \(I_{\lambda ,\mu }\) is bounded below, we obtain that \(C_{\lambda ,\mu }^{+} >- \infty \). □

Lemma 3.3

Assume that hypotheses \((V_{0} )\) and \((H_{1} )\) hold. In addition, if

  1. (i)

    \(h=0\), then \(C_{\lambda ,\mu }^{-} :=\inf_{u\in N_{\lambda ,\mu }^{-} } I_{\lambda ,\mu } (u)>0\) for \(-\infty <\lambda <\lambda _{0} \);

  2. (ii)

    \(h\ne 0\), then \(C_{\lambda ,\mu }^{-} := \inf_{u\in N_{\lambda ,\mu }^{-} } I_{\lambda , \mu } (u)>0\) for \(-\infty <\lambda <\lambda _{0} \), \(0<\mu <\mu _{ \lambda }/2 \), where \(\mu _{\lambda }\) as in (2.6).

Proof

First, by Lemma 3.1, whether case (i) or case (ii), we always have \(N_{\lambda ,\mu }^{-} \ne \emptyset \). For any \(u\in N_{\lambda ,\mu }^{-} \), by \({I}'_{\lambda ,\mu } (u)u=0\), we have

$$ \mu \int _{\mathbb{R}} {h(t)\cdot u(t)\,dt } = \Vert u \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } - \int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } . $$
(3.13)

Thus, by \(( {({I}'_{\lambda ,\mu } u)u})'u<0\), it follows from (3.13) that

$$\begin{aligned} 0 >& 2 \Vert u \Vert _{X^{\alpha }}^{2} -2\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } -q \int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u(t)\,dt } \\ =& \Vert u \Vert _{X^{\alpha }}^{2} -\lambda \int _{ \mathbb{R}} { \vert u \vert ^{2}\,dt } -(q-1) \int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } , \end{aligned}$$

and therefore,

$$\bigl( {1-\lambda S_{2}^{2} } \bigr) \Vert u \Vert _{X^{\alpha }}^{2} \le \Vert u \Vert _{X^{\alpha }} ^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } < (q-1) \int _{\mathbb{R}} {b(t) \vert u \vert ^{q}\,dt } \le (q-1)b_{\infty }S_{q} ^{q} \Vert u \Vert _{X^{\alpha }}^{q} . $$

Thus,

$$ \Vert u \Vert _{X^{\alpha }} > \biggl( {\frac{ ( {1- \lambda S_{2}^{2} } )}{(q-1)b_{\infty }S_{q}^{q} }} \biggr) ^{\frac{1}{q-2}}:=d_{0} >0. $$
(3.14)

On the other hand, by (2.1) together with (3.13) as well as (2.5), we have

$$\begin{aligned} I_{\lambda ,\mu } (u) =& \biggl( {\frac{1}{2}- \frac{1}{q}} \biggr) \biggl( { \Vert u \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u \vert ^{2}\,dt } } \biggr)- \mu \biggl( {1-\frac{1}{q}} \biggr) \int _{\mathbb{R}} {h(t)\cdot u(t)\,dt } \\ \ge & \biggl( {\frac{1}{2}-\frac{1}{q}} \biggr) \bigl( {1-\lambda S _{2}^{2} } \bigr) \Vert u \Vert _{X^{\alpha }}^{2} -\mu \biggl( {1-\frac{1}{q}} \biggr)S_{\infty } \Vert h \Vert _{L^{1}} \Vert u \Vert _{X^{\alpha }} \\ =& \biggl[ { \biggl( {\frac{1}{2}-\frac{1}{q}} \biggr) \bigl( {1- \lambda S_{2}^{2} } \bigr) \Vert u \Vert _{X^{\alpha }} - \mu \biggl( {1-\frac{1}{q}} \biggr)S_{\infty } \Vert h \Vert _{L^{1}} } \biggr] \Vert u \Vert _{X^{\alpha }}. \end{aligned}$$
(3.15)

(i) If \(h=0\), then by (3.14)–(3.15), we obtain

$$I_{\lambda ,\mu } (u)\ge \biggl( {\frac{1}{2}-\frac{1}{q}} \biggr) \bigl( {1-\lambda S_{2}^{2} } \bigr)d_{0} ^{2}>0, $$

and therefore,

$$\inf_{u\in N_{\lambda ,\mu }^{-} } I_{\lambda , \mu } (u)\ge \biggl( { \frac{1}{2}-\frac{1}{q}} \biggr) \bigl( {1- \lambda S_{2}^{2} } \bigr)d_{0} ^{2}>0. $$

(ii) If \(h\ne 0\), then \(\Vert h\Vert _{L^{1}} >0\). For \(0<\mu <\mu _{\lambda }/2 \), by calculation, we can obtain

$$\begin{aligned}& \biggl( {\frac{1}{2}-\frac{1}{q}} \biggr) \bigl( {1-\lambda S_{2} ^{2} } \bigr) \Vert u \Vert _{X^{\alpha }} -\mu \biggl( {1-\frac{1}{q}} \biggr)S_{\infty } \Vert h \Vert _{L^{1}} \\& \quad \ge \biggl( {\frac{1}{2}-\frac{1}{q}} \biggr) \bigl( {1- \lambda S_{2}^{2} } \bigr)d_{0} -\mu \biggl( {1- \frac{1}{q}} \biggr)S_{ \infty } \Vert h \Vert _{L^{1}} >0, \end{aligned}$$

and therefore,

$$I_{\lambda ,\mu } (u)\ge \biggl[ { \biggl( {\frac{1}{2}- \frac{1}{q}} \biggr) \bigl( {1-\lambda S_{2}^{2} } \bigr)d_{0} -\mu \biggl( {1- \frac{1}{q}} \biggr)S_{\infty } \Vert h \Vert _{L^{1}} } \biggr]d _{0} :=d_{1} >0. $$

Consequently, \(\inf_{u\in N_{\lambda ,\mu }^{-} } I _{\lambda ,\mu } (u)\ge d_{1} >0\). □

Lemma 3.4

Assume that \((V_{0} )\), \((H_{1} )\) hold. If \(h\ne 0\), then for \(-\infty <\lambda <\lambda _{0}\), \(0< \mu <\mu _{\lambda }\) (where \(\mu _{\lambda }\) as in (2.6)), \(I_{\lambda ,\mu } \) achieves its minimum at some \(\bar{u} \in N_{\lambda ,\mu }^{+} \) on \(N_{\lambda ,\mu }^{+} \). That is,

$$I_{\lambda ,\mu } (\bar{u})=\inf_{u\in N_{\lambda ,\mu }^{+} } I_{\lambda ,\mu } (u):=C_{ \lambda ,\mu }^{+} . $$

Proof

First, under conditions \((V_{0} )\), \((H_{1} )\) and \(h\ne 0\), for \(\lambda <\lambda _{0}\), \(0<\mu <\mu _{\lambda }\), by Lemmas 3.1–3.2, we know that \(N_{\lambda ,\mu }^{+} \ne \emptyset \), and \(-\infty < C_{\lambda ,\mu }^{+} <0\). Let \(\{ {u_{n} } \} \subset N_{\lambda ,\mu }^{+} \) be a minimizing sequence for \(I_{\lambda ,\mu } \) on \(N_{\lambda ,\mu }^{+} \). That is, \(\lim_{n\to \infty } I_{\lambda ,\mu } (u_{n} )=C_{ \lambda ,\mu }^{+} \). Again, it follows from Lemma 2.4 that \(\{ {u_{n} } \}\) is bounded in \(X^{\alpha }\). Thus, by the reflexivity of \(X^{\alpha }\), there exists a subsequence, still denoted by \(\{ {u_{n} } \}\), such that \(u_{n} \rightharpoonup \bar{u}\) in \(X^{\alpha }\). Thus, by Lemma 2.3, we have

$$ u_{n} \to \bar{u} \quad \mbox{in } L^{q}(\mathbb{R}),p\in [2,\infty ),\qquad u_{n} (t)\to \bar{u}(t), \quad \mbox{a.e. } t\in \mathbb{R}. $$
(3.16)

Again, because \(\{ {u_{n} } \}\) is bounded in \(X^{\alpha }\), there is a constant \(M>0\) such that \(\Vert u_{n} \Vert _{X^{\alpha }} \le M\), \(n\ge 1\), and therefore, it follows from (2.5) that

$$\bigl\vert h(t)\cdot u_{n} (t) \bigr\vert \le \bigl\vert h(t) \bigr\vert \Vert u _{n} \Vert _{\infty }\le \bigl\vert h(t) \bigr\vert S_{\infty } \Vert u_{n} \Vert _{X ^{\alpha }} \le \bigl\vert h(t) \bigr\vert S_{\infty }M \in L^{1}(\mathbb{R}). $$

Thus, by applying the dominated convergence theorem, we have

$$ \lim_{n\to \infty } \int _{\mathbb{R}} {h(t)\cdot u _{n} (t)\,dt } = \int _{\mathbb{R}} {h(t)\cdot \bar{u}(t)\,dt }. $$
(3.17)

Similarly, we also have

$$ \lim_{n \to \infty } \int _{\mathbb{R}} {b(t) \bigl\vert u_{n} (t) \bigr\vert ^{q} \,dt } = \int _{\mathbb{R}} {b(t) \bigl\vert \bar{u}(t) \bigr\vert ^{q} \,dt }. $$
(3.18)

Now, we show that \(\lim_{n\to \infty } \Vert u _{n} \Vert _{X^{\alpha }} =\Vert \bar{u}\Vert _{X^{\alpha }} \).

In fact, if not, then by the weak lower semi-continuity on norm, we have

$$ \Vert \bar{u} \Vert _{X^{\alpha }} < \lim _{\overline{n\to \infty } } \Vert u_{n} \Vert _{X^{\alpha }. } $$
(3.19)

We immediately have

$$ I_{\lambda ,\mu } (\bar{u})< \lim_{n\to \infty } I_{\lambda ,\mu } (u_{n} ) $$
(3.20)

by (3.17)–(3.19).

On the other hand, by \(u_{n} \in N_{\lambda ,\mu }^{+} \), we have

$$\begin{aligned} 0&={I}'_{\lambda ,\mu } (u_{n} )u_{n} = \Vert u_{n} \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u_{n} \vert ^{2} \,dt } - \int _{\mathbb{R}} {b(t) \vert u_{n} \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u_{n} (t)\,dt } , \end{aligned}$$
(3.21)
$$\begin{aligned} 0&< \bigl( {{I}'_{\lambda ,\mu } (u_{n} )u_{n} } \bigr)^{\prime }u _{n} \\ &=2 \Vert u_{n} \Vert _{X^{\alpha }}^{2} -2\lambda \int _{\mathbb{R}} { \vert u_{n} \vert ^{2} \,dt } -q \int _{\mathbb{R}} {b(t) \vert u_{n} \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u_{n} (t)\,dt }. \end{aligned}$$
(3.22)

By (3.21)–(3.22), we have

$$\begin{aligned} (q-1)\mu \int _{\mathbb{R}} {h(t)\cdot u_{n} (t)\,dt } >&(q-2) \biggl( { \Vert u_{n} \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u_{n} \vert ^{2} \,dt } } \biggr) \\ \ge & (q-2) \bigl( {1-\lambda S_{2}^{2} } \bigr) \Vert u_{n} \Vert _{X^{\alpha }}^{2}>0. \end{aligned}$$
(3.23)

Passing to the limit as \(n\to \infty \) on the above inequality, by (3.17) and (3.19), we get

$$(q-1)\mu \int _{\mathbb{R}} {h(t)\cdot \bar{u}(t)\,dt } \ge (q-2) \bigl( {1- \lambda S_{2}^{2} } \bigr)\lim_{\overline{n\to \infty } } \Vert u_{n} \Vert _{X^{\alpha }}^{2} >(q-2) \bigl( {1-\lambda S_{2}^{2} } \bigr) \Vert \bar{u} \Vert _{X^{\alpha }}^{2} . $$

Hence, \(\int _{\mathbb{R}} {h(t)\cdot \bar{u}(t)\,dt } >0\), and therefore, by proof (ii) in Lemma 3.1 and with a derivation similar to (3.8) and (3.9), for \(\phi _{\bar{u}}(s) = I_{\lambda ,\mu } (s \bar{u})\), \(s\in [0,\infty )\), we know that there exist unique \(0< s_{1}^{+} <\bar{s}<s_{2}^{-} \) such that

$$ \begin{aligned} &\phi ' _{\bar{u}} \bigl(s_{1}^{+} \bigr)=\phi ' _{\bar{u}} \bigl(s_{2}^{-} \bigr)=0,\quad s _{1}^{+} \bar{u}\in N_{\lambda ,\mu }^{+} ,s_{2}^{-} \bar{u} \in N_{ \lambda ,\mu }^{-}, \\ & \phi _{\bar{u}} \bigl(s_{1}^{+} \bigr)=\inf _{0\le s\le \bar{s}} \phi _{\bar{u}} (s),\qquad \phi _{\bar{u}} \bigl(s_{2} ^{-} \bigr)=\sup_{s\ge 0} \phi _{\bar{u}} (s). \end{aligned} $$
(3.24)

Similarly, let \(\phi _{u_{n} } (s)={I}_{\lambda ,\mu } (su_{n} )\). Then, by the same reason as (3.8) and (3.9) together with (3.23), from \(\int _{\mathbb{R}} {h(t) \cdot u_{n} (t)\,dt } > 0\), we know that there exist unique \(s_{n}^{+} <\bar{s}_{n} <s_{n}^{-} \) such that

$$\phi _{u_{n}}'\bigl(s_{n} ^{+}\bigr)= \phi _{u_{n}}'\bigl(s_{n} ^{-}\bigr)=0, \qquad \phi _{u_{n}}''(s)>0,\quad s\in (0, \bar{s}_{n});\qquad \phi _{u_{n}}''(s)< 0, \quad s \in ({\bar{s}}_{n}, \infty ) $$

as well as

$$s_{n}^{+} u_{n} \in N_{\lambda ,\mu }^{+} ,\qquad s_{n}^{-} u_{n} \in N_{ \lambda ,\mu }^{-} . $$

Now, the fact that \(u_{n} \in N_{\lambda ,\mu }^{+}\) and the uniqueness for \(s_{n}^{+} \) imply that \(s_{n}^{+} =1\). Thus, \(\bar{s}_{n} >1\).

Now, the fact that \({\phi }''_{u_{n} } (s)>0\), \(s\in (0,\bar{s}_{n} )\) yields that \(\phi ' _{u_{n} } (s)\) is strongly increasing. Combining with \(\phi ' _{u_{n} } (1)=0\) and \(1<\bar{s}_{n}\), we immediately obtain that \(\phi ' _{u_{n} } (s)\le 0\), \(s\in [0,1]\), namely,

$$s \Vert u_{n} \Vert ^{2} _{X^{\alpha }} -\lambda s \int _{\mathbb{R}} { \vert u_{n} \vert ^{2} \,dt } -s^{q-1} \int _{\mathbb{R}} {b(t) \vert u_{n} \vert ^{q}\,dt } -\mu \int _{\mathbb{R}} {h(t)\cdot u_{n} (t)\,dt } \le 0,\quad s \in [0,1]. $$

Passing to the limit on the above inequality and taking into account (3.17)–(3.19), we get

$$\begin{aligned} &\phi ' _{\bar{u}} (s)=s \Vert \bar{u} \Vert _{X^{\alpha }} ^{2} -\lambda s \int _{\mathbb{R}} { \vert \bar{u} \vert ^{2}\,dt } -s^{q-1} \int _{\mathbb{R}} {b(t) \vert \bar{u} \vert ^{q} \,dt } -\mu \int _{ \mathbb{R}} {h(t)\cdot \bar{u}(t)\,dt } < 0, \\ &\quad s\in (0,1]. \end{aligned}$$
(3.25)

On the other hand, by (3.24), \(\phi ' _{\bar{u}} (s_{1}^{+} )=0\), which combined with (3.25) implies that \(s_{1}^{+} >1\). Consequently, \(\bar{s}>s_{1} ^{+}>1\). Again, in terms of (3.24) and (3.20), we have

$$C_{\lambda ,\mu }^{+} = \inf_{u\in N_{\lambda ,\mu } ^{+} } I_{\lambda ,\mu } (u)\le I_{\lambda ,\mu } \bigl(s_{1}^{+} \bar{u}\bigr)= \inf _{0\le s\le \bar{s}} \phi _{\bar{u}} (s)\le \phi _{\bar{u}} (1)=I _{\lambda ,\mu } (\bar{u}) < \lim_{n\to \infty } I _{\lambda ,\mu } (u_{n} )= C_{\lambda ,\mu }^{+} , $$

which is a contradiction. Thus, \(\lim_{\overline{n\to \infty } } \Vert u_{n} \Vert _{X^{\alpha }} =\Vert \bar{u}\Vert _{X^{\alpha }} \), which yields that \(u_{n} \to \bar{u}\) in \(X^{\alpha }\) because \(X^{\alpha }\) is a Hilbert space taking into account that \(u_{n} \rightharpoonup u\) in \(X^{\alpha }\).

Now, by (3.17)–(3.18), it is easy to see that

$$I_{\lambda ,\mu } (\bar{u})= \lim_{n\to \infty } I _{\lambda ,\mu } (u_{n} )=C_{\lambda ,\mu }^{+} . $$

On the other hand, from \({I}'_{\lambda ,\mu } (u_{n} )u_{n} =0\) and \(( {{I}'_{\lambda ,\mu } (u_{n} )u_{n} } )^{\prime }(u _{n} )>0\) together with \(u_{n} \to \bar{u}\) in \(X^{\alpha }\), we immediately have

$${I}'_{\lambda ,\mu } (\bar{u})\bar{u}=0, \qquad \bigl( {{I}'_{\lambda , \mu } (\bar{u})\bar{u}} \bigr)'\bar{u} \ge 0. $$

Again, by Lemma 3.2, \(I_{\lambda ,\mu } (\bar{u})=C_{\lambda ,\mu } ^{+} <0\), and therefore, \(\bar{u}\ne 0\). By Lemma 2.6, \(N_{\lambda , \mu }^{0} =\emptyset \). Hence, \(( {{I}'_{\lambda ,\mu } ( \bar{u})\bar{u}} )'\bar{u}>0\). This means that \(\bar{u}\in N _{\lambda ,\mu }^{+} \). Hence, \(I_{\lambda ,\mu } \) arrives at its minimum in \(\bar{u}\in N_{\lambda ,\mu }^{+} \) on \(N_{\lambda ,\mu } ^{+} \). □

Lemma 3.5

Assume that hypotheses \((V_{0} )\) and \((H_{1} )\) hold. In addition,

  1. 1.

    if \(h=0\), then \(I_{\lambda ,\mu } \) achieves its minimum at some \(\bar{u}\in N_{\lambda ,\mu }^{-} \) on \(N_{\lambda ,\mu }^{-} \) for \(-\infty <\lambda <\lambda _{0} \).

  2. 2.

    if \(h\ne 0\), then \(I_{\lambda ,\mu } \) achieves its minimum at some \(\bar{u}\in N_{\lambda ,\mu }^{-} \) on \(N_{\lambda ,\mu }^{-} \) for \(-\infty <\lambda <\lambda _{0} \), \(0<\mu <\mu _{\lambda }/2 \).

Proof

First, by Lemma 3.3, \(C_{\lambda ,\mu }^{-} >0\). Now, we are going to make an argument similar to that in Lemma 3.4. Let \(\{ {u_{n} } \}\subset N_{\lambda ,\mu }^{-} \) be a minimizing sequence of \(I_{\lambda ,\mu } \) on \(N_{\lambda ,\mu }^{-} \), namely, \(\lim_{n\to \infty } I_{\lambda ,\mu } (u _{n} )= \inf_{u\in N_{\lambda ,\mu }^{-} } I_{\lambda ,\mu } (u):=C_{\lambda ,\mu }^{-} \). Moreover, \(\{ {u_{n} } \}\) is bounded in \(X^{\alpha }\), and therefore, there exists \(\bar{u}\in X^{\alpha }\) such that \(u_{n} \rightharpoonup \bar{u}\) in \(X^{\alpha }\). The same reason as in Lemma 3.4 implies that (3.17)–(3.18) still hold.

Now, we show that \(\lim_{n\to \infty } \Vert u _{n} \Vert _{X^{\alpha }} =\Vert \bar{u}\Vert _{X^{\alpha }} \). In fact, if not, then

$$ \Vert \bar{u} \Vert _{X^{\alpha }} < \lim _{\overline{n\to \infty } } \Vert u_{n} \Vert _{X^{\alpha }. } $$
(3.26)

We claim that \(\bar{u}\ne 0\). In fact, by \(u_{n} \in N_{\lambda , \mu } \), it follows from (2.1) and (2.3) that

$$I_{\lambda ,\mu } (u_{n} )=-\frac{1}{2} \biggl( \Vert u_{n} \Vert _{X^{\alpha }}^{2} -\lambda \int _{\mathbb{R}} { \vert u_{n} \vert ^{2} \,dt } \biggr) + \biggl( {1-\frac{1}{q}} \biggr) \int _{ \mathbb{R}} {b(t) \vert u_{n} \vert ^{q}\,dt } . $$

Hence,

$$\begin{aligned} \biggl( {1-\frac{1}{q}} \biggr) \int _{\mathbb{R}} {b(t) \vert u_{n} \vert ^{q}\,dt } =&I_{\lambda ,\mu } (u_{n} )+\frac{1}{2} \biggl( { \Vert u_{n} \Vert -\lambda \int _{\mathbb{R}} { \vert u_{n} \vert ^{2} \,dt } } \biggr) \\ \ge& C_{\lambda ,\mu }^{-} +\frac{1}{2} \bigl( {1-\lambda S_{2}^{2} } \bigr) \Vert u_{n} \Vert _{X^{ \alpha }} . \end{aligned}$$

Passing to the limit on the above inequality as \(n\to \infty \) and in view of (3.18), (3.26), we have

$$\biggl( {1-\frac{1}{q}} \biggr) \int _{\mathbb{R}} {b(t) \vert \bar{u} \vert ^{q}\,dt } >C_{\lambda ,\mu }^{-} +\frac{1}{2} \bigl( {1-\lambda S _{2}^{2} } \bigr) \Vert \bar{u} \Vert _{X^{\alpha }} . $$

From \(C_{\lambda ,\mu }^{-} >0\), it follows that \(\int _{\mathbb{R}} {b(t)\vert \bar{u}\vert ^{q}\,dt } >0\), and therefore, \(\bar{u}\ne 0\).

Now, by the proof of Lemma 3.4 and with an argument similar to (3.2) and (3.9), we know that there exists unique \(s^{-} >0\) such that

$$ s^{-} \bar{u}\in N_{\lambda ,\mu }^{-} ,\quad I_{\lambda ,\mu } \bigl(s^{-} \bar{u}\bigr)= \sup_{s\ge 0} I_{\lambda ,\mu } (s\bar{u}). $$
(3.27)

Similarly, owing to \(u_{n} \in N_{\lambda ,\mu }^{-} \), we also have that \(I_{\lambda ,\mu } (u_{n} )= \sup_{s\ge 0} I_{ \lambda ,\mu } (su_{n} )\), and therefore,

$$ I_{\lambda ,\mu } \bigl(s^{-} u_{n} \bigr) \leq I_{\lambda ,\mu } (u_{n} ). $$
(3.28)

Once again, (3.17)–(3.18) and (3.26) imply that

$$ I_{\lambda ,\mu } \bigl(s^{-} \bar{u}\bigr)< \lim _{n\to \infty } I_{\lambda ,\mu } \bigl(s^{-} u_{n} \bigr). $$
(3.29)

Hence, by (3.27)–(3.29), we immediately have

$$C_{\lambda ,\mu }^{-} = \inf_{u\in N_{\lambda ,\mu } ^{-} } I_{\lambda ,\mu } (u)\le I_{\lambda ,\mu } \bigl(s^{-} \bar{u}\bigr)< \lim _{n\to \infty } I_{\lambda ,\mu } \bigl(s^{-} u_{n} \bigr)\le \lim_{n\to \infty } I_{\lambda ,\mu } (u_{n} )=C_{\lambda ,\mu } ^{-} , $$

a contradiction. Hence, \(\lim_{n\to \infty } \Vert u_{n} \Vert _{X^{\alpha }} =\Vert \bar{u}\Vert _{X^{\alpha }} \). By an argument similar to that in the proof of Lemma 3.4, we obtain that \(I_{\lambda ,\mu } \) achieves its minimum at ū on \(N_{\lambda ,\mu }^{-} \). □

Now, we are in a position to show our first result on the existence of solution to (1.4).

Theorem 3.1

Assume that hypotheses \((V_{0} )\) and \((H_{1} )\) hold. If \(h=0\), then problem (1.4) has at least one nontrivial weak solution for \(-\infty <\lambda <\lambda _{0} \).

Proof

Under the conditions assumed in this theorem, by Lemma 2.6, Lemma 3.1, Lemma 3.3, and Lemma 3.5, we know that \(N_{\lambda , \mu }^{0} =\emptyset , N_{\lambda ,\mu }^{-} \ne \emptyset \), and \(I_{\lambda ,\mu } \) achieves its minimum at some \(\bar{u}\in N_{ \lambda ,\mu }^{-} \). Precisely, \(I_{\lambda ,\mu } (\bar{u})= \inf_{u\in N_{\lambda ,\mu }^{-} } I_{\lambda , \mu } (u)=C_{\lambda ,\mu }^{-} >0\). Because \(N_{\lambda ,\mu }^{-} \) is an open subset in \(N_{\lambda ,\mu } \) and \(\bar{u}\in N_{\lambda ,\mu }^{-} \), there exists a ball \(B(\bar{u},r)\) centered at ū with radius r satisfying \(B(\bar{u},r) \cap N_{\lambda , \mu } \subset N_{\lambda ,\mu }^{-} \). Thus

$$I_{\lambda ,\mu } (\bar{u})=\inf_{u\in N_{\lambda ,\mu }^{-} } I_{\lambda ,\mu } (u)= \inf_{u\in N_{\lambda ,\mu }^{-} \cap B(\bar{u},r)} I _{\lambda ,\mu } (u)=\inf_{u\in N_{\lambda ,\mu }\cap B(\bar{u},r)} I_{\lambda ,\mu } (u). $$

Hence, by Lemma 2.5 and taking account of the fact that \(N_{\lambda , \mu }^{0} =\emptyset \), we immediately obtain that ū is a nontrivial weak solution of (1.4). This completes the proof. □

We give another result on the multiplicity of solutions to (1.4).

Theorem 3.2

Assume that hypotheses \((V_{0} )\) and \((H_{1} )\) hold. If \(h\ne 0\), then problem (1.4) has at least two nontrivial weak solutions for \(-\infty <\lambda <\lambda _{0} \), \(0<\mu <\mu _{\lambda }/2\).

Proof

Under these conditions, by Lemma 2.6 and Lemmas 3.1–3.5, we know that \(N_{\lambda ,\mu }^{0} =\emptyset , N_{\lambda , \mu }^{-} \ne \emptyset ,N_{\lambda ,\mu }^{+} \ne \emptyset \). Moreover, there exist \(\bar{u}_{1} \in N_{\lambda ,\mu }^{-} \), \(\bar{u} _{2} \in N_{\lambda ,\mu }^{+} \) such that

$$I_{\lambda ,\mu } (\bar{u}_{1} )= \inf_{u \in N_{\lambda ,\mu }^{-} } I_{\lambda ,\mu } (u)>0,\qquad I_{ \lambda ,\mu } (\bar{u}_{2} )= \inf _{u\in N_{\lambda ,\mu }^{+} } I_{\lambda ,\mu } (u)< 0. $$

Because \(N_{\lambda ,\mu }^{+}\), \(N_{\lambda ,\mu }^{-}\) are two open subsets in \(N_{\lambda ,\mu } \) with \(N_{\lambda ,\mu }^{+} \cap N _{\lambda ,\mu }^{-} =\emptyset \) and \(N_{\lambda ,\mu }^{0} =\emptyset \), making an argument similar to that in the proof of Theorem 3.1, we can deduce that \(\bar{u}_{1}\), \(\bar{u}_{2}\) are two nontrivial solutions of problem (1.4). The proof is complete. □

4 Conclusion

In this paper, the authors investigate the existence and multiplicity of solutions for fractional Hamiltonian systems (1.4). Because the energy functional corresponding to system (1.4) is unbounded from below, it is invalid to try finding a critical point by applying the direct method of minimization to the energy functional. To overcome this difficulty, we introduce the Nehari manifold \(N_{\lambda , \mu }\) and split it into three parts \(N_{\lambda , \mu }^{-}\), \(N_{\lambda ,\mu }^{+}\), and \(N_{\lambda , \mu }^{0} \). By looking for minimizer of the energy functional on \(N_{\lambda , \mu }^{-}\) and \(N_{\lambda ,\mu }^{+} \) respectively and with the help of more analysis techniques, the authors establish some results on the existence and multiplicity of solutions to the above problem.