1 Introduction and main results

A Klein–Gordon–Maxwell system arises in a very interesting physical context: a model describing the nonlinear Klein–Gordon field interacting with the electromagnetic field (for more details, see [1, 2]). It has been widely studied on different aspects. Variational methods were firstly used by Benci and Fortunato to consider the following system:

$$ \textstyle\begin{cases} -\Delta u+ [m_{0}^{2}-(\omega +\phi )^{2} u = \vert u \vert ^{p-2}u, & x\in \mathbb{R}^{3}, \\ -\Delta \phi + u^{2}\phi =-\omega u^{2}, & x\in \mathbb{R}^{3}, \end{cases}\displaystyle \quad (\mathrm{KGM}) $$

where \(4< p <6\), \(m_{0}\) and ω are real constants. Infinitely many solitary waves solutions were got for the above system when \(|m_{0}|>|\omega |\), \(p\in (4,6)\) in [2]. For \(p\geq 6\) and \(m_{0}\geq \omega \) or \(p \leq 2\), no-existence result of (KGM) was proved by D’Aprile and Mugnai in [3]. Furthermore, in [4] infinitely many finite energy radial solutions were got if one of the following conditions holds:

  1. (i)

    \(m_{0}>\omega >0\) and \(p\in (4,6)\);

  2. (ii)

    \(m_{0}\sqrt{\frac{p-2}{2}}>\omega >0\) and \(p\in (2,4)\).

For \(p\in (2, 4)\), the existence range of \((m_{0},\omega )\) was extended and a limit case \(m_{0}=\omega \) was also dealt with by Azzollini, Pisani, and Pomponio in [5]. Mugnai in [6] studied the existence of radially symmetric solitary waves for a system of a nonlinear Klein–Gordon equation coupled with Maxwell’s equation in the presence of a positive mass. Ground state solutions, semiclassical state solutions, nonradial solutions have been studied in [7,8,9,10]. The critical exponent case have also been considered in [11,12,13,14]. In[15], via the Ekeland variational principle and the mountain pass theorem, two nontrivial solutions for a nonhomogeneous Klein–Gordon–Maxwell system were got by Chen and Tang. In [16], Jeong and Seok established an abstract critical point theorem about a functional of the mountain-pass type with a small perturbation for the nonlocal term and studied a type of Klein–Gordon–Maxwell system with a very general nonlinear term. A Klein–Gordon–Maxwell system with non-constant potential was firstly considered by He in[17]. Infinitely many solutions for a type of Klein–Gordon–Maxwell system with a coercive potential were got via a variant fountain theorem and the symmetric mountain pass theorem in [17]. The results in [17] were improved and complemented by Li and Tang in [18]. In [19], Under a variant 4-superlinear condition, infinitely many solutions for a nonlinear Klein–Gordon–Maxwell system with sign-changing potential were got by Ding and Li via the symmetric mountain pass theorem.

Inspired by [4, 17,18,19], in this paper, we deal with the following Klein–Gordon–Maxwell system via the variational methods:

$$ \textstyle\begin{cases} -\Delta u+ V(x)u-(2\omega +\phi )\phi u = f(x,u)+\lambda h(x) \vert u \vert ^{q-2}u, & x\in \mathbb{R}^{3}, \\ \Delta \phi = (\omega +\phi )u^{2}, & x\in \mathbb{R}^{3}, \end{cases}\displaystyle \quad (\mathrm{P_{\lambda }}) $$

where \(\omega >0\), the functions V, f and h satisfy the following assumptions.

\((V)\) :

\(V\in C(\mathbb{R}^{3},\mathbb{R})\) with \(\inf_{x\in \mathbb{R}^{3}}V(x)>-\infty \) and there exists a constant \(r>0\) such that

$$ \lim_{ \vert y \vert \rightarrow \infty }\operatorname{meas}\bigl\{ x\in \mathbb{R}^{3}: \vert x-y \vert \leq r, V(x)\leq M\bigr\} =0,\quad \text{for every } M>0; $$
\((F_{1})\) :

\(f\in C(\mathbb{R}^{3}\times \mathbb{R},\mathbb{R})\) and there exist constants \(c_{0}\in (0,\infty )\) and \(p\in (2,6)\) such that

$$ \bigl\vert f(x,t) \bigr\vert \leq c_{0}\bigl( \vert t \vert + \vert t \vert ^{p-1}\bigr),\quad \text{for } (x,t)\in \mathbb{R} ^{3}\times \mathbb{R}; $$
\((F_{2})\) :

there exist constants \(\mu \in (2,\infty )\) and \(R\in (0,\infty )\) such that

$$ f(x,t)t\geq \mu F(x,t)>0,\quad \text{for } (x,t)\in \mathbb{R}^{3} \times \mathbb{R}\quad \text{with } \vert t \vert \geq R,\quad \text{and}\quad \inf_{x\in \mathbb{R}^{3}, \vert u \vert =R }F(x,u)>0, $$

where \(F(x,t):= \int _{0}^{t} f(x,s) \,ds\);

\((F_{3})\) :

\(f(x,-t)=-f(x,t)\), for \((x,t)\in \mathbb{R}^{3} \times \mathbb{R}\);

\((H)\) :

\(h\in L^{\frac{2}{2-q}}(\mathbb{R}^{3})\), for some \(q\in (1,2)\) and \(h(x)>0\) for a.e. \(x\in \mathbb{R}^{3}\).

First of all, we establish the variational framework for (Pλ). As usual, let \(C_{0}^{\infty }(\mathbb{R}^{3})\) denote the collection of smooth functions with compact support and \(D^{1, 2}( \mathbb{R}^{3})\) be the completion of \(C_{0}^{\infty }(\mathbb{R}^{3})\) under the norm

$$ \Vert u \Vert _{D^{1,2}}= \biggl( \int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2} \,dx \biggr) ^{\frac{1}{2}}. $$

Under the condition of (V), we can set \(V_{1}(x)=V(x)+V_{0}\), where \(V_{0}\in ( 1+|\inf_{x\in \mathbb{R}^{3}}V|,\infty )\) is fixed. Define

$$ E := \biggl\{ u\in D^{1,2}\bigl(\mathbb{R}^{3}\bigr)\Big| \int _{\mathbb{R}^{3}}V_{1}(x) u^{2} \,dx< \infty \biggr\} , $$

which is a Hilbert space equipped with the norm

$$\begin{aligned}& \Vert u \Vert = \biggl( \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2} + V_{1}(x)u^{2}\bigr) \,dx \biggr)^{\frac{1}{2}}. \\& L^{p}\bigl(\mathbb{R}^{3}\bigr):= \biggl\{ u: \mathbb{R}^{3}\mapsto \mathbb{R}\Big| u \text{ is Lebesgue measurable}, \int _{\mathbb{R}^{3}} \vert u \vert ^{p} \,dx< \infty \biggr\} \end{aligned}$$

is the usual Lebesgue space equipped with the norm

$$ \vert u \vert _{p}=\biggl( \int _{\mathbb{R}^{3}} \vert u \vert ^{p} \,dx \biggr)^{\frac{1}{p}}. $$

The main results of our paper read as follows.

Theorem 1.1

Let (V), \((F_{1})\)–\((F_{3})\) and \((H)\) hold, then there exists \(\lambda _{*}>0\) such that the system (Pλ) has a sequence of weak solutions \(\{(u_{n},\phi _{n})\}\subset E \times D^{1,2} ( \mathbb{R}^{3})\) satisfying

$$ \begin{aligned} &\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}_{n}+V(x)u^{2}_{n} \bigr)\,dx-\frac{ \omega }{2} \int _{\mathbb{R}^{3}}\phi _{u_{n}}u_{n}^{2} \,dx \\&\quad{}- \int _{\mathbb{R}^{3}}{F}(x,u_{n})\,dx -\frac{\lambda }{q} \int _{\mathbb{R}^{3}}h(x) \vert u_{n} \vert ^{q}\,dx\rightarrow +\infty , \end{aligned} $$

as \(n\to \infty \), for every \(\lambda \in (0,\lambda _{*})\).

If the nonlinearity f also satisfies

\((F_{4})\) :
$$ \lim_{|t|\rightarrow 0}\frac{f(x,t)}{t}=-V_{0},\quad \text{uniformly for } x\in \mathbb{R}^{3}, $$

we get the following result.

Theorem 1.2

Let (V), \((F_{1})\), \((F_{2})\), \((F_{4})\) and \((H)\) hold, then there exists \(\lambda ^{*}>0\) such that the system (Pλ) has at least one nontrivial weak solution for every \(\lambda \in (0,\lambda ^{*})\).

Remark 1.1

In our assumptions, the nonlinearity f just needs to satisfy a classic superquadratic condition at infinity. The 4-superlinear assumption is not necessary. The potential V is sign-changing and F can also change sign. Thus, we extend and improve some existing results to some extent, for example, some results in [17, 19].

Remark 1.2

Since \(-\infty <\inf_{x\in\mathbb{R}^{3}}V(x)<0\), it is a natural idea to add Cu with \(C> |\inf_{x\in\mathbb{R}^{3}}V(x)|\) at both sides of the first equation in the system (Pλ). However, the nonlinearity \(f(x,u)+Cu\) does not satisfy the assumption \((F_{2})\), which brings some difficulty to prove the bounded nature of (PS) sequences. Based on the compact embedding theorem (Lemma 2.1) and the properties of \(\phi _{u}\) (Lemma 2.2), we prove this important property of the functional I in Lemma 3.2. Compared with the second result in [20], our assumption on V is weaker.

Throughout the paper, we denote by C various positive constants, whose value may be different from line to line and is not essential to the problem.

The paper is organized as follows. In Sect. 2, we give some preliminary results. In Sect. 3, we prove our main results.

2 Preliminary

In this section, we give some preliminary results which will be used to prove our main results.

Assumption \((V)\) is similar to the condition introduced by Bartsch, Wang, and Willem in [21] to guarantee the compactness of embedding of the work spaces. Since \(V_{1}\) has positive infimum, it is easy to get the following.

Lemma 2.1

Let \((V)\) be satisfied, the space E is continuously embedded into \(L^{p}(\mathbb{R}^{3})\) for any \(p\in [2,6]\) and compactly embedded into \(L^{p}(\mathbb{R}^{3})\) for any \(p\in [2,6)\).

Since \(2<\frac{12}{5}<3<6\), for any fixed \(u\in E\), the linear operator \(T_{u}:D^{1,2}(\mathbb{R}^{3})\rightarrow \mathbb{R}\) defined by

$$ T_{u}(v):= \int _{\mathbb{R}^{3}}u^{2}v \,dx $$

is continuous in \(D^{1,2}(\mathbb{R}^{3})\). By the Lax–Milgram theorem, there exists \(\phi _{u}\in D^{1,2}(\mathbb{R}^{3})\) such that

$$ \int _{\mathbb{R}^{3}}\bigl(\nabla \phi _{u} \nabla v+u^{2}\phi _{u}v\bigr) \,dx= \int _{\mathbb{R}^{3}}u^{2} v \,dx,\quad \text{for } v\in D^{1,2}\bigl(\mathbb{R} ^{3}\bigr). $$

Therefore, problem (Pλ) can be transformed into a nonlinear Schrödinger equation with a nonlocal term

$$ -\Delta u+ V(x)u-(2\omega +\phi _{u})\phi _{u} u = f(x,u)+ \lambda h(x) \vert u \vert ^{q-2}u, \quad x\in \mathbb{R}^{3}. \quad \bigl(\mathrm{P}_{\lambda }'\bigr) $$

The functional associated to \((\mathrm{P}_{\lambda }')\) is

$$ I(u)=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl(\nabla u^{2}+V(x)u^{2} \bigr)\,dx-\frac{ \omega }{2} \int _{\mathbb{R}^{3}}\phi _{u}u^{2} \,dx - \int _{\mathbb{R} ^{3}}{F}(x,u)\,dx-\frac{\lambda }{q} \int _{\mathbb{R}^{3}}h(x) \vert u \vert ^{q}\,dx. $$

From \((F_{1})\) and Lemma 2.1, it is easy to claim that \(I\in C^{1}(E, \mathbb{R})\) and

$$ \begin{aligned} \bigl\langle I'(u),v\bigr\rangle &= \int _{\mathbb{R}^{3}}\bigl(\nabla u\nabla v+V(x)uv\bigr)\,dx- \int _{\mathbb{R}^{3}}(2\omega +\phi _{u})\phi _{u}uv \,dx \\&\quad{}- \int _{\mathbb{R}^{3}}\bigl(f(x,u)+\lambda h(x) \vert u \vert ^{q-2}u\bigr)v \,dx, \end{aligned} $$

for \(v\in E\). Moreover, the function \(\phi _{u}\) has the following properties.

Lemma 2.2

(see [2, 4])

  1. (i)

    \(-\omega \leq \phi _{u}\leq 0\) on the set \(\{x| u(x) \neq 0\}\);

  2. (ii)

    there exist positive constants C, \(C'\) such that

    $$ \Vert \phi _{u} \Vert _{D^{1,2}}\leq C \Vert u \Vert ^{2} \quad \textit{and}\quad \int _{\mathbb{R}^{3}} \vert \phi _{u} \vert u^{2}\,dx\leq C' \Vert u \Vert ^{4}. $$

Definition 2.1

Let X be a Banach space, we say that the functional \(I\in C^{1}(X, \mathbb{R})\) satisfies Palais–Smale condition at the level \(c\in \mathbb{R}\) (\((\mathrm{PS})_{c}\) in short) if any sequence \(\{u_{n}\} \subset X \) satisfying \(I(u_{n}) \to c\), \(I'(u_{n}) \to 0\) as \(n\to \infty \), has a convergent subsequence. I satisfies the (PS) condition if I satisfies \((\mathrm{PS})_{c}\) condition at any \(c\in \mathbb{R}\).

Lemma 2.3

(Mountain pass theorem, [22])

Let X be a Banach space, \(I\in C^{1}(X,\mathbb{R})\), \(e\in X\) and \(r>0\) be such that \(\|e\|>r\) and

$$ b:=\inf_{\|u\|=r}I(u)>I(0)\geq I(e). $$

If I satisfies the \((\mathrm{PS})_{c}\) condition with

$$ c:=\inf_{\gamma \in \varGamma }\max_{t\in [0,1]}I\bigl(\gamma (t) \bigr), \quad \textit{where } \varGamma :=\bigl\{ \gamma \in C\bigl([0,1],X\bigr)| \gamma (0)=0, \gamma (1)=e\bigr\} , $$

then c is a critical value of I.

Lemma 2.4

([23])

Let E be an infinite dimensional Banach space, and let \(I \in C^{1}(E, \mathbb{R})\) be even, satisfy the (PS) condition and I(0) = 0. Assume that \(E = Y \oplus Z\), where Y is finite dimensional. Suppose that the following hold.

\((I_{1})\) :

There are constants \(\rho , \alpha > 0\) such that \(\inf_{\partial B_{\rho }\cap Z} I \geq \alpha \).

\((I_{2})\) :

For each finite dimensional subspace \(\widetilde{E} \subset E\), there is an \(R(\widetilde{E})\) such that \(I(u) \leq 0\), for \(u\in \widetilde{E}\setminus B_{R(\widetilde{E})}\).

Then I possesses an unbounded sequence of critical values.

3 Proof of main results

In this section, we prove our main results. Firstly, it is easy to check that F satisfies the following properties.

Lemma 3.1

Let \((F_{1})\) and \((F_{2})\) be satisfied, then

  1. (i)

    there exist constants \(c_{1},c_{2}\in (0,\infty )\) such that

    $$ \bigl\vert F(x,t) \bigr\vert \geq c_{1} \vert t \vert ^{\mu }-c_{2} \vert t \vert ^{2}, \quad \textit{for } (x,t)\in \mathbb{R}^{3}\times \mathbb{R}; $$
    (3.1)
  2. (ii)

    for any fixed \(r\in (0,+\infty )\), there exists a positive constant \(c(r)\) such that

    $$ \biggl\vert \frac{1}{\mu }f(x,t)t-F(x,t) \biggr\vert \leq c(r) \vert t \vert ^{2},\quad \textit{for } (x,t) \in \mathbb{R}^{3}\times [-r,r]. $$
    (3.2)

Lemma 3.2

Assume that \((V)\), \((F_{1})\), \((F_{2})\) and \((H)\) hold, then the functional I satisfies the (PS) condition.

Proof

Let \(\{u_{n}\}\) be a (PS) sequence of I, that is, for some \(M>0\),

$$ \bigl\vert I(u_{n}) \bigr\vert \leq M, \quad I'(u_{n}) \to 0,\quad \text{as } n \to \infty \quad \text{in } E. $$

On the one hand, from (i) of Lemma 2.2, \((F_{2})\) and (3.2), since \(\mu >2\), for n large enough

$$ \begin{aligned}[b] & M+1+ \Vert u_{n} \Vert \\ &\quad \geq I(u_{n})- \frac{1}{ \mu } \bigl\langle I'(u_{n}),u _{n}\bigr\rangle \\ &\quad \geq \biggl(\frac{1}{2}-\frac{1}{ \mu }\biggr) \Vert u_{n} \Vert ^{2}-\biggl(\frac{1}{2}- \frac{1}{ \mu }\biggr) \int _{\mathbb{R}^{3}}V_{0}u^{2}_{n}\,dx+ \biggl(\frac{2}{\mu }- \frac{1}{2}\biggr) \int _{\mathbb{R}^{3}}\omega \phi _{u_{n}}u_{n}^{2}\,dx \\ &\qquad{}+ \int _{\mathbb{R}^{3}}\biggl(\frac{1}{ \mu }f(x,u_{n})u_{n}-F(x,u_{n}) \biggr)\,dx+\frac{q- \mu }{q\mu }\lambda \int _{\mathbb{R}^{3}}h(x) \vert u_{n} \vert ^{q}\,dx \\ &\quad \geq \biggl(\frac{1}{2}-\frac{1}{ \mu }\biggr) \Vert u_{n} \Vert ^{2}-\biggl(\frac{1}{2}- \frac{1}{ \mu }\biggr) \int _{\mathbb{R}^{3}}V_{0}u^{2}_{n}\,dx+ \biggl(\frac{2}{\mu }- \frac{1}{2}\biggr) \int _{\{u_{n}\neq 0\}}\omega \phi _{u_{n}}u_{n}^{2}\,dx \\ &\qquad{}+ \int _{\{ \vert u_{n} \vert \leq r_{0}\}}\biggl(\frac{1}{ \mu }f(x,u_{n})u_{n}-F(x,u_{n}) \biggr)\,dx+\frac{q- \mu }{q\mu }\lambda \int _{\mathbb{R}^{3}}h(x) \vert u_{n} \vert ^{q}\,dx \\ &\quad \geq \biggl( \frac{1}{2}-\frac{1}{ \mu }\biggr) \Vert u_{n} \Vert ^{2}-\biggl(\frac{1}{2}- \frac{1}{ \mu }\biggr) \int _{\mathbb{R}^{3}}V_{0}u^{2}_{n}\,dx- \biggl\vert \frac{2}{\mu }- \frac{1}{2} \biggr\vert \omega ^{2} \int _{\mathbb{R}^{3}}u_{n}^{2}\,dx \\ &\qquad{}- \int _{\mathbb{R}^{3}}c(r_{0})u_{n}^{2}\,dx-C \lambda \vert h \vert _{\frac{2}{2-q}} \Vert u_{n} \Vert ^{q} \\ &\quad =:\biggl(\frac{1}{2}-\frac{1}{ \mu }\biggr) \Vert u_{n} \Vert ^{2}-c_{3} \int _{\mathbb{R}^{3}}u_{n}^{2}\,dx-C\lambda \vert h \vert _{\frac{2}{2-q}} \Vert u_{n} \Vert ^{q}, \end{aligned} $$
(3.3)

where \(c_{3}=(\frac{1}{2}-\frac{1}{\mu })V_{0}+|\frac{2}{\mu }-\frac{1}{2}| \omega ^{2}+c(r_{0})\). If \(\{u_{n}\}\) is not bounded in E, there exists a subsequence still denoted by \(\{u_{n}\}\) such that \(\|u_{n}\|\rightarrow \infty \), as \(n\rightarrow \infty \). Set \(w_{n}=\frac{u_{n}}{\|u_{n}\|}\), then there exists \(w\in E\) such that

$$ \begin{aligned}[b] &w_{n}\rightharpoonup w \quad \text{in } E, \\ &w_{n}\rightarrow w \quad \text{in } L^{p}\bigl( \mathbb{R}^{3}\bigr), p\in [2,6), \\ &w_{n}(x)\rightarrow w(x), \quad \text{a.e. } x\in \mathbb{R}^{3}. \end{aligned} $$

Divide \(\|u_{n}\|^{2}\) on both sides of (3.3), we have

$$ \frac{M+1}{ \Vert u_{n} \Vert ^{2}}+\frac{1}{ \Vert u_{n} \Vert }\geq \biggl(\frac{1}{2}- \frac{1}{ \mu }\biggr)-c_{3} \int _{\mathbb{R}^{3}}{w_{n}}^{2}\,dx-C\lambda \vert h \vert _{ \frac{2}{2-q}}\frac{1}{ \Vert u_{n} \Vert ^{2-q}}, $$

then

$$ \int _{\mathbb{R}^{3}}{w}^{2}\,dx\geq \frac{1}{c_{3}}\biggl( \frac{1}{2}-\frac{1}{ \mu }\biggr)>0. $$
(3.4)

On the other hand, since

$$ \int _{\mathbb{R}^{3}}F(x,u_{n})\,dx=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \nabla u_{n}^{2}+V(x)u_{n}^{2} \bigr)\,dx-\frac{\omega }{2} \int _{\mathbb{R} ^{3}}\phi _{u_{n}}u_{n}^{2} \,dx-\frac{\lambda }{q} \int _{\mathbb{R}^{3}}h(x) \vert u _{n} \vert ^{q}\,dx-I(u_{n}), $$

by (3.1), we can get

$$ \begin{aligned}[b] 0&\leq c_{1} \int _{\mathbb{R}^{3}} \vert w_{n} \vert ^{\mu }\,dx \leq \frac{ \int _{\mathbb{R}^{3}}F(x,u_{n})\,dx}{ \Vert u_{n} \Vert ^{\mu }}+\frac{c_{2} \int _{\mathbb{R}^{3}}u^{2}_{n}\,dx}{ \Vert u_{n} \Vert ^{\mu }} \\ &\leq \frac{1}{2 \Vert u_{n} \Vert ^{\mu -2}}+\frac{\omega ^{2}\int _{\mathbb{R}^{3}}u_{n}^{2}\,dx}{2 \Vert u_{n} \Vert ^{\mu }}-\frac{I(u_{n})}{ \Vert u_{n} \Vert ^{\mu }}+ \frac{c_{2} \int _{\mathbb{R}^{3}}u^{2}_{n}\,dx}{ \Vert u_{n} \Vert ^{\mu }} \\ &\rightarrow 0, n\rightarrow \infty , \end{aligned} $$

then \(w=0\), which is a contradiction with (3.4).

The rest of the proof is standard. In fact, since \(\{u_{n}\}\) is bounded in E, we can assume that, up to a subsequence,

$$ \begin{aligned}[b] &u_{n}\rightharpoonup u \quad \text{in } E, \\ &u_{n}\rightarrow u \quad \text{in } L^{p}\bigl( \mathbb{R}^{3}\bigr), p\in [2,6), \\ &u_{n}(x)\rightarrow u(x), \quad \text{a.e. } x\in \mathbb{R}^{3}. \end{aligned} $$

Then

$$ \begin{aligned}[b] & \Vert u_{n}-u \Vert ^{2} \\ &\quad =\bigl\langle I'(u_{n}), u_{n}-u \bigr\rangle - \bigl\langle I'(u), u_{n}-u \bigr\rangle + \int _{\mathbb{R}^{3}}V_{0}(u_{n}-u)^{2}\,dx \\ &\qquad{}+2 \omega \int _{\mathbb{R}^{3}}(\phi _{u_{n}}u_{n}-\phi _{u}u) (u_{n}-u)\,dx+ \int _{\mathbb{R}^{3}}\bigl(\phi _{u_{n}}^{2}u_{n}- \phi _{u}^{2}u\bigr) (u_{n}-u)\,dx \\ &\qquad{}+ \int _{\mathbb{R}^{3}}\bigl(f(x,u_{n})-f(x,u)\bigr) (u_{n}-u)\,dx\\&\qquad{}+\lambda \int _{\mathbb{R}^{3}}h(x) \bigl( \vert u_{n} \vert ^{q-2}u_{n}- \vert u \vert ^{q-2}u\bigr) (u_{n}-u)\,dx. \end{aligned} $$
(3.5)

It follows from \(I'(u_{n})\rightarrow 0\) and \(u_{n}\rightharpoonup u\) in E that the first three parts on the right side of (3.5) converge to zero as \(n\rightarrow \infty \). By the Hölder inequality, (ii) of Lemma 2.2 and Lemma 2.1,

$$ \begin{aligned}[b] & \biggl\vert \int _{\mathbb{R}^{3}}(\phi _{u_{n}}u_{n}-\phi _{u}u) (u_{n}-u) \,dx \biggr\vert \\ &\quad = \biggl\vert \int _{\mathbb{R}^{3}}\phi _{u_{n}}(u_{n}-u)^{2} \,dx + \int _{\mathbb{R} ^{3}}(\phi _{u_{n}}-\phi _{u})u(u_{n}-u) \,dx \biggr\vert \\ &\quad \leq \vert \phi _{u_{n}} \vert _{6} \vert u_{n}-u \vert ^{2}_{\frac{12}{5}} + \vert \phi _{u_{n}}-\phi _{u} \vert _{6} \vert u \vert _{\frac{12}{5}} \vert u_{n}-u \vert _{\frac{12}{5}} \\ &\quad \leq C \bigl( \vert u_{n}-u \vert ^{2} _{\frac{12}{5}} + \vert u \vert _{\frac{12}{5}} \vert u_{n}-u \vert _{\frac{12}{5}}\bigr) \\ &\quad \to 0,\quad n\rightarrow \infty . \end{aligned} $$
(3.6)

Since the sequence \(\{\phi _{u_{n}}^{2}u_{n}\}\) is bounded in \(L^{\frac{3}{2}}(\mathbb{R}^{3})\), by the Hölder inequality,

$$\begin{aligned} & \biggl\vert \int _{\mathbb{R}^{3}}\bigl(\phi _{u_{n}}^{2}u_{n}- \phi _{u}^{2}u\bigr) (u_{n}-u)\,dx \biggr\vert \\ &\quad \leq \bigl\vert \phi _{u_{n}}^{2}u_{n}-\phi _{u}^{2}u \bigr\vert _{\frac{3}{2}} \vert u_{n}-u \vert _{3} \\ &\quad \leq \bigl( \bigl\vert \phi _{u_{n}}^{2}u_{n} \bigr\vert _{\frac{3}{2}}+ \bigl\vert \phi _{u}^{2}u \bigr\vert _{ \frac{3}{2}}\bigr) \vert u_{n}-u \vert _{3} \\ &\quad \rightarrow 0,\quad n\rightarrow \infty . \end{aligned}$$
(3.7)

Then (3.6) and (3.7) imply that the fourth and fifth part on the right side of (3.5) also converge to zero as \(n\rightarrow \infty \). By the Hölder inequality, \((F_{1})\) and Lemma 2.1,

$$ \begin{aligned}[b] & \int _{\mathbb{R}^{3}}\bigl(f(x,u_{n})-f(x,u)\bigr) (u_{n}-u)\,dx \\ &\quad \leq \int _{\mathbb{R}^{3}}\bigl( \bigl\vert f(x,u_{n}) \bigr\vert + \bigl\vert f(x,u) \bigr\vert \bigr) \vert u_{n}-u \vert \,dx \\ &\quad \leq \int _{\mathbb{R}^{3}}c_{0}\bigl( \vert u_{n} \vert ^{p-1}+ \vert u_{n} \vert + \vert u \vert ^{p-1}+ \vert u \vert \bigr) \vert u_{n}-u \vert \,dx \\ &\quad \leq c_{0} \bigl( \bigl( \vert u_{n} \vert ^{p-1}_{p}+ \vert u \vert ^{p-1}_{p} \bigr) \vert u_{n}-u \vert _{p} +\bigl( \vert u_{n} \vert _{2}+ \vert u \vert _{2} \bigr) \vert u_{n}-u \vert _{2} \bigr) \\ &\quad \to 0,\quad n\rightarrow \infty , \end{aligned} $$

thus the sixth part on the right side of (3.5) converges to zero as \(n\rightarrow \infty \). Similarly, by the Hölder inequality, the last part also converges to zero as \(n\rightarrow \infty \). Then \(\|u_{n}-u\|^{2}\to 0\), \(n\rightarrow \infty \). Therefore, I satisfies (PS) condition. □

Let \(\{e_{i}\}^{\infty }_{i=1}\) be an orthonormal basis of E, define \(X_{i}=\mathbb{R}e_{i}\), \(Y_{k}=\bigoplus ^{k}_{i=1}X_{i}\) and \(Z_{k}=\overline{ \bigoplus ^{\infty }_{i=k+1}X_{i}}\), \(k\in \mathbb{N}\).

Lemma 3.3

Assume that \((V)\) and \((F_{1})\) hold, there exist constants \(k_{0}\in \mathbb{N}\) and \(\lambda _{*}>0\) such that \(I|_{\partial B _{\rho _{k_{0,\lambda }}}\cap Z_{k_{0}}}\geq \alpha _{k_{0,\lambda }}\), for every \(\lambda \in (0,\lambda _{*})\) and some \(\rho _{k_{0,\lambda }}, \alpha _{k_{0,\lambda }}\in (0,+\infty )\).

Proof

First of all, in the same way as Lemma 3.8 in [22], it is easy to see that, for \(s\in [2,6)\),

$$ \beta _{k}:=\sup_{u\in Z_{k}, \Vert u \Vert =1} \vert u \vert _{s}\rightarrow 0,\quad \text{as } k\rightarrow \infty . $$

Then there exists \(k_{0}\in \mathbb{N}\) such that

$$ \vert u \vert ^{2}_{2}\leq \frac{1}{2(c_{0}+V_{0})} \Vert u \Vert ^{2}\quad \text{and} \quad \vert u \vert ^{p}_{p}\leq \frac{p}{4c_{0}} \Vert u \Vert ^{p},\quad \text{for } u\in Z_{k _{0}}. $$

Therefore,

$$ \begin{aligned}[b] I(u) &=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl(\nabla u^{2}+V(x)u^{2} \bigr)\,dx-\frac{ \omega }{2} \int _{\mathbb{R}^{3}}\phi _{u}u^{2} \,dx - \int _{\mathbb{R} ^{3}}{F}(x,u)\,dx-\frac{\lambda }{q} \int _{\mathbb{R}^{3}}h(x) \vert u \vert ^{q}\,dx \\ &\geq \frac{1}{2} \Vert u \Vert ^{2}-\frac{V_{0}+c_{0}}{2} \vert u \vert ^{2}_{2}-\frac{c _{0}}{p} \vert u \vert ^{p}_{p}-\frac{C\lambda }{q} \vert h \vert _{\frac{2}{2-q}} \Vert u \Vert ^{q} \\ &\geq \frac{1}{2} \Vert u \Vert ^{2}-\frac{1}{4} \Vert u \Vert ^{2}-\frac{1}{4} \Vert u \Vert ^{p}-\frac{C \lambda }{q} \vert h \vert _{\frac{2}{2-q}} \Vert u \Vert ^{q} \\ &=\frac{1}{4} \Vert u \Vert ^{2}\biggl(1- \Vert u \Vert ^{p-2}-\frac{4C\lambda }{q} \vert h \vert _{\frac{2}{2-q}} \Vert u \Vert ^{q-2}\biggr). \end{aligned} $$

Set \(\eta (t)=1-t^{p-2}-\frac{4C\lambda }{q}|h|_{\frac{2}{2-q}}t^{q-2}\), \(t>0\). Since \(1< q<2<p\), there exists \(t_{0,\lambda }=( \frac{2-q}{p-2}\frac{4C\lambda }{q} |h|_{\frac{2}{2-q}})^{ \frac{1}{p-q}}\) such that \(\eta (t_{0,\lambda })=\max_{t>0} \eta (t)\). Furthermore, we can get \(\eta (t_{0,\lambda })>0\) for \(\lambda \in (0, \lambda _{*})\), where

$$ \lambda _{*}=\biggl(\frac{2-q}{p-q}\biggr)^{\frac{p-q}{p-2}} \frac{p-2}{2-q}\frac{q}{4C \vert h \vert _{ \frac{2}{2-q}}}>0. $$

Choose \(\rho _{k_{0,\lambda }}=t_{0,\lambda }\), then

$$ \begin{aligned} I(u)&\geq \frac{\rho ^{2}_{k_{0,\lambda }}}{4}\biggl(1-\rho ^{p-2}_{k_{0, \lambda }}- \frac{4C\lambda }{q} \vert h \vert _{\frac{2}{2-q}}\rho ^{q-2}_{k_{0, \lambda }} \biggr)\\ &:=\alpha _{k_{0,\lambda }}>0\quad \text{for } u\in Z_{k_{0}} \text{ with } \Vert u \Vert =\rho _{k_{0,\lambda }}. \end{aligned} $$

 □

Lemma 3.4

Assume that \((V)\), \((F_{1})\), \((F_{2})\) and \((H)\) hold, then, for any finite dimensional subspace \(\widetilde{E}\subset E\), there exists \(R(\widetilde{E})>0\) such that \(I(u)\leq 0\) for \(u\in \widetilde{E}\) with \(\|u\|\geq R(\widetilde{E})\).

Proof

By (3.1) and the equivalence of norms in the finite dimensional space Ẽ,

$$ \begin{aligned}[b] I(u) &\leq \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl(\nabla u^{2}+V(x)u^{2} \bigr)\,dx-\frac{ \omega }{2} \int _{\mathbb{R}^{3}}\phi _{u}u^{2} \,dx - \int _{\mathbb{R} ^{3}}{F}(x,u)\,dx \\ &\leq \frac{1}{2} \Vert u \Vert ^{2}+\frac{\omega ^{2}}{2} \vert u \vert ^{2} _{2}-c_{1} \vert u \vert ^{\mu }_{\mu }+c_{2} \vert u \vert ^{2}_{2} \\ &\leq \biggl(\frac{1}{2}+\frac{ \omega ^{2}}{2}+c_{2}\biggr) \Vert u \Vert ^{2}-c_{1}C \Vert u \Vert ^{\mu } \\ &\rightarrow - \infty ,\quad \text{as } \Vert u \Vert \rightarrow \infty . \end{aligned} $$

Therefore, there exists \(R(\widetilde{E})>0\) such that \(I(u)\leq 0\) for \(u\in \widetilde{E}\) with \(\|u\|\geq R(\widetilde{E})\). □

Lemma 3.5

Assume that \((V)\), \((F_{1})\), \((F_{2})\) and \((F_{4})\) hold, then there exists \(\lambda ^{*}>0\) such that

$$ \inf_{\|u\|=r_{\lambda }}I(u)>I(0)\geq I(e_{\lambda }), $$

for every \(\lambda \in (0,\lambda ^{*})\), and some \(r_{ \lambda }\in (0,\infty )\), \(e_{\lambda }\in E\) with \(\|e_{ \lambda }\|>r_{\lambda }\).

Proof

By \((F_{4})\), we can get

$$ \lim_{|t|\rightarrow 0}\frac{F(x,t)+\frac{V_{0}}{2}t^{2}}{t^{2}}=0,\quad \text{uniformly for } x \in \mathbb{R}^{3}. $$

Together with \((F_{1})\), for every \(\varepsilon >0\), there exists \(c(\varepsilon )>0\) such that

$$ \biggl\vert F(x,t)+\frac{V_{0}}{2}t^{2} \biggr\vert \leq \frac{\varepsilon }{2} \vert t \vert ^{2}+c( \varepsilon ) \vert t \vert ^{p},\quad \text{uniformly for } x\in \mathbb{R}^{3}. $$

By (i) of Lemma 2.2, we can get

$$\begin{aligned} I(u) &=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl(\nabla u^{2}+V(x)u^{2} \bigr)\,dx-\frac{ \omega }{2} \int _{\mathbb{R}^{3}}\phi _{u}u^{2} \,dx - \int _{\mathbb{R} ^{3}}{F}(x,u)\,dx-\frac{\lambda }{q} \int _{\mathbb{R}^{3}}h(x) \vert u \vert ^{q}\,dx \\ &\geq \frac{1}{2} \Vert u \Vert ^{2}-\frac{\varepsilon }{2} \vert u \vert ^{2}_{2}-c( \varepsilon ) \vert u \vert ^{p}_{p}-\lambda \vert h \vert _{\frac{2}{2-q}} \Vert u \Vert ^{q} \\ & \geq \frac{1}{2}(1-C_{2}\varepsilon ) \Vert u \Vert ^{2}-C_{p}c(\varepsilon ) \Vert u \Vert ^{p}- \lambda \vert h \vert _{\frac{2}{2-q}} \Vert u \Vert ^{q} \\ &\geq \frac{1}{4} \Vert u \Vert ^{2}\bigl(1-4C_{p}c( \varepsilon ) \Vert u \Vert ^{p-2}-4\lambda \vert h \vert _{\frac{2}{2-q}} \Vert u \Vert ^{q-2}\bigr)\quad (\text{where } \varepsilon \text{ is small enough}). \end{aligned}$$

Then, similar to the proof of Lemma 3.3, there exists \(\lambda ^{*}>0\) such that

$$ \inf_{ \Vert u \Vert =r_{\lambda }}I(u)\geq \frac{r_{\lambda }^{2}}{4}\bigl(1-4C_{p}c( \varepsilon )r_{\lambda }^{p-2}-4\lambda \vert h \vert _{\frac{2}{2-q}}r_{\lambda }^{q-2}\bigr)>0, $$

for every \(\lambda \in (0,\lambda ^{*})\) and some \(r_{\lambda }\in (0,+ \infty )\).

Similar to the proof of Lemma 3.4, for \(u\in E\) with \(\|u\|=1\),

$$ \begin{aligned}[b] I(tu) &\leq \frac{t^{2}}{2} \int _{\mathbb{R}^{3}}\bigl(\nabla u^{2}+V(x)u ^{2} \bigr)\,dx-\frac{\omega }{2} \int _{\mathbb{R}^{3}}\phi _{(tu)}(tu)^{2} \,dx - \int _{\mathbb{R}^{3}}{F}(x,tu)\,dx \\ &\leq \frac{t^{2}}{2} \Vert u \Vert ^{2}+\frac{(t \omega )^{2}}{2} \vert u \vert ^{2}_{2}-c_{1}t^{\mu } \vert u \vert ^{\mu }_{\mu }+c_{2}t^{2} \vert u \vert ^{2} _{2} \\ &\rightarrow -\infty ,\quad \text{as } t\rightarrow \infty . \end{aligned} $$

Thus there exists \(T>r_{\lambda }\) such that \(I(Tu)\leq 0\). By choosing \(e_{\lambda }=Tu\), the proof is completed. □

Proof of Theorem 1.1

By \((F_{1})\) and \((F_{3})\), it is easy to see that \(I(0)=0\) and I is even. Together with Lemma 3.2–3.4, we can see that all the conditions of Lemma 2.4 are satisfied. Thus problem (Pλ) has a sequence of weak solutions \(\{(u_{n},\phi _{n}) \}\subset E \times D^{1,2} (\mathbb{R}^{3})\) satisfying

$$ \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl(\nabla u^{2}_{n}+V(x)u^{2}_{n} \bigr)\,dx-\frac{ \omega }{2} \int _{\mathbb{R}^{3}}\phi _{u_{n}}u_{n}^{2} \,dx - \int _{\mathbb{R}^{3}}{F}(x,u_{n})\,dx\rightarrow +\infty ,\quad \text{as } n\to \infty , $$

for every \(\lambda \in (0,\lambda _{*})\). □

Next, we will give the proof of Theorem 1.2.

Proof of Theorem 1.2

Lemma 3.5 implies that the functional I enjoys the mountain pass structure. from Lemma 3.2, I satisfies the (PS) condition. Hence by Lemma 2.3, problem (Pλ) has at least one nontrivial solution for every \(\lambda \in (0,\lambda ^{*})\). □