Nonexistence and existence of nontrivial solutions for Klein–Gordon–Maxwell systems with competing nonlinearities

Abstract

In this paper, we study the following nonlinear Klein–Gordon–Maxwell system:

$$\begin{aligned} \textstyle\begin{cases} -\Delta u+ u-(2\omega +\phi )\phi u =\lambda a(x) \vert u \vert ^{r-2}u-b(x) \vert u \vert ^{q-2}u, & x\in \mathbb{R}^{3},\\ \Delta \phi = (\omega +\phi )u^{2}, & x\in \mathbb{R}^{3}, \end{cases}\displaystyle \hspace{75pt}(\mathrm{P}) \end{aligned}$$

where ω is a positive constant, $q>2$, $r\in (2,\min \{6,q\})$, $a\in L^{\frac{6}{6-r}}(\mathbb{R}^{3})$ is a positive potential, $b\in L^{1}_{\mathrm{loc}}(\mathbb{R}^{3})$ is also a positive potential. Under some integrability assumption on $\frac{a^{q-2}}{b^{r-2}}$, nonexistence and existence results are obtained depending on λ via variational methods.

1 Introduction and main results

In this paper, we deal with a class of Klein–Gordon–Maxwell systems with a type of concave–convex nonlinearities via variational methods. More precisely, the problem that we study is as follows:

$$ \textstyle\begin{cases} -\Delta u+ u-(2\omega +\phi )\phi u =\lambda a(x) \vert u \vert ^{r-2}u-b(x) \vert u \vert ^{q-2}u, & x\in \mathbb{R}^{3}, \\ \Delta \phi = (\omega +\phi )u^{2}, & x\in \mathbb{R}^{3}, \end{cases}\displaystyle \hspace{76pt}(\mathrm{P}) $$

where ω is a positive constant, $q>2$, $r\in (2,\min \{6,q\})$, the potential functions a and b satisfy the following assumptions.

• (a) $a\in L^{\frac{6}{6-r}}(\mathbb{R}^{3})$ is a positive potential function.

• (b) $b\in L^{1}_{\mathrm{loc}}(\mathbb{R}^{3})$ is a positive potential function.

• (c) $a(\frac{a}{b})^{\frac{r-2}{q-r}}\in L^{\frac{3}{2}}( \mathbb{R}^{3})$.

As a model describing solitary waves for the nonlinear stationary Klein–Gordon equation in the three-dimensional space interacting with the electrostatic field, the following Klein–Gordon–Maxwell system was first introduced in [1]:

$$ \textstyle\begin{cases} -\Delta u+ [m_{0}^{2}-(\omega +e\phi )^{2}] u = \vert u \vert ^{p-2}u, & x\in \mathbb{R}^{3}, \\ -\Delta \phi + e^{2}u^{2}\phi =-\omega eu^{2}, & x\in \mathbb{R}^{3}, \end{cases}\displaystyle \hspace{124pt}(\mathrm{KGM}) $$

where $m_{0}$ and e are the mass and the charge of the particle, respectively, while ω denotes the phase. The unknowns of the system are the field u associated with the particle and the electric potential ϕ. The presence of the nonlinear term simulates the interaction between many particles or external nonlinear perturbations. By variational methods, infinitely many solitary wave solutions were got for the above system when $\vert m_{0}\vert >\vert \omega \vert$, $p\in (4,6)$ in [1]. Furthermore, in [2], infinitely many finite energy radial solutions were got if one of the following conditions holds:

• (i) $m_{0}>\omega >0$ and $p\in (4,6)$;

• (ii) $m_{0}\sqrt{\frac{p-2}{2}}>\omega >0$ and $p\in (2,4)$.

For $p\in (2, 4)$, the existence range of $(m_{0},\omega )$ was improved and a limit case $m_{0}=\omega$ was also dealt with by Azzollini, Pisani, and Pomponio in [3]. Mugnai in [4] studied the existence of radially symmetric solitary waves for a system of a nonlinear Klein–Gordon equation coupled with Maxwell’s equation in presence of a positive mass. Ground state solutions [5, 6], semiclassical states [7], nonradial solutions [8], and the critical exponent case [9,10,11,12] have also been considered. When $p\geq 6$ and $m_{0}\geq \omega >0$ or $p \leq 2$, no-existence result of $(\mathrm{KGM})$ was proved by D’Aprile and Mugnai in [13]. In [14], via Ekeland’s variational principle and mountain pass theorem, two nontrivial solutions for a nonhomogeneous Klein–Gordon–Maxwell system were got by Chen and Tang. In [15], Jeong and Seok established an abstract critical point theorem about a functional of the mountain-pass type with a small perturbation for the nonlocal term and studied a type of Klein–Gordon–Maxwell system with a very general nonlinear term. Klein–Gordon–Maxwell system with a coercive potential was firstly considered by He [16]. In [16], infinitely many solutions were got via a variant fountain theorem and symmetric mountain pass theorem. In [17], Li and Tang improved and complemented the results in [16]. In [18], under a variant 4-superlinear condition, infinitely many solutions for a nonlinear Klein–Gordon–Maxwell system with sign-changing potential were got by Ding and Li via symmetric mountain pass theorem. In [19], the authors studied the existence of cylindrically symmetric electro-magneto-static solitary waves for a system of a nonlinear Klein–Gordon equation coupled with Maxwell’s equations in presence of a positive mass and a nonnegative nonlinear potential. Nonexistence results were got as well.

As to the concave–convex nonlinearities, there is a huge amount of results involving different operators, for example, [20,21,22,23,24,25,26,27,28,29,30,31,32,33,34] and the references therein. Please forgive us for not being able to list all the literature. The first paper is the seminal one by Ambrosetti, Brezis, and Cerami [20], where they considered a concave–convex Dirichlet problem

$$ \textstyle\begin{cases} -\Delta u=\lambda u^{q-1}+u^{p-1}, & x\in \varOmega , \\ u>0, & x\in \varOmega , \\ u=0,& x\in \partial \varOmega , \end{cases}\displaystyle \hspace{193pt}(\mathrm{P}_{\lambda }) $$

where $1< q<2<\frac{2N}{N-2}$, $\lambda >0$, and $\varOmega \subset \mathbb{R}^{N}$ is a smooth bounded domain. They proved that there exists $\lambda _{0} > 0$ such that problem $(\mathrm{P}_{\lambda })$ has at least two solutions for all $\lambda \in (0,\lambda _{0})$, one solution for $\lambda = \lambda _{0}$, and no solution for $\lambda > \lambda _{0}$. In [30], a nonlocal version of problem $(\mathrm{P}_{\lambda })$ was considered

$$ \textstyle\begin{cases} (-\Delta )^{\frac{\alpha }{2}} u=\lambda u^{q}+u^{p}, & x\in \varOmega , \\ u>0, & x\in \varOmega , \\ u=0,& x\in \partial \varOmega , \end{cases} $$

with $0<\alpha <2$, $0< q<1<p<\frac{N+\alpha }{N-\alpha }$, $N>\alpha$, $\lambda >0$, and $\varOmega \subset \mathbb{R}^{N}$ is a smooth bounded domain. The authors also characterized completely the range of parameters for which solutions of the problem exist and proved a multiplicity result. In [22], Alama and Tarantello studied the following Dirichlet problem:

$$ \textstyle\begin{cases} -\Delta u=\lambda u+k(x)u^{q}-h(x)u^{p}, & x\in \varOmega , \\ u>0, & x\in \varOmega , \\ u=0,& x\in \partial \varOmega , \end{cases} $$

where $1< q< p$, $\lambda \in \mathbb{R}$, $N\geq 3$, $\varOmega \subset \mathbb{R}^{N}$ is a bounded open set with smooth boundary and $h, k \in L^{\infty }(\varOmega )$ are nonnegative functions. Existence, nonexistence, and multiplicity results were obtained depending on λ and according to the integrability properties of the ratio $\frac{k^{p-1}}{h^{q-1}}$. In [25], Pucci and Rădulescu considered the following quasilinear problem:

$$ \textstyle\begin{cases} -\operatorname{div}( \vert \nabla u \vert ^{m-2}\nabla u)+a(x) \vert u \vert ^{m-2} u=\lambda w(x) \vert u \vert ^{q-2} u-h(x) \vert u \vert ^{p-2}u, & x\in \mathbb{R}^{N}, \\ u\geq 0, & x\in \mathbb{R}^{N}, \end{cases} $$

where h is a positive continuous function with $\int _{\mathbb{R}^{N}}h(x)^{\frac{q}{q-p}}\,dx<\infty$, $\lambda >0$, $2\leq m< q< m^{*}$ with $m^{*}=\frac{Nm}{N-m}$ if $N>m$ and $m^{*}= \infty$ if $N\leq m$. The nonexistence of nontrivial solutions was got for λ small enough; the existence of at least two nontrivial solutions was obtained for λ large enough. The above results were extended by Autuori and Pucci in [26]. They studied the following quasilinear elliptic equation:

$$ -\operatorname{div}\mathbf{A}(x,\nabla u)+a(x) \vert u \vert ^{p-2} u=\lambda w(x) \vert u \vert ^{q-2} u-h(x) \vert u \vert ^{r-2}u, \quad x\in \mathbb{R}^{N}, $$

which involves a general elliptic operator in divergence form A and two competing nonlinearities. $\max \{2,p\}< q<\min \{r,\frac{Np}{N-p}\}$, the coefficients w and h are related by the integrability condition

$$ \int _{\mathbb{R}^{N}} \biggl(\frac{w^{r}(x)}{h^{q}(x)} \biggr)^{ \frac{1}{r-q}}\,dx \in \mathbb{R}^{+}. $$

(1.1)

Moreover, they proposed two open questions: the relaxation of $\max \{2, p\} < q$ and the replacement of the above integrability condition by the assumption that

$$ w\biggl(\frac{w}{h}\biggr)^{\frac{q-p}{r-q}}\in L^{\frac{N}{p}}\bigl( \mathbb{R}^{N}\bigr). $$

(1.2)

The latter request is weaker than (1.1) which already appeared in [22]. In [28], Autuori and Pucci studied the following elliptic equation involving fractional Laplacian:

$$ (-\Delta )^{s}u +a(x)u =\lambda w(x) \vert u \vert ^{p-2}u-h(x) \vert u \vert ^{r-2}u, \quad x\in \mathbb{R}^{N}, $$

where $\lambda \in \mathbb{R}$, $0< s<1$, $2s< N$, $2< q<\min \{r,2^{*}_{s} \}$ with $2^{*}_{s}=\frac{2N}{N-2s}$. Positive weights w and h are related by condition (1.1) and

$$ w\in L^{\frac{2^{*}_{s}}{2^{*}_{s}-q}}\bigl(\mathbb{R}^{N}\bigr)\cap L_{\mathrm{loc}} ^{\sigma }\bigl(\mathbb{R}^{N}\bigr), \quad \quad \sigma > \frac{2^{*}_{s}}{2^{*}_{s}-q}, \quad \quad h \in L_{\mathrm{loc}}^{1}\bigl( \mathbb{R}^{N}\bigr). $$

(1.3)

In [28], the existence and multiplicity of entire solutions were obtained via variational methods. Recently, in [31] Pucci and Zhang solved the above open problems mentioned in [26] for a class of quasilinear elliptic equations in the setting of variable exponents. More recently, Pucci, Xiang, and Zhang in [32] also gave a positive answer to these open problems in the context of Kirchhoff problems involving the fractional p-Laplacian. In [33], Xiang, Zhang, and Rădulescu studied a perturbed nonlinear elliptic equation driven by the fractional p-Laplacian operator

$$ (-\Delta )_{p}^{s}u +V(x) \vert u \vert ^{p-2}u =\lambda a(x) \vert u \vert ^{r-2}u-b(x) \vert u \vert ^{q-2}u, \quad x\in \mathbb{R}^{N}, $$

where λ is a real parameter, $(-\Delta )_{p}^{s}$ is the fractional p-Laplacian operator with $0 < s<1 <p<\infty$, $p< r<\min \{q, p^{*}_{s}\}$, $p^{*}_{s}=\frac{pN}{N-ps}$, and V is a positive weight with positive infimum, a is a positive weight satisfying

$$ a\in L^{\frac{p^{*}_{s}}{p^{*}_{s}-q}}\bigl(\mathbb{R}^{N}\bigr)\cap L_{\mathrm{loc}} ^{1}\bigl(\mathbb{R}^{N}\bigr), $$

(1.4)

$b\in L_{\mathrm{loc}}^{1}(\mathbb{R}^{N})$ is also a positive weight, a and b are related by

$$ a\biggl(\frac{a}{b}\biggr)^{\frac{r-p}{q-r}}\in L^{\frac{N}{ps}}\bigl( \mathbb{R}^{N}\bigr). $$

(1.5)

Nonexistence and multiplicity results for the above-mentioned equation were obtained by variational methods depending on λ and according to the integrability properties of the ratio $\frac{a^{q-p}}{b ^{r-p}}$. Those results extended the previous work of Autuori and Pucci [28] to the fractional p-Laplacian setting and also solved the above open problems mentioned in [26].

As far as we know, there are few results about Klein–Gordon–Maxwell systems with similar competing nonlinearities to our system $(\mathrm{P})$. Generally, system $(\mathrm{P})$ can be transformed into a single equation with a nonlocal term by dual methods (see Sect. 2). But in contrast to the problems with a purely single equation (similar to [25, 26, 33]), the nonlocal term brings about some difficulties to us. Firstly, the functional associated with system $(\mathrm{P})$ is no longer weakly lower semicontinuous, which is very important to get the global minimizer in [26, 28, 33]. Secondly, since we lack the compact embedding $H^{1}(\mathbb{R}^{3})\hookrightarrow \hookrightarrow L^{p}(\mathbb{R}^{3})$, $p\in (2,6)$, it increases difficulty in verifying that the functional associated with system $(\mathrm{P})$ satisfies the Palais–Smale condition.

The main result of our paper reads as follows.

Theorem 1.1

Let (a)–(c) hold, then there exist $\lambda _{0}$ and $\lambda ^{*}$ with $0<\lambda _{0}\leq \lambda ^{*}$ such that

• (i) system $(\mathrm{P})$ has only the trivial solution for $\lambda < \lambda _{0}$;

• (ii) system $(\mathrm{P})$ has at least two weak nontrivial solutions for $\lambda >\lambda ^{*}$.

Remark 1.1

Our assumption (b) is the same condition as in [28, 33]. Our assumption (c) is (1.5) for $N=3$, $P=2$, $s=1$. Different from (1.3) or (1.4), the local integrability hypothesis of a is not necessary in our paper.

Remark 1.2

If $r<2$, for every $\lambda >0$, $q>1$, it is easy to prove that the functional associated with system $(\mathrm{P})$ is coercive. Similar to our Lemma 3.2, the functional associated with system $(\mathrm{P})$ also satisfies the Palais–Smale condition. Then the functional has a global minimizer via Ekeland’s variational principle and a sequence of solutions with negative energy decreasing to zero via Clark’s theorem.

Throughout the paper, we denote by C various positive constants, whose value may be different from line to line and is not essential to the problem.

2 Preliminary

In this section, we give some preliminary results which will be used to prove our main results.

First of all, we establish the variational framework for system $(\mathrm{P})$.

$H^{1}(\mathbb{R}^{3})=\{u\in L^{2}(\mathbb{R}^{3})\vert \vert \nabla u\vert \in L^{2}(\mathbb{R}^{3})\}$ is the normal Sobolev space with the norm

$$\begin{aligned}& \Vert u \Vert _{H^{1}(\mathbb{R}^{3})}= \biggl( \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+u ^{2}\bigr)\,dx \biggr)^{\frac{1}{2}}, \\& L^{p}\bigl(\mathbb{R}^{3}\bigr):= \biggl\{ u: \mathbb{R}^{3}\mapsto \mathbb{R}\Big|u \text{ is Lebesgue measurabe}, \int _{\mathbb{R}^{3}} \vert u \vert ^{p} \,dx< \infty \biggr\} \end{aligned}$$

equipped with the norm

$$ \vert u \vert _{p}=\biggl( \int _{\mathbb{R}^{3}} \vert u \vert ^{p} \,dx \biggr)^{\frac{1}{p}}. $$

Define

$$ L^{r}\bigl(\mathbb{R}^{3}, a\bigr):= \biggl\{ u: \mathbb{R}^{3}\mapsto \mathbb{R}\Big| u \text{ is Lebesgue measurabe}, \int _{\mathbb{R}^{3}}a(x) \vert u \vert ^{r} \,dx< \infty \biggr\} $$

equipped with the norm

$$ \vert u \vert _{L^{r}(\mathbb{R}^{3}, a)}=\biggl( \int _{\mathbb{R}^{3}}a(x) \vert u \vert ^{r} \,dx \biggr)^{ \frac{1}{r}}. $$

Similarly,

$$ L^{q}\bigl(\mathbb{R}^{3}, b\bigr):= \biggl\{ u: \mathbb{R}^{3}\mapsto \mathbb{R}\Big| u \text{ is Lebesgue measurabe}, \int _{\mathbb{R}^{3}}b(x) \vert u \vert ^{q} \,dx< \infty \biggr\} $$

equipped with the norm

$$ \vert u \vert _{L^{r}(\mathbb{R}^{3}, b)}=\biggl( \int _{\mathbb{R}^{3}}b(x) \vert u \vert ^{q} \,dx \biggr)^{ \frac{1}{q}}. $$

The spaces $L^{r}(\mathbb{R}^{3}, a)$ and $L^{q}(\mathbb{R}^{3}, b)$ are uniformly convex Banach spaces by Proposition A.6 of [26].

The working space of our paper E is the completion of $C^{\infty } _{0}(\mathbb{R}^{3})$ with respect to the norm

$$ \Vert u \Vert _{E}= \Vert u \Vert _{H^{1}(\mathbb{R}^{3})}+ \vert u \vert _{L^{r}(\mathbb{R}^{3}, b)}. $$

E is a reflexive Banach space (see Lemma 2.2 in [33]). Furthermore, $E\hookrightarrow H^{1}(\mathbb{R}^{3})\hookrightarrow L ^{p}(\mathbb{R}^{3})$, $p\in [2,6]$.

Since $2<\frac{12}{5}<6$, thus for $u\in E$ fixed, the linear operator $T_{u}:D^{1,2}(\mathbb{R}^{3})\rightarrow \mathbb{R}$ defined by

$$ T_{u}(v):= \int _{\mathbb{R}^{3}}u^{2}v\,dx $$

is continuous in $D^{1,2}(\mathbb{R}^{3})$. By Lax–Milgram theorem, there exists $\phi _{u}\in D^{1,2}(\mathbb{R}^{3})$ such that, for any $v\in D^{1,2}(\mathbb{R}^{3})$,

$$ \int _{\mathbb{R}^{3}}\bigl(\nabla \phi _{u} \nabla v+u^{2}\phi _{u}v\bigr) \,dx= \int _{\mathbb{R}^{3}}u^{2}v\,dx. $$

Therefore, problem $(\mathrm{P})$ can be transformed into a nonlinear Schrödinger equation with a nonlocal term

$$ -\Delta u+ u-(2\omega +\phi _{u})\phi _{u} u = \lambda a(x) \vert u \vert ^{r-2}u-b(x) \vert u \vert ^{q-2}u, \quad x\in \mathbb{R}^{3}. \hspace{74pt}(\mathrm{P}') $$

The functional associated with $(\mathrm{P}')$ is given by

$$ I(u)=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+u^{2} \bigr)\,dx-\frac{ \omega }{2} \int _{\mathbb{R}^{3}}\phi _{u}u^{2} \,dx + \frac{1}{q} \int _{\mathbb{R}^{3}}b(x) \vert u \vert ^{q}\,dx- \frac{\lambda }{r} \int _{\mathbb{R} ^{3}}a(x) \vert u \vert ^{r}\,dx. $$

Under our assumptions, it is easy to get that $I\in C^{1}(E, \mathbb{R})$ and

$$ \begin{aligned} \bigl\langle I'(u),v\bigr\rangle &= \int _{\mathbb{R}^{3}}(\nabla u\nabla v+uv)\,dx- \int _{\mathbb{R}^{3}}(2\omega +\phi _{u})\phi _{u}uv \,dx \\ &\quad{} + \int _{\mathbb{R}^{3}}b(x) \vert u \vert ^{q-2}uv \,dx-\lambda \int _{\mathbb{R}^{3}}a(x) \vert u \vert ^{r-2}uv \,dx \end{aligned} $$

for $v\in E$.

Moreover, the function $\phi _{u}$ has the following properties.

Lemma 2.1

(see [1, 2])

• (i) $-\omega \leq \phi _{u}\leq 0$ on the set $\{x\vert u(x) \neq 0\}$;

• (ii) There exist positive constants $C_{1}$, $C_{2}$ such that

  $$ \Vert \phi _{u} \Vert _{D^{1,2}}\leq C_{1} \Vert u \Vert ^{2} \quad \textit{and} \quad \int _{\mathbb{R}^{3}} \vert \phi _{u} \vert u^{2} \,dx\leq C_{2} \Vert u \Vert ^{4}. $$

Definition 2.1

Let X be a Banach space, we say that functional $I\in C^{1}(X, \mathbb{R})$ satisfies the Palais–Smale condition at the level $c\in \mathbb{R}$ ((PS)_c in short) if any sequence $\{u_{n}\} \subset X$ satisfying $I(u_{n}) \to c$, $I'(u_{n}) \to 0$ as $n\to \infty$, has a convergent subsequence. I satisfies the (PS) condition if I satisfies the (PS)_c condition at any $c\in \mathbb{R}$.

In order to get the global minimizer, we need the famous Ekeland variational principle.

Lemma 2.2

(Ekeland’s variational principle, [35])

Let $(X,d)$ be a complete space, and let $F:X\rightarrow \mathbb{R}\cup \{+\infty \}$ be a lower semicontinuous functional on X that is bounded below and not identically equal to +∞. Fix $\varepsilon >0$ and a point $u\in X$ such that

$$ F(u)\leq \varepsilon +\inf_{x\in X}F(x). $$

Then there exists a point $v\in X$ such that

$$ F(v)\leq F(u), \quad\quad d(u,v)\leq 1, \quad \textit{and}\quad F(w)>F(v)- \varepsilon d(v,w) \quad \textit{for all } w\neq v. $$

In order to get the second nontrivial solution, we need a modification of mountain pass theorem.

Lemma 2.3

(Theorem A.3 in [26])

Let $(X, \Vert \cdot \Vert )$ and $(Y, \Vert \cdot \Vert )$ be two Banach spaces such that X can be continuously embedded into Y. Let $\varPhi : X\rightarrow \mathbb{R}$ be a $C^{1}$ functional with ${\varPhi (0)=0}$. Suppose that there exist $\rho , \alpha >0$ and $e\in X$ such that $\Vert e\Vert _{Y}>\rho$, $\varPhi (e)< \alpha$ and $\varPhi (u)\geq \alpha$ for all $u\in X$ with $\Vert u\Vert _{Y}= \rho$. Then there exists a sequence $\{u_{n}\}\subset X$ such that, for all n,

$$ c\leq \varPhi (u_{n})\leq c+\frac{1}{n}, \quad \textit{and} \quad \bigl\Vert \varPhi '(u_{n}) \bigr\Vert _{X'}\leq \frac{2}{n}, $$

where $c= \inf_{\gamma \in \varGamma }\max_{t\in [0,1]}\varPhi (\gamma (t))$ and $\varGamma =\{\gamma \in C([0,1]; X)\vert \gamma (0)=0, \gamma (1)=e\}$.

3 Proof of the main results

In this section we will prove our main results. Firstly, in the same spirit of the proof of Lemma 3.1 in [33], we can also get the functional I is coercive. And since $a\in L^{\frac{6}{6-r}}( \mathbb{R}^{3})$ implies $a^{\frac{6}{6-r}}\in L^{1}_{\mathrm{loc}}(\mathbb{R} ^{3})$, the assumption $a\in L_{\mathrm{loc}}^{1}(\mathbb{R}^{3})$ is not necessary in this proof.

Lemma 3.1

(Lemma 3.1 in [33])

Assume that (a)–(c) hold, the functional I is coercive and bounded from below in E.

In contrast to [33], problem $(\mathrm{P}')$ contains a nonlocal term $\phi _{u}$. It brings about some difficulties to us. Since we lack the compact embedding $H^{1}(\mathbb{R}^{3})\hookrightarrow \hookrightarrow L^{p}(\mathbb{R}^{3})$, $p\in (2,6)$, it increases difficulty in verifying that the functional satisfies the Palais–Smale condition. Here we prove that the functional satisfies the Palais–Smale condition under the integrability assumptions on a and b.

Lemma 3.2

Assume that (a)–(c) hold, the functional I satisfies the Palais–Smale condition.

Proof

Let $\{u_{n}\}$ be a (PS) sequence of I, that is, for some $M>0$,

$$ \bigl\vert I(u_{n}) \bigr\vert \leq M, \quad\quad I'(u_{n})\to 0, \quad \text{in } E. $$

Since I is coercive in E, all the (PS) sequences of I are bounded. Then there exists $u\in E$, and up to a subsequence if necessary, we still denote it by $\{u_{n}\}$ satisfying

• $u_{n}\rightharpoonup u$ in $H^{1}(\mathbb{R}^{3})$, $u_{n}\rightharpoonup u$ in E;

• $u_{n}\rightarrow u$ in $L_{\mathrm{loc}}^{s}(\mathbb{R}^{3})$, where $s\in [2,6)$; $u_{n}(x)\rightarrow u(x)$ a.e. in $\mathbb{R}^{3}$;

• and $\vert u_{n}\vert ^{r}\rightharpoonup \vert u\vert ^{r}$, $\vert u_{n}\vert ^{r-2}u_{n}u \rightharpoonup \vert u\vert ^{r}$, $\vert u\vert ^{r-2}uu_{n}\rightharpoonup \vert u\vert ^{r}$ in $L^{\frac{6}{r}}(\mathbb{R}^{3})$.

For every fixed $R\geq 1$, let $\xi _{R}\in C^{\infty }(\mathbb{R}^{3}, [0,1])$ such that

$$ \xi _{R}(x)= \textstyle\begin{cases} 0 , & \text{for } \vert x \vert \leq \frac{R}{2}, \\ 1, & \text{for } \vert x \vert \geq R, \end{cases} $$

and $\vert \nabla \xi _{R}\vert \leq \frac{1}{R}$. Then there exists a positive constant C independent of R such that $\Vert \xi _{R}u\Vert _{E}\leq C\Vert u \Vert _{E}$. Therefore, $\lim_{n\rightarrow \infty }\langle I'(u_{n}),\xi _{R}u_{n}\rangle =0$, uniformly for $R\geq 1$, that is,

$$ \begin{aligned} o_{n}(1) &= \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u_{n} \vert ^{2}+u_{n}^{2}-(2\omega +\phi _{u_{n}})\phi _{u_{n}}u^{2}_{n}\bigr)\xi _{R} \,dx+ \int _{\mathbb{R}^{3}}u _{n}\nabla u_{n}\nabla \xi _{R} \,dx \\ &\quad {} + \int _{\mathbb{R}^{3}}b(x) \vert u_{n} \vert ^{q} \xi _{R}\,dx-\lambda \int _{\mathbb{R}^{3}}a(x) \vert u_{n} \vert ^{r} \xi _{R}\,dx. \end{aligned} $$

Since

$$ \biggl\vert \int _{\mathbb{R}^{3}}u_{n}\nabla u_{n}\nabla \xi _{R} \,dx \biggr\vert \leq \int _{\mathbb{R}^{3}} \vert u_{n} \vert \vert \nabla u_{n} \vert \vert \nabla \xi _{R}\vert \,dx\leq \frac{1}{R} \Vert u_{n} \Vert _{E}^{2} \leq \frac{C}{R} $$

and

$$ \int _{\mathbb{R}^{3}}a(x) \vert u_{n} \vert ^{r} \xi _{R}\,dx\leq \int _{B^{c}_{\frac{R}{2}}(0)}a(x) \vert u_{n} \vert ^{r} \xi _{R}\,dx\leq C \vert a \vert _{L^{ \frac{6}{6-r}}(B^{c}_{\frac{R}{2}}(0))} \Vert u_{n} \Vert _{E} ^{r}\leq C \vert a \vert _{L^{ \frac{6}{6-r}}(B^{c}_{\frac{R}{2}}(0))}, $$

by (i) of Lemma 2.1, we can get that

$$ \int _{B^{c}_{R}(0)}\bigl( \vert \nabla u_{n} \vert ^{2}+u_{n}^{2}+b(x) \vert u_{n} \vert ^{q}\bigr) \,dx \leq \frac{C}{R}+C \vert a \vert _{L^{\frac{6}{6-r}}(B^{c}_{\frac{R}{2}}(0))}+o _{n}(1). $$

Then, for every $\varepsilon >0$, there exist $N>0$ and $R>0$ such that, for every $n\geq N$,

$$ \int _{B^{c}_{R}(0)}\bigl( \vert \nabla u_{n} \vert ^{2}+u_{n}^{2}+b(x) \vert u_{n} \vert ^{q}\bigr) \,dx \leq \frac{\varepsilon }{2}. $$

(3.1)

Since $\{u_{n}\}$ is bounded in E, by (3.1) and the compact embedding $E\hookrightarrow L^{p}_{\mathrm{loc}}(\mathbb{R}^{3})$ for $p\in [2,6)$, we can get that $u_{n}\rightarrow u$ in $L^{2}(\mathbb{R}^{3})$. Furthermore, by the interpolation inequality, we have

$$ u_{n}\rightarrow u, \quad \text{in } L^{p}\bigl( \mathbb{R}^{3}\bigr)\text{ for } p\in [2,6). $$

The rest of proof is standard. In fact,

$$ \begin{aligned}[b] & \Vert u_{n}-u \Vert _{H^{1}(\mathbb{R}^{3})}^{2}+ \int _{\mathbb{R}^{3}}b(x) \bigl( \vert u _{n} \vert ^{q-2}u_{n}- \vert u \vert ^{q-2}u\bigr) (u_{n}-u)\,dx \\ &\quad =\bigl\langle I'(u_{n}), u_{n}-u \bigr\rangle - \bigl\langle I'(u), u_{n}-u \bigr\rangle \\ &\quad\quad {} +2\omega \int _{\mathbb{R}^{3}}(\phi _{u_{n}}u_{n}-\phi _{u}u) (u_{n}-u)\,dx+ \int _{\mathbb{R}^{3}}\bigl(\phi _{u_{n}}^{2}u_{n}- \phi _{u}^{2}u\bigr) (u_{n}-u)\,dx \\ &\quad\quad {} +\lambda \int _{\mathbb{R}^{3}}a(x) \bigl( \vert u_{n} \vert ^{r-2}u_{n}- \vert u \vert ^{r-2}u\bigr) (u _{n}-u)\,dx. \end{aligned} $$

(3.2)

It follows from $I'(u_{n})\rightarrow 0$ and $u_{n}\rightharpoonup u$ in E that the first two parts on the right-hand side of (3.2) converge to zero as $n\rightarrow \infty$. By Hölder’s inequality, we have

$$ \begin{aligned}[b] & \biggl\vert \int _{\mathbb{R}^{3}}(\phi _{u_{n}}u_{n}-\phi _{u}u) (u_{n}-u) \,dx \biggr\vert \\ &\quad = \biggl\vert \int _{\mathbb{R}^{3}}\phi _{u_{n}}(u_{n}-u)^{2} \,dx + \int _{\mathbb{R} ^{3}}(\phi _{u_{n}}-\phi _{u})u(u_{n}-u) \,dx \biggr\vert \\ &\quad \leq \vert \phi _{u_{n}} \vert _{6} \vert u_{n}-u \vert ^{2}_{\frac{12}{5}} + \vert \phi _{u_{n}}-\phi _{u} \vert _{6} \vert u \vert _{\frac{12}{5}} \vert u_{n}-u \vert _{\frac{12}{5}} \\ &\quad \leq C \bigl( \vert u_{n}-u \vert ^{2} _{\frac{12}{5}} + \vert u \vert _{\frac{12}{5}} \vert u_{n}-u \vert _{\frac{12}{5}}\bigr) \\ & \quad \to 0, \quad n\rightarrow \infty . \end{aligned} $$

(3.3)

Since the sequence $\{\phi _{u_{n}}^{2}u_{n}\}$ is bounded in $L^{\frac{3}{2}}(\mathbb{R}^{3})$,

$$ \begin{aligned}[b] & \biggl\vert \int _{\mathbb{R}^{3}}\bigl(\phi _{u_{n}}^{2}u_{n}- \phi _{u}^{2}u\bigr) (u_{n}-u)\,dx \biggr\vert \\ &\quad \leq \bigl\vert \phi _{u_{n}}^{2}u_{n}-\phi _{u}^{2}u \bigr\vert _{\frac{3}{2}} \vert u_{n}-u \vert _{3} \\ &\quad \leq \bigl( \bigl\vert \phi _{u_{n}}^{2}u_{n} \bigr\vert _{\frac{3}{2}}+ \bigl\vert \phi _{u}^{2}u \bigr\vert _{ \frac{3}{2}}\bigr) \vert u_{n}-u \vert _{3} \\ &\quad \rightarrow 0, \quad n\rightarrow \infty . \end{aligned} $$

(3.4)

Then (3.3) and (3.4) imply that the third part and the fourth part on the right-hand side of (3.2) also converge to zero as $n\rightarrow \infty$. For the last part, since $\vert u_{n}\vert ^{r}\rightharpoonup \vert u\vert ^{r}$, $\vert u_{n}\vert ^{r-2}u_{n}u\rightharpoonup \vert u\vert ^{r}$, $\vert u\vert ^{r-2}uu_{n}\rightharpoonup \vert u\vert ^{r}$ in $L^{\frac{6}{r}}(\mathbb{R}^{3})$ and $a\in L^{ \frac{6}{6-r}}(\mathbb{R}^{3})$, we have

$$ \int _{\mathbb{R}^{3}}a(x) \bigl( \vert u_{n} \vert ^{r-2}u_{n}- \vert u \vert ^{r-2}u\bigr) (u_{n}-u)\,dx \rightarrow 0, \quad n\rightarrow \infty . $$

Thus all the parts of on the right-hand side of (3.2) converge to zero as $n\rightarrow \infty$. By the well-known inequality $(\vert s\vert ^{p-2}s-\vert t\vert ^{p-2}t)(s-t) \geq C_{p}\vert s-t\vert ^{p}$, for $s,t\in \mathbb{R}$ and $p \geq 2$, we can get that

$$ \Vert u_{n}-u \Vert _{E}\rightarrow 0, \quad n \rightarrow \infty . $$

Therefore, I satisfies the Palais–Smale condition. □

Though the functional I is no longer weakly lower semicontinuous, which is very important to get the global minimizer in [26, 28, 33], we can still prove that I still has a global minimizer in E via Ekeland’s variational principle.

Lemma 3.3

There exists $\lambda ^{*}>0$ such that I enjoys a global minimizer $u_{\lambda }^{1}\in E$ with $I(u_{\lambda }^{1})<0$ for every $\lambda >\lambda ^{*}$.

Proof

From Lemma 3.1, I is coercive and bounded from below. Set

$$ m_{\lambda }:=\inf_{u\in E}I(u)< +\infty $$

for every $\{u_{n}\}\subset E$ such that $I(u_{n})\rightarrow m_{ \lambda }$ as $n\rightarrow \infty$, we can get I has a (PS)$_{m _{\lambda }}$ sequence for every fixed $\lambda >0$ by Lemma 2.2 (Ekeland’s variational principle). Thus, Lemma 3.2 implies that I has a global minimizer in E. That is to say, there exists $u_{\lambda } ^{1}\in E$ such that

$$ I\bigl(u_{\lambda }^{1}\bigr)=\inf_{u\in E}I(u). $$

Next, we will prove that there exists $\lambda ^{*}>0$ such that $u_{\lambda }^{1}\neq 0$ for $\lambda >\lambda ^{*}$. Set

$$ \lambda ^{*}=\inf_{u\in \mathcal{M}}\biggl(\frac{r}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+ \bigl(1+\omega ^{2}\bigr)u^{2}\bigr)\,dx+\frac{r}{q} \int _{\mathbb{R}^{3}}b(x) \vert u \vert ^{q}\,dx\biggr)= \inf_{u\in \mathcal{M}}rJ(u), $$

where $\mathcal{M}=\{u\in E\vert \int _{\mathbb{R}^{3}}a(x)\vert u(x)\vert ^{r}\,dx=1 \}$. Since J is weakly lower semicontinuous, by Lemma 3.4 in [33], we can get that there exists $u_{0}\in \mathcal{M}$ such that $\lambda ^{*}=rJ(u_{0})>0$. Then, for every $\lambda >\lambda ^{*}$, we have

$$ \lambda \Vert u _{0} \Vert ^{r}_{L^{r}(\mathbb{R}^{3},a)}=\lambda >\lambda ^{*}=rJ(u _{0})=\frac{r}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u_{0} \vert ^{2}+\bigl(1+\omega ^{2}\bigr)u_{0}^{2} \bigr)\,dx+\frac{r}{q} \int _{\mathbb{R}^{3}}b(x) \vert u_{0} \vert ^{q} \,dx. $$

Furthermore,

$$ \begin{aligned} I(u_{0}) &=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u_{0} \vert ^{2}+u_{0} ^{2}\bigr)\,dx-\frac{\omega }{2} \int _{\mathbb{R}^{3}}\phi _{u_{0}}u_{0}^{2} \,dx \\ &\quad{} +\frac{1}{q} \int _{\mathbb{R}^{3}}b(x) \vert u_{0} \vert ^{q} \,dx-\frac{\lambda }{r} \int _{\mathbb{R}^{3}}a(x) \vert u_{0} \vert ^{r} \,dx \\ &\leq J(u_{0})-\frac{\lambda }{r} \int _{\mathbb{R}^{3}}a(x) \vert u_{0} \vert ^{r} \,dx \\ &< 0. \end{aligned} $$

Therefore, $I(u_{\lambda }^{1})=\inf_{u\in E}I(u)\leq I(u_{0})<0$. □

At last, similar to [33], we will prove that, for every $\lambda >\lambda ^{*}$, system $(\mathrm{P})$ has a second weak solution $u_{\lambda }^{2}$ via a modification of the mountain pass theorem of Ambrosetti and Rabinowitz. By choosing $X=E$, $Y=H^{1}(\mathbb{R}^{3})$, we have the following.

Lemma 3.4

Assume that (a)–(c) are satisfied. Then, for every fixed $e\in E \setminus \{0\}$ and $\lambda \in (0,+\infty )$, there exist $\rho _{\lambda }\in (0,\Vert e\Vert _{H^{1}(\mathbb{R}^{3})})$ and $\alpha _{\lambda }>0$ such that

$$ \inf_{u\in E, \Vert u\Vert _{H^{1}(\mathbb{R}^{3})}=\rho _{\lambda }}I(u) \geq \alpha _{\lambda }>0. $$

Proof

Since $a\in L^{\frac{6}{6-r}}(\mathbb{R}^{3})$, by Hölder’s inequality,

$$ \begin{aligned} I(u) &=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+u^{2} \bigr)\,dx-\frac{ \omega }{2} \int _{\mathbb{R}^{3}}\phi _{u}u^{2} \,dx + \frac{1}{q} \int _{\mathbb{R}^{3}}b(x) \vert u \vert ^{q}\,dx- \frac{\lambda }{r} \int _{\mathbb{R} ^{3}}a(x) \vert u \vert ^{r}\,dx \\ &\geq \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+u ^{2}\bigr)\,dx-\frac{\lambda }{r} \int _{\mathbb{R}^{3}}a(x) \vert u \vert ^{r}\,dx \\ & \geq \frac{1}{2} \Vert u \Vert _{H^{1}(\mathbb{R}^{3})}^{2}- \frac{\lambda }{r} \vert a \vert _{ \frac{6}{6-r}} \vert u \vert ^{r}_{6} \\ &\geq \frac{1}{2} \Vert u \Vert _{H^{1}(\mathbb{R} ^{3})}^{2}-C \frac{\lambda }{r} \vert a \vert _{\frac{6}{6-r}} \Vert u \Vert _{H^{1}( \mathbb{R}^{3})}^{r} \\ &= \Vert u \Vert _{H^{1}(\mathbb{R}^{3})}^{2}\biggl( \frac{1}{2}-C \frac{\lambda }{r} \vert a \vert _{\frac{6}{6-r}} \Vert u \Vert _{H^{1}( \mathbb{R}^{3})}^{r-2}\biggr). \end{aligned} $$

Then, for each $e\in E\setminus \{0\}$, by choosing $\rho _{\lambda }= \min \{\frac{1}{2}\Vert e\Vert _{H^{1}(\mathbb{R}^{3})},(\frac{r}{4C\lambda \vert a\vert _{ \frac{6}{6-r}}})^{\frac{1}{r-2}}\}$, we can get

$$ I(u)\geq \frac{1}{4}\rho _{\lambda }^{2}:=\alpha _{\lambda }, \quad u\in E \text{ with } \Vert u \Vert _{H^{1}(\mathbb{R}^{3})}= \rho _{\lambda }. $$

 □

Lemma 3.5

If (a)–(c) hold, then system $(\mathrm{P})$ enjoys a nontrivial solution $u_{\lambda }^{2}\in E$ with $I(u_{\lambda }^{2})=c_{\lambda }>0$ for every $\lambda >\lambda ^{*}$.

Proof

Lemma 3.3 implies that there exists a global nontrivial minimizer $u_{\lambda }^{1}\in E$ of I with $I(u_{\lambda }^{1})<0$ for every $\lambda >\lambda ^{*}$. By choosing $e=u_{\lambda }^{1}$ in Lemma 3.4, I satisfies the geometrical structure of Lemma 2.3. Thus I has a (PS)$_{c_{\lambda }}$ sequence for every $\lambda >\lambda ^{*}$. From Lemma 3.2, we can get that I enjoys a nontrivial solution $u_{\lambda }^{2}$ with $I(u_{\lambda }^{2})=c_{\lambda }>0>I(u_{ \lambda }^{1})$ for every fixed $\lambda \in (\lambda ^{*},+\infty )$ (for more details, see the proof of Theorem A.3 in [26]). □

Proof of Theorem 1.1

In order to complete the proof of (i) in Theorem 1.1, similar to [33], we only need to prove the fact that if system $(\mathrm{P})$ has a nontrivial solution $u\in E$, then there exists $\lambda _{0}>0$ such that $\lambda \geq \lambda _{0}$. By $\langle I'(u),u\rangle =0$ and $-\omega \leq \phi _{u}\leq 0$, we can get that

$$ \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+u^{2} \bigr)\,dx\leq \lambda \int _{\mathbb{R}^{3}}a(x) \vert u \vert ^{r}\,dx- \int _{\mathbb{R}^{3}}b(x) \vert u \vert ^{q} \,dx. $$

(3.5)

It implies that $\lambda >0$. Since for any fixed $k_{1}, k_{2}>0$ and $0<\alpha <\beta$, one can get

$$ k_{1}t^{\alpha }-k_{2}t^{\beta }\leq k_{1}\biggl(\frac{k_{1}}{k_{2}}\biggr)^{\frac{ \alpha }{\beta -\alpha }} \quad \text{for all } t\geq 0. $$

By taking $k_{1}=\lambda a(x)$, $k_{2}=b(x)$, $\alpha =r-2$, $\beta =q-2$, and $t=\vert u(x)\vert$, it is easy to claim that

$$ \lambda a(x) \bigl\vert u(x) \bigr\vert ^{r-2}-b(x) \bigl\vert u(x) \bigr\vert ^{q-2}\leq \lambda ^{ \frac{q-2}{q-r}}\biggl( \frac{a(x)^{q-2}}{b(x)^{r-2}}\biggr)^{\frac{1}{q-r}} \quad \text{for } \textit{a.e. } x\in \mathbb{R}^{3}. $$

(3.6)

By (3.5), (3.6), (c), and Hölder’s inequality,

$$ \Vert u \Vert _{H^{1}(\mathbb{R}^{3})}^{2}\leq \lambda ^{\frac{q-2}{q-r}} \int _{\mathbb{R}^{3}}\biggl(\frac{a(x)^{q-2}}{b(x)^{r-2}}\biggr)^{\frac{1}{q-r}}u ^{2}\,dx\leq S\lambda ^{\frac{q-2}{q-r}} \biggl\vert \biggl( \frac{a(x)^{q-2}}{b(x)^{r-2}}\biggr)^{ \frac{1}{q-r}} \biggr\vert _{\frac{3}{2}} \Vert u \Vert _{H^{1}(\mathbb{R}^{3})}^{2}, $$

where $S:= \inf_{u\in D^{1,2}(\mathbb{R}^{3}), \vert u\vert _{6}=1}\vert \nabla u\vert _{2}^{2}$ is the optimal constant in the Sobolev inequality. Since $u\neq 0$, we have

$$ \lambda \geq \biggl(\frac{1}{S \vert (\frac{a^{q-2}}{b^{r-2}})^{\frac{1}{q-r}} \vert _{ \frac{3}{2}}}\biggr)^{\frac{q-r}{q-2}}:=\lambda _{0}. $$

Therefore, if system $(\mathrm{P})$ has a nontrivial solution, then λ has a positive lower bound $\lambda _{0}$. That is to say, 0 is the only solution of system $(\mathrm{P})$ for every $\lambda <\lambda _{0}$.

(ii) Combine Lemma 3.3 with Lemma 3.5, system $(\mathrm{P})$ enjoys at least two nontrivial solutions for every $\lambda >\lambda ^{*}$. □