The existence of nontrivial solution of a class of Schrödinger–Bopp–Podolsky system with critical growth

Abstract

We consider the following Schrödinger–Bopp–Podolsky problem:

$$ \textstyle\begin{cases} -\Delta u+V(x) u+\phi u=\lambda f(u)+ \vert u \vert ^{4}u,& \text{in } \mathbb{R}^{3}, \\ -\Delta \phi +\Delta ^{2}\phi = u^{2}, & \text{in } \mathbb{R}^{3}. \end{cases} $$

We prove the existence result without any growth and Ambrosetti–Rabinowitz conditions. In the proofs, we apply a cut-off function, the mountain pass theorem, and Moser iteration.

1 Introduction and statement of results

In this paper, we deal with the following Schrödinger–Bopp–Podolsky system with critical growth:

$$ \textstyle\begin{cases} -\Delta u+V(x) u+\phi u=\lambda f(u)+ \vert u \vert ^{4}u, & \text{in } \mathbb{R}^{3}, \\ -\Delta \phi +\Delta ^{2}\phi = u^{2}, &\text{in } \mathbb{R}^{3}, \end{cases} $$

(1.1)

where $\lambda >0$ is a parameter. Systems such as (1.1) have been introduced in [1] as a model describing solitary waves for nonlinear stationary equations of Schrödinger type interacting with an electrostatic field in the Bopp–Podolsky electromagnetic theory and are usually known as Schrödinger–Bopp–Podolsky systems. We refer to [2–7] for a more detailed description of the physical aspects of this problem. In this paper, we suppose that V, f satisfy the following assumptions:

$(V_{1})$:

$V\in C(\mathbb{R}^{3}, \mathbb{R})$, $V_{0}=\inf_{x\in \mathbb{R}^{3}} V(x)>0$.

$(V_{2})$:

For any $T>0$, there exists $r>0$ such that

$$ \lim_{ \vert y \vert \rightarrow \infty }\operatorname{meas}\bigl(\bigl\{ x\in \mathbb{R}^{3}: \vert x-y \vert \leq r, V(x)\leq T\bigr\} \bigr)=0, $$

where $\operatorname{meas}(A)$ is the Lebesgue measure of A.

$(f_{1})$:

$f\in C(\mathbb{R})$ and $f(u)=o(u)$ as $u\rightarrow 0$.

$(f_{2})$:

$f(u)/u\rightarrow +\infty$ as $|u|\rightarrow \infty$.

The solution to (1.1) is understood in the weak sense, that is, a pair $(u, \phi )\in H^{1}(\mathbb{R}^{3})\times \mathcal{D}$ is a solution to (1.1) if

$$\begin{aligned}& \int _{\mathbb{R}^{3}} \bigl(\nabla u\nabla v+V(x)uv+ \phi uv \bigr)\,dx= \int _{\mathbb{R}^{3}} \bigl(\lambda f(u)v+ \vert u \vert ^{4}uv \bigr) \,dx, \quad \forall v\in H^{1}\bigl( \mathbb{R}^{3}\bigr), \\& \int _{\mathbb{R}^{3}}\nabla \phi \nabla \omega \,dx+ \int _{ \mathbb{R}^{3}}\Delta \phi \Delta \omega \,dx= \int _{\mathbb{R}^{3}} \phi u^{2}\,dx,\quad \forall \omega \in \mathcal{D}, \end{aligned}$$

where $\mathcal{D}$ is a function space that will be introduced in Sect. 2. To the best of our knowledge, there are very few papers related to the existence of solutions to problem (1.1). In [1], d’Avenia and Siciliano studied the following Schrödinger–Bopp–Podolsky equation:

$$ \textstyle\begin{cases} -\Delta u+\omega u+q^{2}\phi u= \vert u \vert ^{p-2}u, &\text{in } \mathbb{R}^{3}, \\ -\Delta \phi +a^{2}\Delta ^{2}\phi = 4\pi u^{2}, &\text{in } \mathbb{R}^{3}. \end{cases} $$

(1.2)

The authors give existence and nonexistence results, depending on the parameters p and q. Moreover, they also show that in the radial case, the solutions that they find tend to solutions of the classical Schrödinger–Poisson system as $a\rightarrow 0$.

When $a=0$, (1.2) reduces to the following well-known Schrödinger–Poisson equation

$$ \textstyle\begin{cases} -\Delta u+\omega u+q^{2}\phi u= \vert u \vert ^{p-2}u, &\text{in } \mathbb{R}^{3}, \\ -\Delta \phi = 4\pi u^{2}, &\text{in } \mathbb{R}^{3}, \end{cases} $$

that has been extensively studied in the past few decades. There have been many existence and nonexistence results in the past decades. For some recent results, we refer the readers to [8–13] and the references therein. We now summarize our main results as follows.

Theorem 1.1

Suppose that assumptions $(f_{1})$–$(f_{2})$and $(V_{1})$–$(V_{2})$are satisfied. Then there exists $\lambda _{1}>0$such that, for any $\lambda \in (0, \lambda _{1})$, problem (1.1) has a nontrivial solution.

Remark 1.1

We note that the usual growth condition and the Ambrosetti–Rabinowitz condition are not needed in our result. Moreover, f is allowed to be sign-changing.

Remark 1.2

A typical example of a function satisfying assumptions $(f_{1})$–$(f_{2})$ is given by $f(t)=|t|^{q-2}t$, $q>6$. Furthermore, our conclusion holds for general supercritical nonlinearity.

The proof will be carried out by variational methods. Since the Sobolev embedding $H^{1}(\mathbb{R}^{3})\hookrightarrow L^{s}(\mathbb{R}^{3})$ is not compact, the main difficulty is the lack of compactness. Since we do not assume Ambrosetti–Rabinowitz or growth conditions on f, we first make a suitable modification on f, solve the modified problem, and then check that, for small enough λ, the solutions of the modified problem are also the solutions of the original problem. We note that even for the modified problem it is not easy to obtain compactness in view of the critical growth of the nonlinearity. To overcome the loss of compactness for the energy functional, we shall verify that the Palais–Smale condition is regained when the energy functional is below a suitable level.

The rest of this paper is organized as follows. In Sect. 2, we state some preliminary notations, modify the original problem, and prove the existence result of the modified problem. In Sect. 3, we prove Theorem 1.1.

2 Preliminaries and the modified problem

In this paper, we use the following notation:

• $H^{1}(\mathbb{R}^{3})$ is the usual Sobolev space with an inner product and norm given by

  $$ \langle u, v\rangle _{H^{1}(\mathbb{R}^{3})}:= \int _{\mathbb{R}^{3}}( \nabla u\nabla v+ uv)\,dx,\qquad \Vert \cdot \Vert _{H^{1}(\mathbb{R}^{3})}= \biggl( \int _{\mathbb{R}^{3}} \bigl( \vert \nabla u \vert ^{2}+ \vert u \vert ^{2} \bigr)\,dx \biggr)^{\frac{1}{2}}. $$

• $L^{p}(\mathbb{R}^{3}), 1\leq p\leq +\infty$, denotes a Lebesgue space, and the norm in $L^{p}(\mathbb{R}^{3})$ is denoted by $|\cdot |_{p}$.

• $D^{1,2}(\mathbb{R}^{3})$ is the completion of $C^{\infty }_{0}(\mathbb{R}^{3})$ with respect to the norm

  $$ \Vert u \Vert ^{2}_{D^{1,2}(\mathbb{R}^{3})}= \int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx. $$

• C, $C_{i}$ denote (possible different) any positive constant.

• $H^{-1}$ denotes the dual space of $H^{1}(\mathbb{R}^{3})$.

In this section, we summarize some fundamental properties of the operator $-\Delta +\Delta ^{2}$ and functional space $\mathcal{D}$. The $\mathcal{D}$ is defined by the completion of $C_{0}^{\infty }(\mathbb{R}^{3})$ equipped with the norm $\|\cdot \|_{\mathcal{D}}$ induced by the scalar product

$$ \langle u, v\rangle _{\mathcal{D}}:= \int _{\mathbb{R}^{3}}\nabla u \nabla v \,dx+ \int _{\mathbb{R}^{3}} \Delta u \Delta v\,dx. $$

For more details, we refer the reader to [1].

It is easy to show that $\mathcal{D}$ is a Hilbert space continuously embedded into $D^{1,2}(\mathbb{R}^{3})$ and consequently in $L^{\infty }(\mathbb{R}^{3})$, see [1].

Lemma 2.1

([1, Lemma 3.2])

The space $C_{0}^{\infty }(\mathbb{R}^{3})$is dense in

$$ \mathcal{A}:=\bigl\{ \phi \in D^{1,2}\bigl(\mathbb{R}^{3} \bigr): \Delta \phi \in L^{2}\bigl( \mathbb{R}^{3}\bigr)\bigr\} $$

normed by $\sqrt{\langle \phi , \phi \rangle _{\mathcal{D}}}$and, therefore, $\mathcal{D}=\mathcal{A}$.

For every fixed $u\in H^{1}(\mathbb{R}^{3})$, the Riesz theorem implies that there exists a unique solution $\phi _{u}\in \mathcal{D}$ such that

$$ -\Delta \phi +\Delta ^{2}\phi = u^{2}. $$

In order to write explicitly this solution, we consider

$$ \mathcal{K}(x)=\frac{1-e^{- \vert x \vert }}{4\pi \vert x \vert }, $$

and $\mathcal{K}(x-y)$ is the fundamental solution of the equation $-\Delta \phi +\Delta ^{2}\phi = \delta _{y}$. See [7, formula 2.6] and [1, Lemma 3.3] for more properties of $\mathcal{K}(x)$. Then, the unique solution in $\mathcal{D}$ to the second equation in (1.1) is

$$ \phi _{u}(x):=\mathcal{K}*u^{2}= \frac{1}{4\pi } \int _{\mathbb{R}^{3}} \frac{1-e^{- \vert x-y \vert }}{ \vert x-y \vert }u^{2}(y)\,dy. $$

(2.1)

The function $\phi _{u}$ possesses the following properties (see [1]).

Lemma 2.2

For every $u\in H^{1}(\mathbb{R}^{3})$, we have:

• (i) $\phi _{u}\geq 0$for all $u\in H^{1}(\mathbb{R}^{3})$;

• (ii) $\|\phi _{u}\|_{\mathcal{D}}\leq C\|u\|^{2}$, $\int _{\mathbb{R}^{3}} \phi _{u}u^{2}\,dx\leq C\|u\|_{12/5}^{4}$;

• (iii) if $u_{n}\rightharpoonup u$in $H^{1}(\mathbb{R}^{3})$, then $\phi _{u_{n}}\rightharpoonup \phi _{u}$in $\mathcal{D}$;

• (iv) for every $y\in \mathbb{R}^{3}$, $\phi _{u(\cdot +y)}=\phi _{u}(\cdot +y)$;

• (v) $\phi _{u}$is the unique minimizer of the functional

  $$ E(\phi )=\frac{1}{2} \Vert \nabla \phi \Vert _{2}^{2}+ \frac{1}{2} \Vert \Delta \phi \Vert _{2}^{2}- \int _{\mathbb{R}^{3}}\phi u^{2}\,dx,\quad \phi \in \mathcal{D}. $$

Substituting (2.1) into (1.1), we obtain

$$ -\Delta u+V(x) u+\phi _{u} u=\lambda f(u)+ \vert u \vert ^{4}u,\quad u\in H^{1}\bigl( \mathbb{R}^{3}\bigr). $$

Then we define a smooth functional $\varPhi : H^{1}(\mathbb{R}^{3})\rightarrow \mathbb{R}$ by setting

$$ \varPhi (u)= \int _{\mathbb{R}^{3}}\phi _{u}(x)u^{2}(x)\,dx. $$

(2.2)

In fact, functional Φ possesses the following useful BL-splitting properties, similar to the Brézis–Lieb lemma [14].

Lemma 2.3

Let $u_{n}\rightharpoonup u$in $H^{1}(\mathbb{R}^{3})$and $u_{n}\rightarrow u$a.e. in $\mathbb{R}^{3}$, then

$$ \varPhi (u_{n}-u)=\varPhi (u_{n})-\varPhi (u)+o(1),\quad \textit{as } n \rightarrow \infty , $$

where Φ is defined by (2.2).

Proof

Since $\mathcal{K}(x)\in L^{\tau }(\mathbb{R}^{3})$ for $\tau \in (3,+\infty ]$, together with $u_{n}\rightharpoonup u$ in $H^{1}(\mathbb{R}^{3})$ and $u_{n}\rightarrow u$ a.e. in $\mathbb{R}^{3}$, we have

$$ \phi _{u_{n}-u}=\phi _{u_{n}}-\phi _{u}+o(1). $$

(2.3)

By Lemma 2.2 (v), we obtain that

$$ \langle \phi _{u}, \phi _{u}\rangle _{\mathcal{D}}= \varPhi (u), \quad \forall u\in H^{1}\bigl(\mathbb{R}^{3} \bigr). $$

Consequently, by (2.3) and Lemma 2.2 (ii)–(iii), we obtain that

$$\begin{aligned} \varPhi (u_{n}-u) &=\langle \phi _{u_{n}-u}, \phi _{u_{n}-u}\rangle _{ \mathcal{D}} \\ &=\bigl\langle \phi _{u_{n}}-\phi _{u}+o(1), \phi _{u_{n}}-\phi _{u}+o(1) \bigr\rangle _{\mathcal{D}} \\ &=\langle \phi _{u_{n}}, \phi _{u_{n}}\rangle _{\mathcal{D}}-2 \langle \phi _{u_{n}}, \phi _{u}\rangle _{\mathcal{D}}+\langle \phi _{u}, \phi _{u}\rangle _{\mathcal{D}}+o(1) \\ &=\varPhi (u_{n})-\varPhi (u)+o(1). \end{aligned}$$

The proof is complete. □

We shall search critical points for the functional

$$ I_{\lambda }(u)=\frac{1}{2} \int _{\mathbb{R}^{3}} \bigl( \vert \nabla u \vert ^{2}+V(x)u^{2} \bigr)\,dx+\frac{1}{4}\varPhi (u) - \int _{\mathbb{R}^{3}} \biggl(\lambda F(u)+ \frac{1}{6} \vert u \vert ^{6} \biggr)\,dx, $$

where $F(t)=\int _{0}^{t}f(s)\,ds$, as solutions to (1.1). It is well defined on the Hilbert space

$$ X= \biggl\{ u\in H^{1}\bigl(\mathbb{R}^{3}\bigr): \int _{\mathbb{R}^{3}}V(x)u^{2}\,dx< + \infty \biggr\} $$

and has the inner product and norm

$$ \langle u, v\rangle = \int _{\mathbb{R}^{3}} \bigl(\nabla u\nabla v+V(x)uv \bigr)\,dx, \qquad \Vert u \Vert =\langle u, u\rangle ^{\frac{1}{2}}. $$

It is well known under assumptions $(V_{1})$ and $(V_{2})$ that we have the following compactness lemma see [15] or [16].

Lemma 2.4

Suppose that assumptions $(V_{1})$and $(V_{2})$are satisfied. Then the embedding from X into $L^{s}(\mathbb{R}^{3})$is compact for $s\in [2,6)$.

Since f is continuous, we have $I_{\lambda }\in C^{1}(X, \mathbb{R})$ and

$$ \bigl\langle I'_{\lambda }(u), v\bigr\rangle = \int _{\mathbb{R}^{3}} \bigl( \nabla u\nabla v+V(x)uv+ \phi _{u}uv \bigr)\,dx- \int _{\mathbb{R}^{3}} \bigl(\lambda f(u)v+ \vert u \vert ^{4}uv \bigr) \,dx,\quad \forall u,v \in X. $$

Since $(f_{1})$ and $(f_{2})$ imply that $\lim_{t\rightarrow 0^{+}}\frac{f(t)}{t}=0$, $\lim_{t\rightarrow + \infty }\frac{f(t)}{t}=+\infty$, we can introduce a truncated function. Let $T>0$ be large enough such that $f(T)>0$ according to $(f_{2})$. We set

$$ g_{T}(t)= \textstyle\begin{cases} f(t), & 0\leq t\leq T, \\ C_{T}t^{p-1}, & t>T, \\ 0, & t\leq 0, \end{cases} $$

where $C_{T}=f(T)/T^{p-1}$, $p\in (4,6)$. Based on assumptions $(f_{1})$ and $(f_{2})$ it is easy to show that $g_{T}(t)$ is a continuous function and satisfies the following properties:

$(g_{1})$:

$\lim_{t\rightarrow 0^{+}}\frac{g_{T}(t)}{t}=0$.

$(g_{2})$:

$\lim_{t\rightarrow +\infty }\frac{G_{T}(t)}{t^{4}}=+\infty$, where $G(t)=\int _{0}^{t}g(s)\,ds$.

$(g_{3})$:

$|g_{T}(t)|\leq C_{T}^{*}|t|+C_{T}|t|^{p-1}$, where $C_{T}^{*}=\max_{t\in [0,T]}|f(t)|/t$.

$(g_{4})$:

There exists $\mu =\mu (T)>0$ such that $t g_{T}(t)-4 G_{T}(t)\geq -\mu t^{2}$ for all $t\geq 0$.

Now we obtain the modified problem

$$ \textstyle\begin{cases} -\Delta u+V(x)u+\phi _{u}u=\lambda g_{T}(u)+ \vert u \vert ^{4}u, \quad x\in \mathbb{R}^{3}, \\ u\in X,\qquad u(x)>0. \end{cases} $$

(2.4)

We shall search critical points for the functional

$$ I_{\lambda ,T}(u)=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+V(x)u^{2} \bigr)\,dx+ \frac{1}{4}\varPhi (u) - \int _{\mathbb{R}^{3}} \biggl(\lambda G_{T}(u)+ \frac{1}{6} \vert u \vert ^{6} \biggr)\,dx, $$

as solutions to (2.4). Since $g_{T}$ is continuous, we have $I_{\lambda ,T}\in C^{1}(X, \mathbb{R})$ and, for any $u,v \in X$,

$$ \begin{aligned} \bigl\langle I'_{\lambda ,T}(u),v \bigr\rangle = \int _{\mathbb{R}^{3}} \bigl( \nabla u\nabla v+V(x)uv+ \phi _{u}uv \bigr)\,dx- \int _{\mathbb{R}^{3}} \bigl(\lambda g_{T}(u)v+ \vert u \vert ^{4}uv \bigr) \,dx. \end{aligned} $$

(2.5)

The next lemma shows that the functional $I_{\lambda ,T}(u)$ satisfies the mountain pass geometry[14].

Lemma 2.5

The functional $I_{\lambda ,T}(u)$satisfies the following conditions:

• (i) there exist $\alpha , \rho >0$such that $I_{\lambda ,T}(u)\geq \alpha$with $\|u\|=\rho$;

• (ii) there exists $e\in X$such that $\|e\|>\rho$and $I_{\lambda ,T}(e)<0$.

Proof

For any $u\in X\backslash \{0\}$ and $\epsilon >0$ small, it follows from $(g_{1})$ and $(g_{3})$ that

$$ \bigl\vert g_{T}(t) \bigr\vert \leq \epsilon \vert t \vert +C_{\epsilon } \vert t \vert ^{5} $$

and

$$ \bigl\vert G_{T}(t) \bigr\vert \leq \frac{\epsilon }{2} \vert t \vert ^{2}+\frac{C_{\epsilon }}{6} \vert t \vert ^{6}. $$

Thus

$$ \begin{aligned} I_{\lambda ,T}(u) &\geq \frac{1}{2} \Vert u \Vert ^{2}+\frac{1}{4}\varPhi (u) - \int _{\mathbb{R}^{3}} \biggl(\frac{\lambda \epsilon }{2} \vert u \vert ^{2}+ \frac{1+\lambda C_{\epsilon } }{6} \vert u \vert ^{6} \biggr)\,dx \\ &\geq \frac{1}{2} \Vert u \Vert ^{2}-C\epsilon \Vert u \Vert ^{2}-C \Vert u \Vert ^{6} \end{aligned} $$

by Lemma 2.2 (i) and the Sobolev embedding $X\hookrightarrow L^{s}(\mathbb{R}^{3})$ for $s\in [2, 6]$. Since ϵ is arbitrarily small, there exist $\rho >0$ and $\alpha >0$ such that $I_{\lambda ,T}(u)\geq \alpha >0$ for $\| u\| =\rho$.

Let us check (ii). From $(g_{2})$, for any $M>0$, there exists $r_{M}>0$ such that

$$ G_{T}(t)\geq Mt^{4},\quad \forall t\geq r_{M}. $$

Together with $(g_{1})$ and $(g_{3})$, this implies that, for any $M>0$, there exists a constant $C_{M}>0$ such that

$$ G_{T}(t)\geq Mt^{4}-C_{M}t^{2}, \quad \forall t>0. $$

(2.6)

Then, for each $u\in X\setminus \{0\}$ and $t>0$, we obtain that

$$ \begin{aligned} I_{\lambda ,T}(tu)&\leq \frac{t^{2}}{2} \Vert u \Vert ^{2} +\frac{t^{4}}{4} \varPhi (u)-\lambda Mt^{4} \int _{\mathbb{R}^{3}} \vert u \vert ^{4}\,dx+\lambda C_{M}t^{2} \int _{\mathbb{R}^{3}} \vert u \vert ^{2}\,dx- \frac{t^{6}}{6} \int _{\mathbb{R}^{3}} \vert u \vert ^{6}\,dx. \end{aligned} $$

The step is proved by taking $e=t_{0}u$ with $t_{0}>0$ large enough. □

Now, in view of Lemma 2.5, we can apply a version of the mountain pass theorem without the $(\mathit{PS})$ condition to obtain a sequence $\{u_{n}\}$ such that

$$ I_{\lambda ,T}(u_{n})\rightarrow c_{\lambda ,T}, \qquad \bigl\Vert I'_{\lambda ,T}(u_{n}) \bigr\Vert _{X^{-1}}\rightarrow 0. $$

(2.7)

As in [14], we define

$$ c_{\lambda ,T}=\inf_{\gamma \in \varGamma }\max_{t\in [0,1]}I_{\lambda ,T} \bigl( \gamma (t)\bigr), $$

where

$$ \varGamma =\bigl\{ \gamma \in C\bigl([0,1], H^{1}\bigl( \mathbb{R}^{3}\bigr)\bigr): \gamma (0)=0, I_{ \lambda ,T}\bigl(\gamma (1)\bigr)< 0\bigr\} . $$

Lemma 2.6

Every sequence satisfying (2.7) is bounded in X.

Proof

For every $c\in \mathbb{R}$, let $\{u_{n}\}\subset X$ be a ${(\mathit{PS})_{c}}$ sequence satisfying (2.7). Then, by $(g_{4})$, we deduce that

$$ \begin{aligned}[b] & c_{\lambda ,T}+o_{n}(1) \Vert u_{n} \Vert \\ &\quad\geq I_{\lambda ,T}(u_{n})-\frac{1}{4} I'_{\lambda ,T}(u_{n})u_{n} \\ &\quad= \frac{1}{4} \Vert u_{n} \Vert ^{2}+ \frac{\lambda }{4} \int _{\mathbb{R}^{3}} \bigl[g_{T}(u_{n})u_{n}-4G_{T}(u_{n}) \bigr]\,dx +\frac{1}{12} \int _{ \mathbb{R}^{3}} \vert u_{n} \vert ^{6}\,dx \\ &\quad\geq \frac{1}{4} \Vert u_{n} \Vert ^{2}- \frac{\lambda \mu }{4} \int _{\mathbb{R}^{3}} \bigl\vert u_{n}^{+} \bigr\vert ^{2}\,dx, \end{aligned} $$

(2.8)

where $u_{n}^{+}=\max \{u_{n}(x),0\}$, $u_{n}^{-}=\min \{u_{n}(x),0\}$, $u_{n}(x)=u_{n}^{+}+u_{n}^{-}$. We argue by contradiction that $\|u_{n}\| \rightarrow +\infty$ as $n\rightarrow \infty$. Let $v_{n}=\frac{u_{n}}{\|u_{n}\|}$, $n\geq 1$. According to Lemma 2.4, $X\hookrightarrow L^{s}(\mathbb{R}^{3})$, $s\in [2,6)$ is compact. We may assume that

$$ \begin{aligned} &v_{n}\rightarrow v,\quad \text{a.e. in } \mathbb{R}^{3}, \\ &v_{n}\rightharpoonup v,\quad \text{weakly in } X, \\ &v_{n}\rightarrow v,\quad \text{strongly in } L^{s}\bigl( \mathbb{R}^{3}\bigr), 2\leq s< 6. \end{aligned} $$

Moreover, we have

$$ \begin{aligned} &v^{+}_{n} \rightarrow v^{+},\quad \text{a.e. in } \mathbb{R}^{3}, \\ &v^{+}_{n}\rightharpoonup v^{+},\quad \text{weakly in } X, \\ &v^{+}_{n}\rightarrow v^{+},\quad \text{strongly in } L^{s}\bigl(\mathbb{R}^{3}\bigr), 2\leq s< 6. \end{aligned} $$

(2.9)

From (2.8), we have

$$ o_{n}(1)\geq \frac{1}{4} \biggl(1-\lambda \mu \int _{\mathbb{R}^{3}} \bigl\vert v_{n}^{+} \bigr\vert ^{2}\,dx \biggr)= \frac{1}{4} \biggl(1-\lambda \mu \int _{\mathbb{R}^{3}} \bigl\vert v^{+} \bigr\vert ^{2}\,dx \biggr)+o(1). $$

We conclude that $v^{+}\neq 0$. Then $u_{n}^{+}=v_{n}^{+}\|u_{n}\|\rightarrow +\infty$. By (2.5) and (2.7), we obtain

$$ \begin{aligned}[b] &\frac{I'_{\lambda ,T}(u_{n})u_{n}}{ \Vert u_{n} \Vert ^{4}} \\ &\quad =o_{n}(1)+ \frac{\int _{\mathbb{R}^{3}}\phi _{u_{n}}u_{n}^{2}\,dx}{ \Vert u_{n} \Vert ^{4}} - \int _{\mathbb{R}^{3}} \frac{\lambda g_{T}(u_{n}^{+})u_{n}^{+}}{(u_{n}^{+})^{4}}\bigl(v_{n}^{+} \bigr)^{4}\,dx- \Vert u_{n} \Vert ^{2} \int _{\mathbb{R}^{3}} \vert v_{n} \vert ^{6}\,dx \\ &\quad \leq o_{n}(1)+ \frac{\int _{\mathbb{R}^{3}}\phi _{u_{n}}u_{n}^{2}\,dx}{ \Vert u_{n} \Vert ^{4}} - \int _{\mathbb{R}^{3}} \frac{\lambda g_{T}(u_{n}^{+})u_{n}^{+}}{(u_{n}^{+})^{4}}\bigl(v_{n}^{+} \bigr)^{4}\,dx. \end{aligned} $$

(2.10)

Taking the limit and using Lemma 2.2 (ii), $(g_{2})$, and (2.9), we obtain $0\leq -\infty$, yielding a contradiction. Therefore, $\{u_{n}\}$ is bounded in X. □

Now, we denote by S the best constant of the Sobolev embedding $H^{1}(\mathbb{R}^{3})\hookrightarrow L^{6}(\mathbb{R}^{3})$, i.e.,

$$ S=\inf_{u\in D^{1,2}(\mathbb{R}^{3})} \frac{\int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx}{ (\int _{\mathbb{R}^{3}} \vert u \vert ^{6}\,dx )^{1/3}}. $$

As we will show in the following result, the modified functional satisfies the local compactness condition.

Lemma 2.7

$I_{\lambda ,T}$satisfies the $(PS)_{c}$condition at any level $c_{\lambda ,T}\in (0,\frac{1}{3}S^{\frac{3}{2}})$.

Proof

Let $\{u_{n}\}$ be a $(PS)_{c_{\lambda ,T}}$ sequence satisfying (2.7). By Lemma 2.6, $\{u_{n}\}$ is bounded in X. Up to a subsequence, we may assume that

$$\begin{aligned} \begin{aligned} &u_{n}\rightarrow u,\quad \text{a.e. in } \mathbb{R}^{3}, \\ &u_{n}\rightharpoonup u,\quad \text{weakly in } X, \\ &u_{n}\rightarrow u,\quad \text{strongly in } L^{s}\bigl( \mathbb{R}^{3}\bigr), 2\leq s< 6. \end{aligned} \end{aligned}$$

(2.11)

Since $\phi : L^{12/5}(\mathbb{R}^{3})\rightarrow \mathcal{D}$ is continuous, from (2.11) we obtain that

$$ \begin{aligned} &\phi _{u_{n}}\rightarrow \phi _{u} \quad \text{in } \mathcal{D}, \text{ as } n\rightarrow \infty , \\ & \int _{\mathbb{R}^{3}}\phi _{u_{n}}u_{n}^{2} \,dx\rightarrow \int _{ \mathbb{R}^{3}}\phi _{u}u^{2}\,dx, \quad \text{as } n\rightarrow \infty . \end{aligned} $$

(2.12)

Using (2.11) and [14, Theorem A.1 ], for any $\varphi \in C_{0}^{\infty }(\mathbb{R}^{3})\subset X$, we can obtain that

$$ \begin{aligned} &\int _{\mathbb{R}^{3}}\bigl(\nabla u_{n}\nabla \varphi +V(x)u_{n}\varphi \bigr)\,dx \rightarrow \int _{\mathbb{R}^{3}}\bigl(\nabla u\nabla \varphi +V(x)u \varphi \bigr) \,dx, \\ &\int _{\mathbb{R}^{3}}g_{T}(u_{n})\varphi \,dx \rightarrow \int _{ \mathbb{R}^{3}}g_{T}(u)\varphi \,dx, \\ &\int _{\mathbb{R}^{3}} \vert u_{n} \vert ^{4}u_{n}\varphi \,dx\rightarrow \int _{ \mathbb{R}^{3}} \vert u \vert ^{4}u\varphi \,dx. \end{aligned} $$

(2.13)

From (2.11)–(2.12), the Hölder inequality, and the Sobolev embedding, we obtain

$$\begin{aligned} \int _{\mathbb{R}^{3}}(\phi _{u_{n}}u_{n}-\phi _{u}u)\varphi \,dx &= \int _{\mathbb{R}^{3}}\phi _{u_{n}}(u_{n}-u)\varphi \,dx+ \int _{ \mathbb{R}^{3}}(\phi _{u_{n}}-\phi _{u})u\varphi \,dx \\ &\leq C \Vert \phi _{u_{n}} \Vert _{D^{1,2}(\mathbb{R}^{3})} \vert u_{n}-u \vert _{12/5} \vert \varphi \vert _{12/5} \\ &\quad {}+C \Vert \phi _{u_{n}}-\phi _{u} \Vert _{D^{1,2}(\mathbb{R}^{3})} \vert u \vert _{12/5} \vert \varphi \vert _{12/5} \\ &\rightarrow 0. \end{aligned}$$

(2.14)

By (2.13)–(2.14), the density of $C_{0}^{\infty }(\mathbb{R}^{3})$ in X, and (2.7), we can conclude that $I'_{\lambda ,T}(u_{n})\rightarrow I'_{\lambda ,T}(u)=0$. Let $w_{n}=u_{n}-u$, as $n\rightarrow \infty$. It follows from Lemma 2.3 and the Brezis–Lieb lemma that

$$ I_{\lambda ,T}(w_{n})= I_{\lambda ,T}(u_{n})-I_{\lambda ,T}(u)=c-I_{ \lambda ,T}(u)+o(1):=d+o(1), $$

and $I'_{\lambda ,T}(w_{n})\rightarrow 0$ in $X^{-1}$. We recall that the continuous embedding $X\hookrightarrow L^{s}(\mathbb{R}^{3})$ is compact for $2\leq s < 6$. Hence, up to a subsequence, $w_{n}\rightarrow 0$ in $L^{s}(\mathbb{R}^{3})$, and

$$ \Vert w_{n} \Vert ^{2}+ \int _{\mathbb{R}^{3}}\phi _{w_{n}}w_{n}^{2} \,dx= \int _{ \mathbb{R}^{3}}\lambda g_{T}(w_{n})w_{n} \,dx+ \int _{\mathbb{R}^{3}}w_{n}^{6}\,dx+o(1). $$

(2.15)

By Lemma 2.2(ii), we obtain that

$$ \int _{\mathbb{R}^{3}}\phi _{w_{n}}w_{n}^{2} \,dx\leq C \vert w_{n} \vert _{12/5}^{4} \rightarrow 0. $$

(2.16)

Hence, by $(g_{3})$, (2.15)–(2.16), we have

$$ \Vert w_{n} \Vert ^{2}= \int _{\mathbb{R}^{3}} \vert w_{n} \vert ^{6} \,dx+o(1). $$

Since $w_{n }\subset X$ is bounded, we may assume that as $n\rightarrow \infty$

$$ \Vert w_{n} \Vert ^{2}\rightarrow b\geq 0,\qquad \int _{\mathbb{R}^{3}} \vert w_{n} \vert ^{6}\,dx \rightarrow b\geq 0, $$

up to a subsequence. Suppose by contradiction that $b>0$. By the Sobolev inequality, we have

$$ \Vert w_{n} \Vert ^{2}\geq \int _{\mathbb{R}^{3}} \vert \nabla w_{n} \vert ^{2}\,dx\geq S \vert w_{n} \vert ^{2}_{6} $$

and, therefore, $b\geq S^{\frac{3}{2}}$. Thus

$$ d=\lim_{n\rightarrow \infty }I_{\lambda ,T}(w_{n})= \biggl( \frac{1}{2}- \frac{1}{6} \biggr)b\geq \frac{1}{3}S^{\frac{3}{2}}, $$

which contradicts our assumption. Therefore, $b = 0$ and the proof is complete. □

To obtain the existence result for problem (2.4) by Lemma 2.7, we need to show that the mountain pass value $c_{\lambda ,T}<\frac{1}{3}S^{\frac{3}{2}}$.

Lemma 2.8

For any $\lambda >0$, $c_{\lambda ,T}<\frac{1}{3}S^{\frac{3}{2}}$.

Proof

For $\epsilon >0$, consider the function

$$ U_{\epsilon }(x)= \frac{ (3\epsilon ^{2} )^{\frac{1}{4}}}{ (\epsilon ^{2}+ \vert x \vert ^{2} )^{\frac{1}{2}}}. $$

We recall that $U_{\epsilon }(x)$ satisfies

$$ \textstyle\begin{cases} -\Delta u = \vert u \vert ^{4}u, & \text{in } \mathbb{R}^{3}, \\ u\in D^{1,2}(\mathbb{R}^{3}), u(x)>0, & \text{in } \mathbb{R}^{3}, \end{cases} $$

and

$$ \int _{\mathbb{R}^{3}} \vert \nabla U_{\epsilon } \vert ^{2}\,dx= \int _{\mathbb{R}^{3}} \vert U_{\epsilon } \vert ^{6} \,dx=S^{\frac{3}{2}}. $$

Let $\psi \in C_{0}^{\infty }(\mathbb{R}^{3},[0,1])$ be such that $\psi (x)=1$ for $|x|\leq r$ and $\psi (x)=0$ for $|x|\geq 2r$. Set $u_{\epsilon }(x)=\psi (x)U_{\epsilon }(x)$. Then, the following asymptotic estimates hold if ϵ is small enough (see [14]):

$$\begin{aligned}& \int _{\mathbb{R}^{3}} \vert \nabla u_{\epsilon } \vert ^{2}\,dx=S^{\frac{3}{2}}+O( \epsilon ), \end{aligned}$$

(2.17)

$$\begin{aligned}& \int _{\mathbb{R}^{3}} \vert u_{\epsilon } \vert ^{6} \,dx=S^{\frac{3}{2}}+O\bigl( \epsilon ^{3}\bigr), \end{aligned}$$

(2.18)

$$\begin{aligned}& \int _{\mathbb{R}^{3}} \vert u_{\epsilon } \vert ^{s}\,dx= \textstyle\begin{cases} O(\epsilon ^{\frac{s}{2}}),& s\in [2,3), \\ O(\epsilon ^{\frac{s}{2}} \vert \ln \epsilon \vert ),& s=3, \\ O(\epsilon ^{\frac{6-s}{2}}),& s\in (3,6). \end{cases}\displaystyle \end{aligned}$$

(2.19)

Since $I_{\lambda ,T}(tu_{\epsilon })\rightarrow -\infty$, as $t\rightarrow \infty$, there exists $t_{\epsilon }>0$ such that

$$ I_{\lambda ,T}(t_{\epsilon }u_{\epsilon })=\sup_{t\geq 0}I_{\lambda ,T}(t u_{\epsilon })>0. $$

We claim that $\{t_{\epsilon }\}_{\epsilon >0}$ is bounded from below by a positive constant. Otherwise, there exists a sequence $\{\epsilon _{n}\}\subset \mathbb{R}^{+}$ such that $\lim_{n\rightarrow \infty }t_{\epsilon _{n}}=0$ and

$$ I_{\lambda ,T}(t_{\epsilon _{n}}u_{\epsilon _{n}})=\sup_{t\geq 0}I_{ \lambda ,T}(t u_{\epsilon }). $$

Therefore, $0<\alpha \leq c\leq \lim_{n\rightarrow \infty }I_{\lambda ,T}(t_{ \epsilon _{n}}u_{\epsilon _{n}})=0$, yielding a contradiction. Thus there exists $t_{0}>0$ such that $t_{\epsilon }\geq t_{0}>0$. Moreover, we make the following assertion: $\{t_{\epsilon }\}_{\epsilon >0}$ is bounded from above. In fact, suppose by contradiction that there exists a subsequence $\{t_{\epsilon _{n}}\}$ with $t_{\epsilon _{n}}\rightarrow +\infty$. Then, from (2.17)–(2.19), we obtain

$$ 0< c_{\lambda ,T} \leq I_{\lambda ,T}(t_{\epsilon _{n}}u_{\epsilon _{n}}) \leq C_{1}t_{\epsilon _{n}}^{2}+C_{2}t_{\epsilon _{n}}^{4}-C_{3}t_{ \epsilon _{n}}^{6}. $$

(2.20)

Letting $n\rightarrow \infty$ in (2.20), we obtain $0<-\infty$, which is a contradiction. Therefore, $\{t_{\epsilon }\}_{\epsilon >0}$ is bounded from above. Let

$$ h(t)=\frac{t^{2}}{2} \int _{\mathbb{R}^{3}} \vert \nabla u_{\epsilon } \vert ^{2}\,dx- \frac{t^{6}}{6} \int _{\mathbb{R}^{3}} \vert u_{\epsilon } \vert ^{6} \,dx. $$

It is easy to see that

$$ \sup_{t\geq 0}h(t)=\frac{1}{3}S^{\frac{3}{2}}+O( \epsilon ). $$

(2.21)

By $(V_{2})$, for $|x|\leq r$, there exists $\beta >0$ such that

$$ \bigl\vert V(x) \bigr\vert \leq \beta . $$

(2.22)

From (2.6), (2.21)–(2.22), and (2.19), we obtain

$$ \begin{aligned}[b] c_{\lambda ,T}&\leq I_{\lambda ,T}(t_{\epsilon }u_{\epsilon }) \\ & \leq \sup _{t\geq 0}h(t)+\frac{t_{\epsilon }^{2}}{2}\beta \int _{\mathbb{R}^{3}} \vert u_{ \epsilon } \vert ^{2}\,dx +\frac{t_{\epsilon }^{4}}{4} \int _{\mathbb{R}^{3}} \phi _{u_{\epsilon }}u_{\epsilon }^{2} \,dx \\ &\quad -\lambda M t_{\epsilon }^{4} \int _{\mathbb{R}^{3}} \vert u_{\epsilon } \vert ^{4} \,dx+t_{ \epsilon }^{2}\lambda C_{M} \int _{\mathbb{R}^{3}} \vert u_{\epsilon } \vert ^{2}\,dx \\ & \leq \sup_{t\geq 0}h(t)+C \vert u_{\epsilon } \vert ^{2}_{2}+C \vert u_{\epsilon } \vert ^{4}_{12/5}- \lambda MC \vert u_{\epsilon } \vert ^{4}_{4} \\ &\leq \frac{1}{3}S^{\frac{3}{2}}+CO(\epsilon )+CO\bigl(\epsilon ^{2}\bigr)- \lambda MCO(\epsilon ). \end{aligned} $$

(2.23)

Choosing large enough $M>0$, the conclusion follows from (2.23) for small enough $\epsilon >0$. □

Theorem 2.9

For any $\lambda >0$, $T>0$, problem (2.4) has a nontrivial solution $u_{\lambda }$with $I_{\lambda ,T}(u_{\lambda })=c_{\lambda ,T}$.

Proof

Since the functional $I_{\lambda ,T}$ contains the mountain pass geometry and satisfies the $(\mathit{PS})_{c}$ condition, the mountain pass theorem [14] implies that there exists a critical point $u_{\lambda }\in X$. Moreover, $I_{\lambda ,T}(u_{\lambda })=c_{\lambda ,T}\geq \alpha > 0 = I(0)$, so that $u_{\lambda }$ is a nontrivial solution. □

3 Proof of Theorem 1.1

In this section, we prove our main result. Our approach is based on showing that the solution obtained in Theorem 2.9 satisfies the estimate $|u_{\lambda }|_{\infty }\leq T$. This implies that $u_{\lambda }$ is indeed the solution to the original problem (1.1). The following lemma plays a fundamental role in the study of the existence of the nontrivial solution to problem (1.1), and its proof involves some arguments explored in [17, 18] and involves the use of the Nash–Moser method [19].

Lemma 3.1

If u is a critical point of $I_{\lambda ,T}$, then $u\in L^{\infty }(\mathbb{R}^{3})$and

$$ \vert u \vert _{\infty } \leq C_{0}^{\frac{1}{2(\eta -1)}}\eta ^{\eta /(\eta -1)^{2}} \bigl[\bigl(\lambda C^{*}_{T}+\alpha ( \epsilon , u)\bigr) \bigl(1+ \vert u \vert _{2}\bigr)^{2}+ \lambda C_{T} \vert u \vert _{6}^{p-2} \bigr]^{\frac{1}{2(\eta -1)}} \vert u \vert _{6}^{ \kappa }, $$

where $C_{0}>0$and $\kappa \leq 1$are constants independent of λ and T, $\eta =(8-p)/2$.

Proof

Let $A_{k}=\{x\in \mathbb{R}^{3},|u|^{s-1}\leq k\}$, $B_{k}=\mathbb{R}^{3} \backslash A_{k}$, where $s>1$, $k>0$. Define

$$ u_{k}= \textstyle\begin{cases} u \vert u \vert ^{2(s-1)}, &x\in A_{k}, \\ k^{2}u, & x\in B_{k}, \end{cases} $$

and

$$ w_{k}= \textstyle\begin{cases} u \vert u \vert ^{s-1}, &x\in A_{k}, \\ ku, & x\in B_{k}. \end{cases} $$

Then $u_{k}, w_{k}\in X$, $|u_{k}|\leq |u|^{2s-1}$, and $w_{k}^{2}=u u_{k}\leq |u|^{2s}$. It is easy to check that

$$\begin{aligned}& \nabla u_{k}= \textstyle\begin{cases} (2s-1) \vert u \vert ^{2s-2}\nabla u, &x\in A_{k}, \\ k^{2}\nabla u, & x\in B_{k}, \end{cases}\displaystyle \\& \nabla w_{k}= \textstyle\begin{cases} s \vert u \vert ^{s-1}\nabla u, &x\in A_{k}, \\ k\nabla u, & x\in B_{k}, \end{cases}\displaystyle \end{aligned}$$

and

$$ \int _{\mathbb{R}^{3}}\bigl( \vert \nabla w_{k} \vert ^{2}-\nabla u \nabla u_{k}\bigr)\,dx=(s-1)^{2} \int _{A_{k}} \vert u \vert ^{2(s-1)} \vert \nabla u \vert ^{2}\,dx. $$

(3.1)

Observe that

$$ \begin{aligned}[b] & \int _{\mathbb{R}^{3}}\nabla u \cdot \nabla u_{k}\,dx \\ &\quad=(2s-1) \int _{A_{k}} \vert u \vert ^{2(s-1)} \vert \nabla u \vert ^{2}\,dx+k^{2} \int _{B_{k}} \vert \nabla u \vert ^{2}\,dx \\ &\quad\geq (2s-1) \int _{A_{k}} \vert u \vert ^{2(s-1)} \vert \nabla u \vert ^{2}\,dx. \end{aligned} $$

(3.2)

Therefore, by (3.1)–(3.2), we obtain that $\int _{\mathbb{R}^{3}}\nabla u \cdot \nabla u_{k}\,dx\geq 0$ and

$$ \int _{\mathbb{R}^{3}} \vert \nabla w_{k} \vert ^{2}\,dx\leq s^{2} \int _{\mathbb{R}^{3}} \nabla u \cdot \nabla u_{k}\,dx. $$

(3.3)

Using $u_{k}$ as a test function in (2.5), we obtain

$$ \int _{\mathbb{R}^{3}} \bigl(\nabla u\nabla u_{k} +V(x)u u_{k}+ \phi _{u}uu_{k} \bigr)\,dx= \int _{\mathbb{R}^{3}} \bigl(\lambda g_{T}(u)u_{k}+ \vert u \vert ^{4}uu_{k} \bigr) \,dx. $$

Together with (3.3), this shows that

$$ \int _{\mathbb{R}^{3}} \vert \nabla w_{k} \vert ^{2}\,dx\leq s^{2} \biggl( \int _{ \mathbb{R}^{3}}\lambda g_{T}(u)u_{k}\,dx+ \int _{\mathbb{R}^{3}} \vert u \vert ^{4}uu_{k} \,dx \biggr). $$

By a version of the Brézis–Kato lemma, as in [20, Lemma 2.5], for any $\epsilon > 0$, there exists $\alpha (\epsilon , u)$ such that

$$ \int _{\mathbb{R}^{3}} \vert u \vert ^{4}w_{k}^{2} \,dx\leq \epsilon \int _{ \mathbb{R}^{3}} \vert \nabla w_{k} \vert ^{2}\,dx+\alpha (\epsilon , u) \int _{ \mathbb{R}^{3}} \vert w_{k} \vert ^{2} \,dx. $$

Choosing $\epsilon =\frac{1}{2s^{2}}$, we obtain

$$ \int _{\mathbb{R}^{3}} \vert \nabla w_{k} \vert ^{2}\,dx\leq 2s^{2} \biggl( \int _{ \mathbb{R}^{3}}\lambda g_{T}(u)u_{k}\,dx+ \alpha (\epsilon , u) \int _{ \mathbb{R}^{3}} \vert w_{k} \vert ^{2} \,dx \biggr). $$

(3.4)

By $(g_{3})$ and $w_{k}^{2}=uu_{k}$, we obtain

$$ \bigl\vert g_{T}(u)u_{k} \bigr\vert \leq C^{*}_{T}w_{k}^{2}+ C_{T} \vert u \vert ^{p-2}w_{k}^{2}. $$

(3.5)

By the Sobolev embedding theorem, (3.4)–(3.5), and the Hölder inequality, we obtain

$$ \begin{aligned}[b] \biggl( \int _{A_{k}} \vert w_{k} \vert ^{6} \biggr)^{1/3}&\leq S^{-1} \int _{ \mathbb{R}^{3}} \vert \nabla w_{k} \vert ^{2}\,dx \\ &\leq S^{-1}2s^{2} \biggl[ \int _{\mathbb{R}^{3}}\lambda \bigl(C^{*}_{T}w_{k}^{2}+ C_{T} \vert u \vert ^{p-2}w_{k}^{2} \bigr)\,dx+\alpha (\epsilon , u) \int _{ \mathbb{R}^{3}} \vert w_{k} \vert ^{2} \,dx \biggr] \\ &\leq S^{-1}2s^{2} \bigl[\bigl(\lambda C^{*}_{T}+ \alpha (\epsilon , u)\bigr) \vert w_{k} \vert _{2}^{2}+ \lambda C_{T} \vert u \vert _{6}^{p-2} \vert w_{k} \vert _{2q}^{2} \bigr], \end{aligned} $$

(3.6)

where $q=\frac{6}{8-p}\in (\frac{3}{2},3)$. Recalling that $|w_{k}|\leq |u|^{s}$ and $|w_{k}|= |u|^{s}$ for $x\in A_{k}$, together with (3.6), we obtain that

$$ \biggl( \int _{A_{k}} \vert u \vert ^{6s} \biggr)^{1/3}\leq S^{-1}2s^{2} \bigl[\bigl( \lambda C^{*}_{T}+\alpha (\epsilon , u)\bigr) \vert u \vert _{2s}^{2s}+\lambda C_{T} \vert u \vert _{6}^{p-2} \vert u \vert _{2sq}^{2s} \bigr]. $$

(3.7)

Moreover, by the interpolation inequality, we obtain $|u|_{2s}\leq |u|_{2}^{1-\sigma }|u|_{2qs}^{\sigma }$, where $\sigma \in (0,1)$ satisfying $\frac{1}{2s}=\frac{1-\sigma }{2}+\frac{\sigma }{2sq}$, that is, $\sigma =\frac{q(s-1)}{qs-1}$. Consequently, since $2s(1-\sigma )=2+\frac{2(1-s)}{qs-1}<2$, we obtain

$$ \vert u \vert _{2s}^{2s}\leq \vert u \vert ^{2s(1-\sigma )}_{2} \vert u \vert ^{2s\sigma }_{2sq} \leq \bigl(1+ \vert u \vert _{2}\bigr)^{2} \vert u \vert ^{2s \sigma }_{2sq}. $$

(3.8)

Letting $k\rightarrow \infty$, from (3.7)–(3.8), we obtain

$$ \begin{aligned}[b] \vert u \vert _{6s} & \leq \bigl(S^{-1}2s^{2}\bigr)^{\frac{1}{2s}} \bigl[\bigl( \lambda C^{*}_{T}+ \alpha (\epsilon , u)\bigr) \bigl(1+ \vert u \vert _{2}\bigr)^{2} \vert u \vert _{2sq}^{2s\sigma }+\lambda C_{T} \vert u \vert _{6}^{p-2} \vert u \vert _{2sq}^{2s} \bigr]^{\frac{1}{2s}} \\ &\leq C_{0}^{\frac{1}{2s}}s^{\frac{1}{s}} \bigl[\bigl(\lambda C^{*}_{T}+ \alpha (\epsilon , u)\bigr) \bigl(1+ \vert u \vert _{2}\bigr)^{2}+\lambda C_{T} \vert u \vert _{6}^{p-2} \bigr]^{\frac{1}{2s}} \vert u \vert _{2sq}^{\kappa }, \end{aligned} $$

(3.9)

where $\kappa =\{\sigma ,1\}$, $C_{0}=\max \{2S^{-1},1\}$. Let $\eta =\frac{6}{2q}$, then $\eta \in (1,2)$. We now perform j iterations by setting $s_{j} = \eta ^{j}$ in (3.9) and obtain that

$$ \begin{aligned}[b] & \vert u \vert _{6\eta ^{j}} \\ &\quad\leq C_{0}^{\frac{1}{2}\sum _{j=1}^{\infty } \frac{1}{\eta ^{j}}}\eta ^{\sum _{j=1}^{\infty } \frac{j}{\eta ^{j}}} \bigl[ \bigl(\lambda C^{*}_{T}+\alpha \bigr) \bigl(1+ \vert u \vert _{2}\bigr)^{2}+ \lambda C_{T} \vert u \vert _{6}^{p-2} \bigr]^{\frac{1}{2}\sum _{j=1}^{ \infty }\frac{1}{\eta ^{j}}} \vert u \vert _{6}^{\kappa _{1}\cdots \kappa _{j}}, \end{aligned} $$

(3.10)

where $\sigma _{j}=q(\eta _{j}-1)/(q \eta ^{j}-1)<1$, $\kappa _{j}=\{\sigma _{j},1 \}\leq 1$. By a simple calculation, we obtain that $\sum_{j=1}^{\infty }1/\eta ^{j}=\frac{1}{\eta -1}$, $\sum_{j=1}^{ \infty }j/\eta ^{j}=\frac{\eta }{(\eta -1)^{2}}$. We will divide the study of $|u|_{\infty }$ into two cases.

(i) If $|u|_{6}\geq 1$, then $|u|_{6}^{\kappa _{1}\kappa _{2}\cdots \kappa _{j}}\leq |u|_{6}$. Letting $j\rightarrow \infty$ in (3.10), we obtain

$$ \vert u \vert _{\infty } \leq C_{0}^{\frac{1}{2(\eta -1)}}\eta ^{ \frac{\eta }{(\eta -1)^{2}}} \bigl[\bigl(\lambda C^{*}_{T}+\alpha ( \epsilon , u)\bigr) \bigl(1+ \vert u \vert _{2}\bigr)^{2}+ \lambda C_{T} \vert u \vert _{6}^{p-2} \bigr]^{ \frac{1}{2(\eta -1)}} \vert u \vert _{6}. $$

(ii) If $|u|_{6}< 1$ from $\sigma _{j}=\frac{q(\eta ^{j}-1)}{q\eta ^{j}-1}\geq 1- \frac{1}{\eta ^{j}}$ and $\kappa _{j}=\{\sigma _{j},1\}$, then for any $j\in \mathbb{N}$, we obtain

$$ 0< \sigma _{1}\sigma _{2}\cdots \sigma _{j}\leq \kappa _{1}\kappa _{2} \cdots \kappa _{j}. $$

It can be easily seen that $\ln (1-s)\geq -s-\frac{s^{2}}{2(1-s)^{2}}$, $s\in (0,1)$, implying that

$$ \sum_{i=1}^{j}\ln \kappa _{i} \geq \sum_{i=1}^{j}\ln \sigma _{i} \geq -\sum_{i=1}^{j} \frac{1}{\eta ^{i}}-\frac{1}{2}\sum_{i=1}^{j} \frac{1}{(\eta ^{i}-1)^{2}}. $$

By a direct calculation, we can conclude that

$$ \sum_{i=1}^{\infty }\frac{1}{(\eta ^{i}-1)^{2}}\leq \frac{\eta ^{2}}{(\eta ^{2}-1)(\eta -1)^{2}}. $$

Hence, we obtain that

$$ \sum_{i=1}^{\infty }\ln \kappa _{i} \geq -\frac{1}{\eta -1}- \frac{\eta ^{2}}{2(\eta ^{2}-1)(\eta -1)^{2}}:=\theta . $$

Therefore, $\kappa _{1}\kappa _{2}\cdots \kappa _{j}\geq e^{\theta }$, $\forall j \in \mathbb{N}$. Consequently, by $|u|_{6}< 1$ we obtain that $|u|_{6}^{\kappa _{1}\kappa _{2}\cdots \kappa _{j}}\leq |u|_{6}^{ e^{ \theta }}$. Similarly, letting $j\rightarrow \infty$ in (3.10), we obtain

$$ \vert u \vert _{\infty } \leq C_{0}^{\frac{1}{2(\eta -1)}}\eta ^{ \frac{\eta }{(\eta -1)^{2}}} \bigl[\bigl(\lambda C^{*}_{T}+\alpha ( \epsilon , u)\bigr) \bigl(1+ \vert u \vert _{2}\bigr)^{2}+ \lambda C_{T} \vert u \vert _{6}^{p-2} \bigr]^{ \frac{1}{2(\eta -1)}} \vert u \vert _{6}^{e^{\theta }}. $$

Let $\kappa = 1$ or $\kappa =e^{\theta }\leq 1$. The proof is complete. □

We are now ready to prove the main result of the paper.

Proof of Theorem 1.1

Let $u\in C^{\infty }_{0}(\mathbb{R}^{3})$ and $u(x)\leq 0$, then $G_{T}(u)=0$. Hence,

$$ I_{\lambda ,T}(tu)=\frac{t^{2}}{2} \int _{\mathbb{R}^{3}} \bigl( \vert \nabla u \vert ^{2}+V(x)u^{2} \bigr)\,dx+\frac{t^{4}}{4}\varPhi (u) - \frac{t^{6}}{6} \int _{\mathbb{R}^{3}} \vert u \vert ^{6}\,dx\rightarrow - \infty ,\quad \text{as } t\rightarrow +\infty . $$

Therefore, there exists $t_{0}>0$ such that $I_{\lambda ,T}(t_{0}u)<0$. Let $\gamma (\cdot )=tt_{0}u$, $t\in [0,1]$, we get $\gamma (t)\in \varGamma$. Since $G_{T}(u)=0$, for all $t\in [0,1]$, we obtain

$$ c_{\lambda ,T}\leq \max_{t\in [0,1]}I_{\lambda ,T}\bigl(\gamma (t)\bigr)\leq \max_{t\geq 0} \biggl(\frac{t^{2}}{2} \Vert u \Vert ^{2}+\frac{t^{4}}{4}\varPhi (u)- \frac{t^{6}}{6} \vert u \vert _{6}^{6} \biggr):=D>0, $$

where D is a constant independent of λ and T. From Theorem 2.9, $(g_{4})$, and $(V)$, we obtain

$$ 4D\geq 4c_{\lambda ,T}=4I_{\lambda ,T}- \bigl\langle I'_{\lambda ,T}(u_{ \lambda }), u_{\lambda } \bigr\rangle \geq \frac{1}{2} \Vert u_{\lambda } \Vert ^{2}+ \biggl( \frac{V_{0}}{2}-\lambda \mu \biggr) \vert u_{\lambda } \vert _{2}^{2}. $$

(3.11)

We can choose $\lambda _{0}>0$ such that $\frac{V_{0}}{2}-\lambda _{0}\mu >0$. Therefore, based on (3.11), $\|u_{\lambda }\|\leq 8D$. Hence, we conclude that $|u_{\lambda }|_{2}\leq C_{4}$, $|u_{\lambda }|_{6}^{2}\leq C_{5}$, where $C_{4}, C_{5}>0$ independent of λ, T. From Lemma 3.1, we obtain

$$ \vert u_{\lambda } \vert _{\infty } \leq C_{0}^{\frac{1}{2(\eta -1)}} \eta ^{ \frac{\eta }{(\eta -1)^{2}}} \bigl[\bigl(\lambda C^{*}_{T}+\alpha ( \epsilon , u)\bigr) (1+C_{4})^{2}+\lambda C_{T}C_{5}^{p-2} \bigr]^{ \frac{1}{2(\eta -1)}}C_{5}^{\kappa }. $$

Hence, we first choose $T>0$ large enough such that

$$ C_{0}^{\frac{1}{2(\eta -1)}}\eta ^{\frac{\eta }{(\eta -1)^{2}}} \bigl[\bigl( \alpha ( \epsilon , u)\bigr) (1+C_{4})^{2} \bigr]^{\frac{1}{2(\eta -1)}}C_{5}^{ \kappa } \leq \frac{T}{2}. $$

Since $C^{*}_{T}$, $C_{T}$ are fixed constants for above T, we can choose $\lambda _{1}<\lambda _{0}$ such that

$$ \vert u_{\lambda } \vert _{\infty } \leq C_{0}^{\frac{1}{2(\eta -1)}} \eta ^{ \frac{\eta }{(\eta -1)^{2}}} \bigl[\bigl(\lambda _{1} C^{*}_{T}+ \alpha ( \epsilon , u)\bigr) (1+C_{4})^{2}+\lambda _{1} C_{T}C_{5}^{p-2} \bigr]^{ \frac{1}{2(\eta -1)}}C_{5}^{\kappa }\leq T. $$

Then, for $\lambda \in (0,\lambda _{1})$, we can obtain $|u_{\lambda }|_{\infty }\leq T$, and $u_{\lambda }$ is also a solution to the original problem (1.1). The proof of the theorem is now complete. □