1 Introduction and main results

In this paper, we are concerned with the fractional Schrödinger–Poisson system

$$ \textstyle\begin{cases} (-\Delta )^{s}u+V(x)u+K(x)\phi u=f(x,u), & x \in \mathbb{R}^{3}, \\ (-\Delta )^{t}\phi =K(x)u^{2}, & x \in \mathbb{R}^{3}, \end{cases} $$
(1.1)

where \((-\Delta )^{s}\) is fractional Laplacian operator, \(s,t\in (0,1)\), \(4s+2t>3\).

On the potential \(V(x)\), we make the following assumptions:

(\(V_{1}\)):

\(V(x)\in C(\mathbb{R}^{3},\mathbb{R})\), \(\inf_{x\in \mathbb{R}^{3}}V(x)>0\).

(\(V_{2}\)):

For any \(b>0\) such that the set \(\{x\in \mathbb{R} ^{3}: V(x)< b\}\) is nonempty and has finite Lebesgue measure. In some previous papers, except for \((V_{1})\)\((V_{2})\), the following, \((V_{3})\), is needed.

(\(V_{3}\)):

\(\varOmega =\operatorname{int}V^{-1}(0)\) is nonempty and has smooth boundary and \(\overline{\varOmega }=V^{-1}(0)\).

The potential \(V(x)\) with assumptions \((V_{1})\)\((V_{3})\) are usually referred as the steep well potential. It was firstly proposed by Bartsch and Wang [2] to study a nonlinear Schrödinger equation. Especially, \((V_{1})\)\((V_{2})\) are used to guarantee the compactness of the space.

When \(\phi =0\), the system (1.1) reduces to a fractional Schrödinger equation, which is a fundamental equation of fractional quantum mechanics. It was firstly introduced by Laskin [9, 10] as a result of extending the Feynman path integral, from the Brownian-like to the Lévy-like quantum mechanical paths, where the classical Schrödinger equation changes into the fractional Schrödinger equation. Recently, nonlocal fractional problems have attracted much attention, we refer to [12].

When \(s=t=1\), \(K(x)=1\), the system (1.1) reduces to the following Schrödinger–Poisson system (or Schrödinger–Maxwell system):

$$ \textstyle\begin{cases} -\Delta u+V(x)u+\phi u=f(x,u), & x \in \mathbb{R}^{3} ,\\ - \Delta \phi =u^{2}, & x \in \mathbb{R}^{3}. \end{cases} $$
(1.2)

Due to the real physical meaning, it has been extensively investigated. Benci and Fortunato [4] firstly proposed the system like (1.2) to describe solitary waves for nonlinear Schrödinger type equations and look for the existence of standing waves interacting with unknown electrostatic field. Kristály and Repovš [8] studied a coupled Schrödinger–Maxwell system with the nonlinear term \(f:\mathbb{R}\rightarrow \mathbb{R}\) being superlinear at zero and sublinear at infinity. Under different conditions, they proved a non-existence result and obtained the existence of at least two non-trivial solutions.

There are plenty of results for system (1.2), we refer the interested reader to [3, 5, 13, 17, 19, 25] and the references therein, the main tool is the mountain pass theory [15]. However, to the best of our knowledge, similar results on the fractional Schrödinger–Poisson systems are not so rich as the Schrödinger–Poisson systems (1.2). Zhang, do Ó and Squassina [24] studied the fractional Schrödinger–Poisson system with a general nonlinearity in the subcritical and critical case,

$$ \textstyle\begin{cases} (-\Delta )^{s}u+\lambda \phi u=f(u), & x \in \mathbb{R}^{3}, \\ (- \Delta )^{t}\phi =\lambda u^{2}, & x \in \mathbb{R}^{3}, \end{cases} $$

where \(\lambda > 0\), \(s,t \in [0, 1]\), \(4s + 2t \geq 3\). With some hypotheses, a non-trivial positive radial solution is admitted. Very recently, Teng [21] considered the following nonlinear fractional Schrödinger–Poisson system with critical Sobolev exponent:

$$ \textstyle\begin{cases} (-\Delta )^{s}u+V(x)u+\phi u=\mu \vert u \vert ^{q-1}+ \vert u \vert ^{2^{\ast }_{s}-2}u, & x \in \mathbb{R}^{3}, \\ (-\Delta )^{t}\phi =u^{2}, & x\in \mathbb{R}^{3}, \end{cases} $$
(1.3)

under some appropriate conditions on \(V(x)\), where \(\mu \in \mathbb{R^{+}}\) is a parameter, \(1< q<2^{\ast }_{s}-1= \frac{3+2s}{3-2s}\), \(s,t\in (0,1)\) and \(2s+2t>3\), the existence of a non-trivial ground state solution of system (1) can be proved. Later, Li [11] studied the nonlinear fractional Schrödinger–Poisson equation

$$ \textstyle\begin{cases} (-\Delta )^{s}u+u+\phi u=f(x,u), & x \in \mathbb{R}^{3}, \\ (-\Delta )^{t}\phi =u^{2}, & x \in \mathbb{R}^{3}, \end{cases} $$

where \(s,t\in (0,1]\), \(4s + 2t >3\). Under some assumptions on f, the existence of non-trivial solutions for this system is obtained.

Motivated by all the works just described above, we want to find the existence and multiplicity of non-trivial solutions for the fractional Schrödinger–Poisson with superlinear terms, the following assumptions are needed:

(K):

\(K(x)\in L^{\frac{6}{4s+2t-3}}(\mathbb{R}^{3})\bigcup L ^{\infty }(\mathbb{R}^{3})\), \(s, t\in (0,1)\), \(4s+2t>3\), \(K\geq 0\), \(\forall x\in \mathbb{R}^{3}\).

\((f_{1})\) :

\(\lim_{|t|\rightarrow \infty } F(x,t)/{t^{4}}=+ \infty \) a.e. \(x\in \mathbb{R}^{3}\), and there exists \(r_{1}>0\) such that

$$ F(x,t)\geq 0, \quad \forall x \in \mathbb{R}^{3}, \vert t \vert \geq r_{1}, $$

where \(F(x,t)=\int ^{x}_{0} f(x,t)\,\mathrm{d}x\).

(\(f_{2}\)):

There exist constant \(a>0\), \(p\in (2,2^{\ast }_{2})\) such that

$$\begin{aligned} \bigl\vert f(x,t) \bigr\vert \leq a\bigl(t+{ \vert t \vert }^{p-1}\bigr), \quad \forall (x,t) \in \mathbb{R}^{3} \times \mathbb{R}, \end{aligned}$$

where \(2^{\ast }_{s}=\frac{6}{3-2s}\).

(\(f_{3}\)):

There exists \(L>0\) such that

$$\begin{aligned} \frac{1}{4}f(x,t)-F(x,t)\geq 0, \quad \forall x\in \mathbb{R}, \vert t \vert \geq L. \end{aligned}$$
(\(f_{4}\)):

\(f(x,t)=f(x,-t)\), \(\forall (x,t) \in \mathbb{R} ^{3}\times \mathbb{R}\).

Now we are ready to state the main result of this paper as follows.

Theorem 1.1

Suppose that system (1.1) satisfies \((V_{1})\)\((V_{2})\), \((K)\), and \((f_{1})\)\((f_{4})\), then (1.1) admits infinitely many non-trivial solutions \(\{(u_{k},\phi ^{t}_{k})\}\) such that

1 2 R 3 ( | ( Δ ) s 2 u k | 2 + V ( x ) u k 2 ) d x + 1 4 R 3 K ( x ) ϕ k t u k 2 d x R 3 F ( x , u k ) d x + , k .

2 Variational settings and preliminaries

Let \(L^{r}(\mathbb{R}^{3})(0\leq r<\infty )\) be the usual Lebesgue space with the standard norm \(\|u\|_{r}\) and û as the Fourier transform of u. Firstly let us introduce some necessary variational settings for system (1.1). A complete introduction to fractional Sobolev spaces can be found in [1]. Recall that the fractional Sobolev spaces \(H^{s}( \mathbb{R}^{3})\) can be described by the Fourier transform, that is,

$$ H^{s}\bigl(\mathbb{R}^{3}\bigr)= \biggl\{ u\in L^{2}\bigl(\mathbb{R}^{3}\bigr): \int _{\mathbb{R}^{3}} \vert \xi \vert ^{2s} \bigl\vert \hat{u(\xi )} \bigr\vert ^{2} + \bigl\vert \hat{u(\xi )} \bigr\vert ^{2} \,\mathrm{d} \xi < \infty \biggr\} $$

equipped with the norm

$$ \Vert u \Vert ^{2}_{H^{s}(\mathbb{R}^{3})}: = \biggl( \int _{\mathbb{R}^{3}} \vert \xi \vert ^{2s} \bigl\vert \hat{u(\xi )} \bigr\vert ^{2} + \bigl\vert \hat{u(\xi )} \bigr\vert ^{2} \,\mathrm{d} \xi \biggr) ^{\frac{1}{2}}. $$

According to Plancherel’s theorem [7], we have \(\|u\|_{2}=\|\hat{u}\|_{2}\), \(\|(-\Delta )^{\frac{s}{2}}u\|_{2}=\|\xi ^{s}\hat{u}\|_{2}\). Thus

$$ \Vert u \Vert ^{2}_{H^{s}(\mathbb{R}^{3})}: = \int _{\mathbb{R}^{3}}\bigl( \bigl\vert {(-\Delta )}^{\frac{1}{2}}u \bigr\vert ^{2}+u^{2}\bigr)\,\mathrm{d}x. $$

Following [14], the fractional Laplacian \((-\Delta )^{s}\) can be viewed as

$$ (-\Delta )^{s}u(x)=C(s)P.V. \int _{\mathbb{R}^{3}}{\frac{u(x)-u(y)}{ \vert x-y \vert ^{3+2s}} \,\mathrm{d}y,} $$

where \(P.V\). is the principal value and \(C(s)>0\) is a normalization constant.

For \(s\in (0,1) , D^{s,2}(\mathbb{R}^{3})\) is a homogeneous fractional Sobolev space defined as

$$ D^{s,2}\bigl(\mathbb{R}^{3}\bigr)={u\in L^{2^{\ast }_{s}} \bigl(\mathbb{R}^{3}\bigr): \vert \xi \vert ^{s}\hat{u}( \xi ) \in L^{2^{\ast }_{s}}\bigl(\mathbb{R}^{3}\bigr)} , $$

which is the completion of \(C^{\infty }_{0}(\mathbb{R}^{3})\) with respect to the norm

$$ \Vert u \Vert _{D^{s,2}(\mathbb{R}^{3})}=\biggl( \int _{\mathbb{R}^{3}}{ \vert \xi \vert }^{2s} \bigl\vert \hat{u}(\xi ) \bigr\vert ^{2}\,\mathrm{d}\xi \biggr)^{\frac{1}{2}}= \biggl( \int _{\mathbb{R}^{3}} \bigl\vert (- \Delta )^{\frac{s}{2}}u \bigr\vert ^{2}\,\mathrm{d}x\biggr)^{\frac{1}{2}}. $$

We use “→” and “⇀” to denote strong and weak convergence in the related function spaces, respectively. The symbol “↪” means that a function space is continuously embedded into another function space.

Let

$$ E:=\biggl\{ u\in H^{s}\bigl(\mathbb{R}^{3}\bigr): \int _{\mathbb{R}^{3}}\bigl( \bigl\vert (-\Delta )^{ \frac{s}{2}}u \bigr\vert ^{2}+V(x)u^{2}\bigr)\,\mathrm{d}x< \infty \biggr\} . $$

E is endowed with the following inner product and norm:

$$ (u,v)= \int _{\mathbb{R}^{3}}\bigl((-\Delta )^{\frac{s}{2}}u(-\Delta )^{ \frac{s}{2}}v+V(x)uv\bigr)\,\mathrm{d}x ,\quad \Vert u \Vert =(u,u)^{\frac{1}{2}}. $$

Lemma 2.1

(Lemma 2.3 in [20])

Suppose that \(V(x)\) satisfies \((V_{1})\)\((V_{2})\), the Hilbert space E is compactly embedded in \(L^{r}(\mathbb{R}^{3})\) \((2\leq r < 2^{ \ast }_{s})\).

As a consequence of Lemma 2.1, there is constant \(C_{r}>0\) such that

$$ \Vert u \Vert _{r}\leq C_{r} \Vert u \Vert ,\quad \forall u \in E, r\in [2,2^{\ast }_{s}). $$

For any \(u\in H^{s}(\mathbb{R}^{3})\), one can use the Lax–Milgram theorem [6] to find that there exists a unique \(\phi ^{t}_{u}\in D^{t,2}(\mathbb{R}^{3})\) such that

$$ \int _{\mathbb{R}^{3}}(-\Delta )^{\frac{t}{2}}\phi ^{t}_{u}(- \Delta )^{ \frac{t}{2}}v\,\mathrm{d}x= \int _{\mathbb{R}^{3}}K(x)u^{2} v\,\mathrm{d}x, \quad \forall v\in D^{t,2}\bigl(\mathbb{R}^{3}\bigr). $$
(2.1)

In other words, \(\phi ^{t}_{u}\) is the weak solution of the fractional Poisson equation

$$ (-\Delta )^{t}\phi ^{t}_{u}=K(x)u^{2}, \quad x\in \mathbb{R}^{3}, $$

and the representation formula holds, that is,

$$ \phi ^{t}_{u}(x)=c_{t} \int _{\mathbb{R}^{3}} \frac{K(y)u^{2}(y)}{ \vert x-y \vert ^{3-2t}}\,\mathrm{d}y, \quad x\in \mathbb{R}^{3}, $$

which is called the t-Riesz potential (Chap. 5.1 in [18]), where

$$ c_{t}=\pi ^{-\frac{3}{2}}2^{-2t}\frac{\varGamma (\frac{3-2t}{2})}{\varGamma (t)}. $$

Lemma 2.2

\(\forall u\in H^{s}(\mathbb{R}^{3})\), there exists \(C_{0}>0\) such that

$$ \bigl\Vert \phi ^{t}_{u} \bigr\Vert ^{2}_{D^{t,2}}= \int _{\mathbb{R}^{3}}K(x)\phi ^{t}_{u}u ^{2}\,\mathrm{d}x \leq C_{0} \Vert u \Vert ^{4}. $$

Proof

In (2.1), let \(v=\phi ^{t}_{u}\), using the Hölder inequality,

$$\begin{aligned} \bigl\Vert \phi ^{t}_{u} \bigr\Vert ^{2}_{D^{t,2}(\mathbb{R}^{3})} & = \int _{\mathbb{R} ^{3}}K(x)u^{2}\phi ^{t}_{u} \,\mathrm{d}x \\ & \leq \bigl\Vert K(x) \bigr\Vert _{\frac{6}{4s+2t-3}}\biggl( \int _{\mathbb{R}^{3}} \vert u \vert ^{ \frac{6}{3-2s}}\,\mathrm{d}x \biggr)^{\frac{3-2s}{3}}\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \phi ^{t}_{u} \bigr\vert ^{\frac{6}{3-2t}}\,\mathrm{d}x\biggr)^{\frac{3-2t}{6}} \\ & = \bigl\Vert K(x) \bigr\Vert _{\frac{6}{4s+2t-3}} \Vert u \Vert ^{2}_{\frac{3}{3-2s}} \bigl\Vert \phi ^{t} _{u} \bigr\Vert _{\frac{6}{3-2t}} \\ & \leq C \Vert u \Vert ^{2} \bigl\Vert \phi ^{t}_{u} \bigr\Vert _{D^{t,2}(\mathbb{R}^{3})}. \end{aligned}$$

The result follows. □

The energy functional associated to problem (1.1) is given by

$$ I(u)=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \bigl\vert (-\Delta )^{\frac{s}{2}}u \bigr\vert ^{2}+V(x)u ^{2}\bigr)\,\mathrm{d}x+\frac{1}{4} \int _{\mathbb{R}^{3}}K(x)\phi ^{t}_{u}u ^{2}\,\mathrm{d}x- \int _{\mathbb{R}^{3}}F(x,u)\,\mathrm{d}x. $$

Moreover, its differential is

$$ \bigl\langle I'(u),v \bigr\rangle = \int _{\mathbb{R}^{3}}\bigl((-\Delta )^{ \frac{s}{2}}\phi ^{t}_{u}(-\Delta )^{\frac{s}{2}}v+K(x)\phi ^{t}_{u}uv-f(x,u)v\bigr) \,\mathrm{d}x,\quad \forall v\in E. $$
(2.2)

It is clear that the pair \((u,\phi ^{t}_{u})\) is a solution to the system (1.1) if and only if u is a critical point of \(I(u)\).

To prove Theorem 1.1, we need the following lemma (Theorem 9.12 in [16]).

Lemma 2.3

(Symmetric mountain pass theorem)

Let E be a real infinite dimensional Banach space such that \(E=Y\oplus Z\), where Y is finite dimensional subspace. Suppose \(\varPhi \in C^{1}(E,\mathbb{R})\) is an even functional satisfying the Palais–Smale condition, \(\varPhi (0)=0\); if

  1. (i)

    there exist constant ρ, α such that \(\varPhi |_{\partial {B_{\rho }}\bigcap Z \geq \alpha }\), where \(B_{\rho }\) denotes the open ball in E of radius ρ about 0 and \(\partial B_{\rho }\) denotes its boundary;

  2. (ii)

    for arbitrary finite dimensional subspace \(\tilde{E} \subset E\), there exists constant \(R=R(\tilde{E})>0\) such that \(\varPhi (u) \leq 0\) if \(u\in \tilde{E}/{B_{R}}\);

then the functional Φ possesses an unbounded sequence of critical values.

3 Proof of Theorem 1.1

Lemma 3.1

Under the assumptions \((V_{1})\)\((V_{2})\) and \((f_{1})\)\((f_{3})\), \(I(u)\) satisfies the Palais–Smale condition.

Proof

Let \(\{u_{n}\}\subset E\) be the Palais–Smale sequence of I, we assert that \(\{u_{n}\}\) is bounded. Otherwise, there exists a subsequence (for the sake of convenience, we still write it as \(\{u_{n}\}\)) such that \(\|u_{n}\|\rightarrow \infty (n\rightarrow \infty )\). Define \(\omega _{n}:=u_{n}/\|u_{n}\|\), there exists a subsequence such that

$$ \begin{aligned} &w_{n}\rightharpoonup w\quad \text{in } E, \\ &w\rightarrow w \quad \text{in }L^{r}\bigl(\mathbb{R}^{3} \bigr) \bigl(2\leq r\leq 2^{\ast }_{s}\bigr). \\ &w_{n}(x)\rightarrow w\quad \text{a.e. }x\in \mathbb{R}^{3}. \end{aligned} $$

Case 1. \(\omega =0\). The proof of this case is almost the same as the one of Lemma 2.3 in [23], so we omit it.

Case 2. \(\omega \neq 0\). We have

$$ \bigl\vert F(x,t) \bigr\vert \leq \int ^{1}_{0} \bigl\vert f(x,st)t \bigr\vert \, \mathrm{d}s\leq a\bigl(t^{2}+ \vert t \vert ^{p}\bigr), \quad \forall (x,t)\in \mathbb{R}^{3}\times \mathbb{R}, $$
(3.1)

then

$$ \bigl\vert F(x,t) \bigr\vert \leq a\bigl(1+r^{p-2}_{1} \bigr)t^{2}:=c_{2}t^{2}, \quad \forall x \in \mathbb{R}^{3}, \vert t \vert \leq r_{1}. $$

By \((f_{1})\),

$$ \bigl\vert F(x,t) \bigr\vert \geq -c_{2}t^{2}, \quad \forall (x,t)\in \mathbb{R}^{3}\times \mathbb{R}. $$

Let \(\varOmega _{n}(a,b)=\{x\in \mathbb{R}^{3}:a\leq |u_{n}(x)|< b, 0 \leq a< b \}\); we have

$$ \int _{\varOmega _{n}(0,r_{1})}\frac{F(x,u_{n})}{ \Vert u_{n} \Vert ^{4}}\,\mathrm{d}x \geq - \frac{c_{2}\int _{\varOmega _{n}(0,r_{1})}u^{2}_{n}\,\mathrm{d}x}{ \Vert u ^{2}_{n} \Vert ^{4}}\geq -\frac{c_{2} \Vert u_{n} \Vert ^{2}_{2}}{ \Vert u_{n} \Vert ^{4}}. $$

Take the infimum of the inequality, then

$$ \liminf_{n\rightarrow \infty } \int _{\varOmega (0,r_{1})}\frac{F(x,u_{n})}{ \Vert u _{n} \Vert ^{4}}\,\mathrm{d}x \geq 0. $$
(3.2)

 □

If \(\omega \neq 0\), \(|u_{n}(x)|\rightarrow \infty \ (n\rightarrow \infty )\), then, for n sufficiently large, \(\{x\in \mathbb{R}^{3}:\omega (x)\neq 0\}\subset \varOmega _{n}(r_{1},+\infty )\). By \((f_{1})\) and the Fatou lemma,

$$\begin{aligned} \liminf_{n\rightarrow \infty } \int _{\varOmega (r_{1},\infty )}\frac{F(x,u _{n})}{ \Vert u_{n} \Vert ^{4}}\,\mathrm{d}x & =\liminf_{n\rightarrow \infty } \int _{\mathbb{R}^{3}}\frac{ \vert F(x,u_{n}) \vert }{u_{n}^{4}} \chi _{\varOmega _{n}(r_{1},\infty )}\omega ^{4}_{n}\,\mathrm{d}x \\ & \geq \int _{\mathbb{R}^{3}}\liminf_{n\rightarrow \infty }\frac{ \vert F(x,u _{n}) \vert }{u_{n}^{4}}\chi _{\varOmega _{n}(r_{1},\infty )}\omega ^{4}_{n}\,\mathrm{d}x \\ & =+\infty . \end{aligned}$$

Combined with (3.2) and Lemma 2.2, we have

$$ \begin{aligned}[b] 0 & =\lim_{n\rightarrow \infty } \frac{J(u_{n})}{ \Vert u_{n} \Vert ^{4}} \\ & \leq \lim_{n\rightarrow \infty }\frac{1}{ \Vert u_{n} \Vert ^{4}}\biggl( \frac{ \Vert u_{n} \Vert ^{2}}{2}+C_{0} \Vert u_{n} \Vert ^{4}- \int _{\mathbb{R}^{3}} F(x,u _{n})\,\mathrm{d}x\biggr) \\ & \leq C_{0}-\liminf_{n \rightarrow \infty }{} \int _{\varOmega _{n}(0,r_{1})}+ \int _{\varOmega _{n}(r_{1},+\infty )}\frac{F(x,u _{n})}{ \Vert u_{n} \Vert ^{4}}\,\mathrm{d}x \\ & =-\infty , \end{aligned} $$
(3.3)

a contradiction, so the sequence \({u_{n}}\) is bounded.

Since the sequence \(\{u_{n}\}\) is bounded, there exists a subsequence (we still write it as \(u_{n}\)) such that

$$ \begin{aligned} &u_{n}\rightharpoonup u \quad \text{in } E, \\ &u_{n}\rightarrow u \quad \text{in } L^{r} \bigl( \mathbb{R}^{3}\bigr)\ \bigl(2\leq r< 2^{\ast }_{s} \bigr), \\ &u_{n}(x)\rightarrow u \quad \text{a.e. } x\in \mathbb{R}^{3}. \end{aligned} $$
(3.4)

To prove \(u_{n}\rightarrow u\) in E, we need to prove \(\|u_{n}\| \rightarrow \|u\|\) (this is because E is a Hilbert space). By (2.2)

$$ \begin{aligned}[b] o(1) & = \bigl\langle J'(u_{n})-J'(u),u_{n}-u \bigr\rangle \\ & = (u_{n},u_{n}-u)+ \int _{\mathbb{R}^{3}}K(x)\phi ^{t}_{u_{n}}u_{n}(u_{n}-u) \,\mathrm{d}x- \int _{\mathbb{R}^{3}}f(x,u_{n}) (u_{n}-u)\, \mathrm{d}x \\ & = \bigl\Vert u_{n} \vert ^{2}|-\|u \bigr\Vert ^{2}+ \int _{\mathbb{R}^{3}}K(x)\phi ^{t}_{u_{n}}u_{n}(u_{n}-u) \,\mathrm{d}x- \int _{\mathbb{R}^{3}}f(x,u_{n}) (u_{n}-u)\, \mathrm{d}x. \end{aligned} $$
(3.5)

With \((f_{1})\) and the second limit of (3.4),

$$ \begin{aligned}[b] & \biggl\vert \int _{\mathbb{R}^{3}}f(x,u_{n}) (u_{n}-u)\, \mathrm{d}x \biggr\vert \\ &\quad \leq a \int _{\mathbb{R}^{3}}\bigl( \vert u_{n}\|u_{n}-u \vert + \vert u_{n} \vert ^{p-1} \vert u_{n}-u \vert \bigr) \,\mathrm{d}x \\ &\quad \leq a\bigl( \Vert u_{n} \Vert _{2} \Vert u_{n}-u \Vert _{2}+ \Vert u_{n} \Vert ^{p-1} _{p} \Vert u_{n}-u \Vert _{p}\bigr)\stackrel{n\rightarrow \infty }{\longrightarrow } 0. \end{aligned} $$
(3.6)

For \(K\in L^{\infty }\), by \((K)\), we have

$$ \begin{aligned}[b] & \biggl\vert \int _{\mathbb{R}^{3}}K(x)\phi ^{t}_{u_{n}}u_{n}(u_{n}-u)\,\mathrm{d}x \biggr\vert \\ & \quad \leq \Vert K \Vert _{\infty } \bigl\Vert \phi ^{t}_{u_{n}} \bigr\Vert _{\frac{6}{3-2t}} \Vert u_{n} \Vert _{ \frac{12}{3+2t}} \Vert u_{n}-u \Vert _{\frac{12}{3+2t}}\stackrel{n\rightarrow \infty }{\longrightarrow }0. \end{aligned} $$
(3.7)

For \(K\in L^{\frac{6}{4s+2t-3}}\), we have

$$ \begin{aligned}[b] & \biggl\vert \int _{\mathbb{R}^{3}}K(x)\phi ^{t}_{u_{n}}u_{n}(u_{n}-u)\,\mathrm{d}x \biggr\vert \\ & \quad \leq \Vert K \Vert _{|\frac{6}{4s+2t-3}} \bigl\Vert \phi ^{t}_{u_{n}} \bigr\Vert _{\frac{6}{3-2t}} \Vert u _{n} \Vert _{\frac{6}{3-2s}} \Vert u_{n}-u \Vert _{\frac{6}{3-2s}}\stackrel{n\rightarrow \infty }{\longrightarrow }0. \end{aligned} $$
(3.8)

Combined with (3.5)–(3.8), we can conclude \(\|u_{n}\| \rightarrow \|u\|\).

Lemma 3.2

Suppose that \((V_{1})\)\((V_{2})\) and \((f_{1})\)\((f_{2})\) hold, then, for every finite subspace \(\tilde{E}\subset E\), it follows that

$$ J(u)\rightarrow -\infty , \quad \Vert u \Vert \rightarrow \infty , x\in \tilde{E}. $$

Proof

Suppose there exists \(\{u_{n}\}\subset E\) such that \(\|u_{n}\|\rightarrow \infty \) and \(\inf_{n\rightarrow \infty } J(u_{n})>\infty \). Let \(w_{n}:=u_{n}/\|u_{n}\|\), \(\|w_{n}\|=1\) and there exists \(w\in E \backslash \{0\}\) such that

$$ \begin{aligned} &w_{n}\rightharpoonup w\quad \text{in }E, \\ &w_{n}\rightarrow w \quad \text{in } L^{r}\bigl( \mathbb{R}^{3}\bigr) \bigl(2\leq r \leq 2^{\ast } _{s} \bigr). \\ &w_{n}(x)\rightarrow w \quad \text{a.e. } x\in \mathbb{R}^{3}. \end{aligned} $$

Similarly to the proof of (3.3), we can get a contradiction. □

Corollary 3.3

Suppose that \((V_{1})\)\((V_{2})\) and \((f_{1})\)\((f_{2})\) hold, then, for every finite subspace \(\tilde{E}\subset E\), there exists \(R=R(\tilde{E})>0\) such that

$$ J(u)\leq 0,\quad \forall u\in \tilde{E}, \Vert u \Vert \geq R. $$

Let \(\{e_{i}\}^{\infty }_{i=1}\) be orthogonal bases of space E, let \(X_{i}=\mathbb{R}e_{i}:=\{\alpha e_{i}:\alpha \in \mathbb{R}\}\),

$$ Y_{k}=\bigoplus ^{k}_{i=1}X_{i},\qquad Z_{k}=\bigoplus ^{\infty }_{i=k+1}X_{i}, \quad \forall k\in \mathbb{Z}. $$

Lemma 3.4

(Lemma 3.8 in [22])

Suppose that \((V_{1})\)\((V_{2})\) hold, if \(2\leq r<2^{\ast }\), we have

$$ \beta _{k}(r):=\sup_{u\in Z_{k}, \Vert u \Vert =1} \Vert u \Vert _{r}\rightarrow 0 \quad (k \rightarrow \infty ). $$

Proof

It is obvious that \(0<\beta _{k+1}\leq \beta _{k}\), so that \(\beta _{k} \rightarrow \beta \geq 0\), \(k\rightarrow \infty \). For every \(k\geq 0\), there exists \(u_{k}\in Z_{k}\) such that \(\|u\|=1\) and \(\|u_{k}\|_{r} \geq \beta _{k}/2\). By definition of \(Z_{k}, u_{k} \rightarrow 0\) in \(H^{s}(\mathbb{R}^{3})\). The Sobolev embedding theorem implies that \(u_{k}\rightarrow 0\) in \(L^{r}(\mathbb{R}^{3})\). Thus we have proved that \(\beta =0\). □

Lemma 3.4 shows there exists positive constant \(k_{1},k_{2} \geq 1\) such that

$$ \begin{aligned} &\beta _{k}(2)\leq (2 \sqrt{2a})^{-1}, \quad \forall k\geq k_{1}, \\ &\beta _{k}(p)\leq a^{-\frac{1}{p}}, \quad \forall k\geq k_{2}. \end{aligned} $$
(3.9)

Lemma 3.5

Suppose that \((V_{1})\)\((V_{2})\) and \((f_{2})\) hold, let \(k_{3}= \max \{k_{1},k_{2}\}\), there exists constant \(\beta ,\alpha >0\) such that \(\varPhi |_{\partial B_{\rho }\bigcap Z_{k_{3}}\geq \alpha }\).

Proof

According to (3.1) and (3.9), we have

$$\begin{aligned} J(u) & \geq \frac{1}{2} \Vert u \Vert ^{2}- \int _{\mathbb{R}^{3}} \bigl\vert F(x,u) \bigr\vert \,\mathrm{d}x \\ & \geq \biggl(\frac{1}{2}-a\beta ^{2}_{k}(2)\biggr) \Vert u \Vert ^{2}-a\beta ^{p}_{k}(p) \Vert u \Vert ^{p} \\ & \geq \frac{3}{8} \Vert u \Vert ^{2}-a\beta ^{p}_{k}(p) \Vert u \Vert ^{p} \\ & \geq \Vert u \Vert ^{2}\biggl(\frac{3}{8}- \Vert u \Vert ^{p-2}\biggr). \end{aligned}$$

As a result, let \(\rho :=8^{\frac{1}{(2-p)}}\), we can conclude

$$ J(u)\geq \frac{\rho ^{2}}{4},\quad \forall u\in Z_{k_{3}}, \Vert u \Vert =\rho . $$

 □

Proof of Theorem 1.1

According to Lemma 3.1, Lemma 3.4, Corollary 3.3 and the singularity of f, the functional \(I(u)\) satisfies all assumptions of Lemma 2.3, and the result follows. □