1 Introduction

In this paper we consider the following critical problem involving fractional Laplacian:

$$ \textstyle\begin{cases} (-\Delta )^{s} u=a(x)u^{p-1}+u^{2^{*}_{s}-1} & \text{in } \varOmega , \\ u>0 & \text{in } \varOmega , \\ u=0 & \text{in } \mathbb{R}^{N}\setminus \varOmega , \end{cases} $$
(1.1)

where \(s\in (0,1)\) is fixed and \((-\Delta )^{s}\) is the fractional Laplace operator, \(\varOmega \subset \mathbb{R}^{N}\) (\(N>2s\)) is a smooth bounded domain, \(1< p<2\), \(2^{*}_{s}:=\frac{2N}{N-2s}\), and \(a\in C(\bar{ \varOmega })\) changes sign in Ω.

During the last years there has been an increasing interest in the study of the fractional Laplacian, motivated by great applications and by important advances in the theory of nonlinear partial differential equations, see [3, 7, 11, 14, 15, 17, 20, 21, 24, 25, 35, 36] for details. Nonlinear equations involving fractional Laplacian are currently actively studied. The fractional Laplace operator \((-\Delta )^{s}\) (up to normalization factors) may be defined as

$$ -(-\Delta )^{s}u(x)= \int _{\mathbb{R}^{N}} \bigl(u(x+y)+u(x-y)-2u(x) \bigr)K(y)\,dy, \quad x\in \mathbb{R}^{N}, $$

where \(K(x)=\vert x\vert ^{-(N+2s)}\), \(x\in \mathbb{R}^{N}\). We will denote by \(H^{s}(\mathbb{R}^{N})\) the usual fractional Sobolev space endowed with the so-called Gagliardo norm

$$ \Vert u \Vert _{H^{s}(\mathbb{R}^{N})}= \Vert u \Vert _{L^{2}(\mathbb{R}^{N})}+ \biggl( \int _{\mathbb{R}^{2N}} \bigl\vert u(x)-u(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \biggr)^{1/2}, $$

while \(X_{0}\) is the function space defined as

$$ X_{0}= \bigl\{ u\in H^{s}\bigl(\mathbb{R}^{N} \bigr): u=0\text{ a.e. in }\mathbb{R}^{N}\setminus \varOmega \bigr\} . $$

We refer to [22, 29, 30] for a general definition of \(X_{0}\) and its properties. The embedding \(X_{0}\hookrightarrow L^{q}( \varOmega )\) is continuous for any \(q\in [1,2^{*}_{s}]\) and compact for any \(q\in [1,2^{*}_{s})\). The space \(X_{0}\) is endowed with the norm defined as

$$ \Vert u \Vert _{X_{0}}= \biggl( \int _{\mathbb{R}^{2N}} \bigl\vert u(x)-u(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \biggr) ^{1/2}. $$

By Lemma 5.1 in [29] we have \(C^{2}_{0}(\varOmega )\subset X_{0}\). Thus \(X_{0}\) is nonempty. Note that \((X_{0}, \Vert \cdot \Vert _{X_{0}})\) is a Hilbert space with scalar product

$$ (u,v)_{X_{0}}= \int _{\mathbb{R}^{2N}}\bigl(u(x)-u(y)\bigr) \bigl(v(x)-v(y)\bigr)\,dx\,dy. $$

It is well known that the following critical problem

$$ \textstyle\begin{cases} -\Delta u=u^{2^{*}-1}&\text{in }\varOmega , \\ u>0&\text{in }\varOmega , \\ u=0&\text{on }\partial \varOmega , \end{cases} $$
(1.2)

has no positive solution if Ω is a star-shaped domain, where \(2^{*}=\frac{2N}{N-2}\). For a non-contractible domain Ω, Coron [12] proved that (1.2) has a positive solution. Later, Bahri and Coron [4] improved Coron’s existence result by showing, via topological arguments based upon homology theory, that (1.2) admits a positive solution provided that \(H_{m}(\varOmega ,\mathbb{Z} _{2})\ne \{0\}\) for some \(m>0\). After that, many papers have studied the existence and multiplicity of positive solutions of the problem similar to (1.2), see [16, 18, 37, 39].

It is natural to think that, as in the local case, by assuming suitable geometrical or topological conditions on Ω, one can get the existence of nontrivial solutions for the nonlocal fractional problem. In a recent work, Secchi et al. [28] consider the following nonlocal fractional problem:

$$ \textstyle\begin{cases} (-\Delta )^{s} u=u^{2^{*}_{s}-1}&\text{in }\varOmega , \\ u>0&\text{in }\varOmega , \\ u=0&\text{in }\mathbb{R}^{N}\setminus \varOmega . \end{cases} $$
(1.3)

They proved that (1.3) admits at least a positive solution if there is a point \(x_{0}\in \mathbb{R}^{N}\) and radii \(R_{2}>R_{1}>0\) such that

$$ \bigl\{ R_{1}\le \vert x-x_{0} \vert \le R_{2}\bigr\} \subset \varOmega , \quad\quad \bigl\{ \vert x-x_{0} \vert \le R_{1}\bigr\} \not \subset \bar{\varOmega } $$

and \(R_{2}/R_{1}\) is sufficiently large.

Motivated by the works mentioned above, we study problem (1.1), which involves the critical exponent, the effect of the coefficient \(a(x)\), and the domain with “rich topology”. We try to extend some important results, which are well known for the classical case of the Laplacian (see, e.g., Theorem 1.1 in [39]), to a nonlocal setting.

Taking into account that we are looking for positive solutions, we consider the energy functional associated with (1.1)

$$ \begin{aligned}[b] I(u)&=\frac{1}{2} \int _{\mathbb{R}^{2N}} \bigl\vert u(x)-u(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\&\quad{} - \frac{1}{p} \int _{\varOmega }a(x) \bigl(u^{+}\bigr)^{p}\,dx- \frac{1}{2^{*}_{s}} \int _{\varOmega }\bigl(u^{+}\bigr)^{2^{*}_{s}}\,dx,\end{aligned} $$
(1.4)

where \(u^{+}=\max \{u,0\}\) denotes the positive part of u. By the maximum principle (Proposition 2.2.8 in [33]), it is easy to check that critical points of I are the positive solutions of (1.1).

We make the following assumptions:

  1. (H1)

    There exist three constants \(\rho _{2}>\rho _{1}>\rho _{0}>0\) such that \(\bar{B}_{\rho _{2}}(0)\setminus B_{\rho _{1}}(0)\subset \varOmega \) and \(B_{\rho _{0}}(0)\cap \varOmega =\emptyset \), where \(B_{\rho }(0)=\{x \in \mathbb{R}^{N}: \vert x\vert <\rho \}\) for any \(\rho >0\);

  2. (H2)

    There exists a domain \(\bar{B}_{\rho _{2}}(0)\setminus B_{\rho _{1}}(0)\subset \mathcal{D}\subset \varOmega \) such that \(a(x)>0\) for \(x\in \mathcal{D}\) and \(a(x)\le 0\) for \(x\in \varOmega \setminus \mathcal{D}\).

Theorem 1.1

Assume that (H1), (H2) hold. Then there exists \(\sigma _{0}>0\) such that if \(\vert a^{+}\vert _{q}<\sigma _{0}\), where \(a^{+}(x)=\max \{a(x),0\}\), \(q=\frac{2^{*} _{s}}{2^{*}_{s}-p}\), (1.1) has three positive solutions \(\tilde{u}_{i}(1\le i\le 3)\) such that

$$ \int _{\varOmega }a(x)\tilde{u}^{q}_{i}\,dx>0, \quad i=1,2,3. $$
(1.5)

We should remark that \(\tilde{u}_{2}\) and \(\tilde{u}_{3}\) satisfy \(I(\tilde{u}_{i})< I(\tilde{u}_{1})+\frac{s}{N}S_{s}^{\frac{N}{2s}}\) (\(i=2,3\)), where \(S_{s}\) is the Sobolev constant. It is an interesting task to find the fourth positive solution \(\tilde{u}_{4}\) with \(I(\tilde{u}_{4})>I( \tilde{u}_{1})+\frac{s}{N}S_{s}^{\frac{N}{2s}}\) provided \(\rho _{2}/ \rho _{1}\) is sufficiently large, although we shall not undertake it here.

This paper is organized as follows. In Sect. 2 we introduce Nehari manifold and state technical and elementary lemmas useful along the paper. In Sect. 3 we prove the existence of the first solution of (1.1). In Sect. 4 we establish some essential estimates of energy. In Sect. 5 we prove the existence of the other two solutions by Lusternik–Schnirelmann category. We denote by \(\vert \cdot \vert _{r}\) the \(L^{r}(\varOmega )\)-norm for any \(r>1\), respectively.

2 Preliminaries

Recall that I is unbounded from below; we can get rid of this problem once we restrict I to the Nehari manifold

$$\begin{aligned} \mathcal{N} =&\bigl\{ u\in X_{0}\setminus \{0\}:\bigl\langle I'(u),u\bigr\rangle =0\bigr\} \\ =& \biggl\{ u\in X_{0}\setminus \{0\}: \Vert u \Vert _{X_{0}}^{2}= \int _{\varOmega }a(x) \bigl(u^{+}\bigr)^{p} \,dx+ \int _{\varOmega }\bigl(u^{+}\bigr)^{2^{*}_{s}}\,dx \biggr\} . \end{aligned}$$

Notice that \(u^{+}\not \equiv 0\) for any \(u\in \mathcal{N}\), and on \(\mathcal{N}\) the functional I reads

$$ I(u)= \biggl(\frac{1}{2}-\frac{1}{2^{*}_{s}} \biggr) \Vert u \Vert _{X_{0}}^{2}- \biggl(\frac{1}{p} - \frac{1}{2^{*}_{s}} \biggr) \int _{\varOmega }a(x) \bigl(u ^{+}\bigr)^{p} \,dx. $$
(2.1)

Set

$$ q=\frac{2^{*}_{s}}{2^{*}_{s}-p}. $$

In our context, the Sobolev constant is given by

$$ S_{s}=\inf_{u\in H^{s}(\mathbb{R}^{N})\setminus \{0\}}\frac{ \int _{\mathbb{R}^{2N}}(u(x)-u(y))^{2}K(x-y)\,dx\,dy}{ ( \int _{\mathbb{R}^{N}} \vert u(x) \vert ^{2^{*}_{s}}\,dx )^{2/2^{*}_{s}}}. $$
(2.2)

Lemma 2.1

I is coercive and bounded from below on \(\mathcal{N}\).

Proof

If \(u\in \mathcal{N}\), by (2.1) and the Sobolev inequality,

$$ I(u)\ge \frac{s}{N} \Vert u \Vert _{X_{0}}^{2}- \biggl(\frac{1}{p}-\frac{1}{2^{*} _{s}} \biggr) \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2} \Vert u \Vert _{X_{0}}^{p}. $$
(2.3)

Since \(1< p<2\), we get that I is coercive and bounded from below on \(\mathcal{N}\). □

Define

$$ \psi (u)=\bigl\langle I'(u),u\bigr\rangle . $$

Then, for \(u\in \mathcal{N}\), we have

$$\begin{aligned} \bigl\langle \psi '(u),u\bigr\rangle =&2 \Vert u \Vert _{X_{0}}^{2}-p \int _{\varOmega }a(x) \bigl(u ^{+}\bigr)^{p}\,dx -2^{*}_{s} \int _{\varOmega }\bigl(u^{+}\bigr)^{2^{*}_{s}}\,dx \\ =&(2-p) \Vert u \Vert _{X_{0}}^{2}-\bigl(2^{*}_{s}-p \bigr) \int _{\varOmega }\bigl(u^{+}\bigr)^{2^{*} _{s}}\,dx \end{aligned}$$
(2.4)
$$\begin{aligned} =&\bigl(2^{*}_{s}-p\bigr) \int _{\varOmega }a(x) \bigl(u^{+}\bigr)^{p}\,dx- \bigl(2^{*}_{s}-2\bigr) \Vert u \Vert _{X_{0}}^{2}. \end{aligned}$$
(2.5)

Adopting a method similar to that used in [34], we split \(\mathcal{N}\) into three parts:

$$\begin{aligned}& \mathcal{N}^{+} = \bigl\{ u\in \mathcal{N}: \bigl\langle \psi '(u),u\bigr\rangle >0\bigr\} ; \\& \mathcal{N}^{0} = \bigl\{ u\in \mathcal{N}: \bigl\langle \psi '(u),u\bigr\rangle =0\bigr\} ; \\& \mathcal{N}^{-} = \bigl\{ u\in \mathcal{N}: \bigl\langle \psi '(u),u\bigr\rangle < 0\bigr\} . \end{aligned}$$

Lemma 2.2

Assume that u is a minimizer for I on \(\mathcal{N}\) and \(u\notin \mathcal{N}^{0}\). Then \(\langle I'(u),v\rangle =0\) for any \(v\in X_{0}\).

The proof is similar to that of Theorem 2.3 in [9], we omit it.

Set

$$ \sigma _{1}=\frac{2^{*}_{s}-2}{2^{*}_{s}-p} \biggl(\frac{2-p}{2^{*}_{s}-p} \biggr)^{(2-p)/(2^{*}_{s}-2)}S_{s}^{(2^{*}_{s}-p)/(2^{*}_{s}-2)}. $$

Lemma 2.3

\(\mathcal{N}^{0}=\emptyset \) if \(\vert a^{+}\vert _{q}<\sigma _{1}\).

Proof

Assume by contradiction that there exists \(a\in C(\bar{\varOmega })\) with \(\vert a^{+}\vert _{q}<\sigma _{1}\) such that \(\mathcal{N}\ne \emptyset \). By (2.4) and (2.2), we have

$$ \Vert u \Vert _{X_{0}}^{2}=\frac{2^{*}_{s}-p}{2-p} \int _{\varOmega }\bigl(u^{+}\bigr)^{2^{*} _{s}}\,dx\le \frac{2^{*}_{s}-p}{2-p} S_{s}^{-2^{*}_{s}/2} \Vert u \Vert _{X_{0}} ^{2^{*}_{s}}. $$

Consequently,

$$ \Vert u \Vert _{X_{0}}\ge \biggl(\frac{2-p}{2^{*}_{s}-p}S_{s}^{2^{*}_{s}/2} \biggr) ^{1/(2^{*}_{s}-2)}. $$

Similarly, by (2.5), we have

$$ \Vert u \Vert _{X_{0}}^{2}=\frac{2^{*}_{s}-p}{2^{*}_{s}-2} \int _{\varOmega }a(x) \bigl(u ^{+}\bigr)^{p}\,dx \le \frac{2^{*}_{s}-p}{2^{*}_{s}-2} \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2} \Vert u \Vert ^{p}_{X_{0}}, $$

and so

$$ \Vert u \Vert _{X_{0}}\le \biggl(\frac{2^{*}_{s}-p}{2^{*}_{s}-2} \bigl\vert a^{+} \bigr\vert _{q}S _{s}^{-p/2} \biggr)^{\frac{1}{2-p}}. $$

Thus, we get that \(\vert a^{+}\vert _{q}\ge \sigma _{1}\), which is impossible. □

Define

$$ X_{0}^{+}:=\bigl\{ u\in X_{0}: u^{+} \not \equiv 0\bigr\} . $$

Lemma 2.4

For each \(u\in X_{0}^{+}\), we have

  1. (i)

    if \(\int _{\varOmega }a(x)(u^{+})^{p}\,dx\le 0\), then there exists unique \(t^{-}(u)>t_{\max }\) such that \(t^{-}(u)u\in \mathcal{N}^{-}\) and \(\varphi (t):=I(tu)\) is increasing on \((0,t^{-}(u))\) and decreasing on \((t^{-}(u),+\infty )\), where

    $$ t_{\max }= \biggl(\frac{(2-p) \Vert u \Vert _{X_{0}}^{2}}{(2^{*}_{s}-p)\int _{ \varOmega }(u^{+})^{2^{*}_{s}}\,dx} \biggr)^{\frac{N-2s}{4s}}. $$

    Furthermore,

    $$ \varphi \bigl(t^{-}(u)\bigr)=\sup_{t\ge 0} \varphi (t). $$
    (2.6)
  2. (ii)

    If \(\int _{\varOmega }a(x)(u^{+})^{p}\,dx>0\), then there exist unique \(0< t^{+}(u)< t_{\max }< t^{-}(u)\) such that \(t^{+}(u)u\in \mathcal{N} ^{+}\), \(t^{-}(u)u \in \mathcal{N}^{-}\), and \(\varphi (t)\) is decreasing on \((0,t^{+}(u))\cup (t^{-}(u),+\infty )\) and increasing on \((t^{+}(u),t^{-}(u))\). Furthermore,

    $$ \varphi \bigl(t^{+}(u)\bigr)=\inf_{0\le t\le t^{-}(u)} \varphi (t), \quad\quad \varphi \bigl(t^{-}(u)\bigr)=\sup _{t\ge t^{+}(u)}\varphi (t). $$
    (2.7)
  3. (iii)

    \(t^{-}(u)\) is a continuous function for \(u\in X_{0}^{+}\).

  4. (iv)

    \(\mathcal{N}^{-}= \{u\in X_{0}^{+}: \frac{1}{\Vert u\Vert _{X_{0}}}t ^{-} (\frac{u}{\Vert u\Vert _{X_{0}}} )=1 \}\).

Proof

Fix \(u\in X_{0}^{+}\). We consider the following function:

$$ \gamma (t)=t^{2-p} \Vert u \Vert _{X_{0}}^{2}-t^{2^{*}_{s}-p} \int _{\varOmega }\bigl(u ^{+}\bigr)^{2^{*}_{s}}\,dx, \quad \forall t>0. $$
(2.8)

Clearly, \(tu\in \mathcal{N}\) if and only if \(\gamma (t)=\int _{\varOmega }a(x)(u^{+})^{p}\,dx\). Moreover,

$$ \gamma '(t)=(2-p)t^{1-p} \Vert u \Vert _{X_{0}}^{2}-\bigl(2^{*}_{s}-p \bigr)t^{2^{*}_{s}-p-1} \int _{\varOmega }\bigl(u^{+}\bigr)^{2^{*}_{s}}\,dx. $$
(2.9)

So, it is easy to see that \(tu\in \mathcal{N}^{+}\) (or \(\mathcal{N} ^{-}\)) if and only if \(\gamma '(t)>0\) (or <0). Notice that γ is increasing on \((0,t_{\max })\) and decreasing on \((t_{\max },+\infty )\) and \(\gamma (t)\to -\infty \) as \(t\to +\infty \).

(i) If \(\int _{\varOmega }a(x)(u^{+})^{p}\,dx\le 0\), then \(\gamma (t)= \int _{\varOmega }a(x)(u^{+})^{p}\,dx\) has a unique solution \(t^{-}(u)>t _{\max }\) and \(\gamma '(t^{-}(u))<0\). Thus, \(t^{-}(u)u\in \mathcal{N} ^{-}\). Since

$$ \varphi '(t)=t^{p-1} \biggl[\gamma (t)- \int _{\varOmega }a(x) \bigl(u^{+}\bigr)^{p}\,dx \biggr], $$

we get that (2.6) holds.

(ii) Assume that \(\int _{\varOmega }a(x)\vert u\vert ^{p}\,dx>0\). Direct computation yields that

$$\begin{aligned} \gamma (t_{\max }) =& \biggl(\frac{(2-p) \Vert u \Vert _{X_{0}}^{2}}{(2^{*}_{s}-p) \int _{\varOmega } \vert u \vert ^{2^{*}_{s}}\,dx} \biggr)^{\frac{(N-2s)(2-p)}{4s}} \frac{2^{*} _{s}-2}{2^{*}_{s}-p} \Vert u \Vert _{X_{0}}^{2} \\ \ge &\frac{2^{*}_{s}-2}{2^{*}_{s}-p} \biggl(\frac{2-p}{2^{*}_{s}-p} \biggr) ^{\frac{(N-2s)(2-p)}{4s}}S_{s}^{N(2-p)/(4s)} \Vert u \Vert _{X_{0}}^{p} \\ \ge &\frac{2^{*}_{s}-2}{2^{*}_{s}-p} \biggl(\frac{2-p}{2^{*}_{s}-p} \biggr) ^{\frac{(N-2s)(2-p)}{4s}}S_{s}^{(2^{*}_{s}-p)/(2^{*}_{s}-2)} \bigl\vert a^{+} \bigr\vert _{q} ^{-1} \int _{\varOmega }a(x) \bigl(u^{+}\bigr)^{p}\,dx \\ >& \int _{\varOmega }a(x) \bigl(u^{+}\bigr)^{p}\,dx \end{aligned}$$

since \(\vert a^{+}\vert _{q}<\sigma _{1}\). Thus, \(\gamma (t)=\int _{\varOmega }a(x)(u ^{+})^{p}\,dx\) has exactly two solutions \(t^{+}(u)< t_{\max }< t^{-}(u)\) such that \(\gamma '(t^{+}(u))>0\) and \(\gamma '(t^{-}(u))<0\), and \(\varphi (t)\) is decreasing on \((0,t^{+}(u))\cup (t^{-}(u),+\infty )\) and increasing on \((t^{+}(u),t^{-}(u))\). Consequently, \(t^{+}(u)u \in \mathcal{N}^{+}\) and \(t^{-}(u)u\in \mathcal{N}^{-}\), and (2.7) holds.

(iii) The uniqueness of \(t^{-}(u)\) and its extremal property give that \(t^{-}(u)\) is a continuous function of u.

(iv) Set

$$ \mathcal{S}:= \biggl\{ u\in X_{0}^{+}: \frac{1}{ \Vert u \Vert _{X_{0}}}t^{-} \biggl(\frac{u}{ \Vert u \Vert _{X_{0}}} \biggr)=1 \biggr\} . $$

Let \(v=\frac{u}{\Vert u\Vert _{X_{0}}}\) for any \(u\in \mathcal{N}^{-}\). By (i) and (ii), there exists \(t^{-}(v)>0\) such that \(t^{-}(v)v\in \mathcal{N}^{-}\), that is, \(\frac{t^{-}(v)}{\Vert u\Vert _{X_{0}}}u\in \mathcal{N}^{-}\). Since \(u\in \mathcal{N}^{-}\), we have \(t^{-}(v)=\Vert u \Vert _{X_{0}}\). Hence, we get \(\mathcal{N}^{-}\subset \mathcal{S}\). On the other hand, let \(u\in \mathcal{S}\). Then,

$$ u=t^{-} \biggl(\frac{u}{ \Vert u \Vert _{X_{0}}} \biggr)\frac{u}{ \Vert u \Vert _{X_{0}}} \in \mathcal{N}^{-}. $$

Thus, \(\mathcal{S}\subset \mathcal{N}^{-}\). □

3 Existence of the first solution

Define

$$ m^{+}=\inf_{u\in \mathcal{N}^{+}}I(u) \quad \text{and} \quad m^{-}=\inf_{u\in \mathcal{N}^{-}}I(u). $$

Set

$$ \sigma _{2}=\frac{p}{2}\sigma _{1}. $$

Lemma 3.1

  1. (i)

    \(m^{+}<0\) if function a satisfies \(\vert a^{+}\vert _{q}\in (0,\sigma _{1})\);

  2. (ii)

    there exists positive constant \(c_{0}\) such that \(m^{-}\ge c_{0}\) if \(\vert a^{+}\vert _{q}<\sigma _{2}\). In particular, \(m^{+}=\inf_{u\in \mathcal{N}}I(u)\) if function a satisfies \(\vert a^{+}\vert _{q}\in (0,\sigma _{2})\).

Proof

(i) If \(u\in \mathcal{N}^{+}\), then by (2.5) we get that

$$ \Vert u \Vert _{X_{0}}^{2}< \frac{2^{*}_{s}-p}{2^{*}_{s}-2} \int _{\varOmega }a(x) \bigl(u ^{+}\bigr)^{p} \,dx. $$

Thus, by (2.1),

$$ I(u)< - \biggl(1-\frac{p}{2^{*}_{s}} \biggr) \biggl(\frac{1}{p}- \frac{1}{2} \biggr) \int _{\varOmega }a(x) \bigl(u^{+}\bigr)^{p} \,dx< 0, $$

and so \(m^{+}<0\).

(ii) If \(u\in \mathcal{N}^{-}\), then by (2.4),

$$ \frac{2-p}{2^{*}_{s}-p} \Vert u \Vert _{X_{0}}^{2}< \int _{\varOmega }\bigl(u^{+}\bigr)^{2^{*} _{s}}\,dx\le S_{s}^{-2^{*}_{s}/2} \Vert u \Vert _{X_{0}}^{2^{*}_{s}}. $$

Consequently,

$$ \Vert u \Vert _{X_{0}}>S_{s}^{N/(4s)} \biggl( \frac{2-p}{2^{*}_{s}-p} \biggr) ^{1/(2^{*}_{s}-2)}. $$

By (2.3) and \(\vert a^{+}\vert _{q}<\sigma _{2}\), we have

$$\begin{aligned} I(u) \ge & \Vert u \Vert _{X_{0}}^{p} \biggl[ \frac{s}{N} \Vert u \Vert _{X_{0}}^{2-p}- \biggl( \frac{1}{p}-\frac{1}{2^{*}_{s}} \biggr) \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2} \biggr] \\ \ge &S_{s}^{Np/(4s)} \biggl(\frac{2-p}{2^{*}_{s}-p} \biggr)^{p/(2^{*} _{s}-2)} \biggl[\frac{s}{N}S_{s}^{N(2-p)/(4s)} \biggl(\frac{2-p}{2^{*} _{s}-p} \biggr)^{(2-p)/(2^{*}_{s}-2)} \\ &{}- \biggl(\frac{1}{p}- \frac{1}{2^{*} _{s}} \biggr) \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2} \biggr] \\ >&0. \end{aligned}$$

 □

From now on, we assume that \(\vert a^{+}\vert _{q}\in (0,\sigma _{2})\).

Lemma 3.2

I satisfies the \((\mathit{PS})_{\beta }\) condition in \(X_{0}\) for \(\beta < m ^{+}+\frac{s}{N}S_{s}^{\frac{N}{2s}}\).

Proof

Let \(\{u_{n}\}\) be a \((\mathit{PS})_{\beta }\) sequence for I such that

$$ I(u_{n})\to \beta \quad \text{and} \quad I'(u_{n})\to 0. $$
(3.1)

Then, for n big enough, we have

$$\begin{aligned} \beta +1+ \Vert u_{n} \Vert _{X_{0}} \ge &I(u_{n})-\frac{1}{2^{*}_{s}}\bigl\langle I'(u _{n}),u_{n}\bigr\rangle \\ =& \biggl(\frac{1}{2}-\frac{1}{2^{*}_{s}} \biggr) \Vert u_{n} \Vert _{X_{0}} ^{2}- \biggl( \frac{1}{p}-\frac{1}{2^{*}_{s}} \biggr) \int _{\varOmega }a(x) \bigl(u _{n}^{+} \bigr)^{p}\,dx \\ \ge & \biggl(\frac{1}{2}-\frac{1}{2^{*}_{s}} \biggr) \Vert u_{n} \Vert _{X_{0}} ^{2}- \biggl( \frac{1}{p}-\frac{1}{2^{*}_{s}} \biggr) \bigl\vert a^{+} \bigr\vert _{q}S_{s} ^{-p/2} \Vert u_{n} \Vert _{X_{0}}^{p}. \end{aligned}$$

It follows that \(\Vert u_{n}\Vert _{X_{0}}\) is bounded. Going if necessary to a subsequence, we can assume that

$$\begin{aligned}& u_{n}\rightharpoonup u_{0}\quad \text{in }X_{0}, \\& u_{n}\to u_{0}\quad \text{in }L^{r}(\varOmega )\text{ for }r\in [1,2^{*} _{s}), \\& u_{n}\to u_{0}\quad \text{a.e. in }\varOmega . \end{aligned}$$

We derive from (3.1) that \(\langle I'(u_{0}),v\rangle =0\), \(\forall v\in X_{0}\), i.e., \(u_{0}\) is a solution of (1.1). In particular, \(u_{0}\in \mathcal{N}\). Thus, by Lemma 3.1, we have \(I(u_{0})\ge m^{+}\). Since \(X_{0}\) is a Hilbert space, we have

$$ \Vert u_{n} \Vert _{X_{0}}^{2}= \Vert u_{n}-u_{0} \Vert _{X_{0}}^{2}+ \Vert u_{0} \Vert _{X_{0}} ^{2}+o(1). $$
(3.2)

By Brézis–Lieb’s lemma [8], we get

$$ \int _{\varOmega }\bigl(u_{n}^{+} \bigr)^{2^{*}_{s}}\,dx= \int _{\varOmega }\bigl((u_{n}-u_{0})^{+} \bigr)^{2^{*} _{s}}\,dx+ \int _{\varOmega }\bigl(u_{0}^{+} \bigr)^{2^{*}_{s}}\,dx+o(1). $$
(3.3)

Since \((u_{n}^{+})^{2^{*}_{s}-1}\) is bounded in \(L^{p'}(\varOmega )\) with \(p'=2^{*}_{s}/(2^{*}_{s}-1)\) and \(L^{p'}(\varOmega )\) is a reflexible space, we get \((u_{n}^{+})^{2^{*}_{s}-1}\rightharpoonup (u_{0}^{+})^{2^{*} _{s}-1}\) in \(L^{p'}(\varOmega )\), and so

$$ \int _{\varOmega }\bigl(u_{n}^{+} \bigr)^{2^{*}_{s}-1}u_{0}\,dx\to \int _{\varOmega }\bigl(u _{0}^{+} \bigr)^{2^{*}_{s}}\,dx. $$
(3.4)

Similarly, since \(u_{n}\rightharpoonup u_{0}\) in \(L^{2^{*}_{s}}( \varOmega )\) and \((u_{0}^{+})^{2^{*}_{s}-1}\in L^{p'}(\varOmega )\), we get

$$ \int _{\varOmega }\bigl(u_{0}^{+} \bigr)^{2^{*}_{s}-1}u_{n}\,dx\to \int _{\varOmega }\bigl(u _{0}^{+} \bigr)^{2^{*}_{s}}\,dx. $$
(3.5)

By (3.2)–(3.5), we have

$$ I(u_{n})=I(u_{0})+ \frac{1}{2} \Vert u_{n}-u_{0} \Vert _{X_{0}}^{2}-\frac{1}{2^{*} _{s}} \int _{\varOmega }\bigl(u^{+}\bigr)^{2^{*}_{s}}\,dx+o(1) $$
(3.6)

and

$$\begin{aligned} \begin{aligned}[b] o(1) &=\bigl\langle I'(u_{n})-I'(u_{0}), u_{n}-u_{0}\bigr\rangle \\ &=\Vert u_{n}-u_{0} \Vert _{X_{0}}^{2}- \int _{\varOmega }\bigl((u_{n}-u_{0})^{+} \bigr)^{2^{*} _{s}}\,dx+o(1). \end{aligned} \end{aligned}$$
(3.7)

By (3.6) and (3.7), we have

$$\begin{aligned} \frac{s}{N} \Vert u_{n}-u_{0} \Vert _{X_{0}}^{2} =&I(u_{n})-I(u_{0})+o(1) \\ \le &I(u_{n})-m^{+}+o(1) \\ =&\beta -m^{+}+o(1). \end{aligned}$$

Thus, there exists a positive constant \(\sigma >0\) such that

$$ \Vert u_{n}-u_{0} \Vert _{X_{0}}^{2}< S_{s}^{\frac{N}{2s}}-\sigma $$
(3.8)

for n large enough. By (3.7), (3.8), and Sobolev inequality, we get

$$\begin{aligned} 0 < & \bigl[1- \bigl(S_{s}^{\frac{N}{2s}}-\sigma \bigr)^{(2^{*}_{s}-2)/2}S _{s}^{-2^{*}_{s}/2} \bigr] \Vert u_{n}-u_{0} \Vert _{X_{0}}^{2} \\ \le & \bigl(1-S_{s}^{-2^{*}_{s}/2} \Vert u_{n}-u_{0} \Vert _{X_{0}}^{2^{*}_{s}-2} \bigr) \Vert u_{n}-u_{0} \Vert _{X_{0}}^{2} \\ \le & \Vert u_{n}-u_{0} \Vert _{X_{0}}^{2}- \int _{\varOmega }\bigl((u_{n}-u_{0})^{+} \bigr)^{2^{*} _{s}}\,dx=o(1). \end{aligned}$$

This implies \(\Vert u_{n}-u_{0}\Vert _{X_{0}}\to 0\) in \(X_{0}\). □

Theorem 3.3

There exists a minimizer \(\tilde{u}_{1}\) of the critical problem (1.1), and it satisfies

  1. (i)

    \(\tilde{u}_{1}\in \mathcal{N}^{+}\) and \(I(\tilde{u}_{1})=m^{+}\);

  2. (ii)

    \(\tilde{u}_{1}\in C^{0,s}(\mathbb{R}^{N})\) is a positive solution of (1.1);

  3. (iii)

    \(I(\tilde{u}_{1})\to 0\) as \(\vert a^{+}\vert _{q}\to 0\).

Proof

Applying Ekeland’s variational principle [13] and using the similar argument as the proof of Theorem 1 in [34], we get that there exists \(\{u_{n}\}\subset \mathcal{N}^{+}\) such that

$$ I(u_{n})\to m^{+} \quad \text{and} \quad I'(u_{n})\to 0. $$

By Lemma 3.2, there exist a subsequence (still denoted by \(\{u_{n}\}\)) and \(\tilde{u}_{1}\in \mathcal{N}^{+}\), a solution of (1.1), such that \(u_{n}\to \tilde{u}_{1}\) in \(X_{0}\) and \(m^{+}=I(\tilde{u}_{1})\). By the maximum principle (Proposition 2.2.8 in [33]), \(\tilde{u}_{1}\) is strictly positive in Ω. By Proposition 2.2 in [6], \(u\in L^{\infty }(\varOmega )\). Furthermore, by Proposition 1.1 in [26] (or Proposition 5 in [31]), \(u\in C^{0,s}(\mathbb{R}^{N})\).

By (2.6),

$$ \Vert \tilde{u}_{1} \Vert _{X_{0}}\le \biggl( \frac{2^{*}_{s}-p}{2^{*}_{s}-2} \bigl\vert a ^{+} \bigr\vert _{q}S_{s}^{-p/2} \biggr)^{\frac{1}{2-p}}. $$

This implies \(\Vert \tilde{u}_{1}\Vert _{X_{0}}\to 0\) as \(\vert a^{+}\vert _{q}\to 0\), and so \(I(\tilde{u}_{1})\to 0\) as \(\vert a^{+}\vert _{q}\to 0\). □

4 Estimates of energy

Recall that \(S_{s}\) is defined as

$$ S_{s}:=\inf_{v\in H^{s}(\mathbb{R}^{N})\setminus \{0\}}\frac{ \int _{\mathbb{R}^{2N}} \vert v(x)-v(y) \vert ^{2}K(x-y)\,dx\,dy}{ (\int _{\mathbb{R}^{N}} \vert v \vert ^{2^{*}_{s}}\,dx )^{2/2^{*}_{s}}}. $$

It is well known from [32] that the infimum in the formula above is attained at ũ, where

$$ \tilde{u}(x)=\frac{\kappa }{ (\mu ^{2}+ \vert x-x_{0} \vert ^{2} )^{ \frac{N-2s}{2}}}, \quad x\in \mathbb{R}^{N}, $$
(4.1)

with \(\kappa \in \mathbb{R}\setminus \{0\}\), \(\mu >0\) and \(x_{0}\in \mathbb{R}^{N}\) fixed constants. We suppose \(\kappa >0\) for our convenience. Equivalently, the function ū defined as

$$ \bar{u}=\frac{\tilde{u}}{ \Vert \tilde{u} \Vert _{L^{p}(\mathbb{R}^{N})}} $$

is such that

$$ S_{s}= \int _{\mathbb{R}^{2N}} \bigl\vert \bar{u}(x)-\bar{u}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy. $$

The function

$$ u^{*}(x)=\bar{u} \biggl(\frac{x}{S^{1/(2s)}_{s}} \biggr), \quad x\in \mathbb{R}^{N}, $$

is a solution of

$$ (-\Delta )^{s}u= \vert u \vert ^{p-2}u \quad \text{in }\mathbb{R}^{N}. $$
(4.2)

Now, we consider the family of functions \(U_{\varepsilon }\) defined as

$$ U_{\varepsilon }(x)=\varepsilon ^{-(N-2s)/2}u^{*}(x/ \varepsilon ), \quad x\in \mathbb{R}^{N}, $$
(4.3)

for any \(\varepsilon >0\). The function \(U_{\varepsilon }\) is a solution of problem (4.2) and satisfies

$$ \int _{\mathbb{R}^{2N}} \bigl\vert U_{\varepsilon }(x)-U_{\varepsilon }(y) \bigr\vert ^{2}K(x-y)\,dx\,dy= \int _{\mathbb{R}^{N}} \bigl\vert U_{\varepsilon }(x) \bigr\vert ^{p}\,dx=S^{N/(2s)}_{s}. $$
(4.4)

Let us fix \(\rho _{a}\), \(\rho _{b}\), ρ̃, \(\rho _{c}\), \(\rho _{d}\) such that

$$ \rho _{1}< \rho _{a}< \rho _{b}< \tilde{\rho }< \rho _{c}< \rho _{d}< \rho _{2}. $$
(4.5)

Let \(\eta \in C^{\infty }_{0}(\mathbb{R}^{N})\) be a radially symmetric function such that \(0\le \eta \le 1\) in \(\mathbb{R}^{N}\) and

$$ \eta (x)= \textstyle\begin{cases} 0,& \text{if } \vert x \vert \le \rho _{a}, \\ 1,& \text{if }\rho _{b}\le \vert x \vert \le \rho _{c}, \\ 0,& \text{if } \vert x \vert \ge \rho _{d}. \end{cases} $$
(4.6)

For every \(\varepsilon \in (0,1)\) and \(\mathbf{e}\in \mathbb{S}^{N-1}:= \{x\in \mathbb{R}^{N}:\vert x\vert =1\}\), we denote by \(u_{\varepsilon , \mathbf{e}}\) the following function:

$$ u_{\varepsilon ,\mathbf{e}}(x)=\eta (x)U_{\varepsilon } (x- \tilde{\rho } \mathbf{e} ). $$
(4.7)

Lemma 4.1

There hold

  1. (i)

    \(\int _{\mathbb{R}^{N}}\vert u_{\varepsilon , \mathbf{e}}\vert ^{2^{*}_{s}}=S _{s}^{\frac{N}{2s}}+O(\varepsilon ^{N})\) uniformly in \(\mathbf{e}\in \mathbb{S}^{N-1}\);

  2. (ii)

    \(\Vert u_{\varepsilon ,\mathbf{e}}\Vert _{X_{0}}^{2} =S_{s}^{ \frac{N}{2s}}+O(\varepsilon ^{N-2s})\) uniformly in \(\mathbf{e} \in \mathbb{S}^{N-1}\).

Proof

(i) By Proposition 22 in [32], we have

$$\begin{aligned}& \int _{\mathbb{R}^{N}} \vert u_{\varepsilon ,\mathbf{e}} \vert ^{2^{*}_{s}} \,dx \\ & \quad = \int _{\mathbb{R}^{N}}U^{2^{*}_{s}}_{\varepsilon }(x-\tilde{\rho } \mathbf{e})\,dx+ \int _{\mathbb{R}^{N}} \bigl(\eta ^{2^{*}_{s}}(x)-1 \bigr)U ^{2^{*}_{s}}_{\varepsilon }(x-\tilde{\rho }\mathbf{e})\,dx \\ & \quad = S_{s}^{\frac{N}{2s}}+ \int _{ \vert x \vert < \rho _{b}} \bigl(\eta ^{2^{*}_{s}}(x)-1 \bigr) U^{2^{*}_{s}}_{\varepsilon }(x-\tilde{\rho }\mathbf{e})\,dx + \int _{ \vert x \vert >\rho _{c}} \bigl(\eta ^{2^{*}_{s}}(x)-1 \bigr)U^{2^{*}_{s}} _{\varepsilon }(x-\tilde{\rho }\mathbf{e})\,dx. \end{aligned}$$
(4.8)

Direct computation yields that

$$\begin{aligned}& \biggl\vert \int _{ \vert x \vert < \rho _{b}} \bigl(\eta ^{2^{*}_{s}}(x)-1 \bigr) U ^{2^{*}_{s}}_{\varepsilon }(x-\tilde{\rho }\mathbf{e})\,dx + \int _{ \vert x \vert >\rho _{c}} \bigl(\eta ^{2^{*}_{s}}(x)-1 \bigr)U^{2^{*}_{s}} _{\varepsilon }(x-\tilde{\rho }\mathbf{e})\,dx \biggr\vert \\ & \quad \le C\varepsilon ^{N} \biggl( \int _{ \vert x \vert < \rho _{b}}\frac{dx}{ \vert x- \tilde{\rho }\mathbf{e} \vert ^{2N}} + \int _{ \vert x \vert >\rho _{c}}\frac{dx}{ \vert x- \tilde{\rho }\mathbf{e} \vert ^{2N}} \biggr) \\ & \quad \le C\varepsilon ^{N} \biggl( \int _{ \vert x \vert < \rho _{b}}\frac{dx}{( \tilde{\rho }-\rho _{b})^{2N}} + \int _{ \vert x+\tilde{\rho }\mathbf{e} \vert >\rho _{c}}\frac{dx}{ \vert x \vert ^{2N}} \biggr) \\ & \quad \le C\varepsilon ^{N} \biggl((\tilde{\rho }-\rho _{b})^{-2N} \bigl\vert B_{\rho _{b}}(0) \bigr\vert + \int _{ \vert x \vert >\rho _{c}-\tilde{\rho }} \frac{dx}{ \vert x \vert ^{2N}} \biggr) \\ & \quad \le C'\varepsilon ^{N}. \end{aligned}$$
(4.9)

Thus, by (4.8), we prove (i).

(ii) Set \(\delta =\frac{1}{2}\min \{\tilde{\rho }-\rho _{b},\rho _{c}- \tilde{\rho }\}\). Define

$$\begin{aligned}& \mathcal{D}_{1} = \bigl\{ x\in \mathbb{R}^{N}: \rho _{b}< \vert x \vert < \rho _{c} \bigr\} , \\ & \mathcal{D}_{2} = \bigl\{ x\in \mathbb{R}^{N}: \vert x \vert \le \rho _{b}\text{ or } \vert x \vert \ge \rho _{c} \bigr\} , \\ & \mathcal{D}_{3} = \bigl\{ x\in \mathbb{R}^{N}: \vert x \vert \le \rho _{a}\text{ or } \vert x \vert \ge \rho _{d} \bigr\} , \\ & A_{1} = \bigl\{ (x,y)\in \mathbb{R}^{N}\times \mathbb{R}^{N}: x\in \mathcal{D}_{1}, y\in \mathcal{D}_{1} \bigr\} , \\ & A_{2} = \bigl\{ (x,y)\in \mathbb{R}^{N}\times \mathbb{R}^{N}: x\in \mathcal{D}_{1}, y\in \mathcal{D}_{2}, \vert x-y \vert >\delta \bigr\} , \\ & A_{3} = \bigl\{ (x,y)\in \mathbb{R}^{N}\times \mathbb{R}^{N}: x\in \mathcal{D}_{1}, y\in \mathcal{D}_{2}, \vert x-y \vert \le \delta \bigr\} , \\ & A_{4} = \bigl\{ (x,y)\in \mathbb{R}^{N}\times \mathbb{R}^{N}: x\in \mathcal{D}_{2}, y\in \mathcal{D}_{2} \bigr\} , \\ & B_{1} = \bigl\{ (x,y)\in \mathbb{R}^{N}\times \mathbb{R}^{N}: \vert x \vert \ge \rho _{c}, \vert y \vert \ge \rho _{c}, \bigl\vert tx+(1-t)y \bigr\vert \ge \rho _{c}, \forall t \in [0,1] \bigr\} , \\ & B_{2} = \bigl\{ (x,y)\in \mathbb{R}^{N}\times \mathbb{R}^{N}: \vert x \vert \le \rho _{b}, \vert y \vert \le \rho _{b}, \bigl\vert tx+(1-t)y \bigr\vert \le \rho _{b}, \forall t \in [0,1] \bigr\} , \\ & B_{3} = \bigl\{ (x,y)\in \mathbb{R}^{N}\times \mathbb{R}^{N}: x\in \mathcal{D}_{3}, y\in \mathcal{D}_{3} \bigr\} . \end{aligned}$$

We have

$$\begin{aligned} \Vert u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2} =& \int _{\mathbb{R}^{2N}} \bigl\vert u _{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\ =& \biggl( \int _{A_{1}}+2 \int _{A_{2}}+2 \int _{A_{3}}+ \int _{A_{4}} \biggr) \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy. \end{aligned}$$
(4.10)

We consider the following four cases:

(i) Assume \((x,y)\in A_{4}\). Then \(\vert x-\tilde{\rho }\mathbf{e}\vert \ge \tilde{\rho }-\rho _{b}\) or \(\vert x-\tilde{\rho }\mathbf{e}\vert \ge \rho _{c}- \tilde{\rho }\). Thus, there exists constant \(C>0\) such that

$$ \bigl\vert u_{\varepsilon ,\mathbf{e}}(x) \bigr\vert \le C\varepsilon ^{-\frac{N-2s}{2}} \biggl(\mu ^{2}+\frac{ \vert \xi -\tilde{\rho }\mathbf{e} \vert ^{2}}{ \vert \varepsilon S_{s}^{1/(2s)} \vert ^{2}} \biggr)^{-\frac{N-2s}{2}}\le C \varepsilon ^{\frac{N-2s}{2}}. $$

Consequently,

$$ \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert \le \bigl\vert u _{\varepsilon ,\mathbf{e}}(x) \bigr\vert + \bigl\vert u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert \le C \varepsilon ^{\frac{N-2s}{2}}. $$
(4.11)

Moreover, if \((x,y)\in A_{4}\) and \(\vert x-y\vert \le \frac{1}{2}(\rho _{c}-\rho _{b})\), then \((x,y)\in B_{1}\cup B_{2}\), and so \(\vert \xi -\tilde{\rho } \mathbf{e}\vert \ge \rho _{c}-\tilde{\rho }>0\) or \(\vert \xi -\tilde{\rho } \mathbf{e}\vert \ge \tilde{\rho }-\rho _{b}>0\) for any ξ on the segment joining x and y. By the mean value theorem, there exists ξ̄ on the segment joining x and y such that

$$\begin{aligned}& \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert \\& \quad = \bigl\vert \nabla u_{\varepsilon ,\mathbf{e}}(\bar{\xi }) \bigr\vert \cdot \vert x-y \vert \\& \quad \le \biggl[C\varepsilon ^{-\frac{N-2s}{2}} \biggl(\mu ^{2}+ \frac{ \vert \bar{ \xi }-\tilde{\rho }\mathbf{e} \vert ^{2}}{ \vert \varepsilon S_{s}^{1/(2s)} \vert ^{2}} \biggr) ^{-\frac{N-2s}{2}} \\& \quad\quad{} +C\varepsilon ^{-\frac{N-2s}{2}} \biggl(\mu ^{2}+\frac{ \vert \bar{ \xi }-\tilde{\rho }\mathbf{e} \vert ^{2}}{ \vert \varepsilon S_{s}^{1/(2s)} \vert ^{2}} \biggr) ^{-\frac{N-2s}{2}-1}\frac{ \vert \bar{\xi }-\tilde{\rho }\mathbf{e} \vert }{ \vert \varepsilon S_{s}^{1/(2s)} \vert ^{2}} \biggr] \vert x-y \vert \\& \quad \le C\varepsilon ^{\frac{N-2s}{2}} \vert x-y \vert . \end{aligned}$$

Hence, by (4.11) and the inequality above, we get

$$ \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert \le \textstyle\begin{cases} C\varepsilon ^{\frac{N-2s}{2}} \vert x-y \vert ,&\text{if }(x,y)\in A_{4}\text{ and } \vert x-y \vert \le \frac{1}{2}(\rho _{c}-\rho _{b}), \\ C\varepsilon ^{\frac{N-2s}{2}},&\text{if }(x,y)\in A_{4} \text{ and } \vert x-y \vert >\frac{1}{2}(\rho _{c}-\rho _{b}), \end{cases} $$
(4.12)

or

$$ \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert \le C \varepsilon ^{\frac{N-2s}{2}}\min \bigl\{ 1, \vert x-y \vert \bigr\} . $$
(4.13)

Consequently, by the definition of \(u_{\varepsilon ,\mathbf{e}}\) and (4.13),

$$\begin{aligned}& \int _{A_{4}} \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon , \mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\ & \quad = \int _{A_{4}\cap (\mathbb{R}^{2N}\setminus B_{3})} \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\ & \quad \le 2 \int _{A_{4}\cap (\{(x,y): \vert x \vert \le \rho _{d},y\in \mathbb{R}^{N}\})} \bigl\vert u _{\varepsilon , \mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\ & \quad\le C\varepsilon ^{N-2s} \int _{A_{4}\cap (\{(x,y): \vert x \vert \le \rho _{d},y\in \mathbb{R}^{N}\})} \frac{ \min \{1, \vert x-y \vert ^{2}\}}{ \vert x-y \vert ^{N+2s}}\,dx\,dy \\ & \quad \le C\varepsilon ^{N-2s} \int _{\{(x,y): \vert x \vert \le \rho _{d},y\in \mathbb{R}^{N}\}} \frac{\min \{1, \vert x-y \vert ^{2} \}}{ \vert x-y \vert ^{N+2s}}\,dx\,dy \\ & \quad = C\varepsilon ^{N-2s} \int _{ \vert x \vert \le \rho _{d}}\,dx \int _{\mathbb{R}^{N}} \frac{ \min \{1, \vert y \vert ^{2}\}}{ \vert y \vert ^{N-2s}}\,dy \\ & \quad = C\varepsilon ^{N-2s} \int _{ \vert x \vert \le \rho _{d}}\,dx \biggl( \int _{ \vert y \vert \le 1} \frac{ \vert y \vert ^{2}}{ \vert y \vert ^{N+2s}}\,dy+ \int _{ \vert y \vert >1}\frac{1}{ \vert y \vert ^{N+2s}}\,dy \biggr) \\ & \quad \le C\varepsilon ^{N-2s}. \end{aligned}$$
(4.14)

(ii) Assume \((x,y)\in A_{3}\). Let \(\xi =tx+(1-t)y=y+t(x-y)\) for any \(t\in [0,1]\). If \(\vert y\vert \ge \rho _{c}\), then

$$ \vert \xi \vert = \bigl\vert y+t(x-y) \bigr\vert \ge \vert y \vert - \vert x-y \vert \ge \rho _{c}-\frac{1}{2}(\rho _{c}- \tilde{\rho }) =\frac{1}{2}(\rho _{c}+\tilde{ \rho }), $$

and so

$$ \vert \xi -\tilde{\rho }\mathbf{e} \vert \ge \frac{1}{2}(\rho _{c}+\tilde{\rho })-\tilde{\rho }=\frac{1}{2}(\rho _{c}-\tilde{\rho })>0. $$

If \(\vert y\vert \le \rho _{b}\), then

$$ \vert \xi \vert \le \vert y \vert + \vert x-y \vert \le \rho _{b}+\frac{1}{2}(\tilde{\rho }-\rho _{b}) = \frac{1}{2}(\tilde{\rho }+\rho _{b}), $$

and so

$$ \vert \xi -\tilde{\rho }\mathbf{e} \vert \ge \tilde{\rho }- \vert \xi \vert \ge \tilde{\rho }-\frac{1}{2}(\tilde{\rho }+\rho _{b})= \frac{1}{2}(\tilde{\rho }-\rho _{b})>0. $$

Thus, by the mean value theorem, there exists ξ̄ on the segment joining x and y such that

$$ \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert = \bigl\vert \nabla u_{\varepsilon ,\mathbf{e}}(\bar{\xi }) \bigr\vert \cdot \vert x-y \vert \le C \varepsilon ^{\frac{N-2s}{2}} \vert x-y \vert . $$

Consequently,

$$\begin{aligned}& \int _{A_{3}} \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon , \mathbf{e}}(y) \bigr\vert ^{2} K(x-y)\,dx\,dy \\ & \quad \le C\varepsilon ^{N-2s} \int _{A_{3}}\frac{ \vert x-y \vert ^{2}}{ \vert x-y \vert ^{N+2s}}\,dx\,dy \\ & \quad \le C\varepsilon ^{\frac{N-2s}{2}} \int _{\tilde{A}_{3}} \frac{ \vert x-y \vert ^{2}}{ \vert x-y \vert ^{N+2s}}\,dx\,dy \\ & \quad \le C\varepsilon ^{N-2s} \int _{\mathcal{D}_{1}}\,dx \int _{\{y\in \mathbb{R}^{N}: \vert x-y \vert \le \delta \}} \frac{1}{ \vert x-y \vert ^{N+2s-2}}\,dy \\ & \quad = C\varepsilon ^{N-2s} \int _{\mathcal{D}_{1}}\,dx \int _{ \vert y \vert \le \delta } \frac{1}{ \vert y \vert ^{N+2s-2}}\,dy \\ & \quad \le C\varepsilon ^{N-2s}, \end{aligned}$$
(4.15)

where \(\tilde{A}_{3}= \{(x,y)\in \mathbb{R}^{N}\times \mathbb{R} ^{N}: x\in \mathcal{D}_{1}, y\in \mathbb{R}^{N}, \vert x-y\vert \le \delta \}\).

(iii) Assume \((x,y)\in A_{2}\). Since \(x\in \mathcal{D}_{1}\), we have

$$\begin{aligned}& \int _{A_{2}} \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon , \mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\& \quad = \int _{A_{2}} \bigl\vert U_{\varepsilon }(x-\tilde{\rho } \mathbf{e})-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\& \quad = \int _{A_{2}} \bigl\vert U_{\varepsilon }(x-\tilde{\rho } \mathbf{e})-U_{\varepsilon }(y-\tilde{\rho }\mathbf{e}) +U_{\varepsilon }(y- \tilde{\rho } \mathbf{e})-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2}K(x-y) \,dx\,dy \\& \quad \le \int _{A_{2}} \bigl\vert U_{\varepsilon }(x-\tilde{\rho } \mathbf{e})-U_{ \varepsilon }(y-\tilde{\rho }\mathbf{e}) \bigr\vert ^{2} K(x-y)\,dx\,dy \\& \quad \quad{}+ \int _{A_{2}} \bigl\vert U _{\varepsilon }(y-\tilde{\rho } \mathbf{e})-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\& \quad \quad{} +2 \int _{A_{2}} \bigl\vert U_{\varepsilon }(x-\tilde{\rho } \mathbf{e})-U_{ \varepsilon }(y-\tilde{\rho }\mathbf{e}) \bigr\vert \cdot \bigl\vert U_{\varepsilon }(y- \tilde{\rho }\mathbf{e})-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert K(x-y)\,dx\,dy. \end{aligned}$$
(4.16)

Direct computation yields

$$\begin{aligned}& \int _{A_{2}} \bigl\vert U_{\varepsilon }(y-\tilde{\rho } \mathbf{e})-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2} K(x-y)\,dx\,dy \\& \quad \le \int _{A_{2}}\frac{ ( \vert U_{\varepsilon }(y-\tilde{\rho } \mathbf{e}) \vert +\vert u_{\varepsilon ,\mathbf{e}}(y)\vert )^{2}}{ \vert x-y \vert ^{N+2s}}\,dx\,dy \\& \quad \le 4 \int _{A_{2}} \frac{ \vert U_{\varepsilon }(y-\tilde{\rho }\mathbf{e}) \vert ^{2}}{ \vert x-y \vert ^{N+2s}}\,dx\,dy \\& \quad \le C\varepsilon ^{N-2s} \int _{A_{2}}\frac{1}{ \vert x-y \vert ^{N+2s}}\,dx\,dy \\& \quad = C\varepsilon ^{N-2s} \int _{\mathcal{D}_{1}}\,dx \int _{\{y\in \mathbb{R}^{N}: \vert x-y \vert >\delta \}} \frac{1}{ \vert x-y \vert ^{N+2s}}\,dy \\& \quad \le C\varepsilon ^{N-2s} \int _{\mathcal{D}_{1}}\,dx \int _{\{y\in \mathbb{R}^{N}: \vert y \vert >\delta \}} \frac{1}{ \vert y \vert ^{N+2s}}\,dy \\& \quad \le C\varepsilon ^{N-2s}, \end{aligned}$$
(4.17)

where \(\tilde{A}_{2}=\{(x,y)\in \mathbb{R}^{N}\times \mathbb{R}^{N}: x\in \mathcal{D}_{1}, y\in \mathbb{R}^{N}, \vert x-y\vert >\delta \}\).

For any \((x,y)\in A_{2}\),

$$ \bigl\vert U_{\varepsilon }(x-\tilde{\rho }\mathbf{e})U_{\varepsilon }(y- \tilde{\rho }\mathbf{e}) \bigr\vert \le C\varepsilon ^{\frac{N-2s}{2}} \bigl\vert U_{\varepsilon }(x-\tilde{\rho }\mathbf{e}) \bigr\vert \le C \biggl(\mu ^{2}+ \biggl\vert \frac{x- \tilde{\rho }\mathbf{e}}{\varepsilon S_{s}^{1/(2s)}} \biggr\vert ^{2} \biggr) ^{-\frac{N-2s}{2}}. $$

Therefore, using the change of variable \(\xi =x\), \(\zeta =x-y\), we have that

$$\begin{aligned}& \int _{A_{2}} \bigl\vert U_{\varepsilon }(x-\tilde{\rho } \mathbf{e}) \bigr\vert \cdot \bigl\vert U _{\varepsilon }(y-\tilde{\rho } \mathbf{e})-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert K(x-y)\,dx\,dy \\& \quad \le 2 \int _{A_{2}} \bigl\vert U_{\varepsilon }(x-\tilde{\rho } \mathbf{e}) \bigr\vert \cdot \bigl\vert U_{\varepsilon }(y-\tilde{\rho } \mathbf{e}) \bigr\vert K(x-y)\,dx\,dy \\& \quad \le C \int _{A_{2}} \biggl(\mu ^{2}+ \biggl\vert \frac{x-\tilde{\rho } \mathbf{e}}{\varepsilon S_{s}^{1/(2s)}} \biggr\vert ^{2} \biggr)^{- \frac{N-2s}{2}} \vert x-y \vert ^{-(N+2s)}\,dx\,dy \\& \quad = C \int _{\mathcal{D}_{1}} \biggl(\mu ^{2}+ \biggl\vert \frac{\xi - \tilde{\rho }\mathbf{e}}{\varepsilon S_{s}^{1/(2s)}} \biggr\vert ^{2} \biggr) ^{-\frac{N-2s}{2}}\,d\xi \int _{\{\zeta \in \mathbb{R}^{N}: \vert \zeta \vert > \delta \}}\frac{1}{\zeta ^{N+2s}}\,d\zeta \\& \quad \le C \int _{\{\xi \in \mathbb{R}^{N}: \vert \xi -\tilde{\rho }\mathbf{e} \vert \le \rho _{c}+\tilde{\rho }\}} \biggl(\mu ^{2}+ \biggl\vert \frac{\xi -\tilde{\rho } \mathbf{e}}{\varepsilon S_{s}^{1/(2s)}} \biggr\vert ^{2} \biggr)^{- \frac{N-2s}{2}}\,d\xi \\& \quad \le C\varepsilon ^{N} \int _{\{\xi \in \mathbb{R}^{N}: \vert \xi \vert \le S_{s}^{-1/(2s)}(\rho _{c}+ \tilde{\rho })\varepsilon ^{-1}\}} \bigl(\mu ^{2}+ \vert \xi \vert ^{2} \bigr) ^{-\frac{N-2s}{2}}\,d\xi \\& \quad \le C\varepsilon ^{N} \biggl( \int _{\{x\in \mathbb{R}^{N}: \vert \xi \vert \le 1 \}} + \int _{\{x\in \mathbb{R}^{N}:1\le \vert \xi \vert \le S_{s}^{-1/(2s)}(\rho _{c}+ \tilde{\rho })\varepsilon ^{-1}\}} \biggr) \bigl(\mu ^{2}+ \vert \xi \vert ^{2}\bigr)^{- \frac{N-2s}{2}}\,d\xi \\& \quad \le C\varepsilon ^{N-2s}. \end{aligned}$$
(4.18)

Similar to (4.17), we have

$$\begin{aligned}& \int _{A_{2}} \bigl\vert U_{\varepsilon }(y-\tilde{\rho } \mathbf{e}) \bigr\vert \cdot \bigl\vert U _{\varepsilon }(y-\tilde{\rho } \mathbf{e}) -u_{\varepsilon , \mathbf{e}}(y) \bigr\vert K(x-y)\,dx\,dy \\& \quad \le 2 \int _{A_{2}} \frac{ \vert U_{\varepsilon }(y-\tilde{\rho }\mathbf{e}) \vert ^{2}}{ \vert x-y \vert ^{N+2s}}\,dx\,dy \\& \quad \le C\varepsilon ^{N-2s}. \end{aligned}$$
(4.19)

By (4.16)–(4.19), we get

$$ \begin{aligned}[b] &\int _{A_{2}} \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon , \mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\ &\quad \le \int _{A_{2}} \bigl\vert U_{\varepsilon }(x- \tilde{\rho } \mathbf{e}) -U_{\varepsilon }(y-\tilde{\rho }\mathbf{e}) \bigr\vert ^{2}K(x-y)\,dx\,dy+C \varepsilon ^{N-2s}.\end{aligned} $$
(4.20)

By (4.10), (4.14), (4.15), and (4.20), we have

$$\begin{aligned}& \int _{\mathbb{R}^{2N}} \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2} K(x-y)\,dx\,dy \\& \quad \le \int _{A_{1}} \bigl\vert u_{\varepsilon ,\mathbf{e}}(x)-u_{\varepsilon , \mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\& \quad\quad{} +2 \int _{A_{2}} \bigl\vert U_{\varepsilon }(x- \tilde{\rho } \mathbf{e})-U_{\varepsilon }(y-\tilde{\rho }\mathbf{e}) \bigr\vert ^{2} K(x-y)\,dx\,dy+C\varepsilon ^{N-2s} \\& \quad \le \int _{\mathbb{R}^{2N}} \bigl\vert U_{\varepsilon }(x-\tilde{\rho } \mathbf{e}) -U_{\varepsilon }(y-\tilde{\rho }\mathbf{e}) \bigr\vert ^{2}K(x-y)\,dx\,dy+C \varepsilon ^{N-2s}. \end{aligned}$$
(4.21)

Using the change of variable and (4.13) in [32], we have

$$\begin{aligned} \int _{\mathbb{R}^{2N}} \bigl\vert U_{\varepsilon }(x-\tilde{\rho } \mathbf{e}) -U _{\varepsilon }(y-\tilde{\rho }\mathbf{e}) \bigr\vert ^{2}K(x-y)\,dx\,dy =& \int _{\mathbb{R}^{2N}} \bigl\vert U_{\varepsilon }(x)-U_{\varepsilon }(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\ =& \int _{\mathbb{R}^{N}} \bigl\vert U_{\varepsilon }(x) \bigr\vert ^{2^{*}_{s}}\,dx=S_{s}^{ \frac{N}{2s}} . \end{aligned}$$
(4.22)

By (4.21) and (4.22), we have

$$ \Vert u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2}= \int _{\mathbb{R}^{2N}} \bigl\vert u _{\varepsilon ,\mathbf{e}}(x) -u_{\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \le S_{s}^{\frac{N}{2s}}+C\varepsilon ^{N-2s}. $$
(4.23)

On the other hand, by the definition of \(S_{s}\) and (i), we have

$$ \Vert u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2} \ge S_{s} \biggl( \int _{ \mathbb{R}^{N}} \vert u_{\varepsilon ,\mathbf{e}} \vert ^{2^{*}_{s}} \,dx \biggr) ^{2/2^{*}_{s}} =S_{s}^{\frac{N}{2s}}+o(1). $$
(4.24)

Combining (4.23) and (4.24), we prove (ii). □

Lemma 4.2

Assume that \(a\in C(\bar{\varOmega })\) with \(\vert a^{+}\vert _{q}\in (0,\sigma _{2})\). There exists \(\varepsilon _{0}>0\) such that, for \(\varepsilon <\varepsilon _{0}\),

$$ \sup_{t\ge 0}I(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})< m^{+}+ \frac{s}{N}S_{s}^{\frac{N}{2s}} $$

uniformly in \(\mathbf{e}\in \mathbb{S}^{N-1}\), where \(\tilde{u}_{1}\) is a minimizer of I in Theorem 3.3.

Proof

Since I is continuous in \(X_{0}\) and \(u_{\varepsilon ,\mathbf{e}}\) is uniformly bounded in \(X_{0}\) for ε small enough, there exists \(t_{1}>0\) such that, for \(t\in [0,t_{1}]\),

$$ I(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})< I(\tilde{u}_{1})+ \frac{s}{N}S_{s}^{\frac{N}{2s}}. $$

Since \(u_{\varepsilon ,\mathbf{e}}(x)=0\) for any \(x\in \{x\in \varOmega : a(x)<0\}\), we have

$$\begin{aligned} I(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}}) =&\frac{1}{2} \Vert \tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2}- \frac{1}{p} \int _{\varOmega }a(x) (\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})^{p} \,dx-\frac{1}{2^{*} _{s}} \int _{\varOmega }(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})^{2^{*} _{s}} \,dx \\ =&\frac{1}{2} \Vert \tilde{u}_{1} \Vert _{X_{0}}^{2}+t(\tilde{u}_{1}, u_{ \varepsilon ,\mathbf{e}})_{X_{0}}+ \frac{t^{2}}{2} \Vert u_{\varepsilon , \mathbf{e}} \Vert _{X_{0}}^{2}- \frac{1}{p} \int _{\varOmega }a^{+}(x) ( \tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})^{p} \,dx \\ &{}+\frac{1}{p} \int _{\varOmega }a^{-}(x)\tilde{u}_{1}^{p} \,dx-\frac{1}{2^{*} _{s}} \int _{\varOmega }(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})^{2^{*} _{s}} \,dx. \end{aligned}$$
(4.25)

It is easy to get from Lemma 4.1 that

$$ \int _{\varOmega }u^{2^{*}_{s}}_{\varepsilon ,\mathbf{e}}\,dx\ge \frac{1}{2}S_{s}^{\frac{N}{2s}} $$

for ε small enough. Note that the last term in (4.25) satisfies

$$ \frac{1}{2^{*}_{s}} \int _{\varOmega }(\tilde{u}_{1}+tu_{\varepsilon , \mathbf{e}})^{2^{*}_{s}} \,dx\ge \frac{t^{2^{*}_{s}}}{2^{*}_{s}} \int _{\varOmega }u^{2^{*}_{s}}_{\varepsilon ,\mathbf{e}}\,dx\ge \frac{S _{s}^{\frac{N}{2s}}}{22^{*}_{s}}t^{2^{*}_{s}}. $$

Thus, \(I(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})\to -\infty \) as \(t\to +\infty \) uniformly in ε and e. Consequently, there exists \(t_{2}>t_{1}\) such that \(I(\tilde{u}_{1}+tu _{\varepsilon ,\mathbf{e}})< m^{+}+\frac{s}{N}S_{s}^{\frac{N}{2s}}\) for \(t\ge t_{2}\). Then, we only need to verify the inequality

$$ \sup_{t_{1}\le t\le t_{2}}I(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})< m ^{+}+\frac{s}{N}S_{s}^{\frac{N}{2s}} $$

for ε small enough.

From now on, we assume that \(t\in [t_{1}, t_{2}]\).

There exists a constant \(C>0\) such that

$$\begin{aligned} \int _{\varOmega }(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})^{2^{*}_{s}} \,dx \ge & \int _{\varOmega }\tilde{u}_{1}^{2^{*}_{s}} \,dx+t^{2^{*}_{s}} \int _{ \varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}}\,dx + 2^{*}_{s}t \int _{\varOmega }\tilde{u}_{1}^{2^{*}_{s}-1}u_{\varepsilon ,\mathbf{e}} \,dx \\ &{} +2^{*}_{s}t^{2^{*}_{s}-1} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}} ^{2^{*}_{s}-1} \tilde{u}_{1}\,dx -Ct^{2^{*}_{s}/2} \int _{\varOmega } \tilde{u}_{1}^{2^{*}_{s}/2}u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}/2} \,dx. \end{aligned}$$
(4.26)

We have used the following inequality (see [5, 40] for example): for \(r>2\), there exists a constant \(C_{r}\) (depending on r) such that

$$ (\alpha +\beta )^{r}\ge \alpha ^{r}+\beta ^{r}+r \bigl(\alpha ^{r-1} \beta +\alpha \beta ^{r-1} \bigr) -C_{r}\alpha ^{r/2}\beta ^{r/2} \quad \forall \alpha ,\beta >0. $$

Combining (4.25) and (4.26), and using the fact that \(\tilde{u}_{1}\) is a positive solution of (1.1), we have

$$\begin{aligned}& I(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}}) \\ & \quad \le \frac{1}{2} \Vert \tilde{u}_{1} \Vert _{X_{0}}^{2}+t( \tilde{u}_{1}, u_{\varepsilon , \mathbf{e}})_{X_{0}}+\frac{t^{2}}{2} \Vert u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2}-\frac{1}{p} \int _{\varOmega }a^{+}(x) (\tilde{u}_{1}+tu_{ \varepsilon ,\mathbf{e}})^{p} \,dx \\ & \quad \quad {} +\frac{1}{p} \int _{\varOmega }a^{-}(x)\tilde{u}_{1}^{p} \,dx-\frac{1}{2^{*} _{s}} \int _{\varOmega }\tilde{u}_{1}^{2^{*}_{s}}\,dx- \frac{1}{2^{*}_{s}}t ^{2^{*}_{s}} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}}\,dx \\ & \quad \quad {} -t \int _{\varOmega }\tilde{u}_{1}^{2^{*}_{s}-1}u_{\varepsilon , \mathbf{e}} \,dx -t^{2^{*}_{s}-1} \int _{\varOmega }u_{\varepsilon , \mathbf{e}}^{2^{*}_{s}-1} \tilde{u}_{1} \,dx +Ct^{2^{*}_{s}/2} \int _{ \varOmega }\tilde{u}_{1}^{2^{*}_{s}/2}u_{\varepsilon ,\mathbf{e}}^{2^{*} _{s}/2} \,dx \\ & \quad = \frac{1}{2} \Vert \tilde{u}_{1} \Vert _{X_{0}}^{2}+t \int _{\varOmega }a(x)^{+} \tilde{u}_{1}^{p-1}u_{\varepsilon ,\mathbf{e}} \,dx+\frac{t^{2}}{2} \Vert u _{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2}- \frac{1}{p} \int _{\varOmega }a ^{+}(x) (\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})^{p} \,dx \\ & \quad \quad {} +\frac{1}{p} \int _{\varOmega }a^{-}(x)\tilde{u}_{1}^{p} \,dx-\frac{1}{2^{*} _{s}} \int _{\varOmega }\tilde{u}_{1}^{2^{*}_{s}}\,dx- \frac{1}{2^{*}_{s}}t ^{2^{*}_{s}} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}}\,dx -t ^{2^{*}_{s}-1} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}-1} \tilde{u}_{1} \,dx \\ & \quad \quad {} +Ct^{2^{*}_{s}/2} \int _{\varOmega }\tilde{u}_{1}^{2^{*}_{s}/2}u_{ \varepsilon ,\mathbf{e}}^{2^{*}_{s}/2} \,dx \\ & \quad = I(\tilde{u}_{1})+t \int _{\varOmega }a^{+}(x)\tilde{u}_{1}^{p-1}u_{ \varepsilon ,\mathbf{e}} \,dx+\frac{t^{2}}{2} \Vert u_{\varepsilon , \mathbf{e}} \Vert _{X_{0}}^{2}- \frac{1}{p} \int _{\varOmega }a^{+}(x) ( \tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})^{p} \,dx \\ & \quad \quad {} +\frac{1}{p} \int _{\varOmega }a^{+}(x)\tilde{u}_{1}^{p} \,dx-\frac{1}{2^{*} _{s}}t^{2^{*}_{s}} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*} _{s}}\,dx -t^{2^{*}_{s}-1} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*} _{s}-1} \tilde{u}_{1}\,dx \\ & \quad \quad {} +Ct^{2^{*}_{s}/2} \int _{\varOmega }\tilde{u}_{1}^{2^{*}_{s}/2}u_{ \varepsilon ,\mathbf{e}}^{2^{*}_{s}/2} \,dx \\ & \quad = I(\tilde{u}_{1})+\frac{t^{2}}{2} \Vert u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2}-\frac{1}{2^{*}_{s}}t^{2^{*}_{s}} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}}\,dx \\ & \quad \quad {} -\frac{1}{p} \int _{\varOmega }a^{+}(x) \bigl[(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})^{p} \,dx-\tilde{u}_{1}^{p}-p\tilde{u}_{1}^{p-1}tu_{\varepsilon ,\mathbf{e}} \bigr]\,dx \\ & \quad \quad {} -t^{2^{*}_{s}-1} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*} _{s}-1} \tilde{u}_{1}\,dx +Ct^{2^{*}_{s}/2} \int _{\varOmega }\tilde{u}_{1} ^{2^{*}_{s}/2}u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}/2} \,dx \\ & \quad \le I(\tilde{u}_{1})+\frac{t^{2}}{2} \Vert u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2}-\frac{t^{2^{*}_{s}}}{2^{*}_{s}} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}}\,dx \\ & \quad \quad {} -t^{2^{*}_{s}/2} \biggl(t^{(2^{*}_{s}-2)/2} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}-1} \tilde{u}_{1} \,dx-C \int _{\varOmega }\tilde{u} _{1}^{2^{*}_{s}/2}u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}/2} \,dx \biggr) \\ & \quad \le I(\tilde{u}_{1})+S^{\frac{N}{2s}}_{s} \biggl( \frac{t^{2}}{2}-\frac{t ^{2^{*}_{s}}}{2^{*}_{s}} \biggr) -t^{\frac{2^{*}_{s}}{2}} \biggl(t^{\frac{2^{*}_{s}-2}{2}} \int _{\varOmega }u_{\varepsilon , \mathbf{e}}^{2^{*}_{s}-1} \tilde{u}_{1} \,dx-C \int _{\varOmega } \tilde{u}_{1}^{2^{*}_{s}/2}u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}/2} \,dx \biggr) \\ & \quad \quad {}+O\bigl( \varepsilon ^{N-2s}\bigr) \\ & \quad \le I(\tilde{u}_{1})+\frac{s}{N}S^{\frac{N}{2s}}_{s}-t^{2^{*}_{s}/2} \biggl(t^{\frac{2^{*}_{s}-2}{2}} \int _{\varOmega }u_{\varepsilon , \mathbf{e}}^{2^{*}_{s}-1} \tilde{u}_{1} \,dx-C \int _{\varOmega }\tilde{u} _{1}^{2^{*}_{s}/2}u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}/2} \,dx \biggr) \\ & \quad \quad {}+O\bigl( \varepsilon ^{N-2s}\bigr). \end{aligned}$$
(4.27)

Here we have used the elementary inequality: \((\alpha +\beta )^{p} \ge \alpha ^{p}+p\alpha ^{p-1}\beta \), \(\forall \alpha ,\beta >0\).

Now, we estimate the last but one term in (4.27). By Theorem 3.3, there exists a constant \(C_{1}>0\) such that \(\tilde{u}_{1}(x) \ge C_{1}\) for \(x\in E:=\{x\in \mathbb{R}^{N}:\rho _{b}\le \vert x\vert \le \rho _{c}\}\). Thus,

$$\begin{aligned} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}-1} \tilde{u}_{1} \,dx \ge &C_{1} \int _{E} u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}-1}\,dx \\ \ge &C_{1} \int _{E} U_{\varepsilon }^{2^{*}_{s}-1}(x-\tilde{\rho } \mathbf{e})\,dx \\ \ge &C_{1} \int _{E_{1}}U_{\varepsilon }^{2^{*}_{s}-1}(x)\,dx \\ \ge &C_{1}\varepsilon ^{\frac{N-2s}{2}} \int _{E_{2}}\frac{dx}{ (\mu ^{2}+ \vert x \vert ^{2} )^{\frac{N+2s}{2}}} \\ \ge &C_{2}\varepsilon ^{\frac{N-2s}{2}} \end{aligned}$$
(4.28)

for ε small enough, where

$$\begin{aligned}& E_{1} :=\bigl\{ x\in \mathbb{R}^{N}: \vert x \vert \le \min \{\tilde{\rho }-\rho _{b}, \rho _{c}-\tilde{\rho }\} \bigr\} , \\& E_{2} :=\bigl\{ x\in \mathbb{R}^{N}: \vert x \vert \le \min \{\tilde{\rho }-\rho _{b}, \rho _{c}-\tilde{\rho }\}/ \varepsilon \bigr\} . \end{aligned}$$

Direct computation yields that

$$\begin{aligned} \int _{\varOmega }\tilde{u}_{1}^{2^{*}_{s}/2}u_{\varepsilon ,\mathbf{e}} ^{2^{*}_{s}/2}\,dx \le & C_{3} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}} ^{2^{*}_{s}/2}\,dx \\ \le &C_{3} \int _{D_{1}} U_{\varepsilon }^{2^{*}_{s}/2}(x- \tilde{\rho } \mathbf{e})\,dx \\ \le &C_{3} \int _{D_{2}}U_{\varepsilon }^{2^{*}_{s}/2}(x)\,dx \\ \le &C_{4}\varepsilon ^{\frac{N}{2}} \int _{D_{3}} \frac{1}{(\mu ^{2}+ \vert x \vert ^{2})^{N/2}}\,dx \\ \le &C_{5}\varepsilon ^{\frac{N}{2}} \vert \ln \varepsilon \vert , \end{aligned}$$
(4.29)

where

$$\begin{aligned}& D_{1} := \bigl\{ x\in \mathbb{R}^{N}: \rho _{a} \le \vert x \vert \le \rho _{d}\bigr\} , \\& D_{2} := \bigl\{ x\in \mathbb{R}^{N}: \vert x \vert \le \rho _{d}+\tilde{\rho }\bigr\} , \\& D_{3} := \bigl\{ x\in \mathbb{R}^{N}: \vert x \vert \le (\rho _{d}+\tilde{\rho })/ \varepsilon \bigr\} . \end{aligned}$$

Hence, by (4.28) and (4.29), we have

$$\begin{aligned}& t^{(2^{*}_{s}-2)/2} \int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*} _{s}-1} \tilde{u}_{1}\,dx-C \int _{\varOmega }\tilde{u}_{1}^{2^{*}_{s}/2}u _{\varepsilon ,\mathbf{e}}^{2^{*}_{s}/2}\,dx \\& \quad \ge t_{1}^{\frac{2^{*}_{s}-2}{2}} \int _{\varOmega }u_{\varepsilon , \mathbf{e}}^{2^{*}_{s}-1} \tilde{u}_{1} \,dx-C \int _{\varOmega }\tilde{u} _{1}^{2^{*}_{s}/2}u_{\varepsilon ,\mathbf{e}}^{2^{*}_{s}/2} \,dx \\& \quad \ge t_{1}^{\frac{2^{*}_{s}-2}{2}}C_{2}\varepsilon ^{\frac{N-2s}{2}}-C _{5}\varepsilon ^{\frac{N}{2}} \vert \ln \varepsilon \vert \end{aligned}$$

for \(\varepsilon >0\) small enough. Consequently, by (4.27), we have

$$ \sup_{t_{1}\le t\le t_{2}}I(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})< I( \tilde{u}_{1})+\frac{s}{N}S^{\frac{N}{2s}}_{s} $$
(4.30)

for \(\varepsilon >0\) small enough. □

Let

$$\begin{aligned}& \mathcal{A}_{1} := \biggl\{ u\in X_{0}^{+}: \frac{1}{ \Vert u \Vert _{X_{0}}}t ^{-} \biggl(\frac{u}{ \Vert u \Vert _{X_{0}}} \biggr)>1 \biggr\} , \\& \mathcal{A}_{2} := \biggl\{ u\in X_{0}^{+}: \frac{1}{ \Vert u \Vert _{X_{0}}}t ^{-} \biggl(\frac{u}{ \Vert u \Vert _{X_{0}}} \biggr)< 1 \biggr\} . \end{aligned}$$

Lemma 4.3

Assume that \(a\in C(\bar{\varOmega })\) with \(\vert a^{+}\vert _{q}\in (0,\sigma _{2})\). We have

  1. (i)

    \(X_{0}^{+}=\mathcal{A}_{1}\cup \mathcal{A}_{2}\cup \mathcal{N} ^{-}\);

  2. (ii)

    \(\mathcal{N}^{+}\subset \mathcal{A}_{1}\);

  3. (iii)

    for each \(\varepsilon <\varepsilon _{0}\) (\(\varepsilon _{0}\) is defined in Lemma 4.2), there exists \(t_{0}>1\) such that \(\tilde{u}_{1}+t_{0}u_{\varepsilon ,\mathbf{e}}\in \mathcal{A}_{2}\) for all \(\mathbf{e}\in \mathbb{S}^{N-1}\);

  4. (iv)

    for each \(\varepsilon <\varepsilon _{0}\), there exists \(s_{0} \in (0,1)\) such that \(\tilde{u}_{1}+s_{0}t_{0}u_{\varepsilon , \mathbf{e}}\in \mathcal{N}^{-}\) for all \(\mathbf{e}\in \mathbb{S}^{N-1}\);

  5. (v)

    \(m^{-}< m^{+}+\frac{s}{N}S_{s}^{\frac{N}{2s}}\).

Proof

(i) By Lemma 2.4(iv) we prove (i).

(ii) For any \(u\in \mathcal{N}^{+}\), by (2.6), we get that \(\int _{\varOmega }a(x)(u^{+})^{p}\,dx>0\). Let \(v=\frac{u}{\Vert u\Vert _{X_{0}}}\). By Lemma 2.4, there exists \(t^{-}(v)>0\) such that \(t^{-}(v)v \in \mathcal{N}^{-}\), that is,

$$ \frac{1}{ \Vert u \Vert _{X_{0}}}t^{-} \biggl(\frac{u}{ \Vert u \Vert _{X_{0}}} \biggr)u \in \mathcal{N}^{-}. $$

Hence,

$$ t^{-}(u)=\frac{1}{ \Vert u \Vert _{X_{0}}}t^{-} \biggl(\frac{u}{ \Vert u \Vert _{X_{0}}} \biggr). $$

By Lemma 2.4, we have

$$ 1=t^{+}(u)< t_{\max }(u)< t^{-}(u). $$

Thus, we get \(\mathcal{N}^{+}\subset \mathcal{A}_{1}\).

(iii) We claim that there exists \(C>0\) such that \(\sup_{t \ge 0}t^{-} (\frac{\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}}}{ \Vert \tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}}\Vert _{X_{0}}} )< C\). Assume by contradiction that there exists a sequence \(\{t_{n}\}\) such that \(t_{n}\to +\infty \) and \(t^{-}(v_{n})\to +\infty \) as \(n\to \infty \), where \(v_{n}:=\frac{\tilde{u}_{1}+t_{n}u_{\varepsilon , \mathbf{e}}}{\Vert \tilde{u}_{1}+t_{n}u_{\varepsilon ,\mathbf{e}}\Vert _{X _{0}}}\). Since \(t^{-}(v_{n})v_{n}\in \mathcal{N}^{-}\), by Lebesgue’s dominated convergence theorem, we have

$$ \int _{\varOmega }\bigl(v_{n}^{+} \bigr)^{2^{*}_{s}}\,dx=\frac{1}{ \Vert t^{-1}_{n} \tilde{u}_{1}+u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2^{*}_{s}}} \int _{\varOmega }\bigl(t^{-1}_{n} \tilde{u}_{1}+u_{\varepsilon ,\mathbf{e}}\bigr)^{2^{*} _{s}}\,dx \to \frac{\int _{\varOmega }u_{\varepsilon ,\mathbf{e}}^{2^{*} _{s}}\,dx}{ \Vert u_{\varepsilon ,\mathbf{e}} \Vert ^{2^{*}_{s}}_{X_{0}}} $$

as \(n\to \infty \). Thus,

$$ I\bigl(t^{-}(v_{n})v_{n}\bigr)= \frac{1}{2}\bigl(t^{-}(v_{n})\bigr)^{2}- \frac{(t^{-}(v_{n}))^{p}}{p} \int _{\varOmega }a(x) \bigl(v^{+}_{n} \bigr)^{p}\,dx-\frac{(t ^{-}(v_{n}))^{2^{*}_{s}}}{2^{*}_{s}} \int _{\varOmega }\bigl(v^{+}_{n} \bigr)^{2^{*} _{s}}\,dx\to -\infty $$

as \(n\to \infty \), which is impossible since I is bounded from below on \(\mathcal{N}\) by Lemma 2.1. Set

$$ t_{0}=\frac{ \Vert \tilde{u}_{1} \Vert _{X_{0}}+ ( \Vert \tilde{u}_{1} \Vert _{X_{0}} ^{2}+ \vert C^{2}- \Vert \tilde{u}_{1} \Vert _{X_{0}}^{2} \vert )^{1/2}}{ \Vert u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}}+1. $$

Then

$$\begin{aligned} \Vert \tilde{u}_{1}+t_{0}u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2} =& \Vert \tilde{u}_{1} \Vert _{X_{0}}^{2} +t^{2}_{0} \Vert u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2}+2t_{0} ( \tilde{u}_{1}, u_{\varepsilon ,\mathbf{e}} ) _{X_{0}} \\ >& \Vert \tilde{u}_{1} \Vert _{X_{0}}^{2}+ \bigl\vert C^{2}- \Vert \tilde{u}_{1} \Vert _{X _{0}}^{2} \bigr\vert \\ \ge &C^{2}> \biggl[t^{-} \biggl(\frac{\tilde{u}_{1}+tu_{\varepsilon , \mathbf{e}}}{ \Vert \tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}} \biggr) \biggr] ^{2}. \end{aligned}$$

Hence, we get \(\tilde{u}_{1}+t_{0}u_{\varepsilon ,\mathbf{e}}\in \mathcal{A}_{2}\).

(iv) Define \(\gamma : [0,1]\to \mathbb{R}\) as

$$ \gamma (s):=\frac{1}{ \Vert \tilde{u}_{1}+st_{0}u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}}t^{-} \biggl(\frac{\tilde{u}_{1}+st_{0}u_{\varepsilon , \mathbf{e}}}{ \Vert \tilde{u}_{1}+st_{0}u_{\varepsilon ,\mathbf{e}} \Vert _{X _{0}}} \biggr) \quad \text{for all }s\in [0,1]. $$

By Lemma 2.4(iii), \(\gamma (s)\) is a continuous function of s. Since \(\gamma (0)>1\) and \(\gamma (1)<1\) there exists \(s_{0} \in (0,1)\) such that \(\gamma (s_{0})=1\), that is, \(\tilde{u}_{1}+s _{0}t_{0}u_{\varepsilon ,\mathbf{e}}\in \mathcal{N}^{-}\).

(v) By Lemma 4.2 and (iv), we have \(m^{-}< m^{+}+\frac{s}{N}S _{s}^{\frac{N}{2s}}\). □

Consider the following critical problem:

$$ \textstyle\begin{cases} (-\Delta )^{s}u= \vert u \vert ^{2^{*}_{s}-2}u&\text{in }\varOmega , \\ u=0&\text{in }\mathbb{R}^{N}\setminus \varOmega . \end{cases} $$
(4.31)

We define the energy functional \(J: X_{0}\to \mathbb{R}\) associated with the critical problem (4.31) as

$$ J(u)=\frac{1}{2} \Vert u \Vert _{X_{0}}^{2}- \frac{1}{2^{*}_{s}} \int _{\varOmega } \vert u \vert ^{2^{*} _{s}}\,dx. $$

Set

$$ \mathcal{M}(\varOmega )=\bigl\{ u\in X_{0}\setminus \{0\}: \bigl\langle J'(u),u \bigr\rangle =0\bigr\} $$

and

$$ \gamma (\varOmega )=\inf_{u\in \mathcal{M}(\varOmega )}J(u). $$

Similarly, we define \(J_{\infty }: \dot{H}^{s}(\mathbb{R}^{N})\to \mathbb{R}\) as

$$ J_{\infty }(u)=\frac{1}{2} \int _{\mathbb{R}^{2N}}\bigl(u(x)-u(y)\bigr)^{2}K(x-y)\,dx\,dy - \frac{1}{2^{*}_{s}} \int _{\mathbb{R}^{N}} \vert u \vert ^{2^{*}_{s}}\,dx, $$

where \(\dot{H}^{s}(\mathbb{R}^{N})\) denotes the space of functions \(u\in L^{p}(\mathbb{R}^{N})\) such that \(\int _{\mathbb{R}^{2N}}(u(x)-u(y))^{2}K(x-y)\,dx\,dy< \infty \). Set

$$ \mathcal{M}\bigl(\mathbb{R}^{N}\bigr)=\bigl\{ u\in \dot{H}^{s}\bigl(\mathbb{R}^{N}\bigr):\bigl\langle J_{\infty }(u),u\bigr\rangle =0\bigr\} $$

and

$$ \gamma \bigl(\mathbb{R}^{N}\bigr)=\inf_{u\in \mathcal{M}(\mathbb{R}^{N})}J_{ \infty }(u). $$

It is easy to see that \(\gamma (\mathbb{R}^{N})=\frac{s}{N}S_{s}^{ \frac{N}{2s}}\).

Lemma 4.4

\(\gamma (\varOmega )=\gamma (\mathbb{R}^{N})\) and \(\gamma (\varOmega )\) is never achieved except when \(\varOmega =\mathbb{R}^{N}\).

The proof of Lemma 4.4 can be found in [19], and we give a proof for the reader’s’ convenience although these results are known.

Proof

Since \(\mathcal{M}(\varOmega )\subset \mathcal{M}(\mathbb{R}^{N})\), we have \(\gamma (\mathbb{R}^{N})\le \gamma (\varOmega )\). Conversely, let \(\{u_{n}\}\subset \dot{H}^{s}(\mathbb{R}^{N})\) be a minimizing sequence for \(\gamma (\mathbb{R}^{N})\). By density of \(C^{\infty }_{0}( \mathbb{R}^{N})\) in \(\dot{H}^{s}(\mathbb{R}^{N})\) we may assume that \(u_{n}\in C^{\infty }_{0}(\mathbb{R}^{N})\). We can choose \(y_{n} \in \mathbb{R}^{N}\) and \(\lambda _{n}>0\) such that

$$ u_{n}^{y_{n},\lambda _{n}}(\cdot ):=\lambda ^{\frac{N-2s}{2}}_{n}u_{n}( \lambda _{n}\cdot +y_{n})\in C^{\infty }_{0}( \varOmega ). $$

Since

$$ \bigl\Vert u^{y_{n},\lambda _{n}}_{n} \bigr\Vert _{X_{0}}= \Vert u_{n} \Vert _{\dot{H}(\mathbb{R} ^{N})}, \quad\quad \int _{\varOmega } \bigl\vert u^{y_{n},\lambda _{n}}_{n} \bigr\vert ^{p}\,dx= \int _{\mathbb{R}^{N}} \vert u _{n} \vert ^{p} \,dx, $$

we get \(\gamma (\varOmega )\le \gamma (\mathbb{R}^{N})\). Thus, \(\gamma (\varOmega )=\gamma (\mathbb{R}^{N})\).

Assume by contradiction that \(\varOmega \ne \mathbb{R}^{N}\) and \(u\in X_{0}\) is a minimizer for \(\gamma (\varOmega )\). Let \(t>0\) such that \(t\vert u\vert \in \mathcal{M}(\varOmega )\). Then

$$ t= \biggl(\frac{ \Vert \vert u \vert \Vert ^{2}_{X_{0}}}{\int _{\varOmega } \vert u \vert ^{p}\,dx} \biggr) ^{\frac{1}{p-2}}\le \biggl( \frac{ \Vert u \Vert ^{2}_{X_{0}}}{\int _{\varOmega } \vert u \vert ^{p}\,dx} \biggr) ^{\frac{1}{p-2}}=1. $$

Consequently,

$$ \gamma (\varOmega )\le J\bigl(t \vert u \vert \bigr)=t^{p} \biggl(\frac{1}{2}-\frac{1}{p} \biggr) \int _{\varOmega } \vert u \vert ^{p}\,dx\le \gamma ( \varOmega ). $$

Thus, \(t=1\) and \(\vert u\vert \in \mathcal{M}(\varOmega )\) is another minimizer for \(\gamma (\varOmega )\). For this reason we assume straight away that \(u\ge 0\). Clearly, \(u\in \mathcal{\mathbb{R}^{N}}\) is a minimizer for \(J_{\infty }\). Therefore, we get that \(J'_{\infty }(u)=0\). So that u is a solution of

$$ (-\Delta )^{s}u=u^{p} \quad \text{in }\mathbb{R}^{N}. $$

By the maximum principle (Proposition 2.2.8 in [33]), \(u>0\) in \(\mathbb{R}^{N}\). This is a contradiction. □

Lemma 4.5

If \(u\in \mathcal{N}^{-}\) satisfies \(I(u)\le m^{+}+\frac{s}{N}S_{s} ^{\frac{N}{2s}}\), then \(\int _{\varOmega }a(x)(u^{+})^{p}\,dx>0\).

Proof

Let \(u\in \mathcal{N}^{-}\) with \(I(u)\le m^{+}+\frac{s}{N}S_{s}^{ \frac{N}{2s}}\). Then there exists unique \(t(u)>0\) such that \(t(u)u\in \mathcal{M}(\varOmega )\). Assume by contradiction that \(\int _{\varOmega }a(x)(u^{+})^{p}\,dx\le 0\). By Lemmas 2.4 and 4.4,

$$ \begin{aligned} I(u)&=\sup_{t\ge 0}I(tu)\ge I\bigl(t(u)u\bigr) \ge J\bigl(t(u)u \bigr)-\frac{1}{p} \int _{ \varOmega }a(x) \bigl(t(u)u^{+}\bigr)^{p} \,dx\\&\ge \frac{s}{N}S_{s}^{\frac{N}{2s}}-\frac{t ^{p}(u)}{p} \int _{\varOmega }a(x) \bigl(u^{+}\bigr)^{p}\,dx.\end{aligned} $$

Hence, by Lemma 3.1,

$$ \frac{t^{p}(u)}{p} \int _{\varOmega }a(x) \bigl(u^{+}\bigr)^{p}\,dx \ge -m^{+}>0. $$

We get a contradiction. □

5 Existence of the other two solutions

For \(\mu >0\), we define

$$\begin{aligned}& I_{\mu }(u) =\frac{1}{2} \Vert u \Vert _{X_{0}}^{2}- \frac{\mu }{2^{*}_{s}} \int _{\varOmega } \vert u \vert ^{2^{*}_{s}}\,dx, \\& \mathcal{N}_{\mu } = \bigl\{ u\in X_{0}\setminus \{0\}: \bigl\langle I'_{ \mu }(u), u\bigr\rangle =0 \bigr\} . \end{aligned}$$

Lemma 5.1

For each \(u\in \mathcal{N}^{-}\), we have

  1. (i)

    there exists unique \(t_{\mu }(u)>0\) such that \(t_{\mu }(u)u\in \mathcal{N}_{\mu }\), and

    $$ \sup_{t\ge 0}I_{\mu }(tu)=I_{\mu } \bigl(t_{\mu }(u)u\bigr)=\frac{s}{N} \biggl(\frac{ \Vert u \Vert _{X_{0}}^{2^{*}_{s}}}{\mu \int _{\varOmega } \vert u \vert ^{2^{*}_{s}}\,dx} \biggr) ^{\frac{N-2s}{2s}}; $$
  2. (ii)

    there exists unique \(t(u)>0\) such that \(t(u)u\in \mathcal{M}( \varOmega )\), and for \(c\in (0,1)\),

    $$ J\bigl(t(u)u\bigr)\le (1-c)^{-\frac{N}{2s}} \biggl(I(u)+ \frac{2-p}{2p}c^{ \frac{p}{p-2}} \bigl( \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-\frac{p}{2}} \bigr)^{ \frac{2}{2-p}} \biggr). $$
    (5.1)

Proof

(i) The proof is standard, and we omit it.

(ii) Let \(\mu =(1-c)^{-1}\). Then, by Young’s inequality,

$$\begin{aligned} \int _{\varOmega }a(x) \bigl(t_{\mu }(u)u^{+} \bigr)^{p}\,dx \le & \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2}t ^{p}_{\mu }(u) \Vert u \Vert _{X_{0}}^{p} \\ \le &\frac{2-p}{2} \bigl( \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2}c^{-\frac{p}{2}} \bigr) ^{\frac{2}{2-p}}+\frac{p}{2} \bigl(c^{\frac{p}{2}}t^{p}_{\mu }(u) \Vert u \Vert _{X_{0}}^{p} \bigr)^{\frac{2}{p}} \\ =&\frac{2-p}{2}c^{\frac{p}{p-2}} \bigl( \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2} \bigr) ^{\frac{2}{2-p}} + \frac{pc}{2}t^{2}_{\mu }(u) \Vert u \Vert _{X_{0}}^{2}. \end{aligned}$$

By Lemmas 3.1 and 2.4, we have \(I(u)\ge m^{-}>0\) and \(I(u)=\sup_{t\ge 0}I(tu)\). By (i), we have

$$\begin{aligned} I(u) =&\sup_{t\ge 0}I(tu) \\ \ge &I\bigl(t_{\mu }(u)u\bigr) \\ \ge &\frac{1-c}{2} \bigl\Vert t_{\mu }(u)u \bigr\Vert _{X_{0}}^{2}-\frac{2-p}{2p}c^{ \frac{p}{p-2}} \bigl( \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2} \bigr)^{\frac{2}{2-p}}-\frac{1}{2^{*} _{s}} \int _{\varOmega }\bigl(t_{\mu }(u)u^{+} \bigr)^{2^{*}_{s}}\,dx \\ \ge &(1-c)I_{\mu }\bigl(t_{\mu }(u)u\bigr)- \frac{2-p}{2p}c^{\frac{p}{p-2}} \bigl( \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2} \bigr)^{\frac{2}{2-p}} \\ =&(1-c)^{\frac{N}{2s}}\frac{s}{N} \biggl(\frac{ \Vert u \Vert _{X_{0}}^{2^{*} _{s}}}{\int _{\varOmega } \vert u \vert ^{2^{*}_{s}}\,dx} \biggr)^{\frac{N-2s}{2s}} - \frac{2-p}{2p}c^{\frac{p}{p-2}} \bigl( \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-p/2} \bigr) ^{\frac{2}{2-p}} \\ =&(1-c)^{\frac{N}{2s}}J\bigl(t(u)u\bigr)-\frac{2-p}{2p}c^{\frac{p}{p-2}} \bigl( \bigl\vert a ^{+} \bigr\vert _{q}S_{s}^{-p/2} \bigr)^{\frac{2}{2-p}}. \end{aligned}$$

Thus, we get (5.1). □

Lemma 5.2

There exists \(\delta _{0}>0\) such that, for \(u\in \mathcal{M}(\varOmega )\) with \(J(u)\le \frac{s}{N}S_{s}^{\frac{N}{2s}}+\delta _{0}\), we have

$$ \int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \bigl\vert u(x)-u(y) \bigr\vert ^{2}K(x-y)\,dx\,dy\ne 0. $$
(5.2)

Proof

Assume by contradiction that there exists a sequence \(\{u_{n}\}\subset \mathcal{M}(\varOmega )\) such that

$$ J(u_{n})=\frac{s}{N}S_{s}^{\frac{N}{2s}}+o(1) \quad \text{and} \quad \int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy=0. $$

Without loss of generality, we can assume that \(\{u_{n}\}\) is a \((\mathit{PS})_{\gamma (\varOmega )}\)-sequence (for example, see Lemma 7 in [38]) for J. Since J is coercive on \(\mathcal{M}(\varOmega )\), there exists a subsequence of \(\{u_{n}\}\) (still denoted by \(\{u_{n}\}\)) and \(u_{0}\in X_{0}\) such that \(u_{n}\rightharpoonup u _{0}\) in \(X_{0}\). Since Ω is a bounded domain, we have \(u_{0}\equiv 0\). By Theorem 1.1 in [23] and Lemma 4.4, there exist nontrivial solutions \(v^{1},\ldots,v^{\ell } \in \dot{H}^{s}(\mathbb{R}^{N})\) to

$$ (-\Delta )^{s}u= \vert u \vert ^{2^{*}_{s}-2}u \quad \text{in }\mathbb{R}^{N}, $$
(5.3)

or

$$ (-\Delta )^{s}u= \vert u \vert ^{2^{*}_{s}-2}u \quad \text{in }\mathbb{R}^{N}_{+}, \quad\quad u=0 \quad \text{in }\mathbb{R}^{N}\setminus \mathbb{R}^{N}_{+}, $$
(5.4)

where \(\ell \in \mathbb{N}\), sequences of points \(x^{1}_{n},\ldots, x^{\ell }_{n}\subset \varOmega \) and finitely many sequences of numbers \(r^{1}_{n},\ldots,r^{\ell }_{n}\subset (0,+\infty )\) converging to zero such that, up to a subsequence,

$$ u_{n}=\sum^{\ell }_{j=1} \bigl(r^{j}_{n}\bigr)^{\frac{2s-N}{2}} v^{j} \biggl(\frac{x-x ^{j}_{n}}{r^{j}_{n}} \biggr)+o(1) \quad \text{in } \dot{H}^{s}\bigl( \mathbb{R}^{N}\bigr), $$
(5.5)

and

$$ J(u_{n})=\sum^{\ell }_{j=1}J_{\infty } \bigl(v^{j}\bigr)+o(1). $$
(5.6)

If \(\ell >1\), then by (5.6) we have \(J(u_{n})\to \sum^{\ell }_{j=1}J_{\infty }(v^{j})>\gamma (\varOmega )\), which is a contradiction. Thus, by (5.5),

$$ u_{n}=\bigl(r^{1}_{n} \bigr)^{\frac{2s-N}{2}} v^{1} \biggl(\frac{x-x^{1}_{n}}{r^{1} _{n}} \biggr)+o(1) \quad \text{in } \dot{H}^{s}\bigl(\mathbb{R}^{N}\bigr). $$
(5.7)

By (H1), \(\vert x_{n}^{1}\vert \) is bounded from below. Hence, we may assume \(\frac{x_{n}^{1}}{\vert x_{n}^{1}\vert }\to \mathbf{e}\) as \(n\to \infty \), where \(\vert \mathbf{e}\vert =1\). By Lebesgue’s dominated convergence theorem, we have

$$\begin{aligned} 0 =& \int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\ =& \int _{\mathbb{R}^{2N}}\frac{r_{n}^{1}\tilde{x}+x_{n}^{1}}{ \vert r_{n} ^{1}\tilde{x}+x_{n}^{1} \vert } \bigl\vert v^{1}( \tilde{x})-v^{1}(\tilde{y}) \bigr\vert ^{2}K( \tilde{x}- \tilde{y})\,d\tilde{x}\,d\tilde{y}+o(1) \\ =&\mathbf{e}S_{s}^{\frac{N}{2s}}+o(1), \end{aligned}$$

which is impossible. □

Lemma 5.3

There exists \(\sigma _{0}\in (0,\sigma _{2})\) such that, for \(\vert a^{+}\vert _{q} \in (0,\sigma _{0})\), we have

$$ \int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \bigl\vert u(x)-u(y) \bigr\vert ^{2}K(x-y)\,dx\,dy\ne 0 $$

for all \(u\in \mathcal{N}^{-}\) with \(I(u)< m^{+}+\frac{s}{N}S_{s}^{ \frac{N}{2s}}\).

Proof

For \(u\in \mathcal{N}^{-}\) with \(I(u)< m^{+}+\frac{s}{N}S_{s}^{ \frac{N}{2s}}\), there exists \(t(u)>0\) such that \(t(u)u\in \mathcal{M}( \varOmega )\). By Lemma 5.1(ii), for any \(c\in (0,1)\), we have

$$\begin{aligned} J\bigl(t(u)u\bigr) \le &(1-c)^{-\frac{N}{2s}} \biggl(I(u)+\frac{2-p}{2p}c^{ \frac{p}{p-2}} \bigl( \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-\frac{p}{2}} \bigr)^{ \frac{2}{2-p}} \biggr) \\ < &(1-c)^{-\frac{N}{2s}} \biggl(\frac{s}{N}S_{s}^{\frac{N}{2s}}+ \frac{2-p}{2p}c^{\frac{p}{p-2}} \bigl( \bigl\vert a^{+} \bigr\vert _{q}S_{s}^{-\frac{p}{2}} \bigr) ^{\frac{2}{2-p}} \biggr), \end{aligned}$$

since \(m^{+}<0\) by Lemma 3.1. Thus, there exists \(\sigma _{0} \in (0,\sigma _{2})\) such that, for \(a\in C(\bar{\varOmega })\) with \(\vert a^{+}\vert _{q}\in (0,\sigma _{0})\),

$$ J\bigl(t(u)u\bigr)< \frac{s}{N}S_{s}^{\frac{N}{2s}}+\delta _{0}, $$

where \(\delta _{0}\) is given in Lemma 5.2. Consequently, by Lemma 5.2,

$$ \int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \bigl\vert t(u)u(x)-t(u)u(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \ne 0. $$

Hence, we complete the proof. □

Now, we use Lusternik and Schnirelmann’s theory in order to obtain multiplicity results. The notion of category was introduced by Lusternik and Schnirelmann. It is a topological tool used in the estimate of the lower bounded of the number of critical points of a functional.

Definition 5.4

Let \(\mathfrak{X}\) be a topological space. A closed subset A of \(\mathfrak{X}\) is contractible in \(\mathfrak{X}\) if there exists \(h\in C([0,1]\times A,\mathfrak{X})\) and \(v\in \mathfrak{X}\) such that, for every \(u\in A\),

$$ h(0,u)=u, \quad\quad h(1,u)=v. $$

Definition 5.5

The (L–S) category of A with respect to \(\mathfrak{X}\) (or simply the category of A with respect to \(\mathfrak{X}\)), denoted by \(\operatorname{cat}_{ \mathfrak{X}}(A)\), is the least integer k such that \(A\subset A_{1}\cup \cdots \cup A_{k}\), with \(A_{i}\) (\(i=1,\ldots,k\)) closed and contractible in \(\mathfrak{X}\).

We set \(\operatorname{cat}_{\mathfrak{X}}(\emptyset )=0\) and \(\operatorname{cat}_{\mathfrak{X}}(A)=+ \infty \) if there are no integers with the above property. We will use the notation \(\operatorname{cat}(\mathfrak{X})\) for \(\operatorname{cat}_{\mathfrak{X}}(\mathfrak{X})\). For fundamental properties of Lusternik–Schnirelmann category, we refer to Ambrosetti [2], Schwartz [27], and Chang [10].

Theorem 5.6

(Lusternik–Schnirelmann theorem)

Let M be a smooth Banach–Finsler manifold. Suppose that \(f\in C ^{1}(M,\mathbb{R})\) is a functional bounded from below, satisfying the \((\mathit{PS})\) condition. Then f has at least \(\operatorname{cat}(M)\) critical points.

We say f satisfies the \((\mathit{PS})\) condition if any sequence \(\{u_{n}\} \subset M\), such that

$$ \bigl\vert f(u_{n}) \bigr\vert \le \mathrm{const.} \quad \text{and} \quad f'(u_{n})\to 0, $$

has a converging subsequence.

The following lemma is from [1].

Lemma 5.7

Let \(\mathfrak{X}\) be a topological space. Suppose that there exist two continuous maps

$$ F:\mathbb{S}^{N-1}\to \mathfrak{X}, \quad\quad G:\mathfrak{X}\to \mathbb{S}^{N-1} $$

such that \(G\circ F\) is homotopic to identity map of \(\mathbb{S}^{N-1}\), that is, there exists \(\xi \in C([0,1]\times \mathbb{S}^{N-1}, \mathbb{S}^{N-1})\) such that

$$\begin{aligned}& \xi (0,x)=(G\circ F) (x)\quad \textit{for all }x\in \mathbb{S}^{N-1}, \\& \xi (1,x)=x \quad \textit{for all }x\in \mathbb{S}^{N-1}. \end{aligned}$$

Then

$$ \operatorname{cat}(\mathfrak{X})\ge 2. $$

For \(\varepsilon <\varepsilon _{0}\) (\(\varepsilon _{0}\) is defined in Lemma 4.2), we define a map \(\varPhi : \mathbb{S}^{N-1}\to X_{0}\) by

$$ \varPhi (\mathbf{e})=\tilde{u}_{1}+s_{0}t_{0}u_{\varepsilon ,\mathbf{e}} \quad \text{for all }\mathbf{e}\in \mathbb{S}^{N-1}, $$

where \(s_{0}\), \(t_{0}\) are given in Lemma 4.3.

Lemma 5.8

\(\varPhi (\mathbb{S}^{N-1})\) is compact.

Proof

Let \(\{\mathbf{e}_{n}\}\subset \mathbb{S}^{N-1}\) be a sequence such that \(\mathbf{e}_{n}\to \mathbf{e}_{0}\) as \(n\to \infty \). Using a similar argument as that in the proof of Lemma 4.1 and Lebesgue’s dominated convergence theorem, we obtain \(\Vert u_{\varepsilon , \mathbf{e}_{n}}\Vert _{X_{0}}\to \Vert u_{\varepsilon ,\mathbf{e}_{0}}\Vert _{X _{0}}\) as \(n\to \infty \). Since \(X_{0}\) is a Hilbert space and \(u_{\varepsilon ,\mathbf{e}_{n}}\rightharpoonup u_{\varepsilon , \mathbf{e}_{0}}\), we get \(\Vert u_{\varepsilon ,\mathbf{e}_{n}}-u_{\varepsilon ,\mathbf{e}_{0}}\Vert _{X_{0}}\to 0\). Consequently, \(\varPhi (\mathbf{e}_{n}) \to \varPhi (\mathbf{e}_{0})\). □

For \(c\in \mathbb{R}\), we define

$$ I^{c}:=\bigl\{ u\in \mathcal{N}^{-}: I(u)\le c\bigr\} . $$

Lemma 5.9

There exists \(d_{\varepsilon }\in (0,m^{+}+\frac{s}{N}S_{s}^{ \frac{N}{2s}} )\) such that \(\varPhi (\mathbb{S}^{N-1})\subset I ^{d_{\varepsilon }}\) for each \(\varepsilon \in (0, \varepsilon _{0})\).

Proof

By Lemmas 4.2 and 4.3(iii), for each \(\varepsilon \in (0,\varepsilon _{0})\), we have \(\tilde{u}_{1}+s_{0}t_{0}u_{\varepsilon ,\mathbf{e}}\in \mathcal{N}^{-}\) and

$$ \sup_{t\ge 0}I(\tilde{u}_{1}+tu_{\varepsilon ,\mathbf{e}})< m^{+}+ \frac{s}{N}S_{s}^{\frac{N}{2s}} $$

uniformly in \(\mathbf{e}\in \mathbb{S}^{N-1}\). Since \(\varPhi ( \mathbb{S}^{N-1})\) is compact by Lemma 5.8, there exists \(d_{\varepsilon }\in (0, m^{+}+\frac{S}{N}S_{s}^{\frac{N}{2s}} )\) such that \(\varPhi ( \mathbb{S}^{N-1})\subset I^{d_{\varepsilon }}\). □

Set \(\beta =m^{+}+\frac{s}{N}S_{s}^{\frac{N}{2s}}\) and define \(\varPsi : I^{\beta }\to \mathbb{S}^{N-1}\) by

$$ \varPsi (u)=\frac{\int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \vert u(x)-u(y) \vert ^{2}K(x-y)\,dx\,dy}{ \vert \int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \vert u(x)-u(y) \vert ^{2}K(x-y)\,dx\,dy \vert }. $$

By Lemma 5.3, Ψ is well-defined. Let

$$ \varSigma = \biggl\{ u\in X_{0}\setminus \{0\}: \int _{\mathbb{R}^{2N}} \frac{x}{ \vert x \vert } \bigl\vert u(x)-u(y) \bigr\vert ^{2}K(x-y)\,dx\,dy\ne 0 \biggr\} . $$

We define \(\widetilde{\varPsi }: \varSigma \to \mathbb{S}^{N-1}\) by

$$ \widetilde{\varPsi }(u)=\frac{\int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \vert u(x)-u(y) \vert ^{2}K(x-y)\,dx\,dy}{ \vert \int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \vert u(x)-u(y) \vert ^{2}K(x-y)\,dx\,dy \vert }. $$

Clearly, Ψ̃ is an extension of Ψ.

Lemma 5.10

\(u_{\varepsilon ,\mathbf{e}}\in \varSigma \) for all \(\mathbf{e}\in \mathbb{S}^{N-1}\) and for ε small enough.

Proof

For every \(u_{\varepsilon ,\mathbf{e}}\), one sees immediately that there exists \(t(\varepsilon ,\mathbf{e})>0\) such that \(t(\varepsilon , \mathbf{e})u_{\varepsilon ,\mathbf{e}}\in \mathcal{M}(\varOmega )\). Indeed, \(t(\varepsilon ,\mathbf{e})u_{\varepsilon ,\mathbf{e}}\in \mathcal{M}( \varOmega )\) is equivalent to

$$ \bigl\Vert t(\varepsilon ,\mathbf{e})u_{\varepsilon ,\mathbf{e}} \bigr\Vert _{X_{0}}^{2}= \int _{\varOmega } \bigl\vert t(\varepsilon ,\mathbf{e})u_{\varepsilon ,\mathbf{e}} \bigr\vert ^{2^{*} _{s}}\,dx, $$

which is solved by

$$ t(\varepsilon ,\mathbf{e})= \biggl(\frac{ \Vert u_{\varepsilon ,\mathbf{e}} \Vert _{X_{0}}^{2}}{\int _{\varOmega } \vert u_{\varepsilon ,\mathbf{e}} \vert ^{2^{*} _{s}}\,dx} \biggr)^{1/(2^{*}_{s}-2)}. $$

By Lemma 4.1, we have

$$ \lim_{\varepsilon \to 0}t(\varepsilon ,\mathbf{e})=1 $$

uniformly in \(\mathbf{e}\in \mathbb{S}^{N-1}\). Thus,

$$ \lim_{\varepsilon \to 0}J\bigl(t(\varepsilon ,\mathbf{e})u_{\varepsilon , \mathbf{e}} \bigr)=\frac{s}{N}S_{s}^{\frac{N}{2s}} $$

uniformly in \(\mathbf{e}\in \mathbb{S}^{N-1}\). By Lemma 5.2, we get \(t(\varepsilon ,\mathbf{e})u_{\varepsilon ,\mathbf{e}}\in \varSigma \) for \(\varepsilon >0\) small enough. Consequently, \(u_{\varepsilon , \mathbf{e}}\in \varSigma \). □

Lemma 5.11

\(\varPsi \circ \varPhi : \mathbb{S}^{N-1}\to \mathbb{S}^{N-1}\) is homotopic to the identity.

Proof

By Lemma 5.10, there exists \(\varepsilon ^{*}\in (0,\varepsilon _{0})\) such that, for \(\varepsilon \in (0,\varepsilon ^{*})\), \(u_{\varepsilon ,\mathbf{e}}\in \varSigma \) and \(u_{2(1-\theta )\varepsilon ,\mathbf{e}}\in \varSigma \) for all \(\mathbf{e}\in \mathbb{S}^{N-1}\) and \(\theta \in [\frac{1}{2},1)\). Let \(\gamma : [s_{1},s_{2}] \to \mathbb{S}^{N-1}\) be a regular geodesic between \(\widetilde{\varPsi }(u_{\varepsilon ,\mathbf{e}})\) and \(\widetilde{\varPsi }(\varPhi (\mathbf{e}))\) such that

$$ \gamma (s_{1})=\widetilde{\varPsi }(u_{\varepsilon ,\mathbf{e}}), \quad\quad \gamma (s_{2})=\widetilde{\varPsi }\bigl(\varPhi (\mathbf{e})\bigr). $$

Define \(\xi : [0,1]\times \mathbb{S}^{N-1}\to \mathbb{S}^{N-1}\) by

$$ \xi (\theta ,\mathbf{e})= \textstyle\begin{cases} \gamma (2\theta s_{1}+(1-2\theta )s_{2}))&\text{for }\theta \in [0,\frac{1}{2}), \\ \widetilde{\varPsi }(u_{2(1-\theta )\varepsilon ,\mathbf{e}})&\text{for } \theta \in [\frac{1}{2},1), \\ \mathbf{e}&\text{for }\theta =1. \end{cases} $$

Set \(\tilde{x}=(x-\tilde{\rho }\mathbf{e})/(2(1-\theta )\varepsilon )\), \(\tilde{y}=(y-\tilde{\rho }\mathbf{e})/(2(1-\theta )\varepsilon )\). Then

$$\begin{aligned}& \int _{\mathbb{R}^{2N}}\frac{x}{ \vert x \vert } \bigl\vert u_{2(1-\theta )\varepsilon , \mathbf{e}}(x)-u_{2(1-\theta )\varepsilon ,\mathbf{e}}(y) \bigr\vert ^{2}K(x-y)\,dx\,dy \\& \quad = \int _{\mathbb{R}^{2N}}\frac{\tilde{\rho }\mathbf{e}+2(1-\theta ) \varepsilon \tilde{x}}{ \vert \tilde{\rho }\mathbf{e}+2(1-\theta )\varepsilon \tilde{x} \vert }\eta \bigl(\tilde{\rho } \mathbf{e}+2(1-\theta )\varepsilon \tilde{x}\bigr) \bigl\vert U_{1}( \tilde{x})-U_{1}(\tilde{y}) \bigr\vert ^{2}K( \tilde{x}- \tilde{y})\,d\tilde{x}\,d\tilde{y} \\& \quad \to S_{s}^{\frac{N}{2s}}\mathbf{e}, \end{aligned}$$

as \(\theta \to 1^{-}\) by (4.4) and Lebesgue’s dominated convergence theorem. Consequently,

$$ \lim_{\theta \to 1^{-}}\xi (\theta ,\mathbf{e})=\mathbf{e}. $$

Clearly, \(\xi (\theta ,\mathbf{e})\to \gamma (s-1)=\widetilde{\varPsi }(u _{\varepsilon ,\mathbf{e}})\) as \(\theta \to \frac{1}{2}^{-}\). Thus, \(\xi \in C([0,1]\times \mathbb{S}^{N-1},\mathbb{S}^{N-1})\), and

$$\begin{aligned}& \xi (0,\mathbf{e}) =\widetilde{\varPsi }\bigl(\varPhi (\mathbf{e})\bigr), \\& \xi (1,\mathbf{e}) =\mathbf{e}, \end{aligned}$$

for all \(\mathbf{e}\in \mathbb{S}^{N-1}\). □

Proof of Theorem 1.1

By Lemmas 5.7, 5.9, and 5.11, there exists \(d_{\varepsilon }\in (0,m^{+}+\frac{s}{N}S_{s}^{\frac{N}{2s}} )\) such that

$$ \operatorname{cat} \bigl(I^{d_{\varepsilon }} \bigr)\ge 2. $$

By Lemma 3.2 and Theorem 5.6, I has at least two critical points \(\tilde{u}_{2}\) and \(\tilde{u}_{3}\) in \(\{u\in \mathcal{N}^{-}: I(u)< m^{+}+\frac{s}{N}S_{s}^{\frac{N}{2s}}\}\). By the maximum principle (Proposition 2.2.8 in [33]), \(\tilde{u}_{2}\) and \(\tilde{u} _{3}\) are strictly positive in Ω. By Theorem 3.3, we get three positive solutions \(\tilde{u}_{i}\) (\(i=1,2,3\)) of (1.1). By (2.5) and Lemma 4.5, we have \(\int _{\varOmega }a(x) \tilde{u}^{p}_{i}>0\), \(i=1,2,3\). □