1 Introduction

Recently, a great attention has been devoted to the research of problems involving fractional and nonlocal operators. This type of operators finds many applications in a lot of fields, such as continuum mechanics, phase transition phenomena, population dynamics and game theory, as they are the typical outcome of stochastic stabilization of Lévy processes; see, for instance, [14] and the references therein. There are many works on nonlocal fractional operators and their applications which are very interesting; we refer the interested reader to [514] and the references therein. Here we want to generalize the multiplicity existence results for fractional Laplacian equations in [15] and at the same time fix some bugs there.

In this paper we deal with the following fractional problem in a bounded smooth domain \(\Omega\subset\mathbb{R}^{N}\):

$$ \begin{aligned} & {\mathcal{L}^{p}_{K}}u=\lambda f(x,u) & &\text{in } \Omega, \\ & u=0 & &\text{in } \mathbb{R}^{N}\setminus\Omega, \end{aligned} $$
(1.1)

where \(\lambda>0\), \(1< p<+\infty\). \(\mathcal{L}^{p}_{K} \) is a nonlocal fractional operator defined as follows:

$$\begin{aligned} & {\mathcal{L}^{p}_{K}}\varphi={2}\lim_{\varepsilon\rightarrow0} \int _{\mathbb{R}^{N}\setminus B_{\varepsilon}(x)} \bigl\vert \varphi(x)-\varphi (y) \bigr\vert ^{p-2}\bigl(\varphi(x)-\varphi(y)\bigr)K(x-y)\,dy, \end{aligned}$$

provided that the limit exists and K is a measurable function having the following property:

$$ \textstyle\begin{cases} \gamma K\in L^{1}(\mathbb {R}^{N}) &\text{where }\gamma(x)=\min\{ \vert x \vert ^{p},1\},\\ \text{there exist }\lambda,\Lambda>0,\\ \quad \text{ such that }\lambda < K(x) \vert x \vert ^{N+ps}< \Lambda&\text{for any }x\in\mathbb {R}^{N}\setminus\{0\},\\ K(x)=K(-x) &\text{for any }x\in\mathbb{R}^{N}\setminus\{0\}. \end{cases} $$
(1.2)

Throughout this paper we assume \(N>ps\) with \(s\in(0,1)\). A typical example for K is the singular kernel \(\vert x \vert ^{-(N+ps)}\), in which case problem (1.1) becomes

$$ \begin{aligned} &{(-\Delta)^{s}_{p}}u=\lambda f(x,u)&&\mbox{in }\Omega, \\ &u=0&&\mbox{ in }\mathbb{R}^{N}\setminus\Omega, \end{aligned} $$
(1.3)

where \((-\Delta)^{s}_{p}\) is the fractional p-Laplacian operator defined as

$$\begin{aligned} (-\Delta)^{s}_{p} \varphi(x)={2}\lim_{\varepsilon\rightarrow0} \int _{\mathbb{R}^{N}\setminus B_{\varepsilon}(x)}\frac{ \vert \varphi(x)-\varphi (y) \vert ^{p-2}(\varphi(x)-\varphi(y))}{ \vert x-y \vert ^{N+sp}}\,dy \end{aligned}$$

for all \(x\in\mathbb{R}^{N}\). In the case \(p=2\), problem (1.3) becomes the fractional Laplacian problem

$$ \begin{aligned} &(-\Delta)^{s}u=\lambda f(x,u)&& \mbox{in }\Omega, \\ &u=0&&\mbox{in }\mathbb{R}^{N}\setminus\Omega. \end{aligned} $$
(1.4)

Following [16, 17], we present the main structural assumptions on the nonlinear term f.

\((f_{1})\) :

\(f\in C(\mathbb{R^{N}}\times\mathbb{R})\), \(f(x,0)=0\), \(\liminf_{t\rightarrow 0}\frac{f(x,t)}{ \vert t \vert ^{p-2}t}>-\infty\).

\((f_{2})\) :

\(\lim_{ \vert t \vert \rightarrow\infty}\frac{f(x,t)}{ \vert t \vert ^{p-1}}=0\) uniformly in x.

\((f_{3})\) :

There exist \(t^{-}<0\) and \(t^{+}>0\) such that \(\min\{f(x,t): (x,t)\in \overline{\Omega}\times\{t^{-}\}\}>0\) and \(\max\{f(x,t): (x,t)\in \overline{\Omega}\times\{t^{+}\}\}<0\).

\((f_{4})\) :

There exist \(\hat{t}^{-}\) and \(\hat{t}^{+}\), with \(\hat{t}^{-}< t^{-}\) and \(\hat{t}^{+}>t^{+}\), such that

$$\min\bigl\{ F(x,t): (x,t)\in\overline{\Omega}\times\bigl\{ \hat{t}^{-},\hat{t}^{+} \bigr\} \bigr\} >\max\bigl\{ f(x,t): (x,t)\in\overline{\Omega}\times\bigl[t^{-},t^{+} \bigr]\bigr\} , $$

where \(F(x,t)=\int_{0}^{t}f(x,\xi)\,d\xi\).

A typical example of f satisfying \((f_{1})\)-\((f_{4})\) is

$$f(x,t)= \textstyle\begin{cases} \vert t \vert ^{p-1}( \vert t \vert -1)\operatorname{sgn}(t),& \vert t \vert < 1,\\ ( \vert t \vert -1)^{p-1-\varepsilon}\operatorname{sgn}(t),& \vert t \vert >1, \end{cases} $$

where ε is small.

The natural solution space of problem (1.1) is

$$ X_{0}(\Omega)=\bigl\{ u\in X\bigl(\mathbb{R}^{N}\bigr):u=0 \mbox{ a.e. in } \mathbb {R}^{N}\setminus\Omega\bigr\} , $$
(1.5)

where \(X(\mathbb{R}^{N})\) is the fractional Sobolev space given by

$$X\bigl(\mathbb{R}^{N}\bigr)= \bigl\{ u\in L^{p}\bigl( \mathbb{R}^{N}\bigr):[u]_{s,p}< \infty \bigr\} ,\quad [u]_{s,p}=\biggl( \iint_{\mathbb{R}^{2N}}{ \bigl\vert u(x)-u(y) \bigr\vert ^{p}} {K(x-y)}\, dx\,dy\biggr)^{1/p} $$

and endowed with the norm

$$ \Vert u \Vert _{X(\mathbb{R}^{N})}= \bigl( \Vert u \Vert ^{p}_{L^{p}(\mathbb{R}^{N})}+[u]_{s,p}^{p} \bigr)^{{1}/{p}}. $$

For the basic properties of fractional Sobolev spaces, we refer to [18]. We seek the solutions of (1.1) as the critical points of the functional given by

$$I_{\lambda}(u)=\frac{1}{p}[u]_{s,p}^{p}-\lambda \int_{\Omega}F(x,u)\,dx. $$

Set

$$\alpha_{\lambda}=\inf I_{\lambda} |_{[t^{-},t^{+}]}, $$

where \(t^{-}\) and \(t^{+}\) are the numbers given in \((f_{3})\).

Theorem 1.1

Assume that \((f_{1})\)-\((f_{4})\) hold. Then there exists \(\Lambda>0\) such that, for every \(\lambda\ge\Lambda\), problem (1.1) admits two positive, two negative and two sign-changing solutions in \(Y\setminus[t^{-},t^{+}]\), where \(Y:=X_{0}(\Omega)\cap C^{\alpha/2}(\overline{\Omega})\).

Here, for two real numbers \(t< s\), the symbol \([t,s]\) denotes either an order interval in Y or a usual interval in \(\mathbb{R}\). In this paper there will be no ambiguous meaning concerning this symbol, for example, in Theorem 1.1 \(Y\setminus[t^{-},t^{+}]\) denotes the difference of Y and \([t^{-},t^{+}]\), where \([t^{-},t^{+}]\) is an order interval in Y.

Theorem 1.2

Assume that \((f_{1})\)-\((f_{4})\) hold and \(f(x,t)\) is odd with respect to t. Then, for every \(k\in\mathbb{N}\), there exists positive \(\Lambda _{k}\) such that, for every \(\lambda\ge\Lambda_{k}\), problem (1.1) admits a sign-changing solution \(u_{i}\) with \(I_{\lambda}(u_{i})<\alpha_{\lambda}\) and a sign-changing solution \(v_{i}\) with \(I_{\lambda}(v_{i})\ge\alpha _{\lambda}\), where \(i=1,\ldots,n\).

Next consider the case that \(f(x,t)\) is not odd with respect to t, but oscillates around 0 in the following manner.

\((f_{5})\) :

There exist \(t_{i}^{-}\) and \(t_{i}^{+}\), \(i=1,\ldots,n\), with \(t_{n}^{-}<\cdots <t_{1}^{-}<0<t_{1}^{+}<\cdots<t_{n}^{+}\), such that \(\min\{f(x,t): (x,t)\in\overline{\Omega}\times\{t_{i}^{-}\}\}>0>\max\{ f(x,t): (x,t)\in\overline{\Omega}\times\{t_{i}^{+}\}\}\) for all \(i=1,\ldots,n\).

\((f_{6})\) :

\(\min\{F(x,t): (x,t)\in\overline{\Omega}\times\{t_{i}^{-},t_{i}^{+}\}\}>\max\{ F(x,t): (x,t)\in\overline{\Omega}\times[t_{i-1}^{-},t_{i-1}^{+}]\}\) for \(i=2,\ldots,n\).

Theorem 1.3

Assume that \((f_{1})\)-\((f_{2})\) and \((f_{5})\)-\((f_{6})\) hold. Then there exists \(\Lambda>0\) such that, for every \(\lambda\ge\Lambda\), problem (1.1) admits positive solutions \(u_{i1}\), \(u_{i2}\), negative solutions \(v_{i1}\), \(v_{i2}\), and sign-changing solutions \(w_{i1}\), \(w_{i2}\) in \([t_{i+1}^{-},t_{i+1}^{+}]\setminus[t_{i}^{-},t_{i}^{+}]\), \(i=1,\ldots,n-1\).

At last, we consider the case of f having infinitely many oscillations in both \((-\infty,0)\) and \((0,+\infty)\).

\((f_{7})\) :

There exist a decreasing (an increasing) sequence \((t_{i}^{-})_{i}\) and an increasing (a decreasing) sequence \((t_{i}^{+})_{i}\) such that \(t_{i}^{-}<0<t_{i}^{+}\), and \(f(x,t_{i}^{+})<0<f(x,t_{i}^{-})\) and \(\max\{F(x,t): (x,t)\in\overline{\Omega}\times[t_{i-1}^{-},t_{i-1}^{+}]\} <\min\{F(x,t): (x,t)\in\overline{\Omega}\times\{t_{i}^{-},t_{i}^{+}\}\}\) for every \(i\in\mathbb{N}\).

Theorem 1.4

Assume that \((f_{1})\)-\((f_{2})\) and \((f_{7})\) hold. Then, for every arbitrarily chosen \(k\in\mathbb{N}\), there exists \(\Lambda_{k}>0\) such that, for every \(\lambda\ge\Lambda_{k}\), problem (1.1) admits at least 2k positive, 2k negative, and 2k sign-changing solutions.

Theorem 1.4 is immediate in view of Theorem 1.3.

Remark 1.1

Bartsch and Liu obtained Theorems 1.1-1.4 for p-Laplacian Dirichlet problems in [16]. Fu and Pucci obtained Theorems 1.1-1.4 for fractional Laplacian Dirichlet problems in [15]. Theorems 1.1-1.4 above are generalizations of the corresponding results in [15].

This article is organized as follows. In Section 2, we introduce the fractional Sobolev spaces and some preliminary results. In Section 3, in a suitably chosen framework, we verify that the conditions in the abstract critical point theorems in [16] are satisfied, then we generalize the existence of multiplicity solutions for fractional Laplacian problems to the one for fractional p-Laplacian problems.

2 Preliminaries

Let us now recall some inequalities.

Lemma 2.1

(See [19])

There exist positive constants \(C_{1}\)-\(C_{4}\) such that, for all \(\xi ,\eta\in\mathbb{R}^{N}\),

$$\begin{aligned} &\bigl\vert \vert \xi \vert ^{p-2}\xi- \vert \eta \vert ^{p-2}\eta \bigr\vert \leq C_{1}\bigl( \vert \xi \vert + \vert \eta \vert \bigr)^{p-2} \vert \xi-\eta \vert , \end{aligned}$$
(2.1)
$$\begin{aligned} &\bigl( \vert \xi \vert ^{p-2}\xi- \vert \eta \vert ^{p-2}\eta\bigr) (\xi-\eta)\geq C_{2}\bigl( \vert \xi \vert + \vert \eta \vert \bigr)^{p-2} \vert \xi-\eta \vert ^{2}, \end{aligned}$$
(2.2)
$$\begin{aligned} &\bigl\vert \vert \xi \vert ^{p-2}\xi- \vert \eta \vert ^{p-2}\eta \bigr\vert \leq C_{3} \vert \xi-\eta \vert ^{p-1} ,\quad\textit{if }1< p\leq2, \end{aligned}$$
(2.3)
$$\begin{aligned} &\bigl( \vert \xi \vert ^{p-2}\xi- \vert \eta \vert ^{p-2}\eta\bigr) (\xi-\eta)\geq C_{4} \vert \xi-\eta \vert ^{p},\quad\textit{if } p>2. \end{aligned}$$
(2.4)

The Hölder regularity up to the boundary, strong maximum principles and the Hopf lemma are important in the proof of our results.

Lemma 2.2

(See [20])

Let \(u\in X_{0}\) satisfy \(\vert \mathcal{L}^{p}_{K}u \vert \leq K\) weakly in Ω for some \(K>0\). Then

$$\begin{aligned} \bigl\vert u(x) \bigr\vert \leq(C_{\Omega}K)^{1/p-1} \delta(x)^{s}\quad\textit{a.e. in }\Omega \end{aligned}$$

for some \(C_{\Omega}=C(N,p,s,\Omega)\), \(\delta(x)=\operatorname{dist}(x,\Omega^{C})\). Furthermore, there exists \(\alpha\in(0,s]\) such that, for all weak solutions \(u\in X_{0}\) of problem (1.1), \(u\in C^{\alpha}(\overline{\Omega})\) and

$$\begin{aligned} \Vert u \Vert _{C^{\alpha}(\overline{\Omega})}\leq C_{\Omega} \Vert f \Vert _{L_{\infty}(\Omega)}^{1/p-1}. \end{aligned}$$

Lemma 2.3

(See [8])

If \(u\in X_{0}\) is such that \(u(x)\geq0\) a.e. in Ω and

$$\begin{aligned} & \iint_{\mathbb{R}^{2N}}{ \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}} {\bigl(u(x)-u(y)\bigr)} {\bigl(\varphi(x)-\varphi(y)\bigr)} {K(x-y)}\,dx\,dy\geq0 \end{aligned}$$

for each \(\varphi\in X_{0},\varphi(x)\geq0\) a.e. in Ω, then \(u(x)>0\) a.e. in Ω.

Let \(s\in(0,1)\), \(p\in(1,\infty)\). Consider the problem

$$ \textstyle\begin{cases} (-\Delta)^{s}_{p} u=c(x) \vert u \vert ^{p-2}u\quad\mbox{in }\Omega,\\ u=0\quad\mbox{in }\mathbb{R}^{N}\setminus\Omega. \end{cases} $$
(2.5)

Lemma 2.4

(see [21])

Let Ω be bounded and satisfy the interior ball condition on Ω, \(0\ge c\in C(\overline{\Omega})\), and \(u\in X_{0}\) be a weak solution of (2.5), then either \(u=0\) a.e. in \(\mathbb{R}^{N}\) or

$$\begin{aligned} \liminf_{B_{R}\ni x\rightarrow x_{0}}\frac{u(x)}{{\delta_{R}(x)}^{s}}>0, \end{aligned}$$

where \(\delta_{R}(x)=\operatorname{dist}(x,B_{R}^{C})\).

In the sequel, for any fixed parameter \(\lambda>0\), the space \(X_{0}(\Omega)\) is endowed with the norm

$$\Vert u \Vert _{\lambda}= \bigl([u]_{s,p}^{p}+ \lambda m \Vert u \Vert _{L^{p}(\Omega)}^{p} \bigr)^{{1}/{p}}, $$

and, for brevity, we put \(X_{\lambda}=(X_{0}(\Omega), \Vert \cdot \Vert _{\lambda})\). The number \(m>0\) will be determined in Section 3. \(Y=X_{\lambda}\cap C^{\alpha/2}(\overline{\Omega})\) is endowed with the \(C^{\alpha /2}\)-norm and \(Z=X_{\lambda}\cap C^{\alpha}(\overline{\Omega})\) with the \(C^{\alpha}\)-norm, where \(\alpha>0\) is from Lemma 2.2.

3 Multiplicity of weak solutions

We say that \(u\in X_{\lambda}\) is a (weak) solution of problem (1.1) if

$$\langle u,\varphi\rangle_{s,p} =\lambda \int_{\Omega}f(x,u)\varphi(x)\,dx $$

holds for any \(\varphi\in X_{\lambda}\), where

$$\langle u,\varphi\rangle_{s,p}= \iint_{\mathbb{R}^{2N}}{ \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}} {\bigl(u(x)-u(y)\bigr)} {\bigl(\varphi(x)-\varphi(y)\bigr)} {K(x-y)}\, dx\,dy. $$

Fix \(\lambda>0\) and \(m>0\). Define \(\mathcal{L}(v)={\mathcal {L}^{p}_{K}}v+\lambda m \vert v \vert ^{p-2}v\) and

$$A_{\lambda}:X_{\lambda}\rightarrow X_{\lambda},\quad A_{\lambda}(u)=v, $$

where v is the solution of the linear problem

$$\begin{aligned} \textstyle\begin{cases} {\mathcal{L}^{p}_{K}}v+\lambda m \vert v \vert ^{p-2}v=\lambda (f(x,u)+ m \vert u \vert ^{p-2}u )&\mbox{in } \Omega,\\ v=0&\mbox{in }\mathbb{R}^{N}\setminus\Omega, \end{cases}\displaystyle \end{aligned}$$
(3.1)

which is uniquely determined.

In the following we give a series of lemmas to show that the conditions in the abstract critical point theorems in [16] are satisfied. First we study the properties of the operator \(A_{\lambda}\).

Lemma 3.1

\(A_{\lambda}\in C(X_{\lambda},X_{\lambda})\) is well defined and \(A_{\lambda }(Y)\subset Z\).

Proof

First, from

$$\bigl\langle \mathcal{L}(v),v\bigr\rangle _{X_{\lambda}^{\ast},X_{\lambda}}=\bigl\langle { \mathcal{L}^{p}_{K}}v+\lambda m \vert v \vert ^{p-2}v,v\bigr\rangle _{X_{\lambda}^{\ast},X_{\lambda}}=[v]_{s,p}^{p}+ \lambda m \int_{\Omega} \vert v \vert ^{p} \,dx, $$

we know that

$$\lim_{ \Vert v \Vert _{X_{\lambda}}\rightarrow\infty} \frac{\langle\mathcal{L}(v),v\rangle_{X_{\lambda}^{\ast},X_{\lambda}}}{ \Vert v \Vert _{X_{\lambda}}}=\infty, $$

i.e., the operator \(\mathcal{L}\) is coercive. Second, from

$$ \begin{aligned}[b] &\bigl\langle \mathcal{L}(v_{1})- \mathcal{L}(v_{2}),v_{1}-v_{2}\bigr\rangle _{X_{\lambda}^{\ast},X_{\lambda}} \\ &\quad = \iint_{\mathbb{R}^{2N}}{\bigl( \bigl\vert v_{1}(x)-v_{1}(y) \bigr\vert ^{p-2}} {\bigl(v_{1}(x)-v_{1}(y) \bigr)}-{ \bigl\vert v_{2}(x)-v_{2}(y) \bigr\vert ^{p-2}} \\ &\qquad{}\times {\bigl(v_{2}(x)-v_{2}(y)\bigr)\bigr)} {\bigl( \bigl(v_{1}(x)-v_{1}(y)\bigr)-\bigl(v_{2}(x)-v_{2}(y) \bigr)\bigr)} {K(x-y)}\,dx\,dy \\ &\qquad{} +\lambda m \int_{\Omega}\bigl( \vert v_{1} \vert ^{p-2}v_{1}- \vert v_{2} \vert ^{p-2}v_{2}\bigr) (v_{1}-v_{2})\,dx \\ &\quad >0, \end{aligned} $$
(3.2)

whenever \(v_{1}\neq v_{2}\), we get that \(\mathcal{L}\) is monotone. It is easy to show that \(\mathcal{L}\) is weakly continuous. Thus, by the monotone operator theory, see [22, 23], we conclude that problem (3.1) has a unique (weak) solution in \(X_{\lambda}\). Therefore, \(A_{\lambda}\) is well defined.

Next we prove that \(A_{\lambda}\in C(X_{\lambda},X_{\lambda})\). By \((f_{1})\) and \((f_{2})\), we have \(\vert f(x,u) \vert \le C( \vert u \vert ^{p-1}+1)\) for some constant C depending only on f. Suppose that \(u_{n}\rightarrow u\) in \(X_{\lambda}\). Then \((u_{n})_{n}\) is bounded in \(X_{\lambda}\). By (3.1), Hölder’s and Young’s inequalities, we have

$$\begin{aligned}{} [v_{n}]_{s,p}^{p}+\lambda m \int_{\Omega} \vert v_{n} \vert ^{p}\,dx&= \int_{\Omega }f(x,u_{n})v_{n}\,dx+\lambda m \int_{\Omega} \vert u_{n} \vert ^{p-2}u_{n}v_{n} \, dx \\ &\le C\bigl( \Vert u_{n} \Vert _{L^{p}(\Omega)}^{p-1}+1 \bigr) \Vert v_{n} \Vert _{L^{p}(\Omega)} \\ &\le C\varepsilon \Vert v_{n} \Vert _{L^{p}(\Omega)}^{p}+C( \varepsilon ) \bigl( \Vert u_{n} \Vert _{L^{p}(\Omega)}^{p}+1 \bigr). \end{aligned}$$

Taking \(\varepsilon={\lambda m}/{2C}\), we conclude that \((v_{n})_{n}\) is bounded in \(X_{\lambda}\). Hence, \((v_{n})_{n}\) admits a weakly convergent subsequence, still denoted by \((v_{n})_{n}\). Suppose that \(v_{n}\rightharpoonup v\) weakly in \(X_{\lambda}\). By Lemma 2.5 in [7], we have \(v_{n}\rightarrow v\) in \(L^{p}(\Omega)\) since Ω is bounded. By [24], we get \(f(x,u_{n})\rightarrow f(x,u)\) in \(L^{p}(\Omega)\) by going to a further subsequence if necessary. By (3.1) and Lemma 2.1, using an argument similar to (3.2), we also have

$$\begin{aligned} & \int_{\Omega} \bigl(\lambda f(x,u)+\lambda m \vert u \vert ^{p-2}u-\lambda f(x,u_{n})-\lambda m \vert u_{n} \vert ^{p-2}u_{n} \bigr) \bigl(A_{\lambda}(u)-v_{n} \bigr)\,dx \\ &\quad =\bigl\langle \mathcal{L}^{p}_{K} A_{\lambda}(u)+ \lambda m \bigl\vert A_{\lambda }(u) \bigr\vert ^{p-2}A_{\lambda}(u)- \mathcal{L}^{p}_{K} v_{n}-\lambda m \vert v_{n} \vert ^{p-2}v_{n},A_{\lambda}(u)-v_{n} \bigr\rangle _{X_{\lambda}^{\ast },X_{\lambda}} \\ &\quad = \iint_{\mathbb{R}^{2N}}{\bigl( \bigl\vert v(x)-v(y) \bigr\vert ^{p-2}} {\bigl(v(x)-v(y)\bigr)}-{ \bigl\vert v_{n}(x)-v_{n}(y) \bigr\vert ^{p-2}} \\ &\qquad {}\times{\bigl(v_{n}(x)-v_{n}(y)\bigr)\bigr)} {\bigl( \bigl(v(x)-v(y)\bigr)-\bigl(v_{n}(x)-v_{n}(y)\bigr)\bigr)} {K(x-y)}\,dx\,dy \\ &\qquad {}+\lambda m \int_{\Omega}\bigl( \vert v \vert ^{p-2}v- \vert v_{n} \vert ^{p-2}v_{n}\bigr) (v-v_{n})\,dx \\ &\quad \geq C \iint_{\mathbb{R}^{2N}}{\bigl( \bigl\vert v(x)-v(y) \bigr\vert + \bigl\vert v_{n}(x)-v_{n}(y) \bigr\vert \bigr)^{p-2}}\\ &\qquad {}\times { \bigl\vert \bigl(v(x)-v(y)\bigr)-\bigl(v_{n}(x)-v_{n}(y) \bigr) \bigr\vert ^{2}} {K(x-y)}\,dx\,dy \\ &\qquad {}+C\lambda m \int_{\Omega}\bigl( \vert v \vert + \vert v_{n} \vert \bigr)^{p-2} \vert v-v_{n} \vert ^{2}\,dx \\ &\quad \geq C\biggl([v-v_{n}]_{s,p}^{p}+\lambda m \int_{\Omega} \vert v-v_{n} \vert ^{p}\,dx \biggr). \end{aligned}$$

Letting \(n\rightarrow\infty\), we obtain

$$v_{n}\rightarrow v=A_{\lambda}(u)\quad\mbox{in }X_{\lambda}. $$

This shows that \(A_{\lambda}\in C(X_{\lambda},X_{\lambda})\), as claimed.

If \(u\in Y\), then by \((f_{1})\) and \((f_{2})\) the solution v of (3.1) is in \(L^{\infty}(\Omega)\) thanks to Lemma 2.4. Therefore, \(\Omega\ni x\mapsto g(x)=\lambda (f(x,u(x))+ m \vert u(x) \vert ^{p-2}u(x) )\) is in \(L^{\infty}(\Omega)\) by \((f_{1})\), \((f_{2})\) and the fact that \(u\in L^{\infty}(\Omega)\). Hence, \(v\in C^{\alpha}(\overline{\Omega})\) by Lemma 2.2, since \(v\in X_{\lambda}\), and so \(v\in Z\) by the definition of Z. □

Second, we show that conditions \((J_{1})\) and \((J_{2})\) in [16] are satisfied.

Lemma 3.2

  1. (i)

    If \(1< p\leq2\), then the functional \(I_{\lambda}\) satisfies

    $$\begin{aligned} &\bigl\langle I_{\lambda}^{\prime}(u),u-v\bigr\rangle _{X_{\lambda}^{\ast}, X_{\lambda}} \ge C_{1} \Vert u-v \Vert _{X_{\lambda}}^{2}\bigl( \Vert u \Vert _{X_{\lambda}}+ \Vert v \Vert _{X_{\lambda}} \bigr)^{p-2}, \\ &\bigl\Vert I_{\lambda}^{\prime}(u) \bigr\Vert _{X_{\lambda}^{\ast}}\le C_{2} \Vert u-v \Vert _{X_{\lambda}}^{p-1}. \end{aligned}$$
  2. (ii)

    If \(p\geq2\), then the functional \(I_{\lambda}\) satisfies

    $$\begin{aligned} &\bigl\langle I_{\lambda}^{\prime}(u),u-v\bigr\rangle _{X_{\lambda}^{\ast},X_{\lambda}} \ge C_{3} \Vert u-v \Vert _{X_{\lambda}}^{p}, \\ &\bigl\Vert I_{\lambda}^{\prime}(u) \bigr\Vert _{X_{\lambda}^{\ast}}\le C_{4} \Vert u-v \Vert _{X_{\lambda}}\bigl( \Vert u \Vert _{X_{\lambda}}+ \Vert v \Vert _{X_{\lambda}}\bigr)^{p-2}. \end{aligned}$$

Proof

Let \(u\in X_{\lambda}\). Thus \(v=A_{\lambda}(u)\) implies that

$$\begin{aligned} &\bigl\langle I_{\lambda}^{\prime}(u),u-v\bigr\rangle _{X_{\lambda}^{\ast},X_{\lambda}} \\ &\quad = \iint_{\mathbb{R}^{2N}}{ \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}} {\bigl(u(x)-u(y)\bigr)} {\bigl(\bigl(u(x)-u(y)\bigr)-\bigl(v(x)-v(y)\bigr)\bigr)} {K(x-y)}\,dx\,dy \\ &\qquad {}-\lambda \int_{\Omega}f(x,u) (u-v)\,dx \\ &\quad = \iint_{\mathbb{R}^{2N}}{\bigl( \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}} {\bigl(u(x)-u(y)\bigr)}-{ \bigl\vert v(x)-v(y) \bigr\vert ^{p-2}} {\bigl(v(x)-v(y)\bigr)\bigr)} \\ &\qquad {}\times{\bigl(\bigl(u(x)-u(y)\bigr)-\bigl(v(x)-v(y)\bigr)\bigr)} {K(x-y)}\,dx\,dy\\ &\qquad {}+ \lambda m \int_{\Omega }\bigl( \vert u \vert ^{p-2}u- \vert v \vert ^{p-2}v\bigr) (u-v)\,dx. \end{aligned}$$

If \(p\geq2\), it follows by Lemma 2.1 that

$$\bigl\langle I_{\lambda}^{\prime}(u),u-v\bigr\rangle _{X_{\lambda}^{\ast},X_{\lambda}} \ge C_{3} \Vert u-v \Vert _{X_{\lambda}}^{p}. $$

If \(1< p\leq2\), then from Lemma 2.1 we have

$$\begin{aligned} &\bigl\langle I_{\lambda}^{\prime}(u),u-v\bigr\rangle _{X_{\lambda}^{\ast},X_{\lambda}} \\ &\quad \geq C_{2} \iint_{\mathbb{R}^{2N}}{\bigl( \bigl\vert u(x)-u(y) \bigr\vert + \bigl\vert v(x)-v(y) \bigr\vert \bigr)^{p-2}} { \bigl\vert \bigl(u(x)-u(y)\bigr)-\bigl(v(x)-v(y)\bigr) \bigr\vert ^{2}}\\ &\qquad {}\times {K(x-y)} \,dx\,dy +C_{2}\lambda m \int_{\Omega}\bigl( \bigl\vert u(x) \bigr\vert + \bigl\vert v(x) \bigr\vert \bigr)^{p-2} \bigl\vert u(x)-v(x) \bigr\vert ^{2}\,dx. \end{aligned}$$

By Hölder’s inequality

$$\Vert u-v \Vert _{X_{\lambda}}^{p}\leq C(C_{2},p)\bigl\langle I_{\lambda}^{\prime }(u),u-v\bigr\rangle _{X_{\lambda}^{\ast},X_{\lambda}}^{\frac{p}{2}} \bigl( \Vert u \Vert _{X_{\lambda}}+ \Vert v \Vert _{X_{\lambda}} \bigr)^{p(1-\frac{p}{2})}, $$

and therefore

$$\bigl\langle I_{\lambda}^{\prime}(u),u-v\bigr\rangle _{X_{\lambda}^{\ast}, X_{\lambda}} \ge C_{1} \Vert u-v \Vert _{X_{\lambda}}^{2}\bigl( \Vert u \Vert _{X_{\lambda}}+ \Vert v \Vert _{X_{\lambda}} \bigr)^{p-2}. $$

For \(w\in X_{\lambda}\), we have

$$\begin{aligned} \bigl\langle I_{\lambda}^{\prime}(u),w\bigr\rangle _{X_{\lambda}^{\ast},X_{\lambda}} ={}& \iint_{\mathbb{R}^{2N}}{ \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}} {\bigl(u(x)-u(y)\bigr)} {\bigl(w(x)-w(y)\bigr)} {K(x-y)}\,dx\,dy \\ &{}-\lambda \int_{\Omega}f(x,u)w\,dx \\ ={}& \iint_{\mathbb{R}^{2N}}{\bigl( \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}} {\bigl(u(x)-u(y)\bigr)}-{ \bigl\vert v(x)-v(y) \bigr\vert ^{p-2}} {\bigl(v(x)-v(y)\bigr)\bigr)} \\ &{}\times{\bigl(w(x)-w(y)\bigr)} {K(x-y)}\,dx\,dy+\lambda m \int_{\Omega}\bigl( \vert u \vert ^{p-2}u- \vert v \vert ^{p-2}v\bigr)w\,dx. \end{aligned}$$

If \(1< p\leq2\), then by Lemma 2.1

$$\bigl\Vert I_{\lambda}^{\prime}(u) \bigr\Vert _{X_{\lambda}^{\ast}}\le C_{2} \Vert u-v \Vert _{X_{\lambda}}^{p-1}. $$

If \(p\geq2\), then by Lemma 2.1 and Hölder’s inequality, we have

$$\bigl\Vert I_{\lambda}^{\prime}(u) \bigr\Vert _{X_{\lambda}^{\ast}}\le C_{4} \Vert u-v \Vert _{X_{\lambda}}\bigl( \Vert u \Vert _{X_{\lambda}}+ \Vert v \Vert _{X_{\lambda}}\bigr)^{p-2}. $$

Now we conclude the result. □

Third, we establish the regularity of the critical points of \(I_{\lambda}\).

Lemma 3.3

Let \(K=\{u\in X_{0}(\Omega): I_{\lambda}^{\prime}(u)=0\}\). Then \(K\subset Y\).

Proof

Let u be fixed in K. Then \(u\in L^{\infty}(\Omega)\) by \((f_{1})\), \((f_{2})\) and Lemma 2.2. Hence, \(\Omega\ni x\mapsto\lambda f(x,u(x))\) is in \(L^{\infty}(\Omega)\) again by \((f_{1})\), \((f_{2})\) and the fact that u is in \(L^{\infty}(\Omega )\). Therefore, Lemma 2.2 gives \(u\in C^{\alpha}(\overline{\Omega})\) since \(u\in X_{\lambda}\), and so \(u\in Y\). □

Forth, we get a comparison principle for the operator \(\mathcal{L}\).

Lemma 3.4

If \(u,v\in X_{\lambda}\), with \(v\ge u\) in \(\mathbb{R}^{N}\setminus\Omega\), are such that \(\mathcal{L}v\ge\mathcal {L}u\) in Ω, i.e., \(\langle\mathcal{L}v-\mathcal{L}u,w\rangle _{X_{\lambda}^{\ast},X_{\lambda}}\ge0\) for all \(w\in X_{\lambda}\), with \(w\ge0\) in Ω, then \(v\ge u\) in Ω.

Proof

Let \(u,v\in X_{\lambda}\), with \(v\ge u\) in \(\mathbb {R}^{N}\setminus\Omega\), be such that \(\mathcal{L}v\ge\mathcal{L}u\) in Ω. Take \(\varphi =u-v=[u-v]^{+}-[u-v]^{-}\), \(w=[u-v]^{+}=[\varphi]^{+}\). Clearly, \(w\in X_{\lambda}\) and \(w\ge0\) in Ω. As

$$\vert b \vert ^{p-2}b- \vert a \vert ^{p-2}a=(p-1) (b-a) \int_{0}^{1} \bigl\vert a+t(b-a) \bigr\vert ^{p-2}\,dt, $$

we have

$$\begin{aligned} &\langle\mathcal{L}v-\mathcal{L}u,w\rangle_{X_{\lambda}^{\ast},X_{\lambda}} \\ &\quad = \iint_{\mathbb {R}^{2N}}(p-1) \bigl(\bigl(u(y)-v(y)\bigr)-\bigl(u(x)-v(x) \bigr)\bigr)Q(x,y) \bigl(w(x)-w(y)\bigr){K(x-y)}\,dx\, dy \\ &\qquad {}+\lambda m \int_{\Omega}(p-1) (v-u)R(x)w\,dx \\ &\quad = \iint_{\mathbb{R}^{2N}}(p-1)Q(x,y)\bigl[-\bigl(\varphi^{+}(x)-\varphi ^{+}(y) \bigr)^{2}-\varphi^{-}(x)\varphi^{+}(y)-\varphi^{-}(y)\varphi^{+}(x) \bigr]{K(x-y)}\, dx\,dy \\ &\qquad {}-\lambda m \int_{\Omega}(p-1)R(x) \bigl(\varphi^{+}\bigr)^{2}\,dx \\ &\quad \geq0, \end{aligned}$$

where

$$\begin{aligned} &Q(x,y)= \int_{0}^{1} \bigl\vert \bigl(u(x)-u(y)\bigr)+t \bigl(\bigl(v(x)-v(y)\bigr)-\bigl(u(x)-u(y)\bigr)\bigr) \bigr\vert ^{p-2}\,dt, \\ &R(x)= \int_{0}^{1} \bigl\vert u+t(v-u) \bigr\vert ^{p-2}\,dt. \end{aligned}$$

We see that \(Q(x,y)\geq0\) and \(Q(x,y)=0\) only if \(v(y)=v(x)\) and \(u(y)=u(x)\), and \(R(x)\geq0\). Thus \(\varphi^{+}=0\) in Ω, i.e., \(v\geq u\) in Ω. □

For \(u,v\in Y\), define \(u\ll v\) provided \(u(x)< v(x)\) for any \(x\in \Omega\) and

$$\liminf_{x=x_{0}-d\nu\in\Omega,d\rightarrow0}\frac {u(x)-v(x)}{d^{\alpha /2}}< 0 $$

for any \(x_{0}\in\partial\Omega\), where ν is the unit outward normal vector of Ω at \(x_{0}\). If \(u\le v\) in Ω, then \([u,v]\) denotes the order interval \(\{w\in Y:u\le w\le v\}\).

By \((f_{1})\)-\((f_{3})\), we can choose suitable positive m so that

$$\begin{aligned} &tf(x,t)+m \vert t \vert ^{p}>0\quad\mbox{for } t\neq0, \\ &f(x,t)+m \vert t \vert ^{p-1}t \textstyle\begin{cases} >m \vert t^{-} \vert ^{p-1}t^{-}&\mbox{for } t\ge t^{-},\\ < m \vert t^{+} \vert ^{p-1}t^{+}&\mbox{for } t\le t^{+}. \end{cases}\displaystyle \end{aligned}$$
(3.3)

Fix such an m from now on.

Fifth, we give the following three technical lemmas which are necessary to verify that the conditions in the abstract critical point theorems in [16] (especially \((P_{1})\)-\((P_{3})\) there) are satisfied.

Lemma 3.5

Assume that \((f_{3})\) and \((f_{4})\) also hold. Then there exist ϕ, \(\psi\in Y\) having the following properties.

  1. (1)

    \(t^{-}\le\phi\ll0\ll\psi\le t^{+}\).

  2. (2)

    For every \(u\in X_{\lambda}\cap L^{\infty}(\Omega)\) with \(\phi\le u\), \(\phi\ll A_{\lambda}(u)\).

  3. (3)

    For every \(u\in X_{\lambda}\cap L^{\infty}(\Omega)\) with \(u\le\psi\), \(A_{\lambda}(u)\ll\psi\).

  4. (4)

    For every \(u\in X_{\lambda}\cap L^{\infty}(\Omega)\) with \(t^{-}\le u\), \(\phi\ll A_{\lambda}(u)\).

  5. (5)

    For every \(u\in X_{\lambda}\cap L^{\infty}(\Omega)\) with \(u\le t^{+}\), \(A_{\lambda}(u)\ll\psi\).

  6. (6)

    \(\max\{F(x,t): (x,t)\in\overline{\Omega}\times [- \Vert \phi \Vert _{L^{\infty}(\Omega)}, \Vert \psi \Vert _{L^{\infty}(\Omega)}]\}<\min\{F(x,t): (x,t)\in\overline{\Omega}\times\{s^{-},s^{+}\}\}\).

Proof

Let ψ be the solution of

$$\begin{aligned} &\mathcal{L}^{p}_{K} u+\lambda m \vert u \vert ^{p-2}u=\lambda m \bigl\vert t^{+} \bigr\vert ^{p-2}t^{+}\quad \text{in }\Omega, \\ &u=0\quad \text{in }\mathbb{R}^{N}\setminus\Omega. \end{aligned}$$

Thus \(0\ll\psi\le t^{+}\) again by Lemma 3.4 and the strong maximum principle Lemma 2.3 since \(\mathcal{L}^{p}_{K}(\psi)\ge0\). By Lemmas 3.4 and 2.4, we also have \(\liminf_{B_{R}\ni x\rightarrow x_{0}}\frac{\psi(x)}{{\delta_{R}(x)}^{s}}>0\). Let ϕ be the solution of

$$\begin{aligned} &\mathcal{L}^{p}_{K}u+\lambda m \vert u \vert ^{p-2}u=\lambda m \bigl\vert t^{-} \bigr\vert ^{p-2}t^{-} \quad \text{in }\Omega, \\ &u=0\quad \text{in }\mathbb{R}^{N}\setminus\Omega. \end{aligned}$$

Therefore \(t^{-}\le\phi\ll0\) by Lemma 3.4 and by the strong maximum principle Lemma 2.3 since \(\mathcal{L}^{p}_{K}(-\phi)\ge0\). By Lemmas 3.4 and 2.4, we also have \(\liminf_{B_{R}\ni x\rightarrow x_{0}}\frac{-\varphi(x)}{{\delta_{R}(x)}^{s}}>0\).

Next we prove the results on \(A_{\lambda}\). By (3.3), we can choose \(\delta\in(0,1)\) small enough such that

$$ f(x,t)+m \vert t \vert ^{p-1}t \textstyle\begin{cases} >\delta m \vert t^{-} \vert ^{p-1}t^{-}&\mbox{for } t\ge t^{-},\\ < \delta m \vert t^{+} \vert ^{p-1}t^{+}&\mbox{for } t\le t^{+}. \end{cases} $$

Let \(u\in X_{\lambda}\cap L^{\infty}(\Omega)\) such that \(t^{-}\le u\). Denote \(v=A_{\lambda}(u)\). Then

$$\begin{aligned} \mathcal{L}^{p}_{K} v+\lambda m \vert v \vert ^{p-2}v=\lambda f(x,u)+\lambda m \vert u \vert ^{p-2}u\ge \delta\lambda m \bigl\vert t^{-} \bigr\vert ^{p-2}t^{-} =\delta \bigl[ \mathcal{L}^{p}_{K} \phi+\lambda m \vert \phi \vert ^{p-2}\phi \bigr]. \end{aligned}$$

Hence, \(v\ge\delta^{\frac{1}{p-1}}\phi\gg\phi\) by Lemma 3.4.

Similarly, if \(u\in X_{\lambda}\cap L^{\infty}(\Omega)\) with \(u\le t^{+}\), then \(v\le\delta^{\frac{1}{p-1}}\psi\ll\psi\). This completes the proof of (1)-(5).

Conclusion (6) follows directly from \((f_{2})\) and properties (1)-(3). □

Lemma 3.6

Suppose that \((f_{3})\) and \((f_{4})\) also hold. Then there exists \(\Lambda>0\) such that, for every \(\lambda\ge\Lambda \), there exists \(h\in C([0,1],{\tilde{Y}})\) satisfying \(h(1)\le0\le h(0)\) and

$$ \max_{0\le t\le1}J_{\lambda}\bigl(h(t)\bigr)< \inf _{u\in[\phi,\psi]}J_{\lambda}(u), $$
(3.4)

where \({\tilde{Y}}=X\cap C^{1}(\overline{\Omega})\), ϕ and ψ are the ones in Lemma  3.5. Furthermore, if \(f(x,t)\) is odd with respect to t, then, for every \(k\in\mathbb{N}\), there exists \(\Lambda_{k}>0\) such that, for every \(\lambda\ge\Lambda_{k}\), there exists an odd map \(h_{k}\in C(S^{k},{\tilde{Y}})\) satisfying

$$ \max_{\theta\in S^{k}}J_{\lambda}\bigl(h_{k}(\theta)\bigr)< \inf_{u\in[\phi,\psi ]}J_{\lambda}(u). $$
(3.5)

The proof is a minor modification of the corresponding argument given in order to prove Lemma 3.2 of [16].

Lemma 3.7

For any bounded set \(B\subset Y\) and any order interval \([\phi,\psi]\), there exist \(\phi_{1}\), \(\psi_{1}\in Y\) such that

$$ B\cup[\phi,\psi]\subset\mathop{\operatorname{int}}\limits_{Y}[\phi_{1}, \psi_{1}], $$
(3.6)

and

$$ A_{\lambda}(u)\in[\phi_{1},\psi_{1}]\quad\textit{for every }u\in X \textit{ satisfying } \phi_{1}\le u\le\psi_{1}. $$
(3.7)

Proof

By \((f_{2})\) there exist ε, \(C>0\), which depend on λ, such that

$$\lambda \bigl\vert f(x,t) \bigr\vert \le(\lambda_{1}-2\varepsilon) \vert t \vert ^{p-1}+C\quad\mbox{for every }x\in\Omega, t\in \mathbb{R}, $$

where \(\lambda_{1}\) is the first eigenvalue in Ω of \(\mathcal {L}^{p}_{K}\) with zero Dirichlet boundary condition. By the monotone operator theory, see [22, 23], there exists a unique solution \(\psi_{1}\) to

$$\begin{aligned} &\mathcal{L}^{p}_{K} u-(\lambda_{1}-\varepsilon) \vert u \vert ^{p-2}u=\frac {C}{\mu}\quad\mbox{in }\Omega, \\ &u=0\quad\mbox{in }\mathbb{R}^{N}\setminus\Omega, \end{aligned}$$

where

$$\mu=\frac{\lambda_{1}-2\varepsilon+\lambda m}{\lambda_{1}-\varepsilon +\lambda m}< 1. $$

Let \(\phi_{1}=-\psi_{1}\). Then we get \(\phi_{1}\ll0\ll\psi_{1}\) by Lemma 3.4 and the strong maximum principle lemma. Provided that \(C>0\) is large enough, (3.6) follows.

Next, fix \(u\in X_{\lambda}\) which satisfies \(\phi_{1}\le u\le\psi_{1}\). Denote \(v=A_{\lambda}(u)\). We have

$$\begin{aligned} \mathcal{L}^{p}_{K} v+\lambda m \vert v \vert ^{p-2}v&=\lambda \bigl[ f(x,u)+m \vert u \vert ^{p-2}u \bigr] \\ &\le(\lambda_{1}-2\varepsilon) \vert \psi_{1} \vert ^{p-2}\psi_{1}+C+\lambda m \vert \psi_{1} \vert ^{p-2}\psi_{1} \\ &=\mu \bigl[(\lambda_{1}-\varepsilon) \vert \psi_{1} \vert ^{p-2}\psi _{1}+\lambda m \vert \psi_{1} \vert ^{p-2}\psi_{1}+{C}/{\mu} \bigr] \\ &=\mu \bigl[\mathcal{L}^{p}_{K} \psi_{1}+\lambda m \vert \psi_{1} \vert ^{p-2}\psi _{1} \bigr]. \end{aligned}$$

Hence, \(v\le\mu^{\frac{1}{p-1}}\psi_{1}\ll\psi_{1}\) by Lemma 3.4. Similarly, \(v\gg\phi_{1}\). This completes the proof of (3.7). □

At last, we show that \(I_{\lambda}\) satisfies the Palais-Smale condition which is crucial to guaranteeing the existence of critical points.

Lemma 3.8

Let \((u_{n})_{n}\subset X_{\lambda}\) be a Palais-Smale sequence of \(I_{\lambda}\), i.e., \((I_{\lambda}(u_{n}))_{n}\) is bounded and \(I_{\lambda}^{\prime}(u_{n})\rightarrow0\) as \(n\rightarrow\infty\). Then \((u_{n})_{n}\) admits a strongly convergent subsequence in \(X_{\lambda}\).

Proof

By \((f_{1})\)-\((f_{2})\), for \(\varepsilon>0\) small enough,

$$ \bigl\vert f(x,t) \bigr\vert \le C(\varepsilon)+\frac{1}{p} \varepsilon \vert t \vert ^{p-1},\qquad \bigl\vert F(x,t) \bigr\vert \le C(\varepsilon)+\varepsilon \vert t \vert ^{p}. $$
(3.8)

By Lemma 2.3 in [7],

$$ \Vert u \Vert _{L^{p}(\Omega)}\leq C_{1} [u]_{s,p}, $$
(3.9)

since Ω is bounded.

Let \((u_{n})_{n}\) be a Palais-Smale sequence of \(I_{\lambda}\) in \(X_{\lambda}\). Then there exists \(C>0\) such that, for all n,

$$\begin{aligned} C\ge I_{\lambda}(u_{n}) \ge\frac{1}{p}[u_{n}]_{s,p}^{p}- \lambda\varepsilon \Vert u_{n} \Vert _{L^{p}(\Omega)}^{2}-C( \varepsilon) \vert \Omega \vert \ge\frac{1}{2p}[u_{n}]_{s,p}^{p}-C_{2}, \end{aligned}$$

where \(\varepsilon=1/{2pC_{1}^{2}\lambda}\) in (3.8). Thus \((u_{n})_{n}\) is bounded in \(X_{\lambda}\) by (3.9). So, up to a subsequence, still denoted by \((u_{n})_{n}\), we have \(u_{n}\rightharpoonup u\) weakly in \(X_{\lambda}\). Then \(\langle I'(u_{n}),u_{n}-u\rangle\rightarrow0\), and further we obtain

$$ \begin{aligned}[b] &\bigl\langle I'(u_{n}),u_{n}-u \bigr\rangle \\ &\quad= \iint_{\mathbb{R}^{2N}}{ \bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{p-2}} {\bigl(u_{n}(x)-u_{n}(y) \bigr)} {\bigl[\bigl(u_{n}(x)-u(x)\bigr)-\bigl(u_{n}(y)-u(y) \bigr)\bigr]} \\ &\qquad{}\times {K(x-y)}\,dx\,dy-\lambda \int_{\Omega}f(x,u_{n}) (u_{n}-u)\,dx \\ &\quad \rightarrow0 \end{aligned} $$
(3.10)

as \(n\rightarrow\infty\). Moreover, by Lemma 2.5 in [7], up to a subsequence, \(u_{n}\rightarrow u\) strongly in \(L^{p}(\Omega)\) and a.e. in Ω. Thus, \(f(x,u_{n})(u_{n}-u)\rightarrow0\) a.e. in Ω as \(n\rightarrow\infty\). It is easy to show that the sequence \((f(x,u_{n})(u_{n}-u))_{n}\) is uniformly bounded and equi-integrable in \(L^{1}(\Omega)\). Hence, by the Vitali convergence theorem (see Rudin [25]), we get

$$\begin{aligned} \lim_{n\rightarrow\infty} \int_{\Omega}f(x,u_{n}) (u_{n}-u)\,dx=0. \end{aligned}$$

Therefore, by (3.10), we have

$$\begin{aligned} & \iint_{\mathbb{R}^{2N}}{ \bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{p-2}} {\bigl(u_{n}(x)-u_{n}(y) \bigr)} {\bigl[\bigl(u_{n}(x)-u(x)\bigr)-\bigl(u_{n}(y)-u(y) \bigr)\bigr]} {K(x-y)}\,dx\, dy \\ &\quad \rightarrow0 \end{aligned}$$

as \(n\rightarrow\infty\). Thus, by the weak convergence of \((u_{n})_{n}\) in \(X_{\lambda}\), we get

$$\begin{aligned} & \iint_{\mathbb{R}^{2N}}{\bigl[ \bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{p-2}} {\bigl(u_{n}(x)-u_{n}(y) \bigr)}- \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}\bigl(u(x)-u(y)\bigr) \bigr] \\ &\qquad {}\times{\bigl[\bigl(u_{n}(x)-u(x)\bigr)-\bigl(u_{n}(y)-u(y) \bigr)\bigr]} {K(x-y)}\,dx\,dy \\ &\quad \rightarrow0 \end{aligned}$$

as \(n\rightarrow\infty\). By Lemma 2.1, we obtain, for \(p>2\),

$$ \begin{aligned}[b] & \iint_{\mathbb{R}^{2N}}{ \bigl\vert \bigl(u_{n}(x)-u(x)\bigr)- \bigl(u_{n}(y)-u(y)\bigr) \bigr\vert }^{p}{K(x-y)}\,dx\,dy \\ &\quad \leq C_{1} \iint_{\mathbb{R}^{2N}}{\bigl[ \bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{p-2}} {\bigl(u_{n}(x)-u_{n}(y) \bigr)}- \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}\bigl(u(x)-u(y)\bigr) \bigr] \\ &\qquad {}\times{\bigl[\bigl(u_{n}(x)-u(x)\bigr)-\bigl(u_{n}(y)-u(y) \bigr)\bigr]} {K(x-y)}\,dx\,dy \\ &\quad \rightarrow0 \end{aligned} $$
(3.11)

as \(n\rightarrow\infty\). For \(1< p<2\), we have

$$ \begin{aligned}[b] & \iint_{\mathbb{R}^{2N}}{ \bigl\vert \bigl(u_{n}(x)-u(x)\bigr)- \bigl(u_{n}(y)-u(y)\bigr) \bigr\vert }^{p}{K(x-y)}\,dx\,dy \\ &\quad \leq C_{2}\biggl[ \iint_{\mathbb{R}^{2N}}{\bigl[ \bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{p-2}} {\bigl(u_{n}(x)-u_{n}(y) \bigr)}- \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}\bigl(u(x)-u(y)\bigr) \bigr] \\ &\qquad {}\times{\bigl[\bigl(u_{n}(x)-u(x)\bigr)-\bigl(u_{n}(y)-u(y) \bigr)\bigr]} {K(x-y)}\,dx\,dy\biggr]^{p/2} \\ &\qquad {}\times\biggl[ \iint_{\mathbb{R}^{2N}}{ \bigl\vert (u_{n}(x)-u_{n}(y) \bigr\vert - \bigl\vert u(x)-u(y) \bigr\vert }^{p}{K(x-y)}\,dx \,dy\biggr]^{2-p/2} \\ &\quad \leq C_{3}\biggl[ \iint_{\mathbb{R}^{2N}}{\bigl[ \bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{p-2}} {\bigl(u_{n}(x)-u_{n}(y) \bigr)}- \bigl\vert u(x)-u(y) \bigr\vert ^{p-2}\bigl(u(x)-u(y)\bigr) \bigr]\hspace{-10pt} \\ &\qquad {}\times{\bigl[\bigl(u_{n}(x)-u(x)\bigr)-\bigl(u_{n}(y)-u(y) \bigr)\bigr]} {K(x-y)}\,dx\,dy\biggr]^{p/2} \\ &\quad \rightarrow0 \end{aligned} $$
(3.12)

as \(n\rightarrow\infty\). Combining (3.11) and (3.12), we get that \(u_{n}\rightarrow u\) strongly in \(X_{\lambda}\) as \(n\rightarrow\infty\). Therefore, \(I_{\lambda}\) satisfies the (PS) condition. □

Taking inspiration from [16], we apply Lemmas 3.5-3.8 in order to prove Theorems 1.1-1.3. Let \(\mathrm{int}_{D_{1}}(D_{2})\) refer to the Y-topology on \(D_{1}\).

Proof of Theorem 1.1

Let ϕ, ψ and Λ be the ones in Lemma 3.5 and Lemma 3.6. Fix \(\lambda\ge \Lambda\). Let h be the one in Lemma 3.6. Choose \(B=h([0,1])\) so that \(B\subset\tilde{Y}\subset Y\) by Lemma 3.6. Let the corresponding \(\phi_{1}\) and \(\psi_{1}\) be the ones stated in Lemma 3.7.

Define \(D_{1}^{+}=[0,\psi_{1}]\) and \(D_{2}^{+}=[0,\psi]\). Then

$$\mathrm{int}_{D_{1}^{+}}{D_{2}^{+}}=\{u\in Y:0\le u\ll\psi\}, $$

where \(\mathrm{int}_{D_{1}^{+}}(D_{2}^{+})\) refers to the Y-topology on \(D_{1}^{+}\). By Lemmas 3.5 and 3.7 and Lemma 2.4, conditions \((P_{1})\)-\((P_{3})\) in Section 2 are satisfied for \(D_{2}^{+}\subset D_{1}^{+}\) and \(A=A_{\lambda}\). By Lemmas 3.6-3.8 and Theorem 2.1 in [17], there are two positive and two negative solutions of problem (1.1).

Choose \(D_{1}=[\phi_{1},\psi_{1}]\), \(D_{2}=[\phi,\psi_{1}]\), \(D_{3}=[\phi_{1},\psi]\), and \(D_{4}=[\phi,\psi]\). By Lemmas 3.5-3.8, \(D_{i}\), \(i=1,\ldots,4\), and \(A=A_{\lambda}\) satisfy all the assumptions of Theorem 2.2 in [17]. So there are two sign-changing solutions of problem (1.1).

At last, in view of (3) of Lemma 3.5, we complete the proof. □

Proof of Theorem 1.2

Let \(D_{i}\), \(i=1,\ldots,4\), be the ones in the proof of Theorem 1.1. First, in view of Lemmas 3.5-3.8, all the assumptions of Theorem 2.3 in [17] are satisfied. Second, by (2) and (3) of Lemma 3.5,

$$ \inf_{u\in[t^{-},t^{+}]}I_{\lambda}(u)\le\inf_{u\in D_{4}}I_{\lambda}(u) $$
(3.13)

and

$$ A_{\lambda}\bigl(\bigl[t^{-},t^{+}\bigr]\bigr)\subset\mathrm{int}_{Y}(D_{4}). $$
(3.14)

Hence, (3.13)-(3.14) and Proposition 2.4 of [16] yield the result. □

Proof of Theorem 1.3

Define \(f_{i}(x,t)\), \(i=1,\ldots,n-1\), as in [16]. Consider

$$\begin{aligned} \textstyle\begin{cases} \mathcal{L}^{p}_{k} u=\lambda f_{i}(x,u)&\mbox{in }\Omega,\\ u(x)=0,&\mbox{in }\mathbb{R}^{N}\setminus\Omega. \end{cases}\displaystyle \end{aligned}$$
(3.15)

By Theorem 1.1, problem (3.15) admits two positive, two negative and two sign-changing solutions. Replacing \(t^{-}\), \(t^{+}\) by \(t^{-}_{i}\), \(t^{+}_{i}\), respectively, the six solutions are outside of the order interval \([t^{-}_{i},t^{+}_{i}]\). According to the choice of m, we have

$$\bigl\vert m t^{-}_{i+1} \bigr\vert ^{p-2}m t^{-}_{i+1}< f(x,t)+mt=f_{i}(x,t)+m \vert t \vert ^{p-2}t< m \bigl\vert t^{+}_{i+1} \bigr\vert ^{p-2}t^{+}_{i+1}\quad\mbox{for } t^{-}_{i+1}\le t\le t^{+}_{i+1}. $$

Thus, by the definition of \(f_{i}\), we get

$$\begin{aligned} \begin{aligned} &\mathcal{L}^{p}_{K} u+\lambda m \vert u \vert ^{p-2}>\lambda m \bigl\vert t^{-}_{i+1} \bigr\vert ^{p-2}t^{-}_{i+1}=\mathcal{L}^{p}_{K}t^{-}_{i+1}+ \lambda m \bigl\vert t^{-}_{i+1} \bigr\vert ^{p-2}t^{-}_{i+1} &&\mbox{in }\Omega, \\ &u=0>t^{-}_{i+1}&&\mbox{in }\mathbb{R}^{N}\setminus\Omega. \end{aligned} \end{aligned}$$

Furthermore, by Lemma 3.4 the six solutions of problem (3.15) are inside the order interval \([t^{-}_{i+1},t^{+}_{i+1}]\) and are obviously solutions of problem (1.1). In this way, we manage to get \(2(n-1)\) positive, \(2(n-1)\) negative and \(2(n-1)\) sign-changing solutions of problem (1.1). □

4 Conclusions

The purpose of this paper is to study the existence of multiplicity solutions for fractional p-Laplacian equations with sublinear growth and oscillatory behavior. The key point is the choice of the framework to study the existence of weak solutions. In the suitably chosen framework, we are able to fulfil our strategy and generalize the corresponding results for fractional Laplacian equations.