1 Introduction

In this paper, we focus on the existence of nonnegative weak solutions for the following fractional problem:

$$\begin{aligned} \left\{ \begin{array}{ll} {{(-\Delta )_{p}^ {s_1}u + (-\Delta )_{q}^ {s_2}u\in [\underline{f}(u), \overline{f}(u)] + |u|^{q_{s_{2}}^{*} -2}u}} \, &{} \text{ in } \Omega , \\ u=0 \quad &{} \text{ in } \mathbb {R}^N {\setminus }\Omega , \end{array} \right. \end{aligned}$$
(1.1)

where \(\Omega \subset \mathbb {R}^N\) is a smooth bounded domain, \(0<s_{1}<s_{2}<1, 1<p<q <\frac{N}{s_{2}}\), \(q^*_{s_{2}}:=\frac{Nq}{N-s_{2}q}\) is the fractional critical exponent, \(f\in L^{\infty }_\textrm{loc}(\mathbb {R})\) and

$$\begin{aligned} \underline{f}(t):= \lim _{\varepsilon \rightarrow 0 } \text{ ess } \inf _{|t-\tau | <\varepsilon } f(\tau ) \end{aligned}$$

and

$$\begin{aligned} \overline{f}(t):= \lim _{\varepsilon \rightarrow 0 } \text{ ess } \sup _{|t-\tau | <\varepsilon } f(\tau ). \end{aligned}$$

For \(\alpha \in (0, 1)\) and \(t\in (1, \infty )\), the fractional \((\alpha , t)\)-Laplacian operator \((-\Delta )^{\alpha }_{t}\) is defined up to a normalizing positive constant by setting

$$\begin{aligned} (-\Delta )^{\alpha }_{t}u(x):=2 \, P.V. \int _{\mathbb {R}^{N}} \frac{|u(x)-u(y)|^{t-2}}{|x-y|^{N+\alpha t}}(u(x)-u(y))\textrm{d}y, \end{aligned}$$

for all \(u:\mathbb {R}^{N}\rightarrow \mathbb {R}\) smooth enough. We stress that fractional and nonlocal operators are currently studied in the literature due to their importance in the description of several physical phenomena; see [5, 22] for more details. When the nonlinearity \(f:\mathbb {R}\rightarrow \mathbb {R}\) is continuous, (1.1) falls within the realm of the fractional (p, q)-Laplacian problems of the type

$$\begin{aligned} \left\{ \begin{array}{ll} (-\Delta )_{p}^ {s_1}u + (-\Delta )_{q}^ {s_2}u= g(x, u) \, &{} \text{ in } \Omega , \\ u=0 \quad &{} \text{ in } \mathbb {R}^N {\setminus }\Omega , \end{array} \right. \end{aligned}$$
(1.2)

where g(x, t) is a Carathéodory function in \(\Omega \times \mathbb {R}\) with subcritical or critical growth as \(|t|\rightarrow \infty \). For problems like (1.2), several existence and multiplicity results appeared in the recent literature; see [8, 14, 27] and also [4, 6, 9, 30] for problems in \(\mathbb {R}^N\). We notice that the fractional operator \((-\Delta )_{p}^ {s_1}+ (-\Delta )_{q}^ {s_2}\) in (1.1) is nonhomogeneous in the sense that does not exist any \(\sigma \in \mathbb {R}\) such that

$$\begin{aligned} {[}(-\Delta )_{p}^ {s_1}+ (-\Delta )_{q}^ {s_2}](tu)=t^{\sigma } [(-\Delta )_{p}^ {s_1}+ (-\Delta )_{q}^ {s_2}](u) \quad \text{ for } \text{ all } t>0. \end{aligned}$$

The fractional (p, q)-Laplacian operator can be considered as the fractional counterpart of the (p, q)-Laplacian operator \(-\Delta _{p}-\Delta _{q}\), which appears in the study of reaction-diffusion problems arising in biophysics, plasma physics, and chemical reaction design; see [20]. More precisely, the prototype for these problems can be written in the form

$$\begin{aligned} u_{t}={{\,\textrm{div}\,}}[D(u)\nabla u]+c(x,u), \quad D(u):=|\nabla u|^{p-2}+|\nabla u|^{q-2}. \end{aligned}$$
(1.3)

In this context, the function u in (1.3) denotes a concentration, \({{\,\textrm{div}\,}}[D(u) \nabla u]\) represents the diffusion with a diffusion coefficient D(u), and c(x, u) corresponds to the reaction term related to source and loss processes. Some interesting existence and multiplicity results for (p, q)-Laplacian problems can be found in [10, 12, 24, 29, 35, 38] and the references therein. On the other hand, the functional associated to the (p, q)-Laplacian operator is a particular case of the following double-phase functional

$$\begin{aligned} {\mathcal {F}}_{p, q}(u; \Omega ):=\int _{\Omega } (|\nabla u|^{p}+a(x)|\nabla u|^{q})\, \textrm{d}x, \end{aligned}$$

where \(0\le a(x)\in L^{\infty }(\Omega )\), which was introduced by Zhikov [39, 40] to describe the behavior of strongly anisotropic materials in the context of homogenization phenomena. We also recall that, from a regularity point of view, \({\mathcal {F}}_{p, q}\) belongs to the class of nonuniformly elliptic functionals with nonstandard growth conditions of (p, q)-type, according to Marcellini’s terminology. We refer the interested reader to [32, 33] for a more detailed discussion about double-phase variational problems.

Along this paper, we assume that the nonlinearity \(f:\mathbb {R}\rightarrow \mathbb {R}\) is a measurable function such that \(f(t)=0\) if \(t\le 0\) and satisfies the following conditions:

\((f_1)\):

There are \(C>0\) and \(r\in (q, q^*_{ s_2})\) such that

$$\begin{aligned} |f(t)| \le C(1+ |t|^{ r-1}) \quad \text{ for } \text{ all } t\in \mathbb {R}. \end{aligned}$$
\((f_2)\):

There exists \(\theta \in (q, q^*_{s_2})\) such that

$$\begin{aligned} 0\le \theta F(t) \le t \underline{f}(t) \quad \text{ for } \text{ all } t\in \mathbb {R}, \end{aligned}$$

where \(F(t):=\int _0^t f(\tau )\, \textrm{d}\tau \).

\((f_3)\):

There is \(\beta >0\) that will be fixed later, such that

$$\begin{aligned} H(t-\beta ) \le f(t) \quad \text{ for } \text{ all } t\in \mathbb {R}, \end{aligned}$$

where H is the Heaviside function, i.e.,

$$\begin{aligned} H(t):= \left\{ \begin{array}{l} 1 \quad \text{ if } \, t> 0, \\ 0 \quad \text{ if } \, t< 0. \\ \end{array} \right. \end{aligned}$$
\((f_4)\):

\(\limsup _{t\rightarrow 0} \frac{f(t)}{t^{q-1}} =0\).

A typical example of a function satisfying the conditions \((f_1)\)–\((f_4)\) is given by

$$\begin{aligned} f(t):= \left\{ \begin{array}{ll} 0 \quad &{} \text{ if } \, t\in (-\infty , \frac{\beta }{2}), \\ 1 \quad &{} \text{ if } \, t\in {\mathbb {Q}}\cap [\frac{\beta }{2}, \beta ], \\ 0 \quad &{} \text{ if } \, t\in (\mathbb {R}{\setminus } {\mathbb {Q}})\cap [0, \beta ], \\ \sum _{k=1}^{m} \frac{|t|^{q_{k}-1}}{\beta ^{q_{k}-1}} \quad &{} \text{ if } \, t>\beta , \, m\ge 1 \text{ and } \, q_{k}\in (q, q^{*}_{s_{2}}). \end{array} \right. \end{aligned}$$

Note that the above function has an uncountable set of discontinuity points. We emphasize that elliptic boundary value problems involving discontinuous nonlinearities have been widely investigated by several authors; see [1,2,3, 11, 16, 25, 26] and the references therein. These problems can be used to deal with free-boundary problems arising in mathematical physics, such as the obstacle problem, the seepage surface problem and the Elenbaas equation; see [17,18,19]. On the other hand, in nonlocal fractional framework, only a few papers considered nonlinear problems with discontinuous nonlinearities (see for instance [7, 13, 23, 37]) but none of them involves the fractional (p, q)-Laplacian operator. Strongly motivated by this fact, in this paper we aim to obtain a first result for a critical fractional (p, q)-Laplacian problem with discontinuous nonlinearity. More precisely, our main result can be stated as follows.

Theorem 1.1

Assume that \((f_1)\)–\((f_4)\) hold. Then, (1.1) admits a nonnegative nontrivial weak solution, namely, there exists a couple \((u, \rho )\) where \(u\in W^{s_{2}, q}_{0}(\Omega ){\setminus }\{0\}\) is a nonnegative function such that

$$\begin{aligned}&\iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{p-2} (u(x)-u(y)) (\varphi (x)-\varphi (y)) }{|x-y|^{N+s_1p}}\,\textrm{d}x\textrm{d}y \\&\qquad +\iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{q-2} (u(x)-u(y)) (\varphi (x)-\varphi (y)) }{|x-y|^{N+s_2q}} \, \textrm{d}x \textrm{d}y \\&\quad =\int _{\Omega } \rho \varphi \, \textrm{d}x+ \int _\Omega |u|^{q^*_{s_2} -2}u \varphi \, \textrm{d}x \quad \text{ for } \text{ all } \varphi \in W^{s_{2}, q}_{0}(\Omega ), \end{aligned}$$

and \(\rho \in L^{\frac{r}{r-1}}(\Omega )\) satisfies

$$\begin{aligned} \rho (x)\in [\underline{f}(u(x)), \overline{f}(u(x))] \text{ a.e. } \text{ in } \Omega . \end{aligned}$$

Moreover, the set \(\{x\in \Omega :\, u(x)>\beta \}\) has positive measure.

The proof of Theorem 1.1 is obtained by following the strategy used in [25]. More precisely, we combine the mountain pass theorem for non differentiable functionals [21, 28] and invoke the concentration-compactness lemma by Lions [31] in the fractional setting; see [4, 14, 34]. However, due to the nonlocal character of the involved nonlocal operators, several calculations performed throughout the paper are much more elaborated with respect to the case \(s_{1}=s_{2}=1\) considered in [25]. Moreover, we are able to cover the case \(1<p<q\) which has not been attacked in [25] (where the authors assumed \(2\le p<q\)). Therefore, Theorem 1.1 extends and improves Theorem 1.1 in [25].

The paper is organized as follows. In Sect. 2 we fix the notations and we collect some preliminary results about the fractional Sobolev spaces and critical point theory for locally Lipschitz continuous functionals. In Sect. 3 we provide the proof of Theorem 1.1.

2 Preliminaries

Let \(s\in (0, 1)\) and \(p\in (1, \infty )\). Assume \(N>sp\). Denote by \(D^{s, p}(\mathbb {R}^{N})\) the completion of \(C^{\infty }_{c}(\mathbb {R}^{N})\) with respect to

$$\begin{aligned} {[}u]^{p}_{s, p}:=\iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{p}}{|x-y|^{N+sp}}\textrm{d}x\textrm{d}y, \end{aligned}$$

or equivalently

$$\begin{aligned} D^{s, p}(\mathbb {R}^{N}):=\{u\in L^{p^{*}_{s}}(\mathbb {R}^{N}): [u]_{s, p}<\infty \}, \end{aligned}$$

where \(p^{*}_{s}:=\frac{Np}{N-sp}\) is the fractional critical exponent. Let us introduce the fractional Sobolev space

$$\begin{aligned} W^{s, p}(\mathbb {R}^{N}):=\{u\in L^{p}(\mathbb {R}^{N}): [u]_{s, p}<\infty \} \end{aligned}$$

endowed with the norm

$$\begin{aligned} \Vert u\Vert _{s, p}:=\left( [u]_{s, p}^{p}+|u|^{p}_{p}\right) ^{\frac{1}{p}}. \end{aligned}$$

It is well-known that \(W^{s, p}(\mathbb {R}^{N})\) is continuously embedded into \(L^{t}(\mathbb {R}^{N})\) for all \(t\in [p, p^{*}_{s}]\) and compactly embedded into \(L^{t}(B_{R})\) for all \(t\in [p, p^{*}_{s})\) and for all \(R>0\) (see [22]). Let

$$\begin{aligned} S_{s, p}:=\inf _{u\in D^{s, p}(\mathbb {R}^{N}){\setminus }\{0\}}\frac{[u]^{p}_{s, p}}{|u|^{p}_{p^{*}_{s}}}. \end{aligned}$$

Let us introduce the space

$$\begin{aligned} W_{0}^{s, p}(\Omega ):=\{ u\in W^{s, p}(\mathbb {R}^{N}): u=0 \text{ in } \mathbb {R}^{N}{\setminus } \Omega \} \end{aligned}$$

equipped with the norm

$$\begin{aligned} \Vert u\Vert _{0, s, p}:=[u]_{s, p}. \end{aligned}$$

We observe that \(W_{0}^{s, p}(\Omega )\) is continuously embedded into \(L^{t}(\mathbb {R}^{N})\) for all \(t\in [p, p^{*}_{s}]\) and compactly embedded into \(L^{t}(\mathbb {R}^{N})\) for all \(t\in [p, p^{*}_{s})\); see [22]. Below we recall the relation between \(W_{0}^{s_{1}, p}(\Omega )\) and \(W_{0}^{s_{2}, q}(\Omega )\) when \(0<s_1<s_2<1\) and \(1<p\le q\).

Lemma 2.1

[14, Lemma 2.2] Let \(0<s_1<s_2<1\), \(1<p\le q\) and \(\Omega \subset \mathbb {R}^N\) be a smooth bounded domain in \(\mathbb {R}^N,\) where \(N>s_2q\). Then, \(W_{0}^{s_{2}, q}(\Omega ) \subset W_{0}^{s_{1}, p}(\Omega )\) and there exists a constant \(C=C(|\Omega |, N, p, q, s_1, s_2)>0\) such that

$$\begin{aligned} \Vert u\Vert _{0,s_1, p} \le \Vert u\Vert _{0,s_2, q}, \quad \text{ for } \text{ all } u\in W_{0}^{s_{2}, q}(\Omega ). \end{aligned}$$

In view of Lemma 2.1, we deduce that the right space to study (1.1) is \(W_{0}^{s_{2}, q}(\Omega )\). To deal with the critical growth of the nonlinearity in (1.1), we will use the following variant of the concentration-compactness lemma [31] established in [34] (see also [4, 14] for related results).

Lemma 2.2

[34, Theorem 2.5] Let \(s\in (0,1)\) and \(p\in (1, \infty ).\) Let \((u_n)\) be a bounded sequence in \(W^{s, p}_{0}(\Omega )\). Then, up to a subsequence, there exists \(u\in W^{s, p}_{0}(\Omega )\), two Borel regular measures \(\mu \) and \(\nu \), J denumerable, \(x_{j}\in \overline{\Omega }\), \(\nu _{j}\ge 0\), \(\mu _{j}\ge 0\) with \(\mu _{j}+\nu _{j}>0\), \(j\in J\), such that

$$\begin{aligned}&u_n \rightharpoonup u \text{ in } W^{s,p}_{0}(\Omega ), \quad u_{n}\rightarrow u \text{ in } L^{p}(\Omega ), \\&\int _{\mathbb {R}^{N}} \frac{|u_{n}(x)-u_{n}(y)|^{p}}{|x-y|^{N+sp}}\textrm{d}x\textrm{d}y {\mathop {\rightharpoonup }\limits ^{*}} \mu , \quad |u_n|^{p^*_{s}} {\mathop {\,\,\rightharpoonup }\limits ^{*}} \nu ,\\&\mu \ge \int _{\mathbb {R}^{N}} \frac{|u(x)-u(y)|^{p}}{|x-y|^{N+sp}}\textrm{d}x\textrm{d}y+\sum _{j\in J} \mu _{j}\delta _{x_{j}}, \quad \mu _{j}:=\mu (x_{j}), \\&\nu =|u|^{p^{*}_{s}}+\sum _{j\in J} \nu _{j}\delta _{x_{j}}, \quad \nu _{j}:=\nu (x_{j}), \\&\mu _{j}\ge S_{s, p} \nu _{j}^{\frac{p}{p^{*}_{s}}} \quad \text{ for } \text{ all } j\in J, \end{aligned}$$

where \(\delta _{x_{j}}\) is the Dirac mass at \(x_{j}\).

Hereafter, we collect some results about critical point theory for locally Lipschitz continuous functionals; see [19, 21, 28] for more details.

Let X be a real Banach space endowed with the norm \(\Vert \cdot \Vert \). A functional \(I:X\rightarrow \mathbb {R}\) is locally Lipschitz continuous (in short, \(I\in Lip_{loc}(X, \mathbb {R})\)) if for each \(u\in X\) we can find an open neighborhood \(V:=V_{u}\subset X\) of u and some constant \(K:=K_{u}>0\) such that

$$\begin{aligned} |I(v_{1})-I(v_{2})|\le K\Vert v_{1}-v_{2}\Vert \quad \text{ for } \text{ all } v_{1}, v_{2}\in V. \end{aligned}$$

Let \(I\in Lip_{loc}(X, \mathbb {R})\). The generalized directional derivative of I at \(u\in X\) in the direction \(v\in X\) is defined as

$$\begin{aligned} I^{0}(u; v):=\limsup _{h\rightarrow 0 \, \sigma \downarrow 0} \frac{I(u+h+\sigma v)-I(u+h)}{\sigma }. \end{aligned}$$

Therefore, \(I^{0}(u; \cdot )\) is continuous, convex and its subdifferential at \(z\in X\) is given by

$$\begin{aligned} \partial I^{0}(u; z):=\{\mu \in X^{*}: I^{0}(u; v)\ge I^{0}(u; z)\langle \mu , v-z\rangle \quad \text{ for } \text{ all } v\in X\}, \end{aligned}$$

where \(\langle \cdot , \cdot \rangle \) is the duality pairing between \(X^{*}\) and X. The generalized gradient of I at \(u\in X\) is

$$\begin{aligned} \partial I(u):=\{\mu \in X^{*}: \langle \mu , v\rangle \le I^{0}(u; v) \quad \text{ for } \text{ all } v\in X\}. \end{aligned}$$

Because \(I^{0}(u;0)=0\), \(\partial I(u)\) is the subdifferential of \(I^{0}(u; \cdot )\) at 0. We also have the following facts:

$$\begin{aligned}&\partial I(u)\subset X^{*} \text{ is } \text{ convex, } \text{ not } \text{ empty } \text{ and } \text{ weak }\,^{*}\,\text{-compact, } \\&\lambda (u):=\min \{ \Vert \mu \Vert _{X^{*}}: \mu \in \partial I(u)\}, \\&\partial I(u)=\{I'(u)\} \text{ if } I\in C^{1}(X, \mathbb {R}). \end{aligned}$$

A point \(u_{0}\in X\) is a critical point of I if \(0\in \partial I(u_{0})\). A number \(c\in \mathbb {R}\) is a critical value of I if there exists a critical point \(u_{0}\in X\) such that \(I(u_{0})=c\). We say that I satisfies the nonsmooth Palais–Smale condition at level \(c\in \mathbb {R}\) (nonsmooth \((PS)_{c}\)-condition for short), if every sequence \((u_{n})\subset X\) such that \(I(u_{n})\rightarrow c\) and \(\lambda (u_{n})\rightarrow 0\) has a (strongly) convergent subsequence. We recall the following variant of the mountain pass lemma.

Theorem 2.3

[19, 28] Let X be a real Banach space and \(I\in Lip_{loc}(X, \mathbb {R})\) with \(I(0)=0\). Assume that there exist \(\alpha , r>0\) and \(e\in X\) such that

  1. (i)

    \(I(u)\ge \alpha \) for all \(u\in X\) such that \(\Vert u\Vert =r\),

  2. (ii)

    \(I(e)<0\) and \(\Vert e\Vert >r\).

Let

$$\begin{aligned} c:=\inf _{\gamma \in \Gamma } \max _{t\in [0, 1]} I(\gamma (t)) \text{ and } \Gamma :=\{\gamma \in C^{0}([0, 1], X): \gamma (0)=0 \text{ and } I(\gamma (1))<0\}. \end{aligned}$$

Then \(c\ge \alpha \) and there is a sequence \((u_{n})\subset X\) (named a nonsmooth \((PS)_{c}\)-sequence) such that

$$\begin{aligned} I(u_{n})\rightarrow c \text{ and } \lambda (u_{n})\rightarrow 0. \end{aligned}$$

If, in addition, I satisfies the nonsmooth \((PS)_{c}\)-condition, then c is a critical value of I.

Finally, we have the following result.

Proposition 2.4

[19, 28] Let \(\Psi (u):=\int _{\Omega } F(u)\, \textrm{d}x\). Then, \(\Psi \in Lip_\textrm{loc}(L^{p+1}(\Omega ), \mathbb {R})\) and \(\partial \Psi (u)\subset L^{\frac{p}{p-1}}(\mathbb {R}^{N})\). Moreover, if \(\rho \in \partial \Psi (u)\), we have

$$\begin{aligned} \rho (x)\in [\underline{f}(u(x)), \overline{f}(u(x))] \,\, \text{ for } \text{ a.e. } x\in \Omega . \end{aligned}$$

3 Proof of Theorem 1.1

We will look for nonnegative weak solutions of (1.1) by finding critical points of the Euler–Lagrange functional \(I: W_{0}^{s_{2}, q}(\Omega ) \rightarrow \mathbb {R}\) given by

$$\begin{aligned} I(u):=Q(u)-\Psi (u), \end{aligned}$$

where

$$\begin{aligned} Q(u):= \frac{1}{p} \Vert u\Vert ^p_{0, s_1, p} +\frac{1}{q} \Vert u\Vert ^q_{0, s_2, q} - \frac{1}{q^*_{s_2} } \int _\Omega (u^{+})^ {q^*_{s_2} } \, \textrm{d}x, \end{aligned}$$

and

$$\begin{aligned} \Psi (u):=\int _\Omega F(u)\, \textrm{d}x. \end{aligned}$$

Note that \(I\in {Lip}_{loc}(W_{0}^{s_{2}, q}(\Omega ), \mathbb {R})\) and

$$\begin{aligned} \partial I (u)= \{ Q'(u)\} -\partial \Psi (u) \quad \text{ for } \text{ all } u\in W_{0}^{s_{2}, q}(\Omega ), \end{aligned}$$

where

$$\begin{aligned} \begin{aligned} Q'(u) \varphi&= \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{p-2} (u(x)-u(y)) (\varphi (x)-\varphi (y)) }{|x-y|^{N+s_1p}}\,\textrm{d}x\textrm{d}y \\&\quad +\iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{q-2} (u(x)-u(y)) (\varphi (x)-\varphi (y)) }{|x-y|^{N+s_2q}} \, \textrm{d}x \textrm{d}y \\&\quad - \int _\Omega (u^{+})^{q^*_{s_2} -1} \varphi \, \textrm{d}x. \end{aligned} \end{aligned}$$

Lemma 3.1

The functional I satisfies the (PS)\(_c\) condition for

$$\begin{aligned} c< \left( \frac{1}{\theta }-\frac{1}{q^{*}_{s_{2}}} \right) S^{\frac{N}{s_{2}q}}_{s_{2}, q}. \end{aligned}$$

Proof

Let \((u_n) \subset W_{0}^{s_{2}, q}(\Omega )\) be a \((PS)_c\)-sequence of I, namely

$$\begin{aligned} I(u_n) \rightarrow c \quad \text{ and } \quad \lambda (u_n) \rightarrow 0. \end{aligned}$$
(3.1)

Take \((w_n) \subset \partial I (u_n)\) such that

$$\begin{aligned} \Vert w_n\Vert _* =\lambda (u_n) =o_n(1), \end{aligned}$$

and

$$\begin{aligned} w_n = Q'(u_n) -\rho _n, \end{aligned}$$

where \(\rho _n \in \partial \Psi (u_n).\)

Claim 1

\((u_n)\) is bounded in \(W_{0}^{s_{2}, q}(\Omega )\).

We observe that \((f_2)\) gives

$$\begin{aligned}{} & {} \frac{1}{\theta }\rho _n(x) u_n(x) \ge \frac{1}{\theta }\underline{f} (u_n(x)) u_n(x) \ge F(u_n(x)) \\{} & {} \quad \text{ for } \text{ all } n\in {\mathbb {N}} \text{ and } \text{ for } \text{ a.e. } x\in \Omega . \end{aligned}$$

Then we have

$$\begin{aligned} \begin{aligned} c+1+\Vert u\Vert _{0, s_2, q}&\ge I(u_n) -\frac{1}{\theta }\langle w_n , u_n \rangle \\&\ge \frac{1}{p} \Vert u_n\Vert ^p_{0, s_1, p} + \frac{1}{q} \Vert u_n\Vert ^q_{0, s_2, q} -\frac{1}{q^*_{s_2}} \int _\Omega (u_{n}^{+})^{q^*_{s_2}} \textrm{d}x -\int _\Omega F(u_n) \textrm{d}x \\&\quad -\frac{1}{\theta }\Vert u_n\Vert ^p_{0, s_1, p} - \frac{1}{\theta }\Vert u_n\Vert ^q_{0, s_2, q}+\frac{1}{\theta } \int _\Omega (u_{n}^{+})^{q^*_{s_2}} \textrm{d}x+\frac{1}{\theta }\int _\Omega \rho _n u_n \textrm{d}x \\&= \left( \frac{1}{p} - \frac{1}{\theta }\right) \Vert u_n\Vert ^p_{0, s_1, p} + \left( \frac{1}{q} - \frac{1}{\theta }\right) \Vert u_n\Vert ^q_{0, s_2, q}\\&\quad +\int _\Omega \left( \frac{1}{\theta } \rho _n u_n - F(u_n) \right) \textrm{d}x + \left( \frac{1}{\theta }- \frac{1}{q^*_{s_2}} \right) \int _\Omega (u_{n}^{+})^{q^*_{s_2}} \textrm{d}x\\&\ge \left( \frac{1}{p} - \frac{1}{\theta }\right) \Vert u_n\Vert ^p_{0, s_1, p} + \left( \frac{1}{q} - \frac{1}{\theta }\right) \Vert u_n\Vert ^q_{0, s_2, q}\\&\quad + \left( \frac{1}{\theta }- \frac{1}{q^*_{s_2 }} \right) \int _\Omega (u_{n}^{+})^{q^*_{s_2}} \textrm{d}x\\&\ge \left( \frac{1}{q} - \frac{1}{\theta }\right) \Vert u_n\Vert ^q_{0, s_2, q}, \end{aligned} \end{aligned}$$

where we have used \(\theta>q>p\). Therefore, \((u_n)\) is bounded in \(W_{0}^{s_{2}, q}(\Omega )\). Note that, by Lemma 2.1, \((u_n)\) is also bounded in \(W_{0}^{s_{1}, p}(\Omega )\). Up to a subsequence, we may assume that

$$\begin{aligned} \begin{aligned}&u_{n}\rightharpoonup u \text{ in } W_{0}^{s_{2}, q}(\Omega ), \\&u_{n}\rightarrow u \text{ in } L^{t}(\mathbb {R}^N) \, \text{ for } \text{ all } t\in [1, q^{*}_{s_{2}}), \\&u_{n}\rightarrow u \text{ a.e. } \text{ in } \mathbb {R}^{N}. \end{aligned} \end{aligned}$$
(3.2)

Claim 2

\(u_{n}^{-}\rightarrow 0\) in \(W^{s_{2}, q}_{0}(\Omega )\) and \((u_{n}^{+})\) is a \((PS)_{c}\)-sequence for I. Here \(x^{+}:=\max \{x, 0\}\) and \(x^{-}:=\min \{x, 0\}\) for \(x\in \mathbb {R}\).

Using \(\langle w_{n}, u_{n}^{-}\rangle =o_{n}(1)\), \(f(t)=0\) for \(t\le 0\), and observing that

$$\begin{aligned} |x-y|^{t-2}(x-y)(x^{-}-y^{-})\ge |x^{-}-y^{-}|^{t} \quad \text{ for } \text{ all } x, y\in \mathbb {R} \text{ and } t>1, \end{aligned}$$

we deduce that

$$\begin{aligned} o_{n}(1)&=\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p-2}}{|x-y|^{N+s_{1}p}} (u_{n}(x)-u_{n}(y)) (u^{-}_{n}(x)-u^{-}_{n}(y)) \textrm{d}x\textrm{d}y\\&\quad +\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q-2}}{|x-y|^{N+s_{2}q}} (u_{n}(x)-u_{n}(y)) (u^{-}_{n}(x)-u^{-}_{n}(y)) \textrm{d}x\textrm{d}y \\&\ge \iint _{\mathbb {R}^{2N}} \frac{|u^{-}_{n}(x)-u^{-}_{n}(y)|^{p}}{|x-y|^{N+s_{1}p}} \textrm{d}x\textrm{d}y+\iint _{\mathbb {R}^{2N}} \frac{|u^{-}_{n}(x)-u^{-}_{n}(y)|^{q}}{|x-y|^{N+s_{2}q}} \textrm{d}x\textrm{d}y \end{aligned}$$

which implies that \(u_{n}^{-}\rightarrow 0\) in \(W^{s_{2}, q}_{0}(\Omega )\). In particular, \((u_{n}^{+})\) is bounded in \(W^{s_{2}, q}_{0}(\Omega )\). Combining \(I(u_{n})\rightarrow c\), \(u_{n}=u_{n}^{+}+u_{n}^{-}\), \(u_{n}^{-}\rightarrow 0\) in \(W^{s_{2}, q}_{0}(\Omega )\), and the Brezis–Lieb lemma [15], we obtain

$$\begin{aligned} c+o_{n}(1)=I(u_{n})=I(u_{n}^{+})+o_{n}(1), \end{aligned}$$

that is \(I(u_{n}^{+})\rightarrow c\). Let us now show that \(\lambda (u_{n}^{+})\rightarrow 0\). Take \(\phi \in W^{s_{2}, q}_{0}(\Omega )\) such that \(\Vert \phi \Vert _{0, s_{2}, q}\le 1\). Let \(s\in \{s_{1}, s_{2}\}\) and \(t\in \{p, q\}\). Define

$$\begin{aligned} A_{n}&:=\Bigl | \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{t-2}}{|x-y|^{N+st}} (u_{n}(x)-u_{n}(y)) (\phi (x)-\phi (y))\textrm{d}x\textrm{d}y \\&\quad -\iint _{\mathbb {R}^{2N}} \frac{|u^{+}_{n}(x)-u^{+}_{n}(y)|^{t-2}}{|x-y|^{N+st}} (u^{+}_{n}(x)-u^{+}_{n}(y)) (\phi (x)-\phi (y))\textrm{d}x\textrm{d}y \Bigr |. \end{aligned}$$

In light of \(\lambda (u_{n})\rightarrow 0\), to prove that \(\lambda (u_{n}^{+})\rightarrow 0\), it suffices to verify that \(A_{n}\rightarrow 0\). Let us recall the following inequalities (see [36]):

$$\begin{aligned} \langle |x|^{t-2}x -|y|^{t-2}y, x-y\rangle \le \left\{ \begin{array}{ll} C(|x|+|y|)^{t-2}|x-y| &{} \text{ if } t>2,\\ C|x-y|^{t-1} &{} \text{ if } 1<t\le 2, \end{array} \right. \end{aligned}$$
(3.3)

for all \(x, y\in \mathbb {R}^{N}\). Assume \(t>2\). Using the first relation in (3.3), \(x-x^{+}=x^{-}\) for all \(x\in \mathbb {R}\), the Hölder inequality, \(u_{n}^{-}\rightarrow 0\) in \(W^{s, t}_{0}(\Omega )\) and \((u_{n}^{+})\) is bounded in \(W^{s, t}(\mathbb {R}^{N})\), we see that

$$\begin{aligned} A_{n}&\le C\iint _{\mathbb {R}^{2N}} \frac{[|u_{n}(x)-u_{n}(y)|+|u^{-}_{n}(x)-u^{-}_{n}(y)|]^{t-2}}{|x-y|^{N+st}} |u_{n}^{-}(x)-u_{n}^{-}(y)||\phi (x)\\&\quad -\phi (y)| \textrm{d}x\textrm{d}y \\&\le C[u_{n}]_{s, t}^{t-2}[u_{n}^{-}]_{s, t} [\phi ]_{s, t}\le C[u_{n}^{-}]^{t-2}_{s, t}\rightarrow 0. \end{aligned}$$

Suppose \(1<t\le 2\). Then, exploiting the second relation in (3.3), \(x-x^{+}=x^{-}\) for all \(x\in \mathbb {R}\), the Hölder inequality and \(u_{n}^{-}\rightarrow 0\) in \(W^{s, t}_{0}(\Omega )\), we have that

$$\begin{aligned} A_{n}&\le C\iint _{\mathbb {R}^{2N}} \frac{|u_{n}^{-}(x)-u_{n}^{-}(y)|^{t-1}}{|x-y|^{N+st}} |\phi (x)-\phi (y)| \textrm{d}x\textrm{d}y\le C[u_{n}^{-}]^{t-1}_{s, t} [\phi ]_{s, t}\\&\le C[u_{n}^{-}]^{t-1}_{s, t}\rightarrow 0. \end{aligned}$$

Therefore, \(A_{n}\rightarrow 0\) and so \((u_{n}^{+})\) is a \((PS)_{c}\)-sequence for I. Thus we may assume that \(u_{n}\ge 0\) in \(\mathbb {R}^{N}\) for all \(n\in {\mathbb {N}}\). Clearly, \(u\ge 0\) in \(\mathbb {R}^{N}\).

Claim 3

It holds

$$\begin{aligned} \int _\Omega u_{n}^{q^*_{s_2}}\, \textrm{d}x \rightarrow \int _\Omega u^{q^*_{s_2}}\, \textrm{d}x. \end{aligned}$$

Invoking Lemma 2.2, we can find a denumerable set J, sequences \((x_{j})\subset \overline{\Omega }\), \((\mu _{j}), (\nu _{j})\subset [0, \infty )\), \(j\in J\), such that \(\mu _{j}+\nu _{j}>0\) and

$$\begin{aligned} \begin{aligned} \int _{\mathbb {R}^{N}} \frac{|u_{n}(x)-u_{n}(y)|^{q}}{|x-y|^{N+s_{2}q}}\textrm{d}x\textrm{d}y {\mathop {\rightharpoonup }\limits ^{*}} \mu , \quad u_{n}^{q^{*}_{s_{2}}}{\mathop {\rightharpoonup }\limits ^{*}} \nu , \end{aligned} \end{aligned}$$
(3.4)

and we have

$$\begin{aligned} \begin{aligned}&\mu \ge \int _{\mathbb {R}^{N}} \frac{|u(x)-u(y)|^{q}}{|x-y|^{N+s_{2}q}}\textrm{d}x\textrm{d}y+\sum _{j\in J} \mu _{j}\delta _{x_{j}}, \\&\nu =u^{q^{*}_{s_{2}}}+\sum _{j\in J} \nu _{j}\delta _{x_{j}}, \\&S_{s_{2}, q} \nu _{j}^{\frac{q}{q^{*}_{s_{2}}}}\le \mu _{j} \quad \text{ for } \text{ all } j\in J. \end{aligned} \end{aligned}$$
(3.5)

Fix \(j\in J\). For \(\rho >0\), define \(\psi _{\rho }(x):=\psi (\frac{x-x_{j}}{\rho })\), where \(\psi \in C^{\infty }_{c}(\mathbb {R}^{N})\) is such that \(0\le \psi \le 1\), \(\psi =1\) in \(B_{1}(0)\), \(\psi =0\) in \(\mathbb {R}^{N}{\setminus } B_{2}(0)\) and \(|\nabla \psi |_{\infty }\le 2\). Since \((u_{n}\psi _{\rho })\) is bounded in \(W_{0}^{s_{2}, q}(\Omega )\), we have

$$\begin{aligned} o_{n}(1)&=\langle w_{n}, u_{n}\psi _{\rho }\rangle \\&=\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p-2}}{|x-y|^{N+s_{1}p}} (u_{n}(x)-u_{n}(y)) (u_{n}(x)\psi _{\rho }(x)-u_{n}(y)\psi _{\rho }(y)) \textrm{d}x\textrm{d}y \\&\quad +\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q-2}}{|x-y|^{N+s_{2}q}} (u_{n}(x)-u_{n}(y)) (u_{n}(x)\psi _{\rho }(x)-u_{n}(y)\psi _{\rho }(y)) \textrm{d}x\textrm{d}y \\&\quad -\int _{\Omega } u_{n}^{q^{*}_{s_{2}}} \psi _{\rho }\textrm{d}x-\int _{\Omega } \rho _{n}\psi _{\rho } u_{n}\textrm{d}x, \end{aligned}$$

whence

$$\begin{aligned} \begin{aligned}&\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p}}{|x-y|^{N+s_{1}p}}\psi _{\rho }(x) \textrm{d}x\textrm{d}y+\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q}}{|x-y|^{N+s_{2}q}}\psi _{\rho }(x) \textrm{d}x\textrm{d}y \\&\quad = -\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p-2}}{|x-y|^{N+s_{1}p}} (u_{n}(x)-u_{n}(y)) (\psi _{\rho }(x)-\psi _{\rho }(y))u_{n}(y) \textrm{d}x\textrm{d}y \\&\qquad -\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q-2}}{|x-y|^{N+s_{2}q}} (u_{n}(x)-u_{n}(y)) (\psi _{\rho }(x)-\psi _{\rho }(y))u_{n}(y) \textrm{d}x\textrm{d}y \\&\qquad +\int _{\Omega } u_{n}^{q^{*}_{s_{2}}} \psi _{\rho }\textrm{d}x+\int _{\Omega } \rho _{n}\psi _{\rho } u_{n}\textrm{d}x+o_{n}(1). \end{aligned} \end{aligned}$$
(3.6)

Notice that, by (3.4) and (3.5),

$$\begin{aligned} \begin{aligned}&\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p}}{|x-y|^{N+s_{1}p}}\psi _{\rho }(x) \textrm{d}x\textrm{d}y\ge 0 \quad \text{ for } \text{ all } n\in {\mathbb {N}}, \\&\lim _{n\rightarrow \infty }\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p}}{|x-y|^{N+s_{2}q}}\psi _{\rho }(x) \textrm{d}x\textrm{d}y\\&\quad =\int _{\mathbb {R}^{N}} \psi _{\rho } \textrm{d}\mu \ge \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{p}}{|x-y|^{N+s_{2}q}}\textrm{d}x\textrm{d}y+\mu _{j}. \end{aligned} \end{aligned}$$
(3.7)

On the other hand, using the Hölder inequality and the boundedness of \((u_{n})\) in \(W_{0}^{s_{1}, p}(\Omega )\),

$$\begin{aligned}&\Bigl | \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p-2}}{|x-y|^{N+s_{1}p}} (u_{n}(x)-u_{n}(y)) (\psi _{\rho }(x)-\psi _{\rho }(y))u_{n}(y) \textrm{d}x\textrm{d}y\Bigr | \\&\quad \le [u_{n}]_{s_{1}, p}^{p-1} \left( \iint _{\mathbb {R}^{2N}} \frac{|\psi _{\rho }(x)-\psi _{\rho }(y)|^{p}}{|x-y|^{N+s_{1}p}}|u_{n}(y)|^{p} \textrm{d}x\textrm{d}y \right) ^{\frac{1}{p}} \\&\quad \le C\left( \iint _{\mathbb {R}^{2N}} \frac{|\psi _{\rho }(x)-\psi _{\rho }(y)|^{p}}{|x-y|^{N+s_{1}p}}|u_{n}(y)|^{p} \textrm{d}x\textrm{d}y \right) ^{\frac{1}{p}}. \end{aligned}$$

Thanks to [4, Lemma 2.3], we see that

$$\begin{aligned} \lim _{\rho \rightarrow 0} \lim _{n\rightarrow \infty }\iint _{\mathbb {R}^{2N}} \frac{|\psi _{\rho }(x)-\psi _{\rho }(y)|^{p}}{|x-y|^{N+s_{1}p}}|u_{n}(y)|^{p} \textrm{d}x\textrm{d}y=0, \end{aligned}$$

and so

$$\begin{aligned}{} & {} \lim _{\rho \rightarrow 0} \lim _{n\rightarrow \infty } \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p-2}}{|x-y|^{N+s_{1}p}} (u_{n}(x)-u_{n}(y)) (\psi _{\rho }(x)\nonumber \\{} & {} \quad -\psi _{\rho }(y))u_{n}(y) \textrm{d}x\textrm{d}y=0. \end{aligned}$$
(3.8)

In a similar fashion,

$$\begin{aligned}{} & {} \lim _{\rho \rightarrow 0} \lim _{n\rightarrow \infty } \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q-2}}{|x-y|^{N+s_{2}q}} (u_{n}(x)-u_{n}(y)) (\psi _{\rho }(x)\nonumber \\{} & {} \quad -\psi _{\rho }(y))u_{n}(y) \textrm{d}x\textrm{d}y=0. \end{aligned}$$
(3.9)

Now, by \((f_1)\), we see that

$$\begin{aligned} 0\le \rho _n(x)\le C(1+(u_{n}(x))^{ r-1}) \quad \text{ for } \text{ all } n\in {\mathbb {N}} \text{ and } \text{ for } \text{ a.e. } x\in \Omega . \end{aligned}$$
(3.10)

Hence,

$$\begin{aligned} \Bigl |\int _{B_{2\rho }(0)} \rho _{n}\psi _{\rho }u_{n}\textrm{d}x\Bigr |\le C\left[ \int _{\Omega } \psi _{\rho }u_{n} \textrm{d}x+\int _{\Omega } \psi _{\rho }u_{n}^{r} \textrm{d}x \right] \end{aligned}$$

and exploiting (3.2) and the fact that \(\psi \) has compact support, we infer

$$\begin{aligned} \lim _{\rho \rightarrow 0} \lim _{n\rightarrow \infty } \int _{B_{2\rho }(0)} \rho _{n}\psi _{\rho }u_{n}\textrm{d}x=0. \end{aligned}$$
(3.11)

Finally, due to (3.4) and (3.5), we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{\Omega } u_{n}^{q^{*}_{s_{2}}} \psi _{\rho }\textrm{d}x=\int _{\Omega } \psi _{\rho }\textrm{d}\nu =\int _{\Omega } u^{q^{*}_{s_{2}}} \psi _{\rho }\textrm{d}x+\nu _{j}. \end{aligned}$$
(3.12)

Combining (3.6)–(3.12), we obtain \(\mu _{j}\le \nu _{j}\) which together with (3.5) yields \(\nu _{j}\ge S_{s_{2}, q} \nu _{j}^{\frac{q}{q^{*}_{s_{2}}}}\), that is, \(\nu _{j}=0\) either \(\nu _{j}\ge S^{\frac{N}{s_{2}q}}_{s_{2}, q}\). If the relation \(\nu _{j}\ge S^{\frac{N}{s_{2}q}}_{s_{2}, q}\) holds for some \(j\in J\), then

$$\begin{aligned} c&=\liminf _{n\rightarrow \infty } \left[ I(u_{n})-\frac{1}{\theta }\langle w_{n}, u_{n}\rangle \right] \\&\ge \liminf _{n\rightarrow \infty } \left( \frac{1}{\theta }-\frac{1}{q^{*}_{s_{2}}} \right) \int _{\Omega } u_{n}^{q^{*}_{s_{2}}}\textrm{d}x \\&\ge \liminf _{n\rightarrow \infty } \left( \frac{1}{\theta }-\frac{1}{q^{*}_{s_{2}}} \right) \int _{\Omega } u_{n}^{q^{*}_{s_{2}}}\psi _{\rho }\textrm{d}x \\&\ge \left( \frac{1}{\theta }-\frac{1}{q^{*}_{s_{2}}} \right) \int _{\Omega } \psi _{\rho }\textrm{d}\nu , \end{aligned}$$

and letting \(\rho \rightarrow 0\) we find

$$\begin{aligned} c\ge \left( \frac{1}{\theta }-\frac{1}{q^{*}_{s_{2}}} \right) S^{\frac{N}{s_{2}q}}_{s_{2}, q} \end{aligned}$$

which gives a contradiction. Therefore, \(\nu _{j}=0\) for all \(j\in J\), and this proves the claim.

Claim 4

\(u_n \rightarrow u\) in \(W_{0}^{s_{2}, q}(\Omega )\).

Note that Claim 3 and the Brezis–Lieb lemma [15] yield

$$\begin{aligned} \left| \int _\Omega u_{n}^{q^*_{s_2} -1} (u_n-u) \,\textrm{d}x \right| \le \int _\Omega |u_n|^{q^*_{s_2} -1} |u_n-u | \, \textrm{d}x \le |u_n-u|_{q^*_{s_2} } |u_n|^{q^*_{s_2} -1}_{q^*_{s_2} } =o_n(1). \end{aligned}$$
(3.13)

On the other hand, (3.10) and the boundedness of \((u_{n})\) in \(L^{r}(\Omega )\) ensure that \((\rho _n)\) is bounded in \(L^{r/(r-1)}(\Omega )\) because

$$\begin{aligned} \int _\Omega |\rho _n| ^{r/(r-1)} \textrm{d}x\le C \int _\Omega (1+|u_n|^{ r-1})^{r/(r-1)}\, \textrm{d}x \le C_1 \int _\Omega (1+|u_n|^{ r}) \, \textrm{d}x \le C_1 |\Omega | +C_2. \end{aligned}$$

Hence, by Hölder inequality, we have that

$$\begin{aligned} \int _\Omega \rho _n (u_n -u) \, \textrm{d}x \le |\rho _n|_{r/(r-1)} |u_n -u|_r, \end{aligned}$$

and exploiting Claim 3 and the boundedness of \((\rho _n)\) in \(L^{\frac{r}{r-1}}(\Omega )\), we arrive at

$$\begin{aligned} \int _\Omega \rho _n (u_n -u) \, \textrm{d}x =o_n(1). \end{aligned}$$
(3.14)

Now, since \((u_n-u)\) is bounded in \(W^{s_{2}, q}_{0}(\Omega )\) and \(\Vert w_n\Vert =o_n(1)\), we know that \(\langle w_n, u_n - u \rangle =o_{n}(1)\). Let us recall the following inequalities (see [36]):

$$\begin{aligned} |x-y|^{t} \le \left\{ \begin{array}{ll} C\langle |x|^{t-2}x -|y|^{t-2}y, x-y\rangle &{} \text{ if } t\ge 2,\\ C[\langle |x|^{t-2}x -|y|^{t-2}y, x-y\rangle ]^{\frac{t}{2}}(|x|^{t}+|y|^{t})^{\frac{2-t}{2}} &{} \text{ if } 1<t<2, \end{array} \right. \end{aligned}$$
(3.15)

for all \(x, y\in \mathbb {R}^{N}\). In particular, \(\langle |x|^{t-2}x -|y|^{t-2}y, x-y\rangle \ge 0\) for all \(x, y\in \mathbb {R}^{N}\) and \(t>1\). Then, using (3.13) and (3.14), we have that

$$\begin{aligned} \begin{aligned} 0&\le \iint _{\mathbb {R}^{2N}} \frac{|u_n(x)-u_n(y)|^{q-2} (u_n(x)-u_n(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y)))}{|x-y|^{N+s_2q}} \textrm{d}x\textrm{d}y\\&\quad -\iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{q-2} (u(x)-u(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y))) }{|x-y|^{N+s_2q}}\textrm{d}x\textrm{d}y \\&\le \iint _{\mathbb {R}^{2N}} \frac{|u_n(x)-u_n(y)|^{p-2} (u_n(x)-u_n(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y)))}{|x-y|^{N+s_1p}} \textrm{d}x\textrm{d}y\\&\quad -\iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{p-2} (u(x)-u(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y))) }{|x-y|^{N+s_1p}}\textrm{d}x\textrm{d}y \\&\quad + \iint _{\mathbb {R}^{2N}} \frac{|u_n(x)-u_n(y)|^{q-2} (u_n(x)-u_n(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y)))}{|x-y|^{N+s_2q}} \textrm{d}x\textrm{d}y\\&\quad -\iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{q-2} (u(x)-u(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y))) }{|x-y|^{N+s_2q}}\textrm{d}x\textrm{d}y \\&= \iint _{\mathbb {R}^{2N}} \frac{|u_n(x)-u_n(y)|^{p-2} (u_n(x)-u_n(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y)))}{|x-y|^{N+s_1p}} \textrm{d}x\textrm{d}y \\&\quad +\iint _{\mathbb {R}^{2N}} \frac{|u_n(x)-u_n(y)|^{q-2} (u_n(x)-u_n(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y)))}{|x-y|^{N+s_2q}} \textrm{d}x\textrm{d}y \\&\quad - \int _{\Omega } u_{n}^{q^*_{s_2} -1} (u_n-u) \,\textrm{d}x -\int _\Omega \rho _n (u_n -u)\textrm{d}x+o_{n}(1)\\&= \langle w_n , u_n - u \rangle +o_{n}(1)=o_{n}(1), \end{aligned} \end{aligned}$$

from which

$$\begin{aligned}&\iint _{\mathbb {R}^{2N}} \frac{|u_n(x)-u_n(y)|^{q-2} (u_n(x)-u_n(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y)))}{|x-y|^{N+s_2q}} \textrm{d}x\textrm{d}y\\&\quad -\iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{q-2} (u(x)-u(y)) ((u_n(x)-u(x))-(u_{n}(y)-u(y))) }{|x-y|^{N+s_2q}}\textrm{d}x\textrm{d}y\rightarrow 0. \end{aligned}$$

Now, if \(q\ge 2\), it follows from the first relation in (3.15) that \(\Vert u_{n}-u\Vert _{0, s_{2}, q}\rightarrow 0\). When \(1<q<2\), we use the second relation in (3.15) and the boundedness of \((u_{n})\) in \(W_{0}^{s_{2}, q}(\Omega )\) to deduce that \(\Vert u_{n}-u\Vert _{0, s_{2}, q}\rightarrow 0\). In conclusion, \(u_{n}\rightarrow u\) in \(W_{0}^{s_{2}, q}(\Omega )\). \(\square \)

The next lemma will be used to choose the constant \(\beta >0\) in \((f_3)\).

Lemma 3.2

  1. (i)

    There are \(v\in W_{0}^{s_{2}, q}(\Omega )\) and \(T>0\) such that

    $$\begin{aligned} \max _{t\in [0,T]} I(tv) <c. \end{aligned}$$
    (3.16)
  2. (ii)

    There are \(\gamma , \tau >0\) such that \(I(u)\ge \tau \) for all \(u\in W_{0}^{s_{2}, q}(\Omega )\) with \(\Vert u\Vert _{0,s_2, q} =\gamma \).

  3. (iii)

    There is \(e\in W_{0}^{s_{2}, q}(\Omega )\) such that \(\Vert e\Vert _{0, s_{2}, q}>\gamma \) and \(I(e)<0\).

Proof

Take \(v\in C^\infty _0 (\Omega )\) such that \(v\ge 0\), \(v\not \equiv 0\) and \(\Vert v\Vert _{0,s_2, q} =1\). Let us consider the continuous function \(g:[0, \infty ) \rightarrow \mathbb {R}\) defined as

$$\begin{aligned} g(t):= \frac{t^p}{p} \Vert v\Vert _{0,s_1, p}^{p} + \frac{t^q}{q} - \frac{t^{q^*_{s_2}}}{q^*_{s_2}} |v|^{q^*_{s_2}}_{q^*_{s_2}}. \end{aligned}$$

It is easy to check that g is increasing in \((0,t_*)\) for some \(t_*>0\). Since \(g(t)=o(t)\) as \(t\rightarrow 0\), we can select \(T>0\) such tat

  1. (1)

    \(T<t_{*}\),

  2. (2)

    \(\max _{t\in [0, T]} g(t)\le g(T)<c\),

  3. (3)

    \(g(T)-T\int _{\Omega } v\, \textrm{d}x<0\).

In order to prove (i), we observe that

$$\begin{aligned} \begin{aligned} I(tv)&= \frac{1}{p} \Vert tv\Vert ^p_{0, s_1, p} +\frac{1}{q} \Vert tv\Vert ^q_{0, s_2, q} - \frac{1}{q^*_{s_2} } \int _\Omega |tv|^ {q^*_{s_2} } \, \textrm{d}x - \int _\Omega F(tv)\, \textrm{d}x\\&\le \frac{t^p}{p} \Vert v\Vert ^p_{0, s_1, p} +\frac{t^q}{q} - \frac{t^{q^*_{s_2} } }{q^*_{s_2} } \int _\Omega |v|^ {q^*_{s_2} } \, \textrm{d}x \\&=g(t) \le \max _{\tau \in [0,T]} g(\tau ) \le g(T) <c \quad \text{ for } \text{ all } t\in [0, T]. \end{aligned} \end{aligned}$$

Consequently, (3.16) holds.

Using the growth assumptions on f and the Sobolev embeddings, we deduce that there are \(C_1,C_2, C_3 >0\) such that

$$\begin{aligned} I(u) \ge C_{1} \Vert u \Vert ^q_{0, s_2, q} - C_2 \Vert u\Vert _{0, s_2, q} ^{q^*_{s_2} } - C_3 \Vert u\Vert _{0, s_2, q} ^{r}. \end{aligned}$$

Recalling that \(q<r<q^*_{s_2}\), we easily deduce that (ii) is valid.

Finally, we prove (iii). Using \((f_3)\), we see that

$$\begin{aligned} \begin{aligned} I(Tv)&= \frac{1}{p} \Vert T v\Vert ^p_{0, s_1, p}+\frac{1}{q} \Vert T v\Vert ^q_{0, s_2, q} - \frac{1}{q^*_{s_2} } \int _\Omega (T v)^ {q^*_{s_2} } \, \textrm{d}x - \int _\Omega F(T v)\, \textrm{d}x\\&= g(T) - \int _\Omega F(T v)\, \textrm{d}x\\&\le g(T)-\int _{\Omega } (Tv-\beta )^{+}\textrm{d}x. \end{aligned} \end{aligned}$$

Since \(\int _{\Omega } (Tv-\beta )^{+}\textrm{d}x\rightarrow \int _{\Omega } Tv\,\textrm{d}x\) as \(\beta \rightarrow 0\), there exists \(\beta >0\) small such that \(I(Tv)<0\). \(\square \)

Now we are ready to give the proof of Theorem 1.1.

Proof of Theorem 1.1

In light of Lemmas 3.1 and 3.2, we can apply Theorem 2.3 to infer that (1.1) admits a nonnegative weak solution \((u, \rho )\in W^{s_{2}, q}_{0}(\Omega )\times L^{\frac{r}{r-1}}(\Omega )\). Finally, we verify that the set

$$\begin{aligned} \{ x\in \Omega \,:\, u(x) >\beta \} \end{aligned}$$

has positive measures. Suppose, by contradiction, that \(u(x) \le \beta \) a.e. in \(\Omega .\) Then, since u is a solution of (1.1), we deduce that

$$\begin{aligned} \Vert u\Vert ^p_{0, s_1, p} +\Vert u \Vert ^q_{0, s_2, q} =\int _\Omega \rho u \, \textrm{d}x+\int _\Omega u^{q^*_{s_2} } \,\textrm{d}x. \end{aligned}$$

Now, using \((f_1)\), we have

$$\begin{aligned} \Vert u \Vert ^q_{0, s_2, q}&\le \Vert u\Vert ^p_{0, s_1, p} +\Vert u \Vert ^q_{0, s_2, q} \\&= \int _\Omega \rho u \, \textrm{d}x+\int _\Omega u^{q^*_{s_2} } \,\textrm{d}x \\&\le C \int _\Omega (u + u^r)\, \textrm{d}x+ \int _\Omega u^{q^*_{s_2}} \,\textrm{d}x \\&\le [C(\beta +\beta ^r)+\beta ^{q^{*}_{s_{2}}}] |\Omega |. \end{aligned}$$

Since \(I(u)=c>0\), we can find \(M>0\) such that \(\Vert u \Vert _{0, s_2, q} \ge M\) and so

$$\begin{aligned} M^q \le [C(\beta +\beta ^r)+\beta ^{q^{*}_{s_{2}}}] |\Omega |. \end{aligned}$$

The above inequality is impossible if we choose \(\beta >0\) sufficiently small and thus we get a contradiction. The proof of Theorem 1.1 is now complete.

\(\square \)