Variational methods for nonpositive mixed local–nonlocal operators

Abstract

We prove the existence of a weak solution for boundary value problems driven by a mixed local–nonlocal operator. The main novelty is that such an operator is allowed to be nonpositive definite.

1 Introduction

In this paper we are concerned with semilinear elliptic problems driven by a mixed local–nonlocal operator of the form

$$\begin{aligned} \mathcal {L}_{\alpha }u := -\Delta u +\alpha (-\Delta )^s u\,. \end{aligned}$$

Here $\alpha \in \mathbb {R}$ with no a priori restrictions, $\Delta u$ denotes the classical Laplace operator while $(-\Delta )^s u$, for fixed $s\in (0,1)$ is the fractional Laplacian, usually defined as

$$\begin{aligned} (-\Delta )^s u(x) := C(n,s) \,\text{ P.V. }\int _{\mathbb {R}^n}\frac{u(x)-u(y)}{|x-y|^{n+2s}}\, dy\,, \end{aligned}$$

where P.V. denotes the Cauchy principal value, that is

$$\begin{aligned}{} & {} \text{ P.V. }\int _{\mathbb {R}^n} \frac{u(x)-u(y)}{|x-y|^{n+2s}}\, dy\\ {}{} & {} =\lim _{\epsilon \rightarrow 0}\int _{\{y\in \mathbb {R}^n\,:\,|y-x|\ge \epsilon \}} \frac{u(x)-u(y)}{|x-y|^{n+2s}}\, dy\,. \end{aligned}$$

Clearly, when $\alpha =0$, one recovers the classical Laplacian, while, for $\alpha >0$, one is led to consider a positive operator which can be considered as a particular instance of an infinitesimal generator of a stochastic process involving a Brownian motion and a pure jump Lévy process.

We briefly recall, focusing merely on the more recent (elliptic) PDEs oriented literature, that problems driven by operators of mixed type, even with a nonsingular nonlocal operator [19], have raised a certain interest in the last few years, for example in connection with the study of optimal animal foraging strategies (see e.g. [23] and the references therein). From the pure mathematical point of view, the superposition of such operators generates a lack of scale invariance which may lead to unexpected complications.

At the present stage, and without aim of completeness, the investigations have taken into consideration interior regularity and maximum principles (see e.g. [2, 9, 14, 18, 26, 27]), boundary Harnack principle [16], boundary regularity and overdetermined problems [10, 38], qualitative properties of solutions [3], existence of solutions and asymptotics (see e.g. [6,7,8, 13, 17, 21, 22, 28, 35, 37]) and shape optimization problems [4, 5, 29].

In this paper we deal with the following boundary value problem:

$$\begin{aligned} \left\{ \begin{array}{ll} \mathcal {L}_{\alpha }u = f(x,u)\,, &{}\text {in } \Omega \\ u = 0\,, &{}\text {in } \mathbb {R}^{n}\setminus \Omega \end{array},\right. \end{aligned}$$

(1.1)

where $\Omega \subset \mathbb {R}^n$, with $n>2$, is an open and bounded set with $C^1$-smooth boundary.

In particular, we consider two different sets of assumptions on the nonlinear term $f:\Omega \times \mathbb {R} \rightarrow \mathbb {R}$, which is always supposed to be a Carathéodory function: first, we deal with the asymptotically linear case, and second with the superlinear and subcritical case.

In the first setting, we assume that f has at most linear growth, according to the following assumption: there exist a function $a\in L^{2}(\Omega )$ and $b\in \mathbb {R}$ such that

$$\begin{aligned} |f(x,t)| \le a(x) + b|t|\quad \text {for all } t \in \mathbb {R} \text { and for a.e. } x \in \Omega \,. \end{aligned}$$

(1.2)

In order to state our result, we also need the following measurable functions:

$$\begin{aligned} {\underline{v}}(x) := \liminf _{|t|\rightarrow +\infty }\dfrac{f(x,t)}{t} \le \limsup _{|t|\rightarrow +\infty }\dfrac{f(x,t)}{t}=: {\overline{v}}(x). \end{aligned}$$

(1.3)

A trivial example of such function f is given by $f(x,t)=\lambda t +a(x)$, where $\lambda \in \mathbb {R}$ and $a\in L^2(\Omega )$.

Instead, in the superlinear and subcritical case, we assume:

there exist a function $a\in L^{\infty }(\Omega )_+$, a number $b\in \mathbb {R}$ and $r\in (2,2^*)$ such that

there exist $\mu>2,\,{\tilde{\mu }}>2$, $R>0$, $c>0$, $A\in L^\infty (\Omega )$ and $d\in L^{1}(\Omega )_+$ such that

where $F(x,t)=\int _0^t f(x,\sigma )\,d\sigma$ for any $t\in \mathbb {R}$.

Here $2^*$ denotes the classical Sobolev critical exponent, namely

$$\begin{aligned} 2^*=\dfrac{2n}{n-2}\,. \end{aligned}$$

A trivial example for f in this case is given by $f(x,t)=\lambda t+|t|^{p-2}t$, with $\lambda \in \mathbb {R}$ and $p\in (2,2^*)$.

Remark 1

We remark that condition (iii) means that $f(x,t)-A(x)t$ satisfies the Ambrosetti-Rabinowitz condition [1], where the growth from below is necessary, see [33]. For the model just above we clearly have $A=\lambda$.

As for the asymptotically linear case, we need to introduce asymptotic functions, which are now relevant as $t\rightarrow 0$:

$$\begin{aligned} {\underline{\Theta }}(x) := \liminf _{t\rightarrow 0}\dfrac{f(x,t)}{t} \le \limsup _{t\rightarrow 0}\dfrac{f(x,t)}{t}=: {\overline{\Theta }}(x)\,. \end{aligned}$$

(1.4)

Our aim is to prove the existence of weak solutions to problem (1.1) (see Section 2 for the precise definition). As partially expected, if $\alpha \ge 0$, our existence results (see Theorem 1 and Theorem 2) are either known or applications of standard variational methods (for instance, see [32] or [34] when $\alpha =0$, or [31] for the pure fractional case). A similar behaviour happens to hold if we take

$$\begin{aligned} -\dfrac{1}{C}<\alpha <0\,, \end{aligned}$$

where $C>0$ is the constant of the continuous embedding $H^{1}_{0} \subset H^{s}$ (see e.g. [20]), i.e.

$$\begin{aligned}{}[u]_s^2 := \iint _{\mathbb {R}^{2n}}\dfrac{|u(x)-u(y)|^2}{|x-y|^{n+2s}}\, dx dy \le C \, \int _{\Omega }|\nabla u|^2 \, dx\,. \end{aligned}$$

In this perspective, the probably more interesting case is for $\alpha \le -\tfrac{1}{C}$.

Indeed, the situation becomes suddenly more delicate, mainly because the local–nonlocal operator is not more positive definite. As a consequence, the bilinear form naturally associated to it does not induce a scalar product nor a norm, the variational spectrum may exhibit negative eigenvalues and even the maximum principles may fail, see e.g. [2].

Let $\{\lambda _k\}_{k\in \mathbb {N}}$ be the sequence of eigenvalues of $\mathcal {L}_\alpha$, see Proposition 1 for details. The main result in the asymptotically linear case states as follows.

Theorem 1

Assume that f satisfies (1.2) and that the limits in (1.3) are uniform in x. If either

• ${\overline{v}}(x) < \lambda _{1}$ for a.e. $x \in \Omega$, or

• there exists $k \in \mathbb {N}$ such that $\lambda _{k}<{\underline{v}}(x) \le {\overline{v}}(x)<\lambda _{k+1}$ for a.e. $x \in \Omega$,

then problem (1.1) admits a weak solution.

Its counterpart in the superlinear and subcritical case states instead as follows.

Theorem 2

Assume f satisfies (i), (ii) and (iii) and that the limits in (1.4) are uniform in x. If either

• ${\overline{\Theta }}(x)<\lambda _1$ a.e. $x \in \Omega$ or

• there exists $k \in \mathbb {N}$ such that $\lambda _{k}\le {\underline{\Theta }}(x)\le {\overline{\Theta }}(x)<\lambda _{k+1}$ for a.e. $x \in \Omega$ and

  $$\begin{aligned} F(x,t)\ge \lambda _k \tfrac{t^2}{2}\quad \text {for a.e. }x \in \Omega \text { and for any }t\in \mathbb {R}\,, \end{aligned}$$

  (1.5)

then problem (1.1) admits a weak solution.

We stress that, despite being slightly non-standard, the latter assumption (1.5) is satisfied in the model case previously mentioned.

The paper is organized as follows: in Section 2 we collect all the assumptions and preliminary results, including a description of the variational spectrum (Proposition 1). In Section 3 we prove Theorem 1, while Theorem 2 is proved in Section 4.

2 Assumptions, notations and preliminary results

Let $\Omega \subseteq \mathbb {R}^n$, $n>2$, be a connected and bounded open set with $C^1$-smooth boundary $\partial \Omega$. We define the space of solutions of problem (1.1) as

$$\begin{aligned} \mathbb {X}(\Omega ) := \big \{u\in H^{1}(\mathbb {R}^n):\, u\equiv 0\,\, \text { a.e. on }\mathbb {R}^n\setminus \Omega \big \}. \end{aligned}$$

Thanks to the regularity assumption on $\partial \Omega$ (see [11, Proposition 9.18]), we can identify the space $\mathbb {X}(\Omega )$ with the space $H_0^{1}(\Omega )$ in the following sense:

$$\begin{aligned} u \in H_0^{1}(\Omega )\,\,\Longleftrightarrow \,\, u\cdot \textbf{1}_{\Omega }\in \mathbb {X}(\Omega )\,, \end{aligned}$$

(2.1)

where $\textbf{1}_\Omega$ is the indicator function of $\Omega$. From now on, we shall always identify a function $u\in H_0^{1}(\Omega )$ with ${\hat{u}} := u\cdot \textbf{1}_\Omega \in \mathbb {X}(\Omega )$.

By the Poincaré inequality and (2.1), we get that the quantity

$$\begin{aligned} \Vert u\Vert _{\mathbb {X}} :=\left( \int _{\Omega }|\nabla u|^2\, dx\right) ^{1/2},\quad u\in \mathbb {X}(\Omega )\,, \end{aligned}$$

endows $\mathbb {X}(\Omega )$ with a structure of (real) Hilbert space, which is isometric to $H_0^{1}(\Omega )$. To fix the notation, we denote by $\langle \cdot ,\cdot \rangle _{\mathbb {X}}$ the scalar product which induces the above norm on $\mathbb {X}(\Omega )$. We briefly recall that the space $\mathbb {X}(\Omega )$ is separable and reflexive, $C_0^\infty (\Omega )$ is dense in $\mathbb {X}(\Omega )$ and eventually that $\mathbb {X}(\Omega )$ compactly embeds in $L^{2}(\Omega )$ and in

$$\begin{aligned} H^s_0(\Omega ):=\left\{ H^{s}(\mathbb {R}^n):\,u\equiv 0~ \text {a.e. on}~ \mathbb {R}^n\setminus \Omega \right\} \end{aligned}$$

by [30, Theorem 16.1].

Definition 1

A function $u \in \mathbb {X}(\Omega )$ is called a weak solution of (1.1) if

$$\begin{aligned} \int _{\Omega }\langle \nabla u, \nabla \varphi \rangle \, dx +\alpha \iint _{\mathbb {R}^{2n}}\dfrac{(u(x)-u(y))(\varphi (x)-\varphi (y))}{|x-y|^{n+2s}}\, dxdy = \int _{\Omega }f(x,u)\varphi \, dx \end{aligned}$$

for every $\varphi \in \mathbb {X}(\Omega )$.

As usual, weak solutions of (1.1) can be found as critical points of the functional $\mathcal {J}:\mathbb {X}(\Omega ) \rightarrow \mathbb {R}$ defined as

$$\begin{aligned} \mathcal {J}(u) := \dfrac{1}{2}\int _{\Omega }|\nabla u|^2 \, dx + \dfrac{\alpha }{2}\iint _{\mathbb {R}^{2n}} \dfrac{|u(x)-u(y)|^2}{|x-y|^{n+2s}}\, dxdy - \int _{\Omega }F(x,u)\, dx\,. \end{aligned}$$

Here

$$\begin{aligned} F(x,t) := \int _{0}^{t}f(x,\sigma )\, d\sigma , \quad t \in \mathbb {R}. \end{aligned}$$

The functional $\mathcal {J}$ is Fréchet differentiable and

$$\begin{aligned} \mathcal {J}'(u)(\varphi )&= \int _{\Omega }\langle \nabla u, \nabla \varphi \rangle \, dx + \alpha \iint _{\mathbb {R}^{2n}}\dfrac{(u(x)-u(y))(\varphi (x)-\varphi (y))}{|x-y|^{n+2s}}\, dxdy \\&\quad - \int _{\Omega }f(x,u(x))\varphi (x)\, dx \quad \text {for every }\varphi \in \mathbb {X}(\Omega )\,. \end{aligned}$$

Definition 2

Consider the bilinear form $\mathcal {B}_{\alpha }:\mathbb {X}(\Omega )\times \mathbb {X}(\Omega ) \rightarrow \mathbb {R}$, defined by

$$\begin{aligned} \mathcal {B}_{\alpha }(u,v):= \int _{\Omega }\langle \nabla u, \nabla v\rangle \, dx +\alpha \iint _{\mathbb {R}^{2n}}\dfrac{(u(x)-u(y))(v(x)-v(y))}{|x-y|^{n+2s}}\, dxdy \end{aligned}$$

for any $u,v \in \mathbb {X}(\Omega )$. We say that u and v are $\mathcal {B}$-orthogonal if

$$\begin{aligned} \mathcal {B}_{\alpha }(u,v)=0. \end{aligned}$$

The terminology adopted above is justified by the fact that, for $\alpha >0$, the bilinear form $\mathcal {B}_{\alpha }$ becomes a true scalar product.

We conclude this section dealing with the eigenvalue problem associated to the operator $\mathcal {L}_{\alpha }$, that is the following boundary value problem

$$\begin{aligned} \left\{ \begin{array}{ll} \mathcal {L}_{\alpha }u = \lambda u\,, &{} \text {in } \Omega \\ u= 0\,, &{} \text {in } \mathbb {R}^n \setminus \Omega \end{array}\right. \end{aligned}$$

(2.2)

where $\lambda \in \mathbb {R}$. According to Definition 1, we give the following definition.

Definition 3

A number $\lambda \in \mathbb {R}$ is called a (variational) eigenvalue of $\mathcal {L}_{\alpha }$ if there exists a weak solution $u\in \mathbb {X}(\Omega )$ of (2.2) or, equivalently, if

$$\begin{aligned} \int _{\Omega }\langle \nabla u, \nabla \varphi \rangle \, dx +\alpha \iint _{\mathbb {R}^{2n}}\dfrac{(u(x)-u(y))(\varphi (x)-\varphi (y))}{|x-y|^{n+2s}}\, dxdy =\lambda \int _{\Omega }u\varphi \, dx \end{aligned}$$

for every $\varphi \in \mathbb {X}(\Omega )$. If such function u exists, we call it eigenfunction corresponding to the eigenvalue $\lambda$.

The next result permits a complete description of the eigenvalues and related eigenfunctions of $\mathcal {L}_{\alpha }$.

Proposition 1

The following statements hold true:

• (a) $\mathcal {L}_{\alpha }$ admits a divergent, but bounded from below, sequence of eigenvalues $\{\lambda _k\}_{k\in \mathbb {N}}$, i.e., there exists $C>0$ such that

  $$\begin{aligned} -C < \lambda _1 \le \lambda _2 \le \ldots \le \lambda _k \rightarrow +\infty \,, \quad \text {as } k \rightarrow +\infty . \end{aligned}$$

  Moreover, for every $k\in \mathbb {N}$, $\lambda _{k}$ can be characterized as

  $$\begin{aligned} \lambda _{k} = \min _{{\mathop {\Vert u\Vert _{L^{2}(\Omega )}=1}\limits ^{u \in \mathbb {P}_{k}}}} \left\{ \int _{\Omega }|\nabla u|^2 \, dx +\alpha \iint _{\mathbb {R}^{2n}}\dfrac{|u(x)-u(y)|^2}{|x-y|^{n+2s}}\, dxdy \right\} , \end{aligned}$$

  (2.3)

  where

  $$\begin{aligned} \mathbb {P}_{1} := \mathbb {X}(\Omega ), \end{aligned}$$

  and, for every $k \ge 2$,

  $$\begin{aligned} \mathbb {P}_{k} := \left\{ u \in \mathbb {X}(\Omega ): \mathcal {B}_{\alpha }(u, u_j)=0 \, \text { for every } j = 1, \ldots ,k-1\right\} ; \end{aligned}$$

• (b) for every $k\in \mathbb {N}$ there exists an eigenfunction $u_{k} \in \mathbb {X}(\Omega )$ corresponding to $\lambda _{k}$, which realizes the minimum in (2.3);

• (c) the sequence $\{u_k\}_{k\in \mathbb {N}}$ of eigenfunctions constitutes an orthonormal basis of $L^{2}(\Omega )$. Moreover, the eigenfunctions are $\mathcal {B}$-orthogonal;

• (d) for every $k\in \mathbb {N}$, $\lambda _k$ has finite multiplicity.

Proof

If $\alpha =0$ the result is the classical spectral theorem for the Laplace operator (see e.g. [11]). As already mentioned in the Introduction, the case $\alpha >-\tfrac{1}{C}$ (with $C>0$ being the constant of the continuous embedding $H^1_0 \subset H^s$) is also pretty standard. Therefore, in what follows we assume $\alpha \le -\tfrac{1}{C}$.

By [12, Theorem 1], there exists a positive constant C such that

$$\begin{aligned}{}[u]^2_s\le C\Vert u\Vert ^{2(1-s)}_{L^2(\Omega )}\Vert u\Vert ^{2s}_{H^1(\Omega )} \end{aligned}$$

for any $u\in \mathbb {X}(\Omega )$. We remind that the optimal constant C can be explicitly computed with the help of Fourier transform (see e.g. [15]).

We now combine the previous interpolation estimate with the Young inequality (with exponents $\frac{1}{s}$ and $\frac{1}{1-s}$), i.e. for any $\varepsilon \in \mathbb {R}^+$ there exist positive constants $c_{\varepsilon },c_1,c_2$, depending on s and $\varepsilon$, such that

$$\begin{aligned} \begin{aligned} |\alpha |[u]^2_s&\le |\alpha |C\left( (1-s)c_\varepsilon \Vert u\Vert ^2_{L^2(\Omega )}+s\varepsilon \Vert u\Vert ^2_{H^1(\Omega )}\right) \\&=|\alpha |c_1\varepsilon \Vert u\Vert ^2_{\mathbb {X}(\Omega )}+|\alpha |c_2\Vert u\Vert ^2_{L^2(\Omega )} \end{aligned} \end{aligned}$$

for any $u\in \mathbb {X}(\Omega )$. Therefore, by choosing $\varepsilon =\frac{1}{2c_1|\alpha |}$, we get

$$\begin{aligned} |\alpha |[u]^2_s\le \frac{1}{2}\Vert u\Vert ^2_{\mathbb {X}(\Omega )}+\gamma \Vert u\Vert ^2_{L^2(\Omega )} \end{aligned}$$

for any $u\in \mathbb {X}(\Omega )$, where $\gamma$ only depends on $s,\varepsilon$ and $\alpha$, and so

$$\begin{aligned} \mathcal {B}_\alpha (u,u)+\gamma \Vert u\Vert ^2_{L^2(\Omega )}\ge \frac{1}{2}\Vert u\Vert ^2_{\mathbb {X}(\Omega )}\quad \text {for any }u\in \mathbb {X}(\Omega )\,. \end{aligned}$$

(2.4)

At this point, in a standard fashion (for instance, see [24, Chapter 6]) one can prove the existence of an increasing sequence of eigenvalues $\{\lambda _k\}_k$ of $\mathcal {L}_{\alpha }$, with $\lambda _k\rightarrow \infty$ as $k\rightarrow \infty$, such that every $\lambda _k$ has finite multiplicity and is given by the variational characterization in (2.3). Moreover, if $e_k$ is the eigenfunction associated to $\lambda _k$, we have that $\{e_k\}_k$ is an orthonormal basis of $L^{2}(\Omega )$ with

$$\begin{aligned} \mathcal {B}_\alpha (e_k,e_j)=\lambda _k(e_k,e_j)_{L^{2}(\Omega )}=0 \quad \text {for every }j\ne k\,, \end{aligned}$$

that is, $e_k$ and $e_j$ are also $\mathcal {B}_\alpha$-orthogonal. $\square$

By [25, Theorem 5.2.4] and Proposition 1 (a), we can also infer the existence of a positive integer $N_0 \in \mathbb {N}$ such that $\lambda _{N_0}$ is the first (not necessarily simple) positive eigenvalue. Of course, $\lambda _k >0$ for every $k > N_0$.

We further notice that

$$\begin{aligned} \lambda _{k+1} \int _{\Omega }u^2 \, dx \le \int _{\Omega }|\nabla u|^2 \, dx +\alpha \iint _{\mathbb {R}^{2n}}\dfrac{|u(x)-u(y)|^2}{|x-y|^{n+2s}}\, dxdy \end{aligned}$$

(2.5)

for every $u \in \text {span}(u_1, \ldots , u_k)^{\perp } = \mathbb {P}_{k+1}$ and

$$\begin{aligned} \int _{\Omega }|\nabla u|^2 \, dx +\alpha \iint _{\mathbb {R}^{2n}}\dfrac{|u(x)-u(y)|^2}{|x-y|^{n+2s}}\, dxdy \le \lambda _k \int _{\Omega }u^2 \, dx \end{aligned}$$

(2.6)

for every $u \in \text {span}(u_1, \ldots , u_k)=: H_k$.

While (2.5) directly follows from the variational characterization (2.3), the latter (2.6) can be proved as follows: by assumption, let $u = \sum _{i=1}^{k}c_i u_i$. Then,

$$\begin{aligned} \begin{aligned} \mathcal {B}_{\alpha }(u,u)&=\mathcal {B}_{\alpha }(\sum _{i=1}^{k}c_i u_i, \sum _{j=1}^{k}c_j u_j)= \sum _{i,j=1}^{k}c_i c_j \mathcal {B}_{\alpha }(u_i,u_j) =\sum _{i=1}^{k}c_i^2 \mathcal {B}_{\alpha }(u_i,u_i) \\&= \sum _{i=1}^{k}c_i^2 \lambda _i \int _{\Omega }u_i^2 \, dx \le \lambda _k \int _{\Omega }u^2 \, dx. \end{aligned} \end{aligned}$$

3 The asymptotically linear case

In this section we prove Theorem 1.

Proof of Theorem 1

Case 1: ${\overline{v}}(x)<\lambda _{1}$ for a.e. $x \in \Omega$.

We claim that

$$\begin{aligned} \liminf _{\Vert u\Vert _{\mathbb {X}}\rightarrow +\infty }\dfrac{\mathcal {J}(u)}{\Vert u\Vert ^2_{\mathbb {X}}} >0\,. \end{aligned}$$

(3.1)

Once this is established, we have that functional $\mathcal {J}$ is coercive and sequentially weakly lower semicontinuous in $\mathbb {X}(\Omega )$, since

• the map $u \mapsto \Vert u\Vert _{\mathbb {X}}$ is sequentially weakly l.s.c. in $\mathbb {X}(\Omega )$ (being a norm);

• the map $u \mapsto [u]_s$ is continuous, because $\mathbb {X}(\Omega )$ compactly embeds in $H^s_0(\Omega )$;

• the map $u\mapsto \int _{\Omega }F(x,u)\, dx$ is continuous by (1.2).

By the very definition of ${\overline{v}}$, for every $\varepsilon >0$ there exists $R>0$ such that

$$\begin{aligned} f(x,\sigma ) < ({\overline{v}}(x) + \varepsilon ) \sigma \quad \text {for every } |\sigma |>R\,. \end{aligned}$$

(3.2)

On the other hand, assumption (1.2) readily gives that

$$\begin{aligned} f(x,\sigma ) \le a(x) + bR \quad \text {for every } |\sigma |\le R\,. \end{aligned}$$

(3.3)

Combining (3.2) with (3.3), we then get that

$$\begin{aligned} \limsup _{|\sigma |\rightarrow +\infty } \dfrac{F(x,\sigma )}{\sigma ^2} \le \dfrac{{\overline{v}}(x)+\varepsilon }{2} \quad \mathrm{for ~every}~ \varepsilon >0 \end{aligned}$$

and, passing to the limit as $\varepsilon \rightarrow 0^+$, we finally get that

$$\begin{aligned} \limsup _{|\sigma |\rightarrow +\infty } \dfrac{F(x,\sigma )}{\sigma ^2} \le \dfrac{{\overline{v}}(x)}{2}\,. \end{aligned}$$

(3.4)

Let us now proceed with the proof of (3.1). We take a sequence $\{u_j\}_j \subset \mathbb {X}(\Omega )$ such that $\Vert u_j\Vert _{\mathbb {X}} \rightarrow +\infty$ as $j \rightarrow +\infty$ and we define the normalized sequence $w_j := \tfrac{u_j}{\Vert u_j\Vert _{\mathbb {X}}}$. Then, possibly passing to a subsequence, we can assume the existence of a function $u_0 \in \mathbb {X}(\Omega )$ such that $w_j \rightarrow u_0$ weakly in $\mathbb {X}(\Omega )$, strongly in $L^{2}(\Omega )$ and a.e. in $\Omega$. Moreover, $\Vert u_0\Vert _{\mathbb {X}}\le 1$ and, by (1.2), it holds that

$$\begin{aligned} \dfrac{F(x,u_j)}{\Vert u_j\Vert ^{2}_{\mathbb {X}}} \le \dfrac{a(x)|u_j| + b \tfrac{|u_j|^2}{2}}{\Vert u_j\Vert ^{2}_{\mathbb {X}}} \rightarrow \dfrac{b}{2}u_{0}^{2}(x) \quad \text {in } L^{1}(\Omega )\,. \end{aligned}$$

Now, write

$$\begin{aligned} \Omega = \left\{ x\in \Omega : u_j(x) \text { is bounded}\right\} \cup \left\{ x\in \Omega : |u_j(x)|\rightarrow +\infty \right\} =: \Omega _1 \cap \Omega _2\,. \end{aligned}$$

Of course, if $x \in \Omega _1$, then

$$\begin{aligned} \lim _{j\rightarrow +\infty }\dfrac{F(x,u_j)}{\Vert u_j\Vert ^2_{\mathbb {X}}} = 0 \end{aligned}$$

(3.5)

while, if $x\in \Omega _2$, thanks to (3.4), we get

$$\begin{aligned} \limsup _{j\rightarrow +\infty }\dfrac{F(x,u_j)}{\Vert u_j\Vert ^2_{\mathbb {X}}} = \limsup _{j\rightarrow +\infty } \dfrac{F(x,u_j)}{u_j^{2}(x)} \, \dfrac{u_j^{2}(x)}{\Vert u_j\Vert _{\mathbb {X}}^2} \le \dfrac{{\overline{v}}(x)}{2}u_0^{2}(x)\,. \end{aligned}$$

(3.6)

By the generalized Fatou Lemma, combined with (3.5) and (3.6), we have that

$$\begin{aligned} \limsup _{j\rightarrow +\infty }\int _{\Omega }\dfrac{F(x,u_j)}{\Vert u_j\Vert ^2_{\mathbb {X}}}\, dx \le \int _{\Omega }\limsup _{j\rightarrow +\infty } \dfrac{F(x,u_j)}{\Vert u_j\Vert ^2_{\mathbb {X}}}\, dx \le \int _{\Omega }\dfrac{{\overline{v}}(x)}{2}u_0^{2}(x)\, dx\,. \end{aligned}$$

(3.7)

On the other hand, by the definition of $\lambda _1$, we have

$$\begin{aligned} \begin{aligned} \dfrac{1}{\Vert u_j\Vert _{\mathbb {X}}^2}&\left( \dfrac{1}{2}\int _{\Omega }|\nabla u_j|^2 \, dx + \dfrac{\alpha }{2}\iint _{\mathbb {R}^{2n}}\dfrac{|u_j(x)-u_j(y)|^2}{|x-y|^{n+2s}}\, dxdy \right) \\&\ge \dfrac{\lambda _1}{2}\int _{\Omega }\dfrac{u_j^2(x)}{\Vert u_j\Vert ^2_{\mathbb {X}}}\, dx \rightarrow \dfrac{\lambda _1}{2}\int _{\Omega }u_0^2(x) \, dx\,, \quad \text {as} j \rightarrow +\infty . \end{aligned} \end{aligned}$$

(3.8)

To prove the validity of (3.1), we have to consider two possible situations: either $u_0 \ne 0$ or $u_0 = 0$ a.e. in $\Omega$. If the first happens to be true, then we combine (3.7) and (3.8) with the standing assumption ${\overline{v}}<\lambda _1$ (a.e. in $\Omega$), getting that

$$\begin{aligned} \liminf _{j\rightarrow +\infty }\dfrac{\mathcal {J}(u_j)}{\Vert u_j\Vert ^2_{\mathbb {X}}} \ge \int _{\Omega }\left( \dfrac{\lambda _1}{2} -\dfrac{{\overline{v}}(x)}{2}\right) u_0^2(x)\,dx >0\,. \end{aligned}$$

On the contrary, if $u_0 = 0$ a.e. in $\Omega$, we notice that, by compactness,

$$\begin{aligned} \dfrac{1}{\Vert u_j\Vert ^2_{\mathbb {X}}} \iint _{\mathbb {R}^{2n}}\dfrac{|u_j(x)-u_j(y)|^2}{|x-y|^{n+2s}}\, dxdy \rightarrow 0\,, \quad \text {as }j \rightarrow +\infty \,, \end{aligned}$$

and therefore we have that

$$\begin{aligned} \liminf _{j\rightarrow +\infty }\dfrac{\mathcal {J}(u_j)}{\Vert u_j\Vert ^2_{\mathbb {X}}} \ge \dfrac{1}{2}\,. \end{aligned}$$

In any case, (3.1) holds and an application of the Weierstrass Theorem closes the proof of Case 1.

Case 2: there exists $k \in \mathbb {N}$ such that $\lambda _k< {\underline{v}}(x) \le {\overline{v}}(x) < \lambda _{k+1}$ for a.e. $x \in \Omega$.

Note that, reasoning as in the proof of (3.4) in Case 1, we have that

$$\begin{aligned} \liminf _{|\sigma |\rightarrow +\infty } \dfrac{F(x,\sigma )}{\sigma ^2} \ge \dfrac{{\underline{v}}(x)}{2}\, \quad \text {for a.e. }x \in \Omega \,. \end{aligned}$$

(3.9)

Take a sequence $\{u_j\}_j \subset H_k$ (recall its definition in (2.6)) such that $\Vert u_j\Vert _{\mathbb {X}} \rightarrow +\infty$ as $j \rightarrow +\infty$. Since $H_k$ is finite dimensional, we can infer the existence of a function $u_0 \in H_k$, with $\Vert u_0\Vert _{\mathbb {X}} =1$, and such that the normalized sequence $\tfrac{u_j}{\Vert u_j\Vert _{\mathbb {X}}} \rightarrow u_0$ strongly in $\mathbb {X}(\Omega )$, strongly in $L^2(\Omega )$ and a.e. in $\Omega$. Exploiting now (2.6), combined with the standing assumption on $\lambda _k$ and (3.9), we get

$$\begin{aligned} \limsup _{j \rightarrow +\infty }\dfrac{\mathcal {J}(u_j)}{\Vert u_j\Vert ^2_{\mathbb {X}}} \le \int _{\Omega }\left( \frac{\lambda _k}{2} - \dfrac{{\underline{v}}(x)}{2}\right) u_0^2(x) \,dx < 0\,, \end{aligned}$$

which implies that

$$\begin{aligned} \limsup _{u \in H_k,\, \Vert u\Vert _{\mathbb {X}}\rightarrow +\infty }\dfrac{\mathcal {J}(u)}{\Vert u\Vert ^2_{\mathbb {X}}} <0\,. \end{aligned}$$

(3.10)

Moreover, once again mimicking the argument adopted in Case 1, we have that

$$\begin{aligned} \liminf _{u \in \mathbb {P}_{k+1}, \, \Vert u\Vert _{\mathbb {X}}\rightarrow +\infty }\dfrac{\mathcal {J}(u)}{\Vert u\Vert ^2_{\mathbb {X}}} >0\,. \end{aligned}$$

(3.11)

It follows from (3.11) that for any positive constant $M >0$, there exists a positive constant $R>0$ such that if $u \in \mathbb {P}_{k+1}$ with $\Vert u\Vert _{\mathbb {X}} \ge R$, then $\mathcal {J}(u)\ge M$. On the other hand, if $u \in \mathbb {P}_{k+1}$ with $\Vert u\Vert _{\mathbb {X}} \le R$, a direct application of (1.2), (2.5), Hölder’s and Poincaré’s inequalities gives

$$\begin{aligned} \mathcal {J}(u) \ge \frac{\lambda _{k+1}}{2}\Vert u\Vert ^2_{L^{2}(\Omega )} - \Vert a\Vert _{L^{2}(\Omega )}\Vert u\Vert _{L^{2}(\Omega )} - \dfrac{b}{2}\Vert u\Vert ^2_{L^{2}(\Omega )} \ge -C\,, \end{aligned}$$

where $C=C(R,\Omega ,\Vert a\Vert _{L^2(\Omega )},b)>0$ is a positive constant. Therefore, we have that

$$\begin{aligned} \mathcal {J}(u) \ge M - C, \quad \text {for every}~ u \in \mathbb {P}_{k+1}. \end{aligned}$$

On the other hand, by (3.10), we can choose a positive number $T>0$ such that

$$\begin{aligned} \sup _{u\in H_k, \, \Vert u\Vert _{\mathbb {X}}=T}\mathcal {J}(u) < M - C\,. \end{aligned}$$

It readily implies that

$$\begin{aligned} \sup _{u\in H_k, \, \Vert u\Vert _{\mathbb {X}}=T}\mathcal {J}(u) < M - C \le \inf _{u \in \mathbb {P}_{k+1}}\mathcal {J}(u)\,, \end{aligned}$$

which in turn proves the validity of the topological requirements to apply the Saddle Point Theorem.

It now remains to show the validity of the Palais-Smale condition. Since the space $\mathbb {X}(\Omega )$ compactly embeds in $L^{2}(\Omega )$, it is enough to prove that the Palais-Smale sequences are bounded. Arguing by contradiction, assume that the sequence $\{u_j\}_j$ is unbounded, define the normalized sequence $w_j := \tfrac{u_j}{\Vert u_j\Vert _{\mathbb {X}}}$ and assume the existence of $u_0 \in \mathbb {X}(\Omega )$ such that $w_j \rightarrow u_0$ (up to subsequences) as $j\rightarrow +\infty$, weakly in $\mathbb {X}(\Omega )$, strongly in $L^{2}(\Omega )$ and a.e. in $\Omega$. Since $\{\mathcal {J}'(u_j)\}_j$ is bounded, we can infer the existence of a positive constant $M>0$ such that

$$\begin{aligned} \dfrac{|\mathcal {J}'(u_j)(\varphi )|}{\Vert u_j\Vert _{\mathbb {X}}} = \left| \mathcal {B}_{\alpha }(w_j,\varphi ) - \int _{\Omega }\dfrac{f(x,u_j)}{\Vert u_j\Vert _{\mathbb {X}}}\varphi \, dx \right| \le M \dfrac{\Vert \varphi \Vert _{\mathbb {X}}}{\Vert u_{j}\Vert _{\mathbb {X}}} \quad \text {for any } \varphi \in \mathbb {X}(\Omega )\,.\nonumber \\ \end{aligned}$$

(3.12)

Recalling the standing assumption (1.2) on f, we have that

$$\begin{aligned} \dfrac{|f(x,u_j)|}{\Vert u_j\Vert _{\mathbb {X}}} \le \dfrac{a(x)}{\Vert u_j\Vert _{\mathbb {X}}} + b \dfrac{|u_j|}{\Vert u_j\Vert _{\mathbb {X}}}\,, \end{aligned}$$

(3.13)

and we notice that the r.h.s. of (3.13) is bounded in $L^{2}(\Omega )$. Therefore, there exists $\beta \in L^{2}(\Omega )$ such that (up to subsequences)

$$\begin{aligned} \frac{f(x,u_j)}{\Vert u_j\Vert _{\mathbb {X}}}\quad \text {weakly converges to }\beta \text { in }L^{2}(\Omega )\,,\text { as }j\rightarrow +\infty \,. \end{aligned}$$

Claim A: there exists a measurable function $m:\Omega \rightarrow \mathbb {R}$ such that

• (i) $\beta (x) = m(x) u_0(x)$ for a.e. $x \in \Omega$;

• (ii) ${\underline{v}}(x) \le m(x)\le {\overline{v}}(x)$ for a.e. $x \in \Omega$.

In order to prove the claim, we first notice that, if $x\in \Omega$ is such that $u_0(x)> 0$, then $u_j = w_j \Vert u_j\Vert _{\mathbb {X}}\rightarrow +\infty$ and then, recalling (1.3),

$$\begin{aligned} \liminf _{j\rightarrow +\infty }\dfrac{f(x,u_j)}{\Vert u_j\Vert _{\mathbb {X}}}= \liminf _{j\rightarrow +\infty }\dfrac{f(x,u_j)}{u_j}\dfrac{u_j}{\Vert u_j\Vert _{\mathbb {X}}} \ge {\underline{v}}(x)u_0(x)\,, \end{aligned}$$

while

$$\begin{aligned} \limsup _{j\rightarrow +\infty }\dfrac{f(x,u_j)}{\Vert u_j\Vert _{\mathbb {X}}}= \limsup _{j\rightarrow +\infty }\dfrac{f(x,u_j)}{u_j}\dfrac{u_j}{\Vert u_j\Vert _{\mathbb {X}}} \le {\overline{v}}(x)u_0(x)\,. \end{aligned}$$

If $x\in \Omega$ is such that $u_0(x)<0$, both inequalities are reversed. Now, recall that, if $\{v_j\}_j$ weakly converges to v in $L^2(\Omega )$, $g_j\le v_j$ and $g_j$ converges to g strongly in $L^2(\Omega )$ and a.e. in $\Omega$, then $g\le v$ a.e. in $\Omega$. In this way, setting

$$\begin{aligned} m(x) :={\left\{ \begin{array}{ll} \dfrac{\beta (x)}{u_0(x)}\,, &{} \text { if } u_0(x)\ne 0\\ \dfrac{{\underline{v}}(x)+{\overline{v}}(x)}{2}\,, &{} \text { if } u_0(x) =0 \end{array}\right. } \end{aligned}$$

we complete the proof of (ii), and thus of Claim A, since, if $u_0(x)=0$, then from (3.13) we have

$$\begin{aligned} \dfrac{|f(x,u_j)|}{\Vert u_j\Vert _{\mathbb {X}}} \le \dfrac{a(x)}{\Vert u_j\Vert _{\mathbb {X}}} + b \dfrac{|u_j|}{\Vert u_j\Vert _{\mathbb {X}}} \rightarrow 0\,, \quad \text {as } j\rightarrow +\infty \,. \end{aligned}$$

Then, by passing to the limit in (3.12), we get

$$\begin{aligned} \mathcal {B}_{\alpha }(u_0, \varphi ) - \int _{\Omega }m(x)u_0(x)\varphi (x)\, dx = 0 \quad \text {for every } \varphi \in \mathbb {X}(\Omega )\,. \end{aligned}$$

(3.14)

Claim B: $u_0 =0$ a.e. in $\Omega$. Since $u_0 \in \mathbb {X}(\Omega )$, we can write

$$\begin{aligned} u_0 = u_1 + u_2, \quad u_1 \in H_k, \, u_2 \in \mathbb {P}_{k+1}. \end{aligned}$$

Now, take $\varphi =u_1$ and then $\varphi =u_2$ in (3.14). Comparing, we get

$$\begin{aligned} \mathcal {B}_{\alpha }(u_1,u_1) - \int _{\Omega }m(x)u_1^2(x)\, dx = \mathcal {B}_{\alpha }(u_2,u_2) - \int _{\Omega }m(x)u_2^2(x)\, dx \,. \end{aligned}$$

(3.15)

Keeping in mind (2.5) and (2.6), from (3.15) we find

$$\begin{aligned} \begin{aligned} 0&\ge \int _{\Omega }(\lambda _k - m(x))u_1^2(x)\, dx \ge \mathcal {B}_{\alpha }(u_1,u_1) - \int _{\Omega }m(x)u_1^2(x)\, dx \\&= \mathcal {B}_{\alpha }(u_2,u_2) - \int _{\Omega }m(x)u_2^2(x)\, dx \ge \int _{\Omega }(\lambda _{k+1}-m(x))u_2^2(x) \, dx \ge 0\,, \end{aligned} \end{aligned}$$

but this is impossible unless $u_1 = u_2 =0$ a.e. in $\Omega$, and this proves Claim B.

We now test (3.12) with $\varphi = w_j$, getting

$$\begin{aligned} \left| 1 + \alpha \dfrac{[u_j]_s^2}{\Vert u_j\Vert ^2_{\mathbb {X}}} - \int _{\Omega }\dfrac{f(x,u_j)}{u_{j}}\dfrac{u_j}{\Vert u_j\Vert _{\mathbb {X}}}\right| \le \dfrac{M}{\Vert u_j\Vert _{\mathbb {X}}}\quad \text {for every }j\in \mathbb {N}\,. \end{aligned}$$

Passing to the limit as $j\rightarrow +\infty$ we finally reach the contradiction ”1=0" and this closes the proof. $\square$

4 The superlinear and subcritical case

In this section we prove Theorem 2. We first need the following preliminary result inspired by Rabinowitz [36].

Lemma 1

Let $k\in \mathbb {N}$ be such that

$$\begin{aligned} \lambda _{k}\le {\underline{\Theta }}(x)\le {\overline{\Theta }}(x)<\lambda _{k+1}\quad \text {for a.e. }x\in \Omega \,, \end{aligned}$$

and decompose the space $\mathbb {X}(\Omega )$ as $\mathbb {X}(\Omega )=H_k \oplus \mathbb {P}_{k+1}$, where $H_k :=\text {span}(u_1, \ldots , u_{k})$. Then, there exists a positive constant $\beta$ such that

$$\begin{aligned} \mathcal {B}_\alpha (u,u)-\int _\Omega {\overline{\Theta }}(x)u^2dx\ge \beta \Vert u\Vert _{\mathbb {X}(\Omega )}^2\quad \text {for any }u\in \mathbb {P}_{k+1} \end{aligned}$$

(4.1)

or, equivalently,

$$\begin{aligned} \inf _{u\in \mathbb {P}_{k+1} \setminus \{0\}}\left\{ 1+\frac{\alpha [u]_s^2-\int _\Omega {\overline{\Theta }}(x)u^2dx}{\Vert u\Vert _{\mathbb {X}(\Omega )}^2}\right\} >0\,. \end{aligned}$$

Proof

By contradiction, assume the existence of a sequence $\{u_n\}_n\subset \mathbb {P}_{k+1} \setminus \{0\}$ satisfying

$$\begin{aligned} 1+\frac{\alpha [u_n]_s^2-\int _\Omega {\overline{\Theta }}(x)u_n^2dx}{\Vert u_n\Vert _{\mathbb {X}(\Omega )}^2}\le 0\quad \text {for any }n\in \mathbb {N} \end{aligned}$$

and denote $v_n=\frac{u_n}{\Vert u_n\Vert _{\mathbb {X}(\Omega )}}$ for any $n\in \mathbb {N}$. Then, the sequence $\{v_n\}_n$ is bounded in $\mathbb {P}_{k+1}$ and, by reflexivity, weakly convergent (up to subsequences) to v in $\mathbb {P}_{k+1}$. Therefore, up to a further subsequence, $\{v_n\}_n$ strongly converges to v in $L^2(\Omega )$ and in $H^s(\mathbb {R}^n)$, and so

$$\begin{aligned} 1+\alpha [v_n]_s^2-\int _\Omega {\overline{\Theta }}(x)v_n^2dx\rightarrow 1+\alpha [v]_s^2-\int _\Omega {\overline{\Theta }}(x)v^2dx\le 0. \end{aligned}$$

(4.2)

Therefore, by (2.5) and (4.2),

$$\begin{aligned} 0\le \int _\Omega (\lambda _{k+1}-{\overline{\Theta }}(x))v^2dx\le \mathcal {B}_\alpha (v,v)-\int _\Omega {\overline{\Theta }}(x)v^2dx\le 0. \end{aligned}$$

Since ${\overline{\Theta }}<\lambda _{k+1}$ a.e. in $\Omega$, we get $v=0$ and so, by (4.2)

$$\begin{aligned} 1=1+\alpha [v]_s^2-\int _\Omega {\overline{\Theta }}(x)v^2dx\le 0\,, \end{aligned}$$

which yields a contradiction. $\square$

Proof of Theorem 2

We decompose $\mathbb {X}(\Omega )=H_k \oplus \mathbb {P}_{k+1}$, and we show the existence of $\varrho ,{\tilde{\alpha }}>0$ such that

$$\begin{aligned} \inf _{u\in S_\varrho \cap \mathbb {P}_{k+1}}\mathcal {J}(u)={\tilde{\alpha }}\,. \end{aligned}$$

We first we claim that for any $\varepsilon >0$ there exists $C_\varepsilon >0$ such that

$$\begin{aligned} F(x,t)\le \frac{1}{2}({\overline{\Theta }}(x)+\varepsilon )t^2+C_\varepsilon |t|^r\quad \text {a.e. }x\in \Omega \,\,\text {and for all }t\in \mathbb {R}\,. \end{aligned}$$

(4.3)

Indeed, if we fix $\varepsilon >0$, by (1.4), there exists $\delta =\delta (\varepsilon )>0$ such that

$$\begin{aligned} F(x,t)\le \frac{1}{2}({\overline{\Theta }}(x)+\varepsilon )t^2\quad \text {a.e. }x\in \Omega \text { for all }|t|\le \delta \,. \end{aligned}$$

(4.4)

Moreover, by (ii)

$$\begin{aligned} F(x,t) \le \left| \int _0^t(a(x)+b|\sigma |^{r-1})\,d\sigma \right| \le a(x)|t|+b\frac{|t|^r}{r}\quad \text {a.e. }x\in \Omega \text { for all }t\in \mathbb {R}\,.\nonumber \\ \end{aligned}$$

(4.5)

Combining (4.4) (for $|t|\le \delta$) and (4.5) (for $|t|\ge \delta$), we finally get

$$\begin{aligned} F(x,t)&\le \frac{1}{2}({\overline{\Theta }}(x)+\varepsilon )t^2+a(x)|t|\frac{|t|^{r-1}}{|t|^{r-1}}+\frac{b}{r}|t|^r\\&\le \frac{1}{2}({\overline{\Theta }}(x)+\varepsilon )t^2+(\frac{a(x)}{\delta ^{r-1}}+\frac{b}{r})|t|^r\\&\le \frac{1}{2}({\overline{\Theta }}(x)+\varepsilon )t^2+C_\varepsilon |t|^r\quad \text {a.e. }x\in \Omega \text { for all }t\in \mathbb {R}\,. \end{aligned}$$

Notice that, by (4.3) and the Sobolev inequality, we also have

$$\begin{aligned} \begin{aligned} \int _\Omega F(x,u)\,dx&\le \frac{1}{2}\int _\Omega ({\overline{\Theta }}(x)+\varepsilon )|u|^2\,dx+C_\varepsilon \int _\Omega |u|^r\,dx\\&\le \frac{1}{2}\int _\Omega ({\overline{\Theta }}(x)+\varepsilon )|u|^2\,dx+\overline{C_\varepsilon }\Vert u\Vert _{\mathbb {X}}^r \end{aligned} \end{aligned}$$

(4.6)

for any $u\in \mathbb {X}(\Omega )$.

Now, take $u\in \mathbb {P}_{k+1}$. Then, by (4.1), (4.6) and the Poincaré inequality

$$\begin{aligned} \mathcal {J}(u)&=\frac{1}{2}\mathcal {B}_\alpha (u,u)-\int _\Omega F(x,u)\,dx\\&\ge \frac{1}{2}\mathcal {B}_\alpha (u,u)-\frac{1}{2}\int _\Omega ({\overline{\Theta }}(x)+\varepsilon )|u|^2\,dx-\overline{C_\varepsilon }\Vert u\Vert _{\mathbb {X}}^{r}\\&\ge \frac{1}{2}\beta \Vert u\Vert _{\mathbb {X}}^2-\frac{\varepsilon }{2}\Vert u\Vert _{L^2(\Omega )}^2-\overline{C_\varepsilon }\Vert u\Vert _{\mathbb {X}}^{r}\\&\ge \left( \frac{1}{2}\beta -\frac{\varepsilon }{2\mu _1} - \overline{C_\varepsilon }\Vert u\Vert _{\mathbb {X}}^{r-2}\right) \Vert u\Vert _{\mathbb {X}}^2\,, \end{aligned}$$

where $\mu _1$ denotes the first eigenvalue of $-\Delta$ in $\Omega$. Now we choose $\varepsilon$ and $\varrho >0$ so small that

$$\begin{aligned} \inf _{u\in S_\varrho \cap \mathbb {P}_{k+1}}\mathcal {J}(u)={\tilde{\alpha }}>0\,. \end{aligned}$$

To verify that $\mathcal {J}$ satisfies the geometric condition of the Linking theorem, we show the existence of a radius $\rho >\varrho$ such that

$$\begin{aligned} \sup _{u\in \partial _{H_k \oplus \text {span}(u_{k+1})}\Delta }\mathcal {J}(u)\le 0\,, \end{aligned}$$

where $\Delta :=({\overline{B}}_\rho \cap H_k)\oplus \{tu_{k+1}:t\in [0,\rho ]\}$.

For this, first let us notice that $\mathcal {J}(u)\le 0$ for any $u\in H_k$, as we easily get by combining (2.6) with the assumption (1.5). Therefore,

$$\begin{aligned} \sup _{ H_k}\mathcal {J}= 0\,. \end{aligned}$$

Moreover, notice that there exists a positive constant $c_1$ (if $\alpha <0$, then $c_1=1$) such that

$$\begin{aligned} \mathcal {B}_\alpha (u,u)\le c_1\Vert u\Vert _{\mathbb {X}}^2\quad \text {for any }u\in \mathbb {X}(\Omega ). \end{aligned}$$

(4.7)

Now take $t>0$ and $u\in X_1$. Then, by (iii) and (4.7) we have

$$\begin{aligned} \begin{aligned} \mathcal {J}(u+tu_{k+1})&=\frac{1}{2}\mathcal {B}_\alpha (u+tu_{k+1},u+tu_{k+1})-\int _\Omega F(x,u+tu_{k+1})\,dx\\&\le \frac{c_1t^2}{2}\Vert \frac{u}{t}+u_{k+1}\Vert _\mathbb {X}^2-ct^{{\tilde{\mu }}}\Vert \frac{u}{t}+u_{k+1}\Vert _{L^{{\tilde{\mu }}}(\Omega )}^{{\tilde{\mu }}}+\Vert d\Vert _{L^1(\Omega )}\rightarrow -\infty \end{aligned} \end{aligned}$$

as $t\rightarrow \infty$, having assumed ${\tilde{\mu }}>2$.

In this way the geometric assumptions of the Linking Theorem are satisfied. Notice that, letting $k=0$, we recover the geometric situation of the Mountain Pass Theorem when ${\overline{\Theta }}(x)<\lambda _1$ a.e. $x \in \Omega$.

We conclude by showing the validity of the Palais-Smale condition, that is, that every Palais-Smale sequence, i.e. every sequence $\{u_j\}_j$ in $\mathbb {X}(\Omega )$ such that $\{\mathcal {J}(u_j)\}_j$ is bounded and $\mathcal {J}'(u_j)\rightarrow 0$ in $\mathbb {X}(\Omega )^{-1}$ as $j\rightarrow \infty$, has a strongly converging subsequence in $\mathbb {X}(\Omega )$.

So, let $\{u_j\}_j\subset \mathbb {X}(\Omega )$ be a Palais-Smale sequence. We first prove that $\{u_j\}_j$ is bounded in $\mathbb {X}(\Omega )$.

Assume, by contradiction, that $\{u_j\}_j$ is unbounded in $\mathbb {X}(\Omega )$. Then, there exists $u\in \mathbb {X}(\Omega )$ such that (up to subsequences)

$$\begin{aligned} \frac{u_j}{\Vert u_j\Vert _\mathbb {X}}\rightarrow u\quad \text {weakly in }\mathbb {X}(\Omega ) \hbox { and strongly in}~ H^s_0(\Omega ), \hbox {in}~ L^{{\tilde{\mu }}}(\Omega )\hbox { and a.e. in}~ \Omega . \end{aligned}$$

Now, take $\eta \in (2,\mu )$. Then, by (ii) and (iii), there exists a positive constant $D_R$ such that

$$\begin{aligned} \begin{aligned} 0\leftarrow \frac{\eta \mathcal {J}(u_j)-\mathcal {J}'(u_j)u_j}{\Vert u_j\Vert _X^2}&=\left( \frac{\eta }{2}-1\right) \frac{\mathcal {B}_\alpha (u_j,u_j)}{\Vert u_j\Vert _\mathbb {X}^2}\\&\quad +\frac{\int _{\{|u_j|> R\}}(f(x,u_j)u_j\pm A(x)u_j^2-\eta F(x,u_j))dx}{\Vert u_j\Vert _\mathbb {X}^2}\\&\quad -\frac{\int _{\{|u_j|\le R\}}(\eta F(x,u_j)-f(x,u_j)u_j)dx}{\Vert u_j\Vert _\mathbb {X}^2}\\&\ge \left( \frac{\eta }{2}-1\right) \left( 1+\alpha \frac{[u_j]_s^2}{\Vert u_j\Vert _\mathbb {X}^2}\right) +\dfrac{(\mu -\eta )}{\Vert u_j\Vert _\mathbb {X}^2}\int _\Omega F(x,u_j)\,dx\\&\quad +\left( 1-\frac{\mu }{2}\right) \frac{\int _\Omega Au_j^2dx}{\Vert u_j\Vert _\mathbb {X}^2}-\frac{D_R}{\Vert u_j\Vert _\mathbb {X}^2}\,. \end{aligned} \end{aligned}$$

(4.8)

By Young’s inequality, for every $\varepsilon >0$, there exists a constant ${\tilde{D}}>0$ such that

$$\begin{aligned} \left| \left( 1-\frac{\mu }{2}\right) \right| \Vert A\Vert _{L^\infty (\Omega )}\Vert u_j\Vert ^2_{L^2(\Omega )}\le \varepsilon \int _\Omega |u_j|^{{\tilde{\mu }}}dx+{\tilde{D}}. \end{aligned}$$

(4.9)

By (4.8), (4.9) and (iii) we get

$$\begin{aligned} \begin{aligned} 0\leftarrow \frac{\eta \mathcal {J}(u_j)-\mathcal {J}'(u_j)u_j}{\Vert u_j\Vert _\mathbb {X}^2}&\ge \left( \frac{\eta }{2}-1\right) \left( 1+\alpha \frac{[u_j]_s^2}{\Vert u_j\Vert _\mathbb {X}^2}\right) \\&\quad +[(\mu -\eta )c-\varepsilon ]\int _\Omega \frac{|u_j|^{{\tilde{\mu }}}}{\Vert u_j\Vert _\mathbb {X}^2}dx+o(1)\,, \end{aligned} \end{aligned}$$

(4.10)

where $o(1)\rightarrow 0$ as $j\rightarrow \infty$. Choosing $\varepsilon <(\mu -\eta )c$, we immediately get that

$$\begin{aligned} 1+\alpha [u]_s^2\le 0, \end{aligned}$$

so that $u\ne 0$ if $\alpha <0$, while it is already a contradiction if $\alpha \ge 0$. On the other hand, if in (4.10) we divide by $\Vert u_j\Vert _\mathbb {X}^{{\tilde{\mu }}-2}$ and pass to the limit, we get $u=0$, which yields a contradiction.

Thus, $\{u_j\}_j\subset \mathbb {X}(\Omega )$ is bounded in $\mathbb {X}(\Omega )$ and, by reflexivity and Rellich’s theorem, there exists $u\in \mathbb {X}(\Omega )$ such that (up to subsequences)

$u_j\rightarrow u$ weakly in $\mathbb {X}(\Omega )$ and strongly in $H^s_0(\Omega )$, in $L^p(\Omega )$ $(p<2^*)$ and a.e. in $\Omega$.

Now, it is standard to prove that $u_j\rightarrow u$ strongly in $\mathbb {X}(\Omega )$.

Hence the Mountain Pass Theorem (if ${\overline{\Theta }}(x)<\lambda _1$) or the Linking Theorem can be applied and Theorem 2 holds. $\square$