1 Introduction

In this paper, we shall study the following problem

$$\begin{aligned} \left\{ \begin{array}{ll}-{\mathcal {L}} u+(-\Delta )^su=\lambda \Vert u\Vert _q^{2-q}|u|^{q-2}u~~~\text {on}~\Omega ,\\ u=0~~~\text {on}~{\mathbb {G}}{\setminus }\Omega , \end{array} \right. \end{aligned}$$
(1.1)

where \(\Omega \subset {\mathbb {G}}\) is a bounded subdomain of a stratified Lie group \({\mathbb {G}}\). We shall further assume the following condition on the exponent q

$$\begin{aligned} \text {(C)}:~~~2<Q,~1<q<2^*, \end{aligned}$$

where \(2^*:=\frac{2Q}{Q-2}\) and Q is the homogeneous dimension of the group \({\mathbb {G}}\). This is a new direction of studying the multiphase eigenvalue problem because the problem is considered on a stratified Lie group.

The interest towards the eigenvalue problems like the one in problem (1.1) is not only restricted to within the mathematical community but is also of interest to physicists since it has a relation with the spectral optimization theory, bifurcation theory, fluid and quantum mechanics (see Lê [22], Lindqvist [23]). Interested readers may note that the study of elliptic PDEs involving the p-Laplacian operator is of interest in the theory of non-Newtonian fluids, both for the case \(p \ge 2\) (dilatant fluids) and the case \(1<p<2\) (pseudo-plastic fluids), see Astarita-Marrucci [3]. It is also of geometrical interest for \(p \ge 2\), some of which is discussed in Uhlenbeck [31].

As far as the nonlocal elliptic problems are concerned, we refer to Zhao et al. [34] who considered a nonlocal elliptic problem driven by a nonlinearity, obeying certain conditions. Saoudi et al. [30] proved the existence, multiplicity and regularity of solutions of a nonlocal elliptic PDE, driven by a singular and a power nonlinearity. The reader may also check Bouabdallah et al. [6], and Zhao et al. [34], to understand the trends in research on nonlocal elliptic PDEs, driven by nonlinearities of various types, which were naturally motivated by the literature in the local case. For the latter, we point out for example, Zeddini [32]. However, since our paper is purely of mathematical interest, we further refer the reader to Sect. 2 for the work due to Dipierro-Valdinoci [12], to learn more about the theory of phase transitions and the associated mathematics involved.

Interesting contributions related to (1.1) can be found in the literature, see e.g., Alves-Covei [1], Corrêa et al. [11]. For example, in Alves and Covei [1], the sub-supersolution method was applied to establish the existence of solutions for

$$\begin{aligned} \left\{ \begin{array}{rcl} -a\left( \int _{\Omega }u\right) \Delta u &{} = &{} h_{1}(x,u)f\left( \int _{\Omega }|u|^{p}\right) +h_{2}(x,u)g\left( \int _{\Omega }|u|^{r}\right) \;\;\text{ on }\;\;\Lambda , \\ u &{} = &{} 0\;\;\text{ on }\;\;\partial \Lambda , \end{array} \right. \end{aligned}$$
(1.2)

where \(\Lambda \subset {\mathbb {R}}^n\) is a bounded smooth domain, and \(a,f,g,h_{i}\) (\(i=1,2\)) are given functions with sufficient regularity. In the case of the \(p-\)Laplacian, Corrêa et al. [11] combined the sub-supersolutions method with a classical theorem due to Rabinowitz [28]. As a result, they were able to prove the existence of solutions for the quasilinear problem

figure a

where \(\alpha \) is a nonnegative function defined on \({\bar{\Omega }}.\) The works of Arora-Rădulescu [2], Bahrouni-Rădulescu [4], Garain et al. [16], Gou-Rădulescu [19], Razani-Behboudi [29] and Zhang-Rădulescu [33], where the multiphase problems were studied, are also of interest here.

We now state the first main results of the paper:

Theorem 1.1

Let \(0<s<1\) and \(1<q<2^*\). Assume that \(\lambda >0\) and \(u\in X{\setminus }\{0\}\) is an eigenvalue and the corresponding eigenfunction. Then u is bounded. Furthermore, if u is nonnegative on \(\Omega \), then \(u>0\) on \(\Omega \). Moreover, for every relatively compact subset \(\omega \subset \Omega ,\) there exists a positive constant \(c(\omega )\) such that \(u\ge c>0\) on \(\omega \).

Our second main result is a Pohozaev-like identity for the Brezis-Nirenberg problem, albeit on a Heisenberg group \({\mathbb {H}}^n,\) which is a particular type of stratified Lie group.

Theorem 1.2

The following Pohozaev-like identity corresponding to (1.1) holds

$$\begin{aligned} \begin{aligned}&\frac{Q}{2}\int _{\Omega }G(u)dx-\left( \frac{Q-2}{2}\right) \int _{\Omega }\left| \nabla _{{\mathbb {H}}^n}u\right| ^2dx-\left( \frac{Q-2s}{2}\right) \int _{\Omega }\left| (-\Delta )^su\right| ^2dx\\&\quad =\frac{1}{2}\int _{\partial \Omega }\left| \nabla _{{\mathbb {H}}^n}u\right| ^2\langle Z,{\hat{n}}\rangle dS. \end{aligned} \end{aligned}$$
(1.3)

We complete the introduction by describing the structure of the paper. In Sect. 2, we review the fundamentals of stratified Lie groups. We also prove an embedding result in the Lie group setup. In Sect. 3, we introduce two key operators A and B,  which are necessary for the proof of the first main result. In Sect. 4, we prove the first main result (Theorem 1.1). In Sect. 5, we establish the second main result (Theorem 1.2). For all fundamental material used in this paper we refer the reader to the comprehensive monograph by Papageorgiou et al. [27].

2 Stratified Lie groups

A quick sneak into the basics of stratified Lie groups may be useful to the reader (see Choudhuri-Repovš [9], Choudhuri et al. [10], Folland et al. [14], Ghosh et al. [18], Montgomery [26], and the references therein). This is why we now recall some fundamental definitions from this topics.

Definition 2.1

A Lie group \({\mathbb {G}}\) on \({\mathbb {R}}^n\), is said to be homogeneous, if for every \(\delta >0\) there exists an automorphism \(T_{\delta }:{\mathbb {G}}\rightarrow {\mathbb {G}}\) defined by

$$\begin{aligned} T_{\delta }(x)=(\delta ^{r_1}x_1,\delta ^{r_2}x_2,\ldots ,\delta ^{r_n}x_n), \ \hbox { for every} \ r_i>0, \ i=1,2,\ldots ,n. \end{aligned}$$

The map \(T_{\delta }\) is called a dilation on \({\mathbb {G}}\). Here, \(x=(x_1,x_2,\ldots ,x_n)\).

The number n represents the topological dimension of \({\mathbb {G}}\), whereas the number \(M=r_1+r_2+\cdots +r_n\) represents the homogeneous dimension of the homogeneous Lie group \({\mathbb {G}}\). We shall denote Haar measure by the symbol dx, which is the standard Lebesgue measure on \({\mathbb {R}}^n\). The following is the definition of a stratified Lie group.

Definition 2.2

A homogeneous Lie group \({\mathbb {G}}=({\mathbb {R}}^n,\cdot )\) is called a stratified Lie group (or a homogeneous Carnot group) if the following two conditions hold:

  1. (i)

    The decomposition \({\mathbb {R}}^n={\mathbb {R}}^{n_1}\times {\mathbb {R}}^{n_2}\times \cdots \times {\mathbb {R}}^{n_k}\) holds for some natural numbers \(n_1,n_2,\ldots ,n_k\) such that \(n_1+n_2+\cdots +n_k=n\). Furthermore, for every \(\delta >0\) there exists a dilation of the form \(T_{\delta }(x)=(\delta ^1 x^{(1)},\delta ^2 x^{(2)},\ldots ,\delta ^k x^{(k)})\) which is an automorphism of the group \({\mathbb {G}}\). Here, \(x^{(i)}\in {\mathbb {R}}^{n_i}\) for every \(i=1,2,\ldots ,k\).

  2. (ii)

    Let \(n_1\) be the same as in the above decomposition of \({\mathbb {R}}^n\), and let \({\mathcal {L}}_1,{\mathcal {L}}_2,\ldots ,{\mathcal {L}}_{n_1}\) be the left invariant vector fields on \({\mathbb {G}}\) such that \({\mathcal {L}}_i(0)=\frac{\partial }{\partial x_i}|_{0}\) for \(i=1,2,\ldots ,n_1\). Then the Hörmander rank condition holds for every \(x\in {\mathbb {R}}^n\), i.e., rank(Lie\(\{{\mathcal {L}}_1,{\mathcal {L}}_2,\ldots ,{\mathcal {L}}_{n_1}\})=n\). Roughly speaking, the Lie algebra corresponding to the Lie group \({\mathbb {G}}\) is spanned by the iterated commutators of \({\mathcal {L}}_1,{\mathcal {L}}_2,\ldots ,{\mathcal {L}}_{n_1}\).

Together, the structure \(({\mathbb {R}}^n,\cdot ,T_{\delta })\) is called a stratified Lie group. The number k is called the step of the homogeneous Carnot group. In the case of a stratified Lie group, the homogeneous dimension becomes

$$\begin{aligned} Q=\sum _{i=1}^{k}in_i \end{aligned}$$

Let \({\mathfrak {G}}\) be a Lie algebra associated to a Lie group \({\mathbb {G}}\). A stratification of \({\mathfrak {G}}\) with step k is a direct sum decomposition

$$\begin{aligned} {\mathfrak {G}}=V_1\oplus V_2\oplus \cdots \oplus V_k \end{aligned}$$

of \({\mathfrak {G}}\) with the property that \(V_k\ne \{0\}\) and \([V_2,V_j]=V_{j+1}\), for every \(j=1,2,\ldots ,k\), where we set \(V_{k+1}=\{0\}\). Here \([V,W]:=\text {span}\{[X,Y]:X\in V, Y\in W\}\).

Throughout the paper, we set \(n=n_1\) in Definition 2.2. The operators \({\mathcal {L}}\) and \((-\Delta )^s\) are the Laplacian and the fractional Laplacian, respectively that are defined as follows:

$$\begin{aligned} \begin{aligned} {\mathcal {L}}v&:=\sum _{i=1}^{n_1}X_i^2v,\\ (-\Delta )^sv&:=P.V.\int _{{\mathbb {G}}}|y^{-1}\cdot x|^{Q+2s}dy, \end{aligned} \end{aligned}$$
(2.1)

where for every i the vector field \(X_i\) are left invariant. The subgradient is the \(n_1\)-dimensional vector given by

$$\begin{aligned} \nabla _{{\mathbb {G}}} v(z)=(X_1v,X_2v,\ldots ,X_{n_1}v). \end{aligned}$$

The operators \({\mathcal {L}}\) and \(\nabla _{{\mathbb {G}}}\) are left invariant differential operators. For each real \(\delta \), the naturally associated dilation with a stratified Lie group is given by

$$\begin{aligned} T_{\delta }(z)=(\delta ^1 z^{(1)},\delta ^2 z^{(2)},\ldots ,\delta ^kz^{(k)}), \ \hbox {for every} \ z\in {\mathbb {R}}^{n_1}\times {\mathbb {R}}^{n_2}\times \cdots \times {\mathbb {R}}^{n_k}. \end{aligned}$$

The fractional Sobolev space \(W^{s,2}(\Omega )\), \(0<s<1\), is defined by

$$\begin{aligned} W^{s,2}(\Omega ):=\left\{ u\in L^2(\Omega ):\frac{|v(x)-v(y)|}{|y^{-1}\cdot x|^{\frac{Q}{2}+s}}\in L^2(\Omega \times \Omega )\right\} , \end{aligned}$$

and is equipped with the norm

$$\begin{aligned} \Vert v\Vert _{W^{s,2}}(\Omega ):=\left( \int _{\Omega }|u|^2dx+\iint _{\Omega \times \Omega }\frac{|u(x)-u(y)|^2}{|y^{-1}\cdot x|^{Q+2s}}dxdy\right) ^{1/2}. \end{aligned}$$

However, in order to study the mixed problem we shall consider the space

$$\begin{aligned} X:=\{u\in W^{s,2}({\mathbb {G}}):u|_{\Omega }\in W_0^{1,2}(\Omega ),~u=0~\text {a.e. on}~{\mathbb {G}}{\setminus }\Omega \}, \end{aligned}$$

endowed with the norm

$$\begin{aligned} \Vert u\Vert :=\Vert \nabla _{{\mathbb {G}}} v\Vert _{L^2(\Omega )}+\left\| \frac{v(x)-v(y)}{|y^{-1}\cdot x|^{\frac{Q+2s}{2}}}\right\| _{L^{2}(\Omega \times \Omega )}, \end{aligned}$$

which is more appropriate for our study.

The continuous and compact embedding theorems also hold (see Hasłasz-Koskela [20, Theorem 8.1]). We can now state the result collectively as follows.

Theorem 2.3

Let \(\Omega \subset {\mathbb {G}}\) be a bounded domain and \(1\le p <Q\). Then \(W_0^{1,p}(\Omega )\) is continuously embedded in \(L^q(\Omega )\) for every \(1 \le q \le p^*:=\frac{Qp}{Q-p}\). Moreover, the embedding is compact for every \(1 \le q < p^*\).

We now prove the embedding result along the lines of Buccheri et al. [7].

Theorem 2.4

For every \(p \in (1,\infty )\) and \(s \in (0, 1)\), there exists a constant \(C = C(N,s,\Omega )\) such that \(C_p = C_p(Q,s,\Omega ) \rightarrow C \in (0,\infty )\) as \(p \rightarrow \infty \) and

$$\begin{aligned} \iint _{\Omega \times \Omega }\frac{|{\bar{v}}(x)-{\bar{v}}(y)|^p}{|y^{-1}\cdot x|^{Q+ps}}dy\le C_p\Vert \nabla v\Vert _p, \end{aligned}$$

for every \(v\in W_0^{1,p}(\Omega ),\) where \({\bar{v}}\) is the extension of v to 0 in \(\Omega \).

Proof

Let \({\bar{v}}\) be an extension of \(v\in W_0^{1,p}(\Omega )\) by 0. This implies that \(\Vert \nabla {\bar{v}}\Vert _{L^p({\mathbb {R}}^n)}=\Vert \nabla {\bar{v}}\Vert _{L^p(\Omega )}\). Consider,

$$\begin{aligned} \begin{aligned} \iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{|{\bar{v}}(x)-{\bar{v}}(y)|^p}{|y^{-1}\cdot x|^{Q+ps}}dxdy&=2\iint _{\Omega \times \Omega ^c}\frac{|{\bar{v}}(x)-{\bar{v}}(y)|^p}{|y^{-1}\cdot x|^{Q+ps}}dxdy\\&\quad +\iint _{\Omega \times \Omega }\frac{|{v}(x)-{v}(y)|^p}{|y^{-1}\cdot x|^{Q+ps}}dxdy. \end{aligned} \end{aligned}$$
(2.2)

We shall estimate only the first term since the second one can be estimated similarly. Let \(B_{\rho }(x)\) denote the ball centered at \(x\in \Omega \) with radius \(\rho \). Then we have:

$$\begin{aligned} \iint _{\Omega \times \Omega ^c\cap B_{\rho }(x)}\frac{|{\bar{v}}(x)-{\bar{v}}(y)|^p}{|y^{-1}\cdot x|^{Q+ps}}dxdy\le & {} \iint _{\Omega \times B_{\rho }(x)}\frac{|{\bar{v}}(x)-{\bar{v}}(y)|^p}{|y^{-1}\cdot x|^{Q+ps}}dxdy\nonumber \\\le & {} \iint _{\Omega \times B_{\rho }(0)}\frac{|{\bar{v}}(x)-{\bar{v}}(L_z(x))|^p}{|z^{-1}|^{Q+ps}}dxdz\nonumber \\= & {} \iint _{\Omega \times B_{\rho }(0)}\frac{|\int _0^1\frac{d}{dt}{\bar{v}}(L_x\circ T_{t}(z))dt|^p}{|z|^{Q+ps}}dxdz\nonumber \\\le & {} \iint _{\Omega \times B_{\rho }(0)}\int _0^1|\nabla _{{\mathbb {G}}}{\bar{v}}(L_x\circ T_{t}(z)))|^p\nonumber \\{} & {} \times \left| \frac{d}{dt}(L_x\circ T_{t}(z))\right| ^p\frac{1}{|z|^{Q+ps}}dtdxdz\nonumber \\\le & {} \omega _Q\Vert \nabla _{{\mathbb {G}}}{\bar{v}}\Vert _{L^p({\mathbb {G}})}^2\int _0^\rho r^{p(1-s)-1}dr\nonumber \\= & {} \frac{\omega _Q}{p(1-s)}\rho ^{p(1-s)}\Vert \nabla _{{\mathbb {G}}}{\bar{v}}\Vert _{L^p({\mathbb {G}})}^p , \end{aligned}$$
(2.3)

where \(L_z, T_t\), respectively, stand for translation by z and multiplication by t, and \(\omega _Q\) stands for the volume of a unit ball in a stratified Lie group whose homogeneous dimension is Q.

On the other hand, we have by the Poincaré inequality that

$$\begin{aligned} \begin{aligned} \iint _{\Omega \times \Omega ^c\cap B_{\rho }(x)^c}\frac{|{\bar{v}}(x)-{\bar{v}}(y)|^2}{|y^{-1}\cdot x|^{Q+ps}}dxdy\le&\iint _{\Omega \times B_{\rho }(x)^c}\frac{|{\bar{v}}(x)-{\bar{v}}(y)|^p}{|y^{-1}\cdot x|^{Q+ps}}dxdy\\ \le&\frac{\omega _Q}{ps}\rho ^{-ps}\Vert w\Vert _{L^p(\Omega )}^p\le \frac{\omega _Q(C(\Omega ,p))^p}{ps}\rho ^{-ps}\Vert \nabla _{{\mathbb {G}}} w\Vert _{L^p(\Omega )}^p. \end{aligned} \end{aligned}$$
(2.4)

This completes the proof of Theorem 2.4. \(\square \)

Remark 2.5

X is a real separable and reflexive Banach space (see Ghosh et al. [18]).

Remark 2.6

For fine bounds on the best constants of the Sobolev embedding \(W_0^{s,p}(\Omega )\hookrightarrow L^q(\Omega )\) in the Euclidean setup one may refer to Cassini-Du [8].

3 Operators \(A:X\rightarrow X^*\) and \(B:L^q(\Omega )\rightarrow (L^q(\Omega ))^*\)

We define an eigenpair for problem (1.1).

Definition 3.1

We say that \((\lambda ,u)\in {\mathbb {R}}\times X{\setminus }\{0\}\) is an eigenpair for problem (1.1) if for every \(\phi \in X\), we have

$$\begin{aligned} \begin{aligned} \int _{\Omega }\nabla _{{\mathbb {G}}}u\cdot \nabla _{{\mathbb {G}}}\phi dx+\iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{(u(x)-u(y))(\phi (x)-\phi (y))}{|y^{-1}\cdot x|^{Q+2s}}dxdy&=\lambda \Vert u\Vert _{L^q(\Omega )}^{2-q}\int _{\Omega }|u|^{q-2}u\phi dx. \end{aligned} \end{aligned}$$
(3.1)

Note that Theorem 2.4 guarantees that eigenpairs are well-defined. Next, we define the operators \(A:X\rightarrow X^*\) by

$$\begin{aligned} \begin{aligned} \langle Av,w\rangle&=\int _{\Omega }\nabla _{{\mathbb {G}}} v\cdot \nabla _{{\mathbb {G}}} wdx+\iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{(v(x)-v(y))(w(x)-w(y))}{|y^{-1}\cdot x|^{Q+2s}}dxdy, \end{aligned} \end{aligned}$$
(3.2)

and \(B:L^q(\Omega )\rightarrow (L^q(\Omega ))^*\) by

$$\begin{aligned} \begin{aligned} \langle Bv,w\rangle&=\int _{\Omega }|u|^{q-2}uwdx, \end{aligned} \end{aligned}$$
(3.3)

where the symbols \(X^*\), \((L^q(\Omega ))^*\) denote the dual of X, \(L^q(\Omega ),\) respectively. We prove the following theorem about their properties.

Theorem 3.2

The operators \(A:X\rightarrow X^*\) and \(B:L^q(\Omega )\rightarrow (L^q(\Omega ))^*\) are continuous. Moreover, A is bounded, coercive, and monotone.

Proof

Continuity: Suppose that \(v_j\in X\) is such that \(v_j\rightarrow v\) in the norm of X. Thus, a combination of Egoroff’s theorem and the Sobolev embedding, we have up to a subsequence \(\nabla _{{\mathbb {G}}} v_j(x)\rightarrow \nabla _{{\mathbb {G}}} v(x)\) a.e. in \(\Omega \). We note that

$$\begin{aligned} \Vert \nabla _{{\mathbb {G}}} v_j\Vert _{L^{2}(\Omega )} \le c, \end{aligned}$$
(3.4)

for some constant \(c>0\) independent of j. Therefore, up to a subsequence, we have

$$\begin{aligned} \nabla _{{\mathbb {G}}} v_j\rightharpoonup \nabla _{{\mathbb {G}}} v~\text {in}~L^{2}(\Omega ). \end{aligned}$$
(3.5)

In addition, we also have

$$\begin{aligned} \frac{v_j(x)-v_j(y)}{|y^{-1}\cdot x|^{\frac{Q+2s}{2}}}\rightharpoonup \frac{v(x)-v(y)}{|y^{-1}\cdot x|^{\frac{Q+2s}{2}}}~\text {on}~L^2({\mathbb {R}}^n), \end{aligned}$$
(3.6)

and since this weak limit is independent of the choice of the subsequence, it follows by (3.5) and (3.6) that

$$\begin{aligned} \underset{j\rightarrow \infty }{\lim }\langle Av_j,\phi \rangle =\langle Av,\phi \rangle , \ \hbox {for every} \ \phi \in X. \end{aligned}$$

This proves that A is continuous. It follows by a similar argument that B is also continuous.

Boundedness: By the Cauchy–Schwarz and the Hölder inequalities, we have

$$\begin{aligned} \begin{aligned} \langle Av,\phi \rangle =&\int _{\Omega }\nabla _{{\mathbb {G}}} v\cdot \nabla _{{\mathbb {G}}}\phi dx+\iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{(v(x)-v(y))(\phi (x)-\phi (y))}{|y^{-1}\cdot x|^{Q+2s}}dxdy\\ \le&\Vert \nabla v\Vert _{2}\Vert \nabla \phi \Vert _2\\&+\left( \iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{|v(x)-v(y)|^2}{|y^{-1}\cdot x|^{Q+2s}}dxdy\right) ^{1/2}\left( \iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{|\phi (x)-\phi (y)|^2}{|y^{-1}\cdot x|^{Q+2s}}dxdy\right) ^{1/2}\\ \le&\left( \Vert \nabla v\Vert _2+\left( \iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{|v(x)-v(y)|^2}{|y^{-1}\cdot x|^{Q+2s}}dxdy\right) ^{1/2}\right) \Vert \phi \Vert _{X}\\ \le&\left( \Vert \nabla v\Vert _2^2+\left( \iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{|v(x)-v(y)|^2}{|y^{-1}\cdot x|^{Q+2s}}dxdy\right) ^{2}\right) ^{1/2}\Vert \phi \Vert _{X}=\Vert v\Vert _X\Vert \phi \Vert _X, \end{aligned} \end{aligned}$$
(3.7)

hence,

$$\begin{aligned} \Vert Av\Vert _{X^*}=\underset{\Vert \phi \Vert _X\le 1}{\sup }|\langle Av,\phi \rangle |\le \Vert v\Vert _X\Vert \phi \Vert _X\le \Vert v\Vert _X, \end{aligned}$$

therefore A is bounded.

Coercivity: We notice that

$$\begin{aligned} \langle Av,v\rangle&=\int _{\Omega }|\nabla _{{\mathbb {G}}} v|^2dx+\iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{|v(x)-v(y)|^2}{| y^{-1} \cdot x |^{Q+2s} }dxdy=\Vert v\Vert _X^2, \end{aligned}$$
(3.8)

which provides that A is coercive.

Monotonicity: For \(u\in X\), let

$$\begin{aligned} {\mathcal {O}}(u(x,y))=u(x)-u(y),~~d\mu =\frac{dxdy}{|y^{-1}\cdot x|^{Q+2s}}. \end{aligned}$$

Considering the following

$$\begin{aligned} \begin{aligned}&\langle Av-A\phi ,v-\phi \rangle =\int _{\Omega }|\nabla _{{\mathbb {G}}} v-\nabla _{{\mathbb {G}}}\phi |^2dx\\&\quad +\iint _{{\mathbb {G}}\times {\mathbb {G}}}({\mathcal {O}}(v(x,y))-{\mathcal {O}}(w(x,y)))((v(x)-w(x))-(v(y)-w(y)))d\mu \ge 0, \end{aligned} \end{aligned}$$
(3.9)

we conclude that A is monotone. This completes the proof of Theorem 3.2. \(\square \)

4 Proof of Theorem 1.1

We shall need the following results in the sequel.

Theorem 4.1

Let \(0<s<1\) and \(1<q<2^*\). Then the following properties hold:

  1. 1.

    There exists a sequence \((w_j)\subset X\cap L^q(\Omega )\) such that \(\Vert w_j\Vert _{L^q(\Omega )}=1\) and for every \(\phi \in X\), we have

    $$\begin{aligned}{} & {} \int _{\Omega }\nabla _{{\mathbb {G}}}w_j\cdot \nabla _{{\mathbb {G}}}\phi dx+\iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{(w_j(x)-w_j(y))(\phi (x)-\phi (y))}{|y^{-1}\cdot x|^{Q+2s}}dxdy \nonumber \\{} & {} \quad =\mu _n\Vert w_j\Vert _{L^q(\Omega )}^{2-q}\int _{\Omega }|w_j|^{q-2}w_j\phi dx, \end{aligned}$$
    (4.1)

    where

    $$\begin{aligned} \mu _j\ge \lambda :=\inf \{\int _{\Omega }|\nabla _{{\mathbb {G}}}u|^2dx+\iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{|u(x)-u(y)|^2}{|y^{-1}\cdot x|^{Q+2s}}dxdy:u\in X\cap L^q(\Omega ), \Vert u\Vert _{L^q(\Omega )}=1\}. \end{aligned}$$
  2. 2.

    The sequences \((\mu _j)\) and \((\Vert w_{j+1}\Vert ^p)\) satisfying (4.1) are nonincreasing and they converge to the same limit \(\mu \), which is bounded below by \(\lambda \). Moreover, there exists a subsequence \((n_j)\) such that both \((w_{j_i})\) and \((w_{j_{i+1}})\) converge in X to the same limit \(w\in X\cap L^q(\Omega )\) with \(\Vert w\Vert _{L^q(\Omega )}=1,\) and \((\mu ,w)\) is an eigenpair for problem (1.1).

Proof

One can follow the argument of the proof of Garain-Ukhlov [17, Theorem 2.2]. \(\square \)

Theorem 4.2

Let \(0<s<1\) and \(1<q<2^*\). Suppose that \((u_j)\subset X\cap L^q(\Omega )\) is such that \(\Vert u_j\Vert _{L^q(\Omega )}=1\) and \(\underset{j\rightarrow \infty }{\lim }\Vert u_j\Vert ^p=\lambda \). Then there exists a subsequence \((u_{j_i})\) which converges weakly in X to \(u\in X\cap l^q(\Omega )\) with \(\Vert u\Vert _{L^q(\Omega )}=1\) such that

$$\begin{aligned} \lambda =\int _{\Omega }|\nabla _{{\mathbb {G}}}u|^pdx+\iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{|u(x)-u(y)|^{2}}{|y^{-1}\cdot x|^{Q+2s}}dxdy. \end{aligned}$$

Moreover, \((\lambda ,u)\) is an eigenpair for problem (1.1) and every associated eigenfunction of \(\lambda \) is a scalar multiple of the vector at which \(\lambda \) is reached.

Proof

One can follow the argument of the proof of Garain-Ukhlov [17, Theorem 2.2]. However, we prefer to apply Ercole [13, Proposition 2] in place of Ercole [13, Theorem 1]. \(\square \)

We are now in a position to prove our first main result.

Proof of Theorem 1.1

Since (1.1) is homogeneous, we can without loss of generality, let \(\Vert u\Vert _q=1\). Furthermore, for every \(k\ge 1,\) let \(\Omega _k:=\{x\in \Omega :u(x)>k\}\) and \(v:=(u-k)_+\) be a test function for (3.1). With these choices we obtain

$$\begin{aligned} \begin{aligned} \int _{\Omega _k}|\nabla _{{\mathbb {G}}}u|^2dx&+\iint _{{\mathbb {G}}\times {\mathbb {G}}}\frac{(u(x)-u(y))((u-k)_+(x)-(u-k)_+(y))}{ |y^{-1}\cdot x|^{Q+2s}}dxdy\\&=\lambda \int _{\Omega }u^{q-1}(u-k)_+dx. \end{aligned} \end{aligned}$$
(4.2)

The second term on the left hand side of (4.2) is nonnegative and hence

$$\begin{aligned} \begin{aligned} \int _{\Omega _k}|\nabla _{{\mathbb {G}}}u|^2dx&\le \lambda \int _{\Omega }u^{q-1}(u-k)_+dx. \end{aligned} \end{aligned}$$
(4.3)

We split the proof into two parts.

First case:  \(q\le 2\). Since \(k\ge 1\), we have \(u^{q-1}\le u\) on \(\Omega _k\). From (4.3) we have

$$\begin{aligned} \begin{aligned} \int _{\Omega _k}|\nabla _{{\mathbb {G}}}u|^2dx&\le \lambda \int _{\Omega }u(u-k)dx=\lambda \int _{\Omega _k}(2(u-k)^2+2k(u-k))dx, \end{aligned} \end{aligned}$$
(4.4)

where the last inequality is obtained using \(a+b\le 2(a+b)\). Using the Sobolev inequality we get

$$\begin{aligned} \begin{aligned} (1-2S\lambda |\Omega _k|^{2/N})\int _{\Omega _k}(u-k)^pdx&\le k\lambda |\Omega _k|^{2/N}\int _{\Omega _k}(u-k)dx, \end{aligned} \end{aligned}$$
(4.5)

where S is the best Sobolev constant. Note that \(\Vert u\Vert _{1}\ge k|\Omega _k|\) and therefore, for every \(k\ge k_* = \) \((4S\lambda )^{\frac{N}{2}} \Vert u\Vert _{1},\) we have

$$\begin{aligned} 2S\lambda |\Omega (k)| ^\frac{2}{N} \le \frac{1}{2}. \end{aligned}$$

Using (4.4), for every \( k\ge \max \{ k_*, 1\},\) we obtain

$$\begin{aligned} \int _{L(k)} (u-k)^2 dx \le 4S\lambda k |\Omega _k|^\frac{2}{N} \int _{\Omega _k} (u-k)dx, \end{aligned}$$
(4.6)

and invoking the Hölder inequality and the estimate (4.6), we find

$$\begin{aligned} \int _{\Omega _k} (u-k)dx \le (4S\lambda )k|\Omega _k|^{1+\frac{2}{N}}, \end{aligned}$$
(4.7)

so taking into account (4.7), we conclude by invoking Ladyzhenskaya-Ural’tseva [21, Lemma 5.1] (which holds irrespective of the group structure on \({\mathbb {R}}^n\)), to get \( u \in L^{\infty } (\Omega ).\)

Second case: \(q>2.\) Using the inequality

$$\begin{aligned} (a+b)^{q-1} \le 2^{q-1} (a^{q-1}+ b^{q-1}), \ \hbox {for every} \ a,b \ge 0, \end{aligned}$$

in (4.1),  we get

$$\begin{aligned} \int _{\Omega _k} |\nabla _{{\mathbb {G}}} u|^2 dx \le \lambda \int _{\Omega _k} (2^{q-1} (u-k)^q) + 2^{q-1}k^{q-1}(u-k))dx \end{aligned}$$
(4.8)

and using the Sobolev inequality with \(r = q\) in estimate (4.8), we find

$$\begin{aligned} \left( \int _{\Omega _k} (u-k)^q dx \right) ^\frac{2}{q} \le S \lambda |\Omega _k|^{2(\frac{1}{q}-\frac{1}{2}+ \frac{1}{N} )} \int _{\Omega _k} (2^{q-1}(u-k)^q + 2^{q-1} k^{q-1} (u-k))dx, \nonumber \\ \end{aligned}$$
(4.9)

where \(S>0\) is the Sobolev constant. Since

$$\begin{aligned} \int _{\Omega _k} (u-k)^q dx \le \Vert u\Vert ^q _{q} =1 \end{aligned}$$

and \( q>2,\) the quantity on the left-hand side of (4.9) can be estimated from below as follows:

$$\begin{aligned} \left( \int _{\Omega _k} (u-k)^q dx \right) ^\frac{2}{q} = \left( \int _{\Omega _k} (u-k)^q dx \right) ^{\frac{2-q}{q} +1} \ge \int _{\Omega _k} (u-k)^q dx. \end{aligned}$$
(4.10)

Using (4.10) in (4.9), we get

$$\begin{aligned}{} & {} (1- S \lambda 2^{q-1} |\Omega _k|^{p(\frac{1}{q}-\frac{1}{2}+ \frac{1}{N} )}) \int _{\Omega _k} (u-k)^q dx \nonumber \\{} & {} \quad \le S \lambda 2^{q-1} k^{q-1} |\Omega _k|^{2(\frac{1}{q}-\frac{1}{2}+ \frac{1}{N} )} \int _{\Omega _k} (u-k) dx. \end{aligned}$$
(4.11)

Let

$$\begin{aligned} \alpha = 2\left( \frac{1}{q} -\frac{1}{2} + \frac{1}{N}\right) , \end{aligned}$$

and note that it is positive since \(1<q<2^*.\) Choose \(K_1 =(S\lambda 2^q)^{\frac{1}{\alpha }}\Vert u\Vert _{1}.\) Then since \( k|\Omega _k| \le \Vert u\Vert _{1}, \) for every \(k \ge k_*,\) we have

$$\begin{aligned} S \lambda 2^{q-1} |\Omega _k|^{\alpha } \le \frac{1}{2}. \end{aligned}$$

Using this property in (4.11), we obtain

$$\begin{aligned} \int _{L(k)} (u-k)^q dx \le S \lambda 2^q k^{q-1} |L(k)|^{\alpha } \int _{L(k)} (u-k) dx, \end{aligned}$$
(4.12)

so by the Hölder inequality and the estimate (4.12), we arrive at the following estimate

$$\begin{aligned} \int _{L(k)} (u-k) dx \le (S \lambda 2^q)^{\frac{1}{q-1}}k|L(k)|^{1+\frac{\alpha }{q-1}}. \end{aligned}$$
(4.13)

Taking into account (4.13) and invoking Ladyzhenskaya-Ural’tseva [21, Lemma 5.1], we get \(u \in L^{\infty } (\Omega ).\)

The other assertion follows by Garain-Kinnunen [15, Theorem 8.4] which considers the following problem:

$$\begin{aligned} \begin{aligned} -\Delta _pu+(-\Delta _{p,K})^su=&0~\text {in}~\Omega , \end{aligned} \end{aligned}$$
(4.14)

where \(1<p<\infty \) and

$$\begin{aligned} (-\Delta _{p,K})^su=\text {P.V.} \int _{{\mathbb {R}}^n}|u(x)-u(y)|^{p-2}(u(x)-u(y))K(x,y)dy, \end{aligned}$$

here P.V. denotes the principle value. The weak Harnack inequality proved in Garain-Kinnunen [15] also works for the Dirichlet boundary condition \(u|_{\partial \Omega }=0\). In addition, we note that the proof of Garain-Kinnunen [15, Theorem 8.4] works even for a sub-Laplacian \({\mathcal {L}}\) that acts on functions defined on a stratified Lie group since the group structure does not affect the proof of the result. Let us denote v to be a nonnegative solution of problem (1.1). On subtracting the weak formulation of (4.14) (for \(p=2\)) from (1.1) we get

$$\begin{aligned} \begin{aligned} \int _{\Omega }\nabla _{{\mathbb {G}}}(v-u)&\cdot \nabla _{{\mathbb {G}}}(v-u)^-dx\\&+\iint _{\Omega \times \Omega }((v-u)(x)-(v-u)(y))((v-u)^-(x)-(v-u)^-(y))|y^{-1}\cdot x|^{-Q-2s}dx\\ =&\lambda \Vert u\Vert _q^{2-q}\int _{\Omega }v^{q-1}(v-u)^-dx, \end{aligned} \end{aligned}$$
(4.15)

which implies

$$\begin{aligned} \begin{aligned} 0\ge -\int _{\Omega }|\nabla _{{\mathbb {G}}}(v-u)^-|^2dx&-\iint _{\Omega \times \Omega }\frac{|(v-u)^-(x)-(v-u)^-(y)|^2}{|y^{-1}\cdot x|^{Q+2s}}dx\\ =&\lambda \Vert u\Vert _q^{2-q}\int _{\Omega }v^{q-1}(v-u)^-dx\ge 0, \end{aligned} \end{aligned}$$
(4.16)

hence the Lebesgue measure of the set \(\{x\in \Omega :v(x)\le u(x)\}\) is zero and therefore \(v\ge u\) a.e. in \(\Omega \). By [15], we already have \(u>0\) on any compact subset of \(\Omega \) and hence we have the same conclusion for v. This completes the proof of Theorem 1.1. \(\square \)

5 Proof of Theorem 1.2

A natural question to raise at this juncture is what happens when \(q=2^*\)? Of course, it is impossible to figure out the answer for a general stratified Lie group involving a fractional Laplacian - owing to the unavailability of the derivative of the distance function. However, in order to be able to answer this question, we shall study the Brezis-Nirenberg type of problem on a Heisenberg group \({\mathbb {G}}:={\mathbb {H}}^n \) (see Molica Bisci-Repovš [25]) for which we shall establish a Pohozaev-like identity. The problem is as follows:

$$\begin{aligned} \left\{ \begin{array}{ll}-{\mathcal {L}} u+(-\Delta )^su=\lambda \Vert u\Vert _{2^*}^{2-2^*}|u|^{2^*-2}u=:g(u)~~~\text {on}~\Omega \\ u=0~~~\text {on}~{\mathbb {R}}^n{\setminus }\Omega .\end{array} \right. \end{aligned}$$
(5.1)

Let

$$\begin{aligned} G(u):=\int _0^ug(t)dt \end{aligned}$$

be the primitive of the continuous function g. The vector fields

$$\begin{aligned} X_i=\frac{\partial }{\partial x_i}-\frac{y_i}{2}\frac{\partial }{\partial z}, Y_i=\frac{\partial }{\partial y_i}+\frac{x_i}{2}\frac{\partial }{\partial z}, Z=\frac{\partial }{\partial z}, \ \hbox {where} \ i=1,2,\ldots ,n \end{aligned}$$

generate the Lie algebra corresponding to the Heisenberg group \({\mathbb {H}}^n\) of topological dimension \(2n+1\). The corresponding group law for \((a,b,c),(a',b'c')\in {\mathbb {R}}^{2n}\times {\mathbb {R}}\) is

$$\begin{aligned} (a,b,c)\circ (a',b'c')^{-1}=(a+a',b+b',c+c'+2^{-1}(ab'-ba')). \end{aligned}$$

Under this group law, the inverse is \((a,b,c)^{-1}=(-a,-b,-c)\) and the identity is (0, 0, 0). The distance function for a Heisenberg group is defined as

$$\begin{aligned} |(a,b,c)|=[(|a|^2+|b|^2)^2+|c|^2]^{1/4}. \end{aligned}$$

However, before we establish a Pohozaev-like identity, we need to prove Theorem 5.1.

Theorem 5.1

Suppose that \(u_1,u_2\in W^{1,1}({{\mathbb {H}}^n})\) have disjoint compact supports, say \(\Omega _1, \Omega _2,\) respectively. Then

$$\begin{aligned}{} & {} \int _{\Omega _1}({\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} u_1)(-\Delta )^su_2dx+\int _{\Omega _2}({\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} u_2)(-\Delta )^su_1dx \nonumber \\{} & {} \quad =\left( \frac{2s-Q}{2}\right) \int _{\Omega _1}u_1(-\Delta )^su_2dx+\left( \frac{2s-Q}{2}\right) \int _{\Omega _2}u_2(-\Delta )^su_1dx, \end{aligned}$$
(5.2)

where \({\bar{x}}=(x_1^{(1)},x_2^{(1)},\ldots ,x_{N_1}^{(1)},\ldots ,rx_1^{(r)},rx_2^{(r)}, \cdots ,rx_{N_r}^{(r)})\) and \(\nabla _{{\mathbb {H}}^n}\) is the subgradient corresponding to the Heisenberg group \({\mathbb {H}}^n\).

Proof of Theorem 5.1

We first claim that

$$\begin{aligned} (-\Delta )^s({\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} u_j)&={\bar{x}}\cdot (-\Delta )^s\nabla _{{\mathbb {H}}^n} u_j+2s(-\Delta )^su_j~\text {in}~{\mathbb {H}}^n{\setminus }\Omega _i, \ \hbox {for every} \ i=1,2. \end{aligned}$$
(5.3)

Clearly, we have \(u_j\equiv 0\) on \({\mathbb {H}}^n{\setminus }\Omega _i\) and hence using the definition of \((-\Delta )^s\) for \(x\in {\mathbb {H}}^n{\setminus }\Omega _i,\) we obtain

$$\begin{aligned} (-\Delta )^s({\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} u_i(x))= & {} c_{N,s}\int _{\Omega _i}\frac{-{\bar{y}}\cdot \nabla _{{\mathbb {H}}^n} u_i(y)}{|y^{-1}\circ x|^{Q+2s}}dy \nonumber \\= & {} c_{N,s}\int _{\Omega _i}\frac{\sum _{k=1}^r\sum _{l=1}^{N_j}k(x_l^{(k)}-y_l^{(k)})\frac{\partial }{\partial y_l^{(k)}}u_i(y)}{|y^{-1}\circ x|^{Q+2s}}dy \nonumber \\{} & {} +c_{N,s}\int _{\Omega _i}\frac{\sum _{k=1}^r\sum _{l=1}^{N_j}kx_l^{(k)}\frac{\partial }{\partial y_l^{(k)}}u_i(y)}{|y^{-1}\circ x|^{Q+2s}}dy \nonumber \\= & {} c_{N,s}\int _{\Omega _i}-\sum _{k=1}^r\sum _{l=1}^{N_j}k\frac{(x_l^{(k)}-y_l^{(k)})}{|y^{-1}\circ x|^{Q+2s}}\frac{\partial }{\partial y_l^{(k)}}u_i(y)+{\bar{x}}\cdot (-\Delta )^s\nabla _{{\mathbb {H}}^n} u_i(x) \nonumber \\= & {} c_{N,s}\int _{\Omega _i}-\frac{2s}{|y^{-1}\circ x|^{Q+2s}}u_i(y)+{\bar{x}}\cdot (-\Delta )^s\nabla _{{\mathbb {H}}^n} u_i(x) \nonumber \\= & {} 2s(-\Delta )^su_i(y)+{\bar{x}}\cdot (-\Delta )^s\nabla _{{\mathbb {H}}^n} u_i(x), \ \hbox {for every} \ i=1,2, \end{aligned}$$
(5.4)

hence

$$\begin{aligned} \begin{aligned} \int _{\Omega _1}({\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} u_1)(-\Delta )^su_2dx=&-Q\int _{\Omega _1}u_1(-\Delta )^su_2dx-\int _{\Omega _1}u_1{\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} (-\Delta )^su_2dx, \end{aligned} \end{aligned}$$
(5.5)

so integrating by parts along with (5.4), we obtain

$$\begin{aligned} \begin{aligned} \int _{\Omega _1}u_1{\bar{x}}\cdot \nabla _{{\mathbb {H}}^n}(-\Delta )^su_2dx=&\int _{\Omega _1}u_1(-\Delta )^s({\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} u_2)dx-2s\int _{\Omega _1}u_1(-\Delta )^su_2dx\\ =&\int _{\Omega _1}(-\Delta )^su_1({\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} u_2)dx-2s\int _{\Omega _1}u_1(-\Delta )^su_2dx, \end{aligned} \end{aligned}$$
(5.6)

therefore by (5.5) and (5.6),

$$\begin{aligned} \begin{aligned} \int _{\Omega _1}({\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} u_1)(-\Delta )^su_2dx=&-\int _{\Omega _1}(-\Delta )^su_1({\bar{x}}\cdot \nabla _{{\mathbb {H}}^n} u_2)dx+(2s-Q)\int _{\Omega _1}u_1(-\Delta )^su_2dx, \end{aligned} \end{aligned}$$
(5.7)

thus, by again integrating by parts,

$$\begin{aligned} \begin{aligned} \int _{\Omega _1}u_1(-\Delta )^su_2dx=&\frac{1}{2}\int _{\Omega _1}u_1(-\Delta )^su_2dx+\frac{1}{2}\int _{\Omega _2}u_2(-\Delta )^su_1dx, \end{aligned} \end{aligned}$$
(5.8)

and the equality (5.2) follows. This completes the proof of Theorem 5.1\(\square \)

Proof of Theorem 1.2

We shall follow the standard technique of deriving the identity - by multiplying the PDE in (5.1) by

$$\begin{aligned} Zu:=\sum _{j=1}^r\sum _{i=1}^{n_j}ix_i^{(j)}\frac{\partial u}{\partial x_i^{(j)}}. \end{aligned}$$

We note of that \(\text {div}Z=Q\). Invoking Bonfiglioli et al. [5, Proposition 1.6.1] and integrating by parts, we get

$$\begin{aligned}{} & {} -\int _{\Omega }\Delta _{{\mathbb {G}}}uZudx+\int _{\Omega }(-\Delta )^suZudx=\int _{\Omega }g(u)Zudx \nonumber \\{} & {} \quad = \int _{\Omega }\sum _{i=1}^{n_1}ix_i\frac{\partial }{\partial x_i}G(u)dx =-\frac{Q}{2}\int _{\Omega }G(u)dx. \end{aligned}$$
(5.9)

Combining (5.9) with Theorem 5.1 and Louidice [24, Equation (4.15) in Theorem 4.1], the identity can be stated as follows:

$$\begin{aligned}{} & {} \frac{Q}{2}\int _{\Omega }G(u)dx-\left( \frac{Q-2}{2}\right) \int _{\Omega }|\nabla _{_{{\mathbb {H}}^n}}u|^2dx-\left( \frac{Q-2s}{2}\right) \int _{\Omega }|(-\Delta )^su|^2dx\nonumber \\{} & {} \quad =\frac{1}{2}\int _{\partial \Omega }|\nabla _{_{{\mathbb {H}}^n}}u|^2\langle Z,{\hat{n}}\rangle dS. \end{aligned}$$
(5.10)

This completes the proof of Theorem 1.2. \(\square \)

Remark 5.2

Using Theorem 1.2, one can show that problem (5.1) when defined on a star-shaped domain \(\Omega ,\) has no nontrivial solutions for any \(\lambda <0\). This is because Eq. (5.2) yields

$$\begin{aligned} 0<\frac{1}{2}\int _{\partial \Omega }|\nabla _{_{{\mathbb {H}}^n}}u|^2\langle Z,{\hat{n}}\rangle dS= & {} \frac{Q}{2}\int _{\Omega }G(u)dx-\frac{Q}{2}\int _{\Omega }g(u)udx \nonumber \\{} & {} +\left( \int _{\Omega }|\nabla _{_{{\mathbb {H}}^n}}u|^2dx+s\int _{\Omega }|(-\Delta )^su|^2dx\right) \nonumber \\\le & {} \frac{Q}{2}\int _{\Omega }G(u)dx-\frac{Q}{2}\int _{\Omega }g(u)udx\nonumber \\{} & {} +\left( \int _{\Omega }|\nabla _{_{{\mathbb {H}}^n}}u|^2dx+\int _{\Omega }|(-\Delta )^su|^2dx\right) \nonumber \\= & {} \frac{Q}{2}\int _{\Omega }G(u)dx-\frac{Q}{2}\int _{\Omega }g(u)udx+\int _{\Omega }g(u)udx\nonumber \\= & {} \frac{Q}{2}\int _{\Omega }G(u)dx+\left( 1-\frac{Q}{2}\right) \int _{\Omega }g(u)udx< 0. \end{aligned}$$
(5.11)

This is absurd and hence there exists no nontrivial solution to the problem if \(\lambda <0\) and the domain \(\Omega \) is star-shaped.