Nonlocal Kirchhoff Problems with Singular Exponential Nonlinearity

Abstract

In this paper, we first develop the fractional Trudinger–Moser inequality in singular case and then we use it to study the existence and multiplicity of solutions for a class of perturbed fractional Kirchhoff type problems with singular exponential nonlinearity. Under some suitable assumptions, the existence of two nontrivial and nonnegative solutions is obtained by using the mountain pass theorem and Ekeland’s variational principle as the nonlinear term satisfies critical or subcritical exponential growth conditions. Moreover, the existence of ground state solutions for the aforementioned problems without perturbation and without the Ambrosetti–Rabinowitz condition is investigated.

1 Introduction and Main Results

Let $N\ge 2$ and assume that $\Omega \subset {\mathbb {R}}^N$ is a bounded domain with Lipschitz boundary and $0\in \Omega$. Given $s\in (0,1)$, we study the following fractional Kirchhoff type problem with exponential growth:

$$\begin{aligned} {\left\{ \begin{array}{ll} M\big (\Vert u\Vert ^{N/s}\big ){\mathcal {L}}_{\mathcal K}u=\displaystyle \frac{f(x,u)}{|x|^\beta } +\lambda h(x)\,\, \ \ &{}\mathrm{in}\ \Omega ,\\ u=0\ \ &{}\mathrm{in}\ {\mathbb {R}}^N\setminus \Omega , \end{array}\right. } \end{aligned}$$

(1.1)

where

$$\begin{aligned} \Vert u\Vert =\left( \iint _{{\mathbb {R}}^{2N}}|u(x)-u(y)|^{N/s}\mathcal K(x-y)dxdy\right) ^{{s}/{N}}, \end{aligned}$$

$M:[0,\infty )\rightarrow [0,\infty )$ is a continuous function, $\beta \in [0,N)$, $\lambda >0$ is a parameter, $h:{\mathbb {R}}^N\rightarrow [0,\infty )$ is a perturbed function which belongs to the dual space $(W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ))^*$ of $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ (see Sect. 2), $f:\Omega \times {\mathbb {R}}\rightarrow {\mathbb {R}}$ is a continuous function, and ${\mathcal {L}}_K$ is the associated nonlocal integro-differential operator which, up to a normalization constant, is defined as

$$\begin{aligned} {\mathcal {L}}_{{\mathcal {K}}}\varphi (x)=2 \lim _{\varepsilon \rightarrow 0^+}\int _{{\mathbb {R}}^N\setminus B_\varepsilon (x)}|\varphi (x)-\varphi (y)|^{N/s-2}(\varphi (x)-\varphi (y))\mathcal K(x-y)\,dy,\quad x\in {\mathbb {R}}^{N}, \end{aligned}$$

along functions $\varphi \in C_0^\infty ({\mathbb {R}}^N)$. Henceforward $B_\varepsilon (x)$ denotes the ball of ${\mathbb {R}}^N$ centered at $x\in {\mathbb {R}}^N$ and radius $\varepsilon >0$. Throughout the paper, we always assume that the singular kernel ${\mathcal {K}}:{\mathbb {R}}^N\setminus \{0\}\rightarrow {\mathbb {R}}^+$ satisfies the following properties:

$(K_1)$:

$m {\mathcal {K}}\in L^1({\mathbb {R}}^N)$, where $m(x)=\min \{1,|x|^{N/s}\}$;

$(K_2)$:

there exists ${\mathcal {K}}_0>0$ such that ${\mathcal {K}}(x)\ge {\mathcal {K}}_0|x|^{-2N}$ for all $x\in {\mathbb {R}}^N\setminus \{0\}$.

Obviously, if ${\mathcal {K}}(x)=|x|^{-2N}$, then ${\mathcal {L}}_{{\mathcal {K}}}$ reduces to the fractional N/s-Laplacian $(-\Delta )^s_{N/s}$.

Equations of the type (1.1) are important in many fields of science, notably continuum mechanics, phase transition phenomena, population dynamics, minimal surfaces and anomalous diffusion, since they are the typical outcome of stochastically stabilization of Lévy processes, see [2, 8, 25] and the references therein. Moreover, such equations and the associated fractional operators allow us to develop a generalization of quantum mechanics and also to describe the motion of a chain or an array of particles that are connected by elastic springs as well as unusual diffusion processes in turbulent fluid motions and material transports in fractured media, for more details see [2, 8] and the references therein. Indeed, the nonlocal fractional operators have been extensively studied by many authors in many different cases: bounded domains and unbounded domains, different behavior of the nonlinearity, and so on. In particular, many works focus on the subcritical and critical growth of the nonlinearity which allow us to treat the problem variationally using general critical point theory.

Recently, some authors have paid considerable attention in the limiting case of the fractional Sobolev embedding. Let $\omega _{N-1}$ be the $N-1$-dimensional measure of the unit sphere in ${\mathbb {R}}^N$ and let $\Omega \subset {\mathbb {R}}^N$ be a bounded domain and define $W_0^{s,N/s}(\Omega )$ as the completion of $C_0^\infty (\Omega )$ with respect to the norm $[\cdot ]_{s,N/s}$ which is defined as

$$\begin{aligned}{}[u]_{s,N/s}= \left( \iint _{{\mathbb {R}}^{2N}}\frac{|u(x)-u(y)|^{N/s}}{|x-y|^{2N}}dxdy\right) ^{s/N}. \end{aligned}$$

In [29], Martinazzi obtained that there exist positive constants

$$\begin{aligned} \alpha _{N,s} =\frac{N}{\omega _{N-1}}\left( \frac{\Gamma ((N-s)/2)}{\Gamma (s/2)2^s\pi ^{N/2}}\right) ^{-\frac{N}{N-s}} \end{aligned}$$

and $C_{N,s}$ depending only on N and s such that

$$\begin{aligned} \mathop {\mathop {\sup }\limits _{u\in W_0^{s,N/s}(\Omega )}}\limits _{[u]_{s,N/s}\le 1}\int _\Omega \exp (\alpha |u|^{\frac{N}{N-s}})dx\le C_{N,s}|\Omega |, \end{aligned}$$

(1.2)

for all $\alpha \in [0,\alpha _{N,s}]$ and there exists $\alpha _{N,s}^*\ge \alpha _{N,s}$ such that the supremum in (1.2) is $\infty$ for $\alpha >\alpha _{N,s}^*$. However, it still an open problem whether or not $\alpha _{N,s}=\alpha _{N,s}^*?$ For more details about Trudinger–Moser inequality, we also refer to [23] and [39].

On one hand, in the setting of the fractional Laplacian, Iannizzotto and Squassina in [22] investigated existence of solutions for the following Dirichlet problem

$$\begin{aligned} {\left\{ \begin{array}{ll} (-\Delta )^{\frac{1}{2}} u=f(u)\ \ &{}\mathrm{in}\ (0,1),\\ u=0\quad \ \ &{}\mathrm{in}\ {\mathbb {R}}\setminus (0,1), \end{array}\right. } \end{aligned}$$

(1.3)

where $(-\Delta )^{\frac{1}{2}}$ is the fractional Laplacian and f(u) behaves like $\exp (\alpha |u|^2)$ as $u\rightarrow \infty$. Using the mountain pass theorem, the authors obtained the existence of solutions for problem (1.3). The existence of ground state solutions for (1.3) was discussed in [16]. Subsequently, Giacomoni, Mishra and Sreenadh in [21] studied the multiplicity of solutions for problems like (1.3) by using the Nehari manifold method. For more recent results for problem (1.3) in the higher dimension case, we refer the interested reader to [41] and the references therein. For the general fractional p-Laplacian in unbounded domains, Souza in [13] considered the following nonhomogeneous fractional p-Laplacian equation

$$\begin{aligned} (-\Delta )_p^su+V(x)|u|^{p-2}u=f(x,u)+\lambda h\ \ \mathrm{in}\ {\mathbb {R}}^N, \end{aligned}$$

(1.4)

where $(-\Delta )_p^s$ is the fractional p-Laplacian and the nonlinear term f satisfies exponential growth. The author obtained a nontrivial weak solution of the equation (1.4) by using fixed point theory. Li and Yang [27] studied the following equation

$$\begin{aligned} (-\Delta )_p^\zeta u+V(x)|u|^{p-2}u=\lambda A(x) |u|^{q-2}u+f(u)\ \ \mathrm{x\in {\mathbb {R}}^N}, \end{aligned}$$

where $p\ge 2$, $0<\zeta <1$, $1<q<p$, $\lambda >0$ is a real parameter, A is a positive function in $L^{\frac{p}{p-q}}({\mathbb {R}}^N)$, $(-\Delta )_p^\zeta$ is the fractional p-Laplacian and f satisfies exponential growth.

On the other hand, Li and Yang in [26] studied the following Schrödinger–Kirchhoff type equation

$$\begin{aligned}&\left( \int _{{\mathbb {R}}^N}(|\nabla u|^N+V(x)|u|^N)dx\right) ^k(-\Delta _N u +V(x)|u|^{N-2}u)\nonumber \\&\quad =\lambda A(x)|u|^{p-2}u+ f(u)\quad \text{ in } {\mathbb {R}}^N, \end{aligned}$$

(1.5)

where $\Delta _N u=\mathrm{div}(|\nabla u|^{N-2}\nabla u)$ is the N-Laplaician, $k>0$, $V:{\mathbb {R}}^N\rightarrow (0,\infty )$ is continuous, $\lambda >0$ is a real parameter, A is a positive function in $L^{\frac{p}{p-q}}({\mathbb {R}}^N)$ and f satisfies exponential growth. By using the mountain pass theorem and Ekeland’s variational principle, the authors obtained two nontrivial solutions of (1.5) as the parameter $\lambda$ small enough. Mingqi, Rădulescu and Zhang studied the following problem

$$\begin{aligned} {\left\{ \begin{array}{ll} M\big (\Vert u\Vert ^{N/s}\big )(-\Delta )_p^s u=f(x,u)\,\, \ \ &{}\mathrm{in}\ \Omega ,\\ u=0\ \ &{}\mathrm{in}\ {\mathbb {R}}^N\setminus \Omega , \end{array}\right. } \end{aligned}$$

where f behaves like $\exp (\alpha |t|^{N/(N-s)})$ as $t\rightarrow \infty$ for some $\alpha >0$. Under suitable assumption on M and f, the authors obtained the existence of ground state solutions by using the mountain pass lemma combined with the fractional Trudinger–Moser inequality. Actually, the study of Kirchhoff-type problems, which arise in various models of physical and biological systems, have received more and more attention in recent years. More precisely, Kirchhoff established a model governed by the equation

$$\begin{aligned} \rho \frac{\partial ^2u}{\partial t^2}-\left( \frac{\rho _{0}}{h}+\frac{E}{2L}\int _0^L\left| \frac{\partial u}{\partial x}\right| ^2dx\right) \frac{\partial ^2u}{\partial x^2}=0, \end{aligned}$$

(1.6)

for all $x\in (0,L),t\ge 0$, where $u=u(x,t)$ is the lateral displacement at the coordinate x and the time t, E is the Young modulus, $\rho$ is the mass density, h is the cross-section area, L is the length and $\rho _0$ is the initial axial tension. Eq. (1.6) extends the classical D’Alembert wave equation by considering the effects of the changes in the length of the strings during the vibrations. Recently, Fiscella and Valdinoci in [18] have proposed a stationary Kirchhoff model driven by the fractional Laplacian by taking into account the nonlocal aspect of the tension, see [18, Appendix A] for more details. Related results in the case of critical nonlinearities have been obtained by Fiscella and Pucci [19] and Miyagaki and Pucci [35].

It is worth mentioning that when $s\rightarrow 1$ and $M\equiv 1$, the equation in problem (1.1) becomes

$$\begin{aligned} -\Delta _N u=f(x,u)+\lambda h(x), \end{aligned}$$

which studied by many authors by using variational methods, see for example, [1, 12, 15, 20, 24].

Inspired by the above works, we are devoted to the existence and multiplicity of solutions for problem (1.1) and overcome the lack of compactness due to the presence of critical exponential growth terms as well as the degenerate nature of the Kirchhoff coefficient. To the best of our knowledge, there are no results for (1.1) in such a generality.

Throughout the paper, without explicit mention, we assume that $M:{\mathbb {R}}^+_0\rightarrow {\mathbb {R}}^+_0$ is assumed to be continuous and to verify

$(M_1)$:

for any $\tau >0$ there exists $\kappa =\kappa (\tau )>0$ such that $M(t)\ge \kappa$ for all $t\ge \tau$;

$(M_2)$:

there exists $\theta >1$ such that

$$\begin{aligned} \theta {\mathscr {M}}(t)\ge M(t)t\ \ \mathrm{for all}\ t\ge 0, \end{aligned}$$

where ${\mathscr {M}}(t)=\int _0^\tau M(\tau )d\tau$.

A typical example of M is given by $M(t)=a+b\theta \,t^{\theta -1}$ for $t\ge 0$, where $a,b\ge 0$ and $a+b>0$. When M is of this type, problem (1.1) is said to be degenerate if $a=0$, while it is called non-degenerate if $a>0$. Recently, the fractional Kirchhoff problems have received more and more attention. Some new existence results of solutions for fractional non-degenerate Kirchhoff problems were given, for example, in [42,43,44, 49]. On some recent results concerning the degenerate case of Kirchhoff-type problems, we refer to [3, 9, 30, 45, 50] and the references therein. It is worth pointing out that the degenerate case in Kirchhoff theory is rather interesting, for example, it was treated in the seminal paper [11]. In the large literature on degenerate Kirchhoff problems, the transverse oscillations of a stretched string, with nonlocal flexural rigidity, depends continuously on the Sobolev deflection norm of u via $M(\Vert u\Vert ^2)$. From a physical point of view, the fact that $M(0)=0$ means that the base tension of the string is zero. Clearly, assumptions $(M_1)$–$(M_2)$ cover the degenerate case.

Define

$$\begin{aligned} \lambda _1=\inf \left\{ \frac{\Vert u\Vert ^{\theta N/s}}{\int _\Omega \frac{1}{|x|^\beta }|u|^{\theta N/s}dx}: u\in W_{0,\mathcal K}^{s,N/s}(\Omega )\setminus \{0\}\right\} . \end{aligned}$$

Clearly, by $0\le \beta <N$ and the fractional Sobolev embedding, we obtain that $\lambda _1>0$.

First in bounded domain $\Omega$, we assume that the nonlinear term $f:{\overline{\Omega }}\times {\mathbb {R}}\rightarrow {\mathbb {R}}$ is a continuous function, with $f(x,t)\equiv 0$ for $t\le 0$ and $x\in \Omega$. In the following, we also require the following assumptions $(f_1)$–$(f_3)$.

$(f_1)$:

f satisfies subscritical growth, i.e., for any $\alpha >0$ there holds

$$\begin{aligned} \lim _{t\rightarrow \infty }f(x,t)\exp (-\alpha |t|^{N/(N-s)})= 0, \end{aligned}$$

uniformly in $\Omega$.

$(f_1^\prime )$:

f satisfies critical growth, i.e., there exists $\alpha _0>0$ such that,

$$\begin{aligned} \lim _{t\rightarrow \infty }f(x,t)\exp (-\alpha |t|^{N/(N-s)})= {\left\{ \begin{array}{ll} 0,\ \ \ \forall \alpha >\alpha _0,\\ \infty ,\ \ \forall \alpha <\alpha _0, \end{array}\right. } \end{aligned}$$

uniformly in $\Omega$.

$(f_2)$:

There exists $\mu >\theta N/s$ such that

$$\begin{aligned} 0<\mu F(x,t)\le f(x,t)t,\quad F(x,t)=\int _0^tf(x,\tau )d\tau , \end{aligned}$$

whenever $x\in \Omega$ and $t>0$, and there exists some $T>0$ such that $\inf _{x\in \Omega }F(x,T)>0$.

$(f_3)$:

There holds:

$$\begin{aligned} \limsup _{t\rightarrow 0^+}\frac{F(x,t)}{|t|^{\theta N/s}}<\frac{s{\mathscr {M}}(1)}{N}\lambda _1\ \ \mathrm{uniformly\ in}\ x\in \Omega . \end{aligned}$$

$(f_4)$:

There exist $q_0>\theta N/s$ and ${\mathcal {C}}_0>0$ such that

$$\begin{aligned} F(x,t)\ge \frac{{\mathcal {C}}_0}{q_0} t^{q_0}\ \ \mathrm{for\ all}\ x\in \Omega \ \mathrm{and}\ t\ge 0, \end{aligned}$$

where

$$\begin{aligned} {\mathcal {C}}_0>\left( \frac{4\mu (sq_0-N\theta )}{q(s\mu -N\theta )}\right) ^{\frac{q_0s-N\theta }{N\theta }} \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{\frac{(N-s)(q_0s-N\theta )}{Ns}}C_{q_0}^{\frac{sq_0}{N\theta }}, \end{aligned}$$

(1.7)

and $C_{q_0}>0$ is defined by

$$\begin{aligned} C_{q_0}=\inf _{u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\setminus \{0\}}\left\{ \Vert u\Vert ^{N/s}: \int _\Omega \frac{1}{|x|^\beta } |u|^{q_0}dx=1\right\} . \end{aligned}$$

A simple example of f, verifying $(f_1)$–$(f_2)$, is given by

$$\begin{aligned} f(x,t)=t^{\theta N/s}\left[ \exp (|t|^{N/(N-s)})-1\right] +{\mathcal {C}}_0t^{\theta N/s-1}, \end{aligned}$$

where ${\mathcal {C}}_0$ is a positive constant.

Definition 1.1

We say that $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ is a (weak) solution of problem (1.1), if

$$\begin{aligned} M(\Vert u\Vert ^{N/s})\langle u,\varphi \rangle _{s,N/s}= & {} \int _{\Omega }\left( \frac{ f(x,u)}{|x|^\beta }+ \lambda h(x)\right) \varphi dx,\\ \langle u,\varphi \rangle _{s,N/s}= & {} \iint _{{\mathbb {R}}^{2N}}|u(x)-u(y)|^{\frac{N}{s}-2}[u(x)-u(y)]\\&\cdot \big [\varphi (x)-\varphi (y)\big ]K(x-y) dxdy, \end{aligned}$$

for all $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$, where $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ will be introduced in Sect. 2.

Now we are in a position to state our results concerning the subcritical case.

Theorem 1.1

Assume that f satisfies $(f_1)$–$(f_3)$ and M fulfills $(M_1)$–$(M_2)$. Let $0\le h\in (W_{0,\mathcal K}^{s,N/s}(\Omega ))^*$. Then there exists $\lambda ^*>0$ such that for all $0<\lambda <\lambda ^*$, problem (1.1) admits at least two nontrivial and nonnegative solutions in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$, where one is a mountain pass type solution and another is a local least energy solution.

For the critical case, we have the following result.

Theorem 1.2

Assume that f satisfies $(f_1^\prime )$, $(f_2)$–$(f_4)$ and $M=a+b\theta t^{\theta -1}$ with $a\ge 0,\ b>0$ and $\theta >1$. Let $0\le h\in (W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ))^*$. Then there exists $\lambda _*>0$ such that for all $0<\lambda <\lambda _*$, problem (1.1) admits at least two nontrivial and nonnegative solutions in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$, where one is a mountain pass type solution and another is a local least energy solution.

Let us simply give an sketch of the proofs of Theorems 1.1 and 1.2. Since the problems discussed here satisfies singular exponential growth conditions, the fractional Trudinger–Moser inequality is not available directly. Thus, we first obtain the fractional Trudinger–Moser inequality in singular case. Then, two theorems are proved by using the mountain pass lemma and the Eekand variational principle combined with the singular fractional Trudinger–Moser inequality. To applying the mountain pass theorem and the Ekeland variational principle, we must verify that the associated functional satisfies the Palais–Smale conditions. However, since the nonlinear term satisfies the critical exponential growth, it becomes more difficulty to get the compactness of the energy functional. To overcome the loss of compactness of the energy functional, we have to estimate the range of level value of energy functional. This is the key point to obtain the existence of solutions for the critical case.

Finally, we consider the following problem with critical exponential growth

$$\begin{aligned} {\left\{ \begin{array}{ll} M\big (\Vert u\Vert ^{N/s}\big ){\mathcal {L}}_{{\mathcal {K}}}u=\displaystyle \frac{f(x,u)}{|x|^\beta }\,\, \ \ &{}\mathrm{in}\ \Omega ,\\ u=0\ \ &{}\mathrm{in}\ {\mathbb {R}}^N\setminus \Omega . \end{array}\right. } \end{aligned}$$

(1.8)

To get the existence of ground state solutions for problem (1.8), we also need the following hypotheses:

$(M_3)$:

There exists $\theta >1$ such that $\displaystyle \frac{M(t)}{t^{\theta -1}}$ is nonincreasing for $t>0$.

$(M_4)$:

${\mathscr {M}}$ is superadditive, i.e., for any $t_1,t_2\ge 0$ there holds

$$\begin{aligned} {\mathscr {M}}(t_1)+{\mathscr {M}}(t_2)\le {\mathscr {M}}(t_1+t_2). \end{aligned}$$

$(f_5)$:

There exists $\beta _0>\frac{M\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}}{\frac{\omega _{N-1}R_0^{N-\beta }}{N-\beta }}$ such that

$$\begin{aligned} \lim _{t\rightarrow \infty }\frac{f(x,t)t}{\exp \left( \frac{\alpha _0\alpha _{N,s}^*}{\alpha _{N,s}} t^{N/(N-s)}\right) }\ge \beta _0\ \ \ \mathrm{uniformly\ in}\ x\in \Omega , \end{aligned}$$

where $R_0$ is the radius of the largest open ball centered at zero contained in $\Omega$.

$(f_6)$:

For each $x\in \Omega$, $\displaystyle \frac{f(x,t)}{t^{\frac{\theta N}{s}-1}}$ is increasing for $t>0$, where $\theta >1$ is given by $(M_3)$.

Remark 1.1

If M is a nondecreasing function, then $(M_4)$ holds. Indeed, for any $0\le t_1\le t_2<\infty$

$$\begin{aligned} {\mathscr {M}}(t_1+t_2)=\int _0^{t_1+t_2} M(t)dt =\int _0^{t_1}M(t)dt+\int _{t_1}^{t_1+t_2}M(t)dt \ge {\mathscr {M}}(t_1)+{\mathscr {M}}(t_2). \end{aligned}$$

In terms of $(M_3)$ and Remark 1.1 of [33], we can obtain that

$$\begin{aligned} \theta {\mathscr {M}}(t)-M(t)t\ \ \mathrm{is\ nondecreasing\ for\ }\ t>0. \end{aligned}$$

In particular, we have

$$\begin{aligned} \theta {\mathscr {M}}(t)-M(t)t\ge 0,\ \ \forall t\ge 0. \end{aligned}$$

(1.9)

Moreover, from $(M_3)$ one can deduce that

$$\begin{aligned} \lim _{t\rightarrow \infty }{\mathscr {M}}(t)=\infty . \end{aligned}$$

Remark 1.2

According to $(f_1^\prime )$, for some $0<\alpha <\alpha _0$ we have

$$\begin{aligned} \lim _{t\rightarrow \infty }\frac{f(x,t)}{\exp (\alpha t^{\frac{N}{N-s}})}=\infty , \end{aligned}$$

uniformly in $\Omega$. Then

$$\begin{aligned} \lim _{t\rightarrow \infty }\frac{f(x,t)}{t^{\theta \frac{N}{s}-1}}=\infty , \end{aligned}$$

uniformly in $\Omega$. Furthermore, we deduce

$$\begin{aligned} \lim _{t\rightarrow \infty }\frac{F(x,t)}{t^{\theta \frac{N}{s}}}=\infty , \end{aligned}$$

(1.10)

uniformly in $\Omega$.

Using $(f_6)$ and the same discussion as [33], one can deduce that for each $x\in \Omega$,

$$\begin{aligned} tf(x,t)-\frac{N\theta }{s} F(x,t) \ \ \mathrm{is\ increasing\ for}\ t>0. \end{aligned}$$

(1.11)

In particular, $tf(x,t)-\frac{ N\theta }{s}F(x,t)\ge 0$ for all $(x,t)\in \Omega \times [0,\infty )$.

Theorem 1.3

Assume that f satisfies $(f_1^\prime )$, $(f_3)$, $(f_5)$ and $(f_6)$, and M fulfills $(M_1)$, $(M_3)$ and $(M_4)$. Then problem (1.8) has a ground state solution in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$.

To get the existence of ground state solutions for problem (1.8), we first show that problem (1.8) has a nonnegative mountain pass solution, and then prove that the mountain pass solution is a ground state solution. The main difficulty is that how we can get the strong convergence of $(u_n)_n$ and how to prove that the limit of $(u_n)_n$ is the ground state solution of problem (1.8). In the process of proving our main results, some ideas are inspired from papers [17] and [33].

To the best of our knowledge, Theorems 1.1–1.3 are the first results for the Kirchhoff equations involving singular Trudinger–Moser nonlinearities in the fractional setting.

The paper is organized as follows. In Sect. 2, we present the functional setting and show preliminary results. In Sect. 3, by using the mountain pass theorem and Ekeland’ variational principle, we obtain the existence of two nontrivial nonnegative solutions for problem (1.1) with subcritical exponential growth conditions as $\lambda$ small. In Sect. 4, we get the existence of two nonnegative solutions for problem (1.1) with critical exponential nonlinearity. In Sect. 5, we investigate the existence of ground state solutions for problem (1.8) without perturbation term and the Ambrosetti–Rabinowitz condtion.

2 Preliminary Results

In this section, we give the variational framework of problem (1.1) and prove several necessary results which will be used later.

Define $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ as

$$\begin{aligned} W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )=\left\{ u\in L^{N/s}(\Omega ): [u]_{s,{\mathcal {K}}}<\infty ,\ u=0 \ \mathrm{a.e.\ in }\ {\mathbb {R}}^N\setminus \Omega \right\} , \end{aligned}$$

where

$$\begin{aligned}{}[u]_{s,{\mathcal {K}}}= \left( \iint _{{\mathbb {R}}^{2N}}|u(x)-u(y)|^{N/s}\mathcal K(x-y)dxdy\right) ^{s/N}. \end{aligned}$$

Equip $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ with respect to the norm

$$\begin{aligned} \Vert u\Vert =[u]_{s,{\mathcal {K}}}, \end{aligned}$$

here we apply $(k_1)$. By a similar discussion as in [44], we know that $(W_{0,{\mathcal {K}}}^{s,N/s}({\mathbb {R}}^N),\Vert \cdot \Vert )$ is a reflexive Banach space. Clearly, the embedding $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\hookrightarrow W_{0}^{s,N/s}(\Omega )$ is continuous, being

$$\begin{aligned}{}[u]_{s,N/s}\le {\mathcal {K}}_0^{-s/N}[u]_{s,{\mathcal {K}}}\quad \text{ for } \text{ all } u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ), \end{aligned}$$

by $(k_2)$.

Theorem 2.1

(see [14, Theorem 6.10]) Let $s\in (0,1)$ and $N\ge 1$. Let $\Omega \subset {\mathbb {R}}^N$ be a bounded domain with Lipschitz boundary. Then there exists a positive constant $C=C(N,s,\Omega )$ such that for any $u\in W_0^{s,N/s}(\Omega )$ there holds

$$\begin{aligned} \Vert u\Vert _{L^q(\Omega )}\le C[u]_{s,N/s} \end{aligned}$$

for any $q\in [1,\infty )$, i.e. the space $W_0^{s,N/s}(\Omega )$ is continuously embedded in $L^q(\Omega )$ for any $q\in [1,\infty )$.

For $\nu \ge 1$ and $\beta <N$, we define

$$\begin{aligned} L^\nu (\Omega ,|x|^{-\beta })=\left\{ u:\Omega \rightarrow {\mathbb {R}}\ \ \mathrm{is\ measurable}\bigg | \int _\Omega \frac{1}{|x|^\beta }|u(x)|^\nu dx<\infty \right\} , \end{aligned}$$

endowed with the norm

$$\begin{aligned} |u|_{\nu ,\beta }=\left( \int _\Omega \frac{1}{|x|^\beta }|u(x)|^\nu dx\right) ^{\frac{1}{\nu }}. \end{aligned}$$

To prove the existence of weak solutions for problem (1.1), we shall use the following embedding theorem.

Theorem 2.2

(Compact embedding) Let $s\in (0,1)$, $N\ge 1$ and $0\le \beta <N$. Assume that $\Omega$ is a bounded domain in ${\mathbb {R}}^N$ with Lipschitz boundary $\partial \Omega$. Then, for any $\nu \ge 1$ the embeddings $W_0^{s,N/s}(\Omega )\hookrightarrow L^\nu (\Omega )$ and $W_0^{s,N/s}(\Omega )\hookrightarrow L^\nu (\Omega ,|x|^{-\beta })$ are compact.

Proof

By [33], we know that the embedding $W_0^{s,N/s}(\Omega )\hookrightarrow L^\nu (\Omega )$ is compact for any $\nu \in [1,\infty )$.

Next we show that $W_0^{s,N/s}(\Omega )\hookrightarrow L^\nu (\Omega ,|x|^{-\beta })$ is compact. To this aim, we choose $t>1$ close to 1 such that $\beta t<N$. Then for any bounded sequence $(u_n)_n$ in $W_0^{s,N/s}(\Omega )$, we have

$$\begin{aligned} \int _\Omega \frac{1}{|x|^\beta }|u_n-u|^\nu dx \le&\left( \int _\Omega \frac{1}{|x|^{\beta t}} dx\right) ^{\frac{1}{t}} \left( \int _\Omega |u_n-u|^{\frac{\nu t}{t-1}} dx\right) ^{\frac{t-1}{t}} \\ \le&\, C\left( \int _\Omega |u_n-u|^{\frac{\nu t}{t-1}} dx\right) ^{\frac{t-1}{t}}. \end{aligned}$$

Note that the embedding $W_0^{s,N/s}(\Omega )\hookrightarrow L^{\frac{\nu t}{t-1}}(\Omega )$ is compact. Thus,

$$\begin{aligned} \int _\Omega \frac{1}{|x|^\beta }|u_n-u|^\nu dx\rightarrow 0. \end{aligned}$$

This proves the theorem. $\square$

Theorem 2.3

Let $N\ge 2$ and let $\Omega$ be a bounded domain in ${\mathbb {R}}^N$ containing the origin. Assume $u\in W_0^{s,N/s}(\Omega )$. Then for any $\alpha \ge 0$ and $\beta \in [0,N)$ there holds

$$\begin{aligned} \int _\Omega \frac{\exp (\alpha |u|^{N/(N-s)})}{|x|^\beta }dx<\infty . \end{aligned}$$

Moreover, for all $0\le \alpha <\left( 1-\frac{\beta }{N}\right) \alpha _{N,s}$ there holds

$$\begin{aligned} \sup _{[u]_{s,N/s}\le 1}\int _\Omega \frac{\exp (\alpha |u|^{N/(N-s)})}{|x|^\beta }dx<\infty , \end{aligned}$$

and the supremum is $\infty$ for $\alpha >\left( 1-\frac{\beta }{N}\right) \alpha _{N,s}^*$.

Proof

Choose $\sigma >1$ such that $\sigma \beta <N$. Then by the Hölder inequality and the fractional Trudinger–Moser inequality, we have

$$\begin{aligned} \int _\Omega \frac{\exp (\alpha |u|^{N/(N-s)})}{|x|^\beta }dx&\le \left( \int _\Omega \exp \left( \alpha \frac{\sigma }{\sigma -1}|u|^{N/(N-s)}\right) dx\right) ^{\frac{\sigma -1}{\sigma }} \left( \int _\Omega \frac{1}{|x|^{\sigma \beta }}dx\right) ^{\frac{1}{\sigma }}<\infty , \end{aligned}$$

being $\beta \sigma <N.$ If $\alpha <(1-\frac{\beta }{N})\alpha _{N,s}$, we can choose $\sigma >1$ is sufficiently close 1 such that $\sigma \alpha <\alpha _{N,s}$ and $\sigma (\sigma -1)^{-1}<\frac{N}{\beta }$. Then by the Hölder inequality and the fractional Trudinger–Moser inequality, we deduce that

$$\begin{aligned} \sup _{\Vert u\Vert \le 1}\int _\Omega \frac{\exp (\alpha |u|^{N/(N-s)})}{|x|^\beta }dx \le&\sup _{\Vert u\Vert \le 1}\left( \int _\Omega \exp (\alpha \sigma |u|^{N/(N-s)})dx\right) ^{\frac{1}{\sigma }} \left( \int _{\Omega }\frac{1}{|x|^{\beta \frac{\sigma }{\sigma -1}}}dx\right) ^{\frac{\sigma -1}{\sigma }}\\ <&\infty . \end{aligned}$$

Now we define the Moser functions which have been used in [40]:

$$\begin{aligned} {\widetilde{G}}_n(x)= \frac{1}{\gamma _{s,N}^{s/N}}{\left\{ \begin{array}{ll} |\ln n|^{\frac{N-s}{N}}\ \ &{}\mathrm{if}\ |x|\le \frac{1}{n},\\ \frac{|\ln |x||}{|\ln n|^{\frac{s}{N}}}\ \ &{}\mathrm{if}\ \frac{1}{n}<|x|<1,\\ 0\ \ &{}\mathrm{if}\ |x|\ge 1, \end{array}\right. } \end{aligned}$$

where

$$\begin{aligned} \gamma _{s,N}: =\frac{2(N\omega _N)^2\Gamma \left( \frac{N}{s}+1\right) }{N!}\sum _{k=0}^\infty \frac{N+k-1}{k!} \frac{1}{(N+2k)^{N+s}}. \end{aligned}$$

By the result in [40], we get

$$\begin{aligned} \lim _{n\rightarrow \infty }[{\widetilde{G}}_n(x)]_{s,N/s}^{N/s}=1. \end{aligned}$$

Choose $R>\varepsilon >0$ such that $B_R(0)\subset \Omega$ and define

$$\begin{aligned} G_n(x)={\widetilde{G}}_n(x/R), \end{aligned}$$

then $G_n(x)\in W^{s,N/s}_0(\Omega )$, the support of $G_n(x)$ is the ball $B_R(0)$ and

$$\begin{aligned} \lim _{n\rightarrow 0}[G_n]_{s,N/s}=1. \end{aligned}$$

(2.1)

Consider $\omega _n=\frac{G_n}{[G_n]_{s,N/s}}$, then we can write

$$\begin{aligned} \omega _n^{N/(N-s)} =\gamma _{s,N}^{-s/(N-s)}\ln n+d_n\ \ \mathrm{for}\ |x|\le \frac{R}{n}. \end{aligned}$$

Moreover, we have

$$\begin{aligned}&\frac{d_n}{\ln n}\rightarrow 0 \ \ \mathrm{as}\ \ n\rightarrow \infty . \end{aligned}$$

(2.2)

Thus, for $\alpha >(N-\beta )\gamma _{s,N}^{\frac{s}{N-s}}$, we deduce that

$$\begin{aligned}&\int _\Omega \frac{\exp (\alpha |\omega _n|^{N/(N-s)})}{|x|^\beta }dx\\&\quad \ge \int _{B_{R/n}(0)}\frac{\exp (\alpha |\omega _n|^{N/(N-s)})}{|x|^\beta }dx\\&\quad =\exp [\alpha (\gamma _{s,N}^{-s/(N-s)}\ln n+d_n)]\frac{(\frac{R}{n})^{N-\beta }\omega _{N-1}}{N-\beta }\\&\quad =R^{N-\beta }\omega _{N-1}\exp [(\alpha \gamma _{s,N}^{-s/(N-s)}-N+\beta )\ln n+\alpha d_n]\rightarrow \infty \ \ \mathrm{as }\ \ n\rightarrow \infty , \end{aligned}$$

which together with

$$\begin{aligned} \sup _{[u]_{s,N/s}\le 1}\int _\Omega \frac{\exp (\alpha |u|^{N/(N-s)})}{|x|^\beta }dx \ge \int _\Omega \frac{\exp (\alpha |\omega _n|^{N/(N-s)})}{|x|^\beta }dx \end{aligned}$$

yields that

$$\begin{aligned} \sup _{[u]_{s,N/s}\le 1}\int _\Omega \frac{\exp (\alpha |u|^{N/(N-s)})}{|x|^\beta }dx=\infty . \end{aligned}$$

It follows from [40] that $\alpha _{N,s}^*=N\gamma _{s,N}^{s/(N-s)}$. In conclusion, the proof is complete. $\square$

We give a singular fractional version of theorem of P.L. Lions ( [28]).

Theorem 2.4

Let $(u_n)_n$ be sequence in $W_0^{s,N/s}(\Omega )$ satisfying $[u_n]_{s,N/s}=1$ and converging weakly to a nonzero function u. Then for any $\alpha <(1-\frac{\beta }{N})\alpha _{N,s}(1-[u]_{s,N/s}^{N/s})^{-s/(N-s)}$ and $0\le \beta <N$,

$$\begin{aligned} \sup _n\int _\Omega \frac{\exp \left( \alpha |u_n|^{N/(N-s)}\right) }{|x|^\beta }dx<\infty . \end{aligned}$$

Proof

By the Hölder inequality, we obtain

$$\begin{aligned} \int _\Omega \frac{\exp \left( \alpha |u_n|^{N/(N-s)}\right) }{|x|^\beta }dx \le \left( \int _\Omega \exp \left( t\alpha |u_n|^{N/(N-s)}\right) dx\right) ^{\frac{1}{t}} \left( \int _\Omega \frac{1}{|x|^{\beta \frac{t}{t-1}}}dx\right) ^{\frac{t-1}{t}}. \end{aligned}$$

where $t>\frac{N}{N-\beta }$ sufficiently close to $\frac{N}{N-\beta }$ such that $\alpha t<\alpha _{N,s}(1-[u]_{s,N/s}^{N/s})^{-s/(N-s)}$. By Theorem 2.2 in [41], we have

$$\begin{aligned} \sup _{n}\left( \int _\Omega \exp \left( t\alpha |u_n|^{N/(N-s)}\right) dx\right) ^{\frac{1}{t}}<\infty . \end{aligned}$$

Clearly, from $t>\frac{N}{N-\beta }$, one can deduce that

$$\begin{aligned} \int _\Omega \frac{1}{|x|^{\beta \frac{t}{t-1}}}dx<\infty . \end{aligned}$$

Therefore, the desired result holds true. $\square$

To study the nonnegative solutions of problems (1.1) and (1.8), we define the associated functionals $I_\lambda , I:W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\rightarrow {\mathbb {R}}$ as

$$\begin{aligned}&I_\lambda (u)=\frac{s}{N}{\mathscr {M}}(\Vert u\Vert ^{N/s}) -\int _{\Omega }\frac{1}{|x|^\beta }F(x,u)dx- \lambda \int _{\Omega }h(x)udx \end{aligned}$$

and

$$\begin{aligned}&I(u)=\frac{s}{N}{\mathscr {M}}(\Vert u\Vert ^{N/s}) -\int _{\Omega }\frac{1}{|x|^\beta }F(x,u)dx. \end{aligned}$$

Since f is continuous and satisfies $(f_1)$ (or $(f_1^\prime )$) and $(f_3)$, for any $0<\varepsilon <\lambda _1$, $\alpha >\alpha _0$ and $q\ge 0$, there exists $C=C(\varepsilon ,\alpha ,q)>0$ such that

$$\begin{aligned} |F(x,t)|\le \frac{s}{N}{\mathscr {M}}(1)(\lambda _1-\varepsilon ) |t|^{\frac{\theta N}{s}}+C|t|^{q}\exp (\alpha |t|^{\frac{N}{N-s}})\ \ \forall (x,t)\in \Omega \times {\mathbb {R}}. \end{aligned}$$

(2.3)

If $(f_1^\prime )$ holds, then the $\alpha >0$ in (2.3) is arbitrary. Using (2.3), Theorem 2.3 and the assumption on ${\mathcal {K}}$, one can verify that the functionals $I_\lambda$ and I are well defined, of class $C^1(W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ),{\mathbb {R}})$. Moreover,

$$\begin{aligned} \langle I_\lambda ^\prime (u),v\rangle = M(\Vert u\Vert ^{N/s})&\langle u,v\rangle _{s,N/s}-\int _{\Omega }\frac{f(x,u)}{|x|^\beta }vdx- \lambda \int _{\Omega }hvdx \end{aligned}$$

and

$$\begin{aligned} \langle I^\prime (u),v\rangle = M(\Vert u\Vert ^{N/s})&\langle u,v\rangle _{s,N/s}-\int _{\Omega }\frac{f(x,u)}{|x|^\beta }vdx \end{aligned}$$

for all $u,v\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. From now on, $\langle \cdot ,\cdot \rangle$ denotes the duality pairing between $\big (W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\big )'$ and $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Evidently, the critical points of $I_\lambda$ and I are exactly the weak solutions of problem (1.1) and problem (1.8), respectively. Moreover, the following lemma shows that any nontrivial weak solution of problem (1.1) or problem (1.8) is nonnegative.

Lemma 2.1

If $h(x)\ge 0$ for almost every $x\in \Omega$, then for all $\lambda >0$ any nontrivial critical point of functional $I_\lambda$ is nonnegative. Similarly, any nontrivial critical point of functional I is also nonnegative.

Proof

Fix $\lambda >0$ and let $u_\lambda \in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\setminus \{0\}$ be a critical point of functional $I_\lambda$. Clearly, $u_\lambda ^-=\max \{-u,0\}\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Then $\langle I_\lambda ^\prime (u_\lambda ),-u_\lambda ^-\rangle =0$, a.e.

$$\begin{aligned} M(\Vert u_\lambda \Vert ^{N/s})\langle u_\lambda ,-u_\lambda ^-\rangle _{s,N/s} =\int _{\Omega }\frac{1}{|x|^\beta }f(x,u_\lambda )(-u_\lambda ^-)dx +\lambda \int _{\Omega }h(-u_\lambda ^-)dx. \end{aligned}$$

Observe that for a.e. $x, y\in \Omega$,

$$\begin{aligned}&|u_\lambda (x)-u_\lambda (y)|^{N/s-2}(u_\lambda (x)-u_\lambda (y))(-u_\lambda ^-(x)+u_\lambda (y)^-)\\&\quad =|u_\lambda (x)-u_\lambda (y)|^{N/s-2}u_\lambda ^+(x)u_\lambda ^-(y)\\&\qquad +|u_\lambda (x)-u_\lambda (y)|^{N/s-2}u_\lambda ^-(x)u_\lambda ^+(y)+[u_\lambda ^-(x)-u_\lambda ^-(y)]^p\\&\quad \ge |u_\lambda ^--u_\lambda ^-(y)|^{N/s}, \end{aligned}$$

$f(x,u_\lambda )u_\lambda ^-=0$ a.e. $x\in \Omega$ by assumption and $h(-u_\lambda ^-)\le 0$ a.e. in $\Omega$. Hence,

$$\begin{aligned} M(\Vert u_\lambda \Vert ^{N/s})\Vert u_\lambda ^-\Vert ^{N/s}\le 0. \end{aligned}$$

This, together with $\Vert u_\lambda \Vert >0$ and $(M_1)$, implies that $u_\lambda ^-\equiv 0$, that is $u_\lambda \ge 0$ a.e. in $\Omega$.

Similarly, one can verify that any nontrivial critical point of functional I is nonnegative. $\square$

3 The Subcritical Case

Let us recall that $I_\lambda$ satisfies the $(PS)_c$ condition in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ at level $c\in {\mathbb {R}}$, if any $(PS)_c$ sequence $(u_n)_{n}\subset W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$, namely a sequence such that $I_\lambda (u_n)\rightarrow c$ and $I_\lambda ^\prime (u_n)\rightarrow 0$ as $n\rightarrow \infty$, admits a strongly convergent subsequence in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$.

In the sequel, we shall make use of the well-known mountain pass theorem. For the reader’s convenience, we state it as follows (see for example [46]).

Theorem 3.1

Let X be a real Banach space and $J\in C^1(X,{\mathbb {R}})$ with $J(0)=0$. Suppose that

(i):

there exist $\rho ,\alpha >0$ such that $J(u)\ge \alpha$ for all $u\in X$, with $\Vert u\Vert _{X}=\rho$;

(ii):

there exists $e\in X$ satisfying $\Vert e\Vert _{X}>\rho$ such that $J(e)<0$.

Define $\Gamma =\{\gamma \in C^1([0,1];X):\gamma (0)=1,\gamma (1)=e\}$. Then

$$\begin{aligned} c=\inf _{\gamma \in \Gamma }\max _{0\le t\le 1} J(\gamma (t))\ge \alpha \end{aligned}$$

and there exists a $(PS)_c$ sequence $(u_n)_n\subset X$.

To find a mountain pass solution of problem (1.1), let us first verify the validity of the conditions of Theorem 3.1.

Lemma 3.1

(Mountain Pass Geometry I) Assume that $(f_1)$ and $(f_4)$ hold. Then there exist $\Lambda ^*>0$, $\rho >0$ and $\sigma >0$ such that $I_\lambda (u)\ge \sigma$ for any $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ with $\Vert u\Vert =\rho$, and all $0<\lambda <\Lambda ^*$.

Proof

Since f satisfies subcritical growth condition $(f_1)$, for $q>\theta N/s$ and any $\alpha >0$, we have

$$\begin{aligned} \int _\Omega F(x,u)dx&\le \frac{s}{N}{\mathscr {M}}(1)\left( \lambda _1-\varepsilon \right) \int _\Omega \frac{1}{|x|^\beta }|u|^{\theta N/s}dx+C \int _\Omega |u|^q\frac{1}{|x|^\beta }\exp (\alpha |u|^{\frac{N}{N-s}})dx\nonumber \\&\le \frac{s}{N}{\mathscr {M}}(1)\left( 1-\frac{\varepsilon }{\lambda _1}\right) \Vert u\Vert ^{\theta N/s}\nonumber \\&\qquad +C\left( \int _\Omega \frac{1}{|x|^\beta }|u|^{2q}dx\right) ^{1/2} \left( \int _\Omega \frac{1}{|x|^\beta } \exp (2\alpha \Vert u\Vert ^{\frac{N}{N-s}}(u/\Vert u\Vert )^{\frac{N}{N-s}})dx\right) ^{1/2}, \end{aligned}$$

(3.1)

for all $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ and any $\varepsilon \in (0,\lambda _1)$. Since $0\le \beta <N$, we can choose $\nu >1$ close to 1 such that $\beta \nu <N$. It follows from Theorem 2.1 and $(K_2)$ that there exists $C>0$ such that

$$\begin{aligned} \left( \int _\Omega \frac{1}{|x|^\beta }|u|^{2q}dx\right) ^{1/2}\le \left( \int _\Omega \frac{1}{|x|^{\nu \beta }}dx\right) ^{\frac{1}{2\nu }} \left( \int _\Omega |u|^{\frac{2q\nu }{\nu -1}}dx\right) ^{\frac{\nu -1}{2\nu }} \le C\Vert u\Vert ^q. \end{aligned}$$

Thus, we deduce from (3.1) that

$$\begin{aligned} \int _\Omega F(x,u)dx&\le \frac{s}{N}{\mathscr {M}}(1)\left( 1-\frac{\varepsilon }{\lambda _1}\right) \Vert u\Vert ^{\theta N/s}\nonumber \\&\quad +C\Vert u\Vert ^q \left( \int _\Omega \frac{1}{|x|^\beta } \exp (2\alpha \Vert u\Vert ^{\frac{N}{N-s}}(u/\Vert u\Vert )^{\frac{N}{N-s}})dx\right) ^{1/2}, \end{aligned}$$

(3.2)

for all $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. On the other hand, by $(M_2)$ one can deduce

$$\begin{aligned} {\mathscr {M}}(t)\ge {\mathscr {M}}(1)t^\theta \quad \text{ for } \text{ all } t\in [0,1]. \end{aligned}$$

(3.3)

Thus, combining (3.2) with (3.3), we obtain

$$\begin{aligned} I_\lambda (u) \ge&\frac{s{\mathscr {M}}(1)\varepsilon }{N\lambda _1}\Vert u\Vert ^{\theta N/s}-C\Vert u\Vert ^q \left( \int _\Omega \frac{1}{|x|^\beta }\exp (2\alpha \rho _1^{\frac{N}{N-s}}(u/\Vert u\Vert )^{\frac{N}{N-s}})dx\right) ^{1/2}\\&-\lambda \Vert h\Vert _*\Vert u\Vert , \end{aligned}$$

for all $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ with $\Vert u\Vert \le \rho _1\le 1$, and $\varepsilon \in (0,\lambda _1)$. Here $\Vert h\Vert _*$ denotes $\Vert h\Vert _{(W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ))^*}$. Choosing $2\alpha \rho _1^{N/(N-s)}\le \left( 1-\beta /N\right) \alpha _{N,s}$ and using Theorem 2.3, we get

$$\begin{aligned} I_\lambda (u)\ge \Vert u\Vert ^{\theta N/s}\frac{s{\mathscr {M}}(1)\varepsilon }{N\lambda _1} -C\Vert u\Vert ^{q}-\lambda \Vert h\Vert _*\Vert u\Vert . \end{aligned}$$

Fix $\varepsilon \in (0,\lambda _1)$ and define

$$\begin{aligned} g(t)=\frac{s{\mathscr {M}}(1)\varepsilon }{N\lambda _1}t^{\frac{\theta N}{s}-1}-Ct^{q-1}, \ \ \mathrm{for\ all}\ t\in [0,\rho _1]. \end{aligned}$$

Due to $\theta N/s<q$, we can choose $0<\rho \le \rho _1<1$ such that $g(\rho )>0$. Thus, $I_\lambda (u)\ge \sigma := \rho \left( g(\rho )-\lambda \Vert h\Vert _*\right) >0$ for all $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ with $\Vert u\Vert =\rho$, and $0<\lambda <\Lambda ^*:=\frac{g(\rho )}{\Vert h\Vert _*}$. $\square$

Lemma 3.2

(Mountain Pass Geometry II) Assume that $(f_1)$–$(f_2)$ hold. Then there exists a nonnegative function $e\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ independent of $\lambda$, such that $I_\lambda (e)<0$ and $\Vert e\Vert \ge \rho$ for all $\lambda \in {\mathbb {R}}^+$, where $\rho$ is given in Lemma 3.1.

Proof

By $(M_2)$, one can deduce that

$$\begin{aligned} {\mathscr {M}}(t)\le {\mathscr {M}}(1)t^\theta \quad \text{ for } \text{ all } t\ge 1. \end{aligned}$$

(3.4)

On the other hand, using $(f_2)$ and the continuity of f, there exist positive constants $C_1,C_2>0$ such that

$$\begin{aligned} F(x,t)\ge C_1 t^{\mu }-C_2\ \ \mathrm{for\ all}\ x\in \Omega \ \mathrm{and}\ t\ge 0. \end{aligned}$$

(3.5)

Now, choose nonnegative function $v_0\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ with $\int _\Omega \frac{1}{|x|^\beta }|v_0|^\mu dx>0$ and $\Vert v_0\Vert =1$. Then for all $t\ge 1$, we have

$$\begin{aligned} I_\lambda (tv_0)&\le \frac{s}{N}{\mathscr {M}}(1)t^{\theta N/s}\Vert v_0\Vert ^{\theta N/s} -C_1t^{\mu }\int _{\Omega }\frac{1}{|x|^\beta }|v_0|^\mu dx\\&\quad +C_2\int _\Omega \frac{1}{|x|^\beta }dx-t\lambda \int _\Omega hv_0dx\\&\le \frac{s}{N}{\mathscr {M}}(1)t^{\theta N/s} -C_1t^{\mu }\int _{\Omega }\frac{1}{|x|^\beta }|v_0|^\mu dx+C\ \ \rightarrow -\infty \ \ \mathrm{as}\ t\rightarrow \infty , \end{aligned}$$

thanks to $\theta N/s<\mu$. The lemma is proved by taking $e=T_0v_0$, with $T_0>0$ so large that $\Vert e\Vert \ge \rho$ and $I_\lambda (e)<0$. $\square$

Lemma 3.3

(The $(PS)_{c}$ condition) Let $(M_1)-(M_2)$ and $(f_1)$, $(f_2)$, $(f_4)$ hold. Then the functional $I_\lambda$ satisfies the $(PS)_{c}$ condition for all $c\in {\mathbb {R}}$.

Proof

Let $(u_n)_n$ be a $(PS)_c$ sequence in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Then

$$\begin{aligned} I_\lambda (u_n)\rightarrow c\ \ \mathrm{and}\ I^\prime _\lambda (u_n)\rightarrow 0. \end{aligned}$$

If $d:=\inf _{n\ge 1}\Vert u_n\Vert =0$, either 0 is an isolated point or accumulation point of the sequence $(\Vert u_n\Vert )_n$. If 0 is an isolated point, then there is a subsequence $(u_{n_k})_k$ such that

$$\begin{aligned} \inf _{k\in {\mathbb {N}}}\Vert u_{n_k}\Vert =d>0. \end{aligned}$$

Otherwise, 0 is an accumulation point of the sequence $(\Vert u_n\Vert )_n$ and so there exists a subsequence $(u_{n_k})_k$ of $(u_{n})_n$ such that $u_{n_k}\rightarrow 0$ strongly in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ as $k\rightarrow \infty$. Thus, we need only consider the case $d:=\inf _{n\ge 1}\Vert u_n\Vert >0$.

In the following, we assume that $d:=\inf _{n\ge 1}\Vert u_n\Vert >0$. We first show that $(u_n)_n$ is bounded in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Using $(M_1), (M_2)$ and $(f_2)$ with $\mu >\frac{\theta N}{s}$, we get

$$\begin{aligned} C+\Vert u_n\Vert&\ge I_\lambda (u_n)-\frac{1}{\mu }\langle I_\lambda ^\prime (u_n),u_n\rangle \nonumber \\&\ge \left( \frac{s}{N}-\frac{1}{\mu }\right) M(\Vert u_n\Vert ^{N/s})\Vert u_n\Vert ^{N/s}- \left( 1-\frac{1}{\mu }\right) \lambda \Vert h\Vert _*\Vert u_n\Vert \nonumber \\&\ge \left( \frac{s}{N\theta }-\frac{1}{\mu }\right) \kappa \Vert u_n\Vert ^{N/s} -\left( 1-\frac{1}{\mu }\right) \lambda \Vert h\Vert _*\Vert u_n\Vert . \end{aligned}$$

(3.6)

It follows from (3.6) that $(u_n)_n$ is bounded in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$.

Next we show that $(u_n)_n$ has a convergence subsequence in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Going if necessary to a subsequence, there exists a function $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ such that

$$\begin{aligned}&u_n\rightharpoonup u\ \ \mathrm{weakly in}\ W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ),\nonumber \\&u_n\rightarrow u\ \ \mathrm{strongly in}\ L^{\nu }(\Omega )( \nu \ge 1),\nonumber \\&u_n\rightarrow u\ \ \mathrm{a.e. in}\ \Omega . \end{aligned}$$

(3.7)

Here we have used the compact embedding from $W_0^{s,N/s}(\Omega )$ to $L^\nu (\Omega )$ for any $\nu \ge 1$ (see Theorem 2.2) and the embedding $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\hookrightarrow W_0^{s,N/s}(\Omega )$ is continuous.

Next we show that

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _\Omega \frac{1}{|x|^\beta }f(x,u_n)(u_n-u)dx=0. \end{aligned}$$

(3.8)

Choose $\nu >1$ close to 1 and $\alpha$ small enough such that $\nu \alpha \Vert u_n\Vert ^{N/(N-s)}<\delta <\frac{N-\beta }{N}\alpha _{N,s}$. Thus, it follows from $(f_1)$ and $(f_4)$ that

$$\begin{aligned}&\left| \int _\Omega \frac{1}{|x|^\beta }f(x,u_n)(u_n-u)dx\right| \\&\quad \le C\left( \int _\Omega \frac{1}{|x|^\beta }|u_n|^{\theta N/s-1}|u_n-u|dx+\int _\Omega \frac{1}{|x|^\beta }|u_n-u|\exp (\alpha |u_n|^{N/(N-s)})dx\right) \\&\quad \le C\left[ \left( \int _\Omega \frac{|u_n-u|^{\frac{N\theta }{s}}}{|x|^\beta }dx\right) ^{\frac{s}{N\theta }}+\left( \int _\Omega \frac{|u_n-u|^{\frac{\nu }{\nu -1}}}{|x|^\beta } dx\right) ^{\frac{\nu -1}{\nu }}\right. \\&\qquad \left. \left( \int _{\Omega }\frac{1}{|x|^\beta }\exp [\nu \alpha \Vert u_n\Vert ^{\frac{N}{N-s}}(\frac{u_n}{\Vert u_n\Vert })^{\frac{N}{N-s}}]dx\right) ^{\frac{1}{\nu }}\right] \\&\quad \le C\left[ \left( \int _\Omega \frac{|u_n-u|^{\frac{N\theta }{s}}}{|x|^\beta }dx\right) ^{\frac{s}{N\theta }}+\left( \int _\Omega \frac{|u_n-u|^{\frac{\nu }{\nu -1}}}{|x|^\beta } dx\right) ^{\frac{\nu -1}{\nu }}\right] \rightarrow 0 \end{aligned}$$

as $n\rightarrow \infty$, thanks to Theorem 2.2. Thus, (3.8) holds true.

Since $(u_n)_n$ is a bounded $(PS)_{c}$ sequence, we get as $n\rightarrow \infty$

$$\begin{aligned} \langle I_\lambda ^\prime (u_n),u_n-u\rangle =&M(\Vert u_n\Vert ^{N/s})\langle u_n,u_n-u\rangle _{s,N/s} -\int _{\Omega }\frac{1}{|x|^\beta }f(x,u_n)(u_n-u)dx\\&-\lambda \int _\Omega h (u_n-u)dx\rightarrow 0, \end{aligned}$$

which implies that

$$\begin{aligned} M(\Vert u_n\Vert ^{N/s})\langle u_n,u_n-u\rangle _{s,N/s}\rightarrow 0. \end{aligned}$$

Moreover, one can prove that $\langle u,u_n-u\rangle _{s,N/s}\rightarrow 0$. Hence we obtain that

$$\begin{aligned} M(\Vert u_n\Vert ^{N/s})\left[ \langle u_n,u_n-u\rangle _{s,N/s}-\langle u,u_n-u\rangle _{s,N/s}\right] \rightarrow 0. \end{aligned}$$

By using a similar discussion as [33], we have $u_n\rightarrow u$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. This ends the proof. $\square$

Proof of Theorem 1.1

By Lemmas 3.1 and 3.2, we know that there exists a threshold $\lambda ^*>0$ such that for all $0<\lambda <\lambda ^*$, $I_\lambda$ satisfies all the assumptions of Theorem 3.1. Hence there exists a $(PS)_{c}$ sequence. Moreover, by Lemma 3.3, for all $\lambda <\lambda ^*$ the functional $I_\lambda$ admits a nontrivial critical point $u_1\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. The critical point $u_1$ is a nontrivial mountain pass solution of problem (1.1). Furthermore, Lemma 2.1 shows that $u_1$ is nonnegative.

Next we show that problem has another nontrivial and nonnegative solution. Define

$$\begin{aligned} c_\lambda =\inf _{u\in {\overline{B}}_{\rho }}I_\lambda (u)\ \ \mathrm{and}\ \inf _{x\in \partial B_{\rho }}I_\lambda (u)>0, \end{aligned}$$

where $\rho >0$ is given by Lemma 3.1 and $B_{\rho }=\{u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ):\Vert u\Vert <\rho \}$. Now we claim that $c_\lambda <0$. Consider the following problem

$$\begin{aligned} {\left\{ \begin{array}{ll} {\mathcal {L}}_K v=h(x)\ \ &{}\mathrm{in }\ \Omega ,\\ v=0\ \ \ \ &{}\mathrm{on}\ {\mathbb {R}}^N\setminus \Omega . \end{array}\right. } \end{aligned}$$

By the direct method and $0\le h\in (W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ))^*$, one can verify that the above problem has a unique nonnegative solution $v\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Moreover, $\Vert v\Vert ^{N/s}=\int _{\Omega }h(x)vdx>0$. Then

$$\begin{aligned} I_\lambda (tv)\le \left( \max _{0\le \tau \le 1}M(\tau )\right) \frac{st^{N/s}}{N} -\lambda t\int _{\Omega }h(x)vdx \end{aligned}$$

for all $0\le t\le 1$ small enough. Since $N/s>1$, it follows that $I_\lambda (tv)<0$ for $t\in (0,1)$ small enough. Thus, the claim is true. By Ekeland’s principle and a standard argument, there exists a sequence $(u_n)_n\subset B_{\rho }$ such that $I_\lambda (u_n)\rightarrow c_\lambda <0$ and $I^\prime _\lambda (u_n)\rightarrow 0$ as $n\rightarrow \infty$. Furthermore, Lemma 3.3 yields that $(u_n)_n$ converges to some $u_2$ strongly in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$, and so $u_2$ is a nontrivial and nonnegative solution of problem (1.1). Clearly, $u_1$ and $u_2$ are two distinct solutions. $\square$

4 The Critical Case

In this section, we consider the critical case of problem (1.1). Without further mentioning, we always assume that f satisfies $(f_1^\prime ), (f_2)-(f_4)$, and $M(t)=t^{\theta -1}$ with $\theta >1$. To prove Theorem 1.2, we first give several necessary results.

Lemma 4.1

Under assumptions $(f_1^\prime ), (f_2), (f_3)$, the functional $I_\lambda$ satisfies the conditions of the mountain pass theorem:

(1):

$I(0)=0$;

(2):

there exist $\Lambda _2>0$, $\rho _2>0$ and $\sigma _2>0$ such that for $0<\lambda <\Lambda _2$, $I_\lambda (u)\ge \sigma _2>0$ for any $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$, with $\Vert u\Vert =\rho _2$. Furthermore, $\rho _2$ can be chosen small enough such that $\rho _2<(\frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0})^{(N-s)/N}$;

(3):

there exists a nonnegative function $e\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ independent of $\lambda$, such that $I_\lambda (e)<0$ and $\Vert e\Vert \ge \rho _2$ for all $\lambda \in {\mathbb {R}}^+$.

Proof

Clearly $I(0)=0$. The rest of proofs are similar to the proofs of Lemmas 3.1–3.2. $\square$

Lemma 4.2

There exists $\Lambda _3>0$ such that for all $0<\lambda <\Lambda _3$, the functional $I_\lambda$ satisfies the $(PS)_c$ condition for $c<\frac{1}{4}(\frac{s}{N\theta }-\frac{1}{\mu })\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{\frac{(N-s)\theta }{s}}$.

Proof

Assume $(u_n)_n\subset W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ satisfies

$$\begin{aligned} I_\lambda (u_n)\rightarrow c,\ \ I_\lambda ^\prime (u_n)\rightarrow 0 \ \ \mathrm{as}\ n\rightarrow \infty . \end{aligned}$$

We first consider $c>0$. By $(f_2)$ and the assumption on M, we have

$$\begin{aligned} c+o(1)\Vert u_n\Vert&\ge I_\lambda (u_n)-\frac{1}{\mu }\langle I_\lambda ^\prime (u_n),u_n\rangle \\&\ge \left( \frac{s}{N\theta }-\frac{1}{\mu }\right) M(\Vert u_n\Vert ^{N/s})\Vert u_n\Vert ^{N/s} -\lambda \left( 1-\frac{1}{\mu }\right) \Vert h\Vert _*\Vert u_n\Vert , \end{aligned}$$

which means that $(u_n)_n$ is bounded in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Thus, we get

$$\begin{aligned} \left( \frac{s}{N\theta }-\frac{1}{\mu }\right) \Vert u_n\Vert ^{\theta \frac{N}{s}}\le c+o(1)\Vert u_n\Vert + \lambda \left( 1-\frac{1}{\mu }\right) \Vert h\Vert _{*}\Vert u_n\Vert . \end{aligned}$$

(4.1)

For any $\varepsilon >0$, by the Young inequality we have

$$\begin{aligned} \lambda \left( 1-\frac{1}{\mu }\right) \Vert h\Vert _{*}\Vert u_n\Vert \le \varepsilon \Vert u_n\Vert ^{\frac{N\theta }{s}} +\varepsilon ^{-\frac{s}{N\theta -s}} \left( \lambda \left( 1-\frac{1}{\mu }\right) \Vert h\Vert _{*}\right) ^{\frac{N\theta }{N\theta -s}}. \end{aligned}$$

Taking $\varepsilon =\frac{1}{2}\left( \frac{s}{N\theta }-\frac{1}{\mu }\right)$ in above inequality and putting it into (4.1), we obtain

$$\begin{aligned} \frac{1}{2}\left( \frac{s}{N\theta }-\frac{1}{\mu }\right) \Vert u_n\Vert ^{\theta \frac{N}{s}}&\le c+o(1)\Vert u_n\Vert + \left( \frac{1}{2}\left( \frac{s}{N\theta }-\frac{1}{\mu }\right) \right) ^{-\frac{s}{N\theta -s}}\\&\quad \times \left( \lambda \left( 1-\frac{1}{\mu }\right) \Vert h\Vert _{*}\right) ^{\frac{N\theta }{N\theta -s}}. \end{aligned}$$

It follows that

$$\begin{aligned} \limsup _{n\rightarrow \infty }\Vert u_n\Vert \le \left[ \frac{c}{\frac{1}{2}(\frac{s}{N\theta }-\frac{1}{\mu })} + \left( \frac{1}{2}\left( \frac{s}{N\theta }-\frac{1}{\mu }\right) \right) ^{-\frac{N\theta }{N\theta -s}} \left( \lambda \left( 1-\frac{1}{\mu }\right) \Vert h\Vert _{*}\right) ^{\frac{N\theta }{N\theta -s}}\right] ^{\frac{s}{N\theta }} \end{aligned}$$

Set

$$\begin{aligned} \Lambda _3^\prime =\frac{(s\mu -N\theta )}{2N\theta (\mu -1)\Vert h\Vert _{*}} \left[ \frac{1}{2}\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{\frac{(N-s)\theta }{s}}\right] ^{\frac{N\theta -s}{N\theta }}. \end{aligned}$$

Then for all $0<\lambda <\Lambda _3^\prime$, we get

$$\begin{aligned} \limsup _{n\rightarrow \infty } \Vert u_n\Vert <\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{\frac{N-s}{N}}, \end{aligned}$$

(4.2)

thanks to $c<\frac{1}{4}(\frac{s}{N\theta }-\frac{1}{\mu })\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{\frac{(N-s)\theta }{s}}$.

If $c<0$, then with a similar discussion as above, one can easily get that there exists $\Lambda ^{\prime \prime }_3>0$ such that the (PS) sequence satisfies (4.2).

Therefore, there exists $\Lambda _3=\min \{\Lambda _3^\prime ,\Lambda _3^{\prime \prime }\}$ such that (4.2) holds true.

It follows from (4.2) that there exist $n_0\in {\mathbb {N}}$ and $\delta >0$ such that $\Vert u_n\Vert ^{N/(N-s)}<\delta <\frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}.$ Choosing $\nu >1$ close to 1 and $\alpha >\alpha _0$ close to $\alpha _0$ such that we still have $\nu \alpha \Vert u_n\Vert ^{N/(N-s)}<\delta <\frac{N-\beta }{N}\alpha _{N,s}$. Thus, it follows from (2.2) with $q=1$ that

$$\begin{aligned}&\left| \int _\Omega \frac{1}{|x|^\beta }f(x,u_n)(u_n-u)dx\right| \\&\quad \le C\left( \int _\Omega \frac{1}{|x|^\beta }|u_n|^{\theta N/s-1}|u_n-u|dx+\int _\Omega \frac{1}{|x|^\beta }|u_n-u|\exp (\alpha |u_n|^{N/(N-s)})dx\right) \\&\quad \le C\left[ \left( \int _\Omega \frac{|u_n-u|^{\frac{N\theta }{s}}}{|x|^\beta }dx\right) ^{\frac{s}{N\theta }}+\left( \int _\Omega \frac{|u_n-u|^{\frac{\nu }{\nu -1}}}{|x|^\beta } dx\right) ^{\frac{\nu -1}{\nu }}\right. \\&\qquad \left. \left( \int _{\Omega }\frac{1}{|x|^\beta }\exp [\nu \alpha \Vert u_n\Vert ^{\frac{N}{N-s}}(\frac{u_n}{\Vert u_n\Vert })^{\frac{N}{N-s}}]dx\right) ^{\frac{1}{\nu }}\right] \\&\quad \le C\left[ \left( \int _\Omega \frac{|u_n-u|^{\frac{N\theta }{s}}}{|x|^\beta }dx\right) ^{\frac{s}{N\theta }}+\left( \int _\Omega \frac{|u_n-u|^{\frac{\nu }{\nu -1}}}{|x|^\beta } dx\right) ^{\frac{\nu -1}{\nu }}\right] \rightarrow 0 \end{aligned}$$

as $n\rightarrow \infty$, thanks to Theorem 2.2. Then using a similar discussion as Lemma 3.3, one can prove that $u_n\rightarrow u$ strongly in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$.

If $\inf _{n\ge 1}\Vert u_n\Vert =0$, we can proceed as in Lemma 3.3. $\square$

Proof of Theorem 1.2

By Lemma 4.1 and Theorem 3.1, there exists a sequence $(u_n)_n\subset W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ such that $I_\lambda (u_n)\rightarrow c_1$ and $I^\prime _\lambda (u_n)\rightarrow 0$, where

$$\begin{aligned} c_1=\inf _{\gamma \in \Gamma }\max _{0\le t\le 1} I_\lambda (\gamma (t))\ge \alpha \end{aligned}$$

and $\Gamma =\left\{ \gamma \in C^1([0,1];W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )):\gamma (0)=1,\gamma (1)=e\right\}$. Next we show that

$$\begin{aligned} c_1<\frac{1}{4}\left( \frac{s}{N\theta }-\frac{1}{\mu }\right) \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{\frac{(N-s)\theta }{s}}. \end{aligned}$$

(4.3)

Set

$$\begin{aligned} C_{q_0}:=\inf _{\varphi \in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\setminus \{0\} } \left\{ \Vert \varphi \Vert ^{N\theta /s}:\int _\Omega \frac{|\varphi |^{q_0}}{|x|^\beta } dx=1\right\} . \end{aligned}$$

Clearly, $C_{q_0}>0$. By Theorem 2.2, one can easily verify that there exists a nonnegative function $\varphi _0\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\setminus \{0\}$ such that

$$\begin{aligned} \Vert \varphi _0\Vert ^{N\theta /s} =C_{p_0}\ \ \mathrm{and}\ \ |\varphi _0|_{q_0,\beta }^{q_0}=1. \end{aligned}$$

In view of the proof of Lemma 4.1, we take $\gamma (t)=tT\varphi _0$, where $T>0$ is sufficiently large such that $e=T\varphi _0$. Hence, it follows from the definition of $c_1$ that

$$\begin{aligned} c_1<\max _{t\ge 0}I_\lambda (t\varphi _0), \end{aligned}$$

which implies that

$$\begin{aligned} c_1\le \max _{t\ge 0}\left\{ \frac{s}{N\theta }\Vert t\varphi _0\Vert ^{N\theta /s} -\int _\Omega \frac{1}{|x|^\beta }F(x,t\varphi _0)dx\right\} . \end{aligned}$$

Furthermore, from $(f_4)$, we obtain

$$\begin{aligned} c_1&\le \max _{t\ge 0}\left\{ t^{\frac{N\theta }{s}}\frac{s}{N\theta }\Vert \varphi _0\Vert ^{N\theta /s} -t^{q_0}\frac{{\mathcal {C}}_0}{q_0}\int _\Omega \frac{1}{|x|^\beta }|\varphi _0|^{q_0}dx\right\} \\&=\max _{t\ge 0}\left\{ t^{\frac{N\theta }{s}}\frac{s}{N\theta }C_{p_0} -t^{q_0}\frac{{\mathcal {C}}_0}{q_0}\right\} \\&=C_{p_0}^{\frac{q_0}{q_0-\frac{N\theta }{s}}} {\mathcal {C}}_0^{-\frac{N\theta }{sq_0-N\theta }} \left( \frac{s}{N\theta }-\frac{1}{q_0}\right) . \end{aligned}$$

By the assumption on ${\mathcal {C}}_0$, (4.3) holds.

Thus, it follows from Lemma 4.2 that there exists $\Lambda _4=\min \{\Lambda _2,\Lambda _3\}$ such that problem (1.1) has a nontrivial nonnegative solution.

To show that problem has another solution, we set

$$\begin{aligned} c_2=\inf _{u\in {\overline{B}}_{\rho _2}}I_\lambda (u), \end{aligned}$$

where $\rho _2>0$ is given by Lemma 4.1 and $B_{\rho _2}=\{u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ):\Vert u\Vert <\rho _2\}$. Then $\inf _{x\in \partial B_{\rho _2}}I_\lambda (u)>0$. With a similar discussion as the proof of Theorem 1.1, we can prove that $c_2<0$. By Lemma 4.1, we obtain

$$\begin{aligned} \rho _2<\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/N}. \end{aligned}$$

By Ekeland’s variational principle, there exists a sequence $(v_n)_n\subset B_{\rho _2}$ such that $I_\lambda (v_n)\rightarrow c_2\le 0$ and $I^\prime _\lambda (v_n)\rightarrow 0$, as $n\rightarrow \infty$. Observing that

$$\begin{aligned} \Vert v_n\Vert \le \rho _2<\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/N}, \end{aligned}$$

by Lemma 4.2, for all $\lambda \in (0,\Lambda _4)$, $(v_n)_n$ has a convergent subsequence still denoted by $(v_n)_n$ such that $v_n\rightarrow u_\lambda$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Thus, $u_\lambda$ is a nontrivial nonnegative solution with $I_\lambda (u_\lambda )<0$. Thus, the proof is complete. $\square$

5 Problem (1.1) Without Perturbation

In this section, we consider problem (1.8), i.e. problem (1.1) without perturbation term h and the Ambrosetti-Rabinowitz condition.

The following version of the mountain pass theorem, which will be used later, shows us the existence of a Cerami sequence at the mountain pass level.

Theorem 5.1

(See [10]) Let X be a real Banach space with its dual space $E^*$ and assume that $J\in C^1(X,{\mathbb {R}})$ satisfies

$$\begin{aligned} \max \{J(0),J(e)\}\le \varrho <\sigma \le \inf _{\Vert u\Vert _X=\rho }J(u), \end{aligned}$$

for some $\varrho ,\sigma ,\rho >0$ and $e\in X$ with $\Vert e\Vert _X>\rho$. Let c be characterized by

$$\begin{aligned} c=\inf _{\gamma \in \Gamma }\max _{0\le t\le 1}J(\gamma (t)), \end{aligned}$$

where $\Gamma =\{\gamma \in C([0,1],X):\gamma (0)=0, \gamma (1)=e\}$. Then there exists a Cerami sequence $(u_n)_n$ in X, that is,

$$\begin{aligned} J(u_n)\rightarrow c\ge \sigma ,\ \ (1+\Vert u_n\Vert _X)\Vert J^\prime (u_n)\Vert _{X^*} \rightarrow 0, \end{aligned}$$

as $n\rightarrow \infty$.

To this aim, let us first verify the validity of the conditions of Theorem 5.1.

Lemma 5.1

(Mountain Pass Geometry I) Assume that $(f_1^\prime )$ and $(f_3)$ hold. Then there exist $\rho >0$ and $\varrho >0$ such that $I(u)\ge \varrho$ for any $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ with $\Vert u\Vert =\rho$.

Proof

By (3.2) and (3.3), we obtain

$$\begin{aligned} I(u)&\ge \frac{s{\mathscr {M}}(1)\varepsilon }{N\lambda _1}\Vert u\Vert ^{\theta N/s}-C\Vert u\Vert ^q \left( \int _\Omega \frac{1}{|x|^\beta }\exp (2\alpha \rho _1^{\frac{N}{N-s}}(u/\Vert u\Vert )^{\frac{N}{N-s}})dx\right) ^{1/2}, \end{aligned}$$

for all $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ with $\Vert u\Vert \le \rho _1\le 1$, and $\varepsilon \in (0,\lambda _1)$. Choosing $2\alpha \rho _1^{N/(N-s)}\le \left( 1-\beta /N\right) \alpha _{N,s}$ and using Theorem 2.3, we get

$$\begin{aligned} I(u)\ge \frac{s{\mathscr {M}}(1)\varepsilon }{N\lambda _1}\Vert u\Vert ^{\theta N/s} -C\Vert u\Vert ^{q}. \end{aligned}$$

Fix $\varepsilon \in (0,\lambda _1)$. By virtue of $\theta N/s<q$, we can choose $0<\rho \le \rho _1<1$ such that $\frac{s{\mathscr {M}}(1)\varepsilon }{N\lambda _1} \rho ^{\theta N/s}-C\rho ^{q}>0$. Thus, $I(u)\ge \varrho := \varrho ^{\theta N/s}\frac{s{\mathscr {M}}(1)\varepsilon }{N\lambda _1} -C\rho ^{q}>0$ for all $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ with $\Vert u\Vert =\rho$. $\square$

Lemma 5.2

(Mountain Pass Geometry II) Assume that $(f_1^\prime )$, $(f_2)$ and $(f_3)$ hold. Then there exists a nonnegative function $e\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ such that $I(e)<0$ and $\Vert e\Vert \ge \rho$, where $\rho$ is given in Lemma 5.1.

Proof

Choose a nonnegative function $v_0\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ with $\int _\Omega \frac{1}{|x|^\beta }|v_0|^\mu dx>0$ and $\Vert v_0\Vert =1$. Then for all $t\ge 1$, we have by (3.4) and (3.5) that

$$\begin{aligned} I(tv_0)&\le \frac{s}{N}{\mathscr {M}}(1)t^{\theta N/s}\Vert v_0\Vert ^{\theta N/s} -C_1t^{\mu }\int _{\Omega }\frac{1}{|x|^\beta }|v_0|^\mu dx+C_2\int _\Omega \frac{1}{|x|^\beta }dx\\&\le \frac{s}{N}{\mathscr {M}}(1)t^{\theta N/s} -C_1t^{\mu }\int _{\Omega }\frac{1}{|x|^\beta }|v_0|^\mu dx+C\ \ \rightarrow -\infty \ \ \mathrm{as}\ t\rightarrow \infty , \end{aligned}$$

thanks to $\theta N/s<\mu$. The lemma is proved by taking $e=T_0v_0$, with $T_0>0$ so large that $\Vert e\Vert \ge \rho$ and $I(e)<0$. $\square$

By Theorem 5.1, there exists a Cerami sequence $(u_n)_n\subset W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ such that

$$\begin{aligned} I(u_n)\rightarrow c_* \ \ \mathrm{and}\ (1+\Vert u_n\Vert )\Vert I^\prime (u_n)\Vert \rightarrow 0\ \ \mathrm{as}\ n\rightarrow \infty , \end{aligned}$$

where

$$\begin{aligned} c_*=\inf _{\gamma \in \Gamma }\max _{t\in [0,1]}I(\gamma (t)), \end{aligned}$$

(5.1)

where $\Gamma =\left\{ \gamma \in C([0,1];W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )):\gamma (0)=0,\ \gamma (1)=e\right\} .$ Obviously, $c_*>0$ by Lemma 5.1. To get more precisely estimate of $c_*$, we first obtain the following result.

Lemma 5.3

Assume that $(f_1^\prime ), (f_3)$ and $(f_5)$ hold. Then there exists $n>0$ such that

$$\begin{aligned} \max _{t\ge 0}I(tG_n)< \frac{s}{N}{\mathscr {M}}\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) , \end{aligned}$$

where $G_n$ is given by Theorem 2.3.

Proof

Arguing by contradiction, we assume that

$$\begin{aligned} \max _{t\ge 0}I(tG_n)\ge \frac{s}{N}{\mathscr {M}}\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) . \end{aligned}$$

(5.2)

Since I possesses the mountain pass geometry, for each n, $\max _{t\ge 0} I(tG_n)$ is attained at some $t_n>0$, that is,

$$\begin{aligned} I(t_nG_n)=\max _{t\ge 0} I(tG_n). \end{aligned}$$

Using $F(x,t)\ge 0$ for all $(x,t)\in \Omega \times {\mathbb {R}}$, one can deduce that

$$\begin{aligned} {\mathscr {M}}\left( t_n^{\frac{N}{s}}\Vert G_n\Vert ^{N/s}\right) \ge {\mathscr {M}}\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) . \end{aligned}$$

Since ${\mathscr {M}}:[0,\infty )\rightarrow [0,\infty )$ is a nondecreasing function by $(M_1)$, we get

$$\begin{aligned} t_n^{\frac{N}{s}}\Vert G_n\Vert ^{N/s}\ge \left( \frac{N-\beta }{N} \frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}. \end{aligned}$$

It follows from $\Vert G_n\Vert ^{N/s}\rightarrow 1$ that

$$\begin{aligned} \liminf _{n\rightarrow \infty }t_n^{N/s}\ge \left( \frac{N-\beta }{N} \frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}. \end{aligned}$$

(5.3)

Due to

$$\begin{aligned} \frac{d}{dt}I(tG_n)\big |_{t=t_n}=0, \end{aligned}$$

we deduce

$$\begin{aligned} M(t_n^{N/s}\Vert G_n\Vert ^{N/s})t_n^{N/s}\Vert G_n\Vert ^{N/s}=&\int _\Omega \frac{1}{|x|^\beta }f(x,t_nG_n)t_nG_ndx\nonumber \\ \ge&\int _{B_{R_0}(0)}\frac{1}{|x|^\beta }f(x,t_nG_n). t_nG_n dx. \end{aligned}$$

(5.4)

Next we show that $(t_n)_n$ is bounded. Using change of variable, we deduce from (5.4) that

$$\begin{aligned}&M(t_n^{N/s}\Vert G_n\Vert ^{N/s})t_n^{N/s}\Vert G_n\Vert ^{N/s}\\&\quad \ge R_0^N\int _{B_{1}(0)}\frac{1}{|R_0x|^\beta }f(R_0x,t_n{\widetilde{G}}_n)t_n{\widetilde{G}}_ndx\\&\quad \ge R_0^N\int _{B_{\frac{1}{n}}(0)}\frac{1}{|R_0x|^\beta } f(R_0x,t_n\frac{1}{\gamma _{s,N}^{s/N}}(\ln n)^{(N-s)/N})t_n\frac{1}{\gamma _{s,N}^{s/N}}(\ln n)^{(N-s)/N}dx. \end{aligned}$$

Note that (5.3) implies that

$$\begin{aligned} \frac{t_n}{\gamma _{s,N}^{s/N}}(\ln n)^{(N-s)/N}\rightarrow \infty \ \ \mathrm{as}\ n\rightarrow \infty . \end{aligned}$$

It follows from $(f_5)$ that given $\delta >0$ there exists $t_\delta >0$ such that

$$\begin{aligned} f(x,t)t\ge (\beta _0-\delta )\exp \left( \frac{\alpha _0\alpha _{N,s}^*}{\alpha _{N,s}}t^{N/(N-s)}\right) \ \ \forall (x,t)\in \Omega \times [t_\delta ,\infty ). \end{aligned}$$

(5.5)

Thus, there exists $n_0\in {\mathbb {N}}$ such that

$$\begin{aligned}&f(R_0 x,t_n\frac{1}{\gamma _{s,N}^{s/N}}(\ln n)^{(N-s)/N})t_n\frac{1}{\gamma _{s,N}^{s/N}}(\ln n)^{(N-s)/N}\\&\quad \ge (\beta _0-\delta )\exp \left( \frac{\alpha _0\alpha _{N,s}^*}{\alpha _{N,s}}t_n^{N/(N-s)}\frac{1}{\gamma _{s,N}^{s/(N-s)}}\ln n\right) , \end{aligned}$$

for all $n\ge n_0$. Hence,

$$\begin{aligned}&M(t_n^{N/s}\Vert G_n\Vert ^{N/s})t_n^{N/s}\Vert G_n\Vert ^{N/s}\nonumber \\&\quad \ge (\beta _0-\delta )R_0^{N-\beta }\exp \left( \frac{\alpha _0\alpha _{N,s}^*}{\alpha _{N,s}}t_n^{N/(N-s)}\frac{1}{\gamma _{s,N}^{s/(N-s)}}\ln n\right) \omega _{N-1}\frac{1}{n^{N-\beta }}\nonumber \\&\quad =(\beta _0-\delta )\omega _{N-1}R_0^{N-\beta }\exp \left( \frac{\alpha _0\alpha _{N,s}^*}{\alpha _{N,s}}t_n^{N/(N-s)}\frac{1}{\gamma _{s,N}^{s/(N-s)}}\ln n\right) \exp (-(N-\beta )\ln n)\nonumber \\&\quad =(\beta _0-\delta )\omega _{N-1}R_0^{N-\beta }\exp \left[ \left( \frac{\alpha _0}{\alpha _{N,s}}t_n^{N/(N-s)}N-N+\beta \right) \ln n\right] . \end{aligned}$$

(5.6)

From $(M_2)$ and (5.3), we can conclude that

$$\begin{aligned} \frac{M(t_n^{N/s}\Vert G_n\Vert ^{N/s})t_n^{N/s}\Vert G_n\Vert ^{N/s}}{\exp \left[ \left( \frac{\alpha _0}{\alpha _{N,s}}t_n^{N/(N-s)}N-N+\beta \right) \ln n\right] }\rightarrow 0\ \ \mathrm{as}\ n\rightarrow \infty , \end{aligned}$$

which contradicts (5.6). Thus,

$$\begin{aligned} \limsup _{n\rightarrow \infty }t_n^{N/s}\le \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}, \end{aligned}$$

which together with (5.3) yields that

$$\begin{aligned} \lim _{n\rightarrow \infty }t_n^{N/s}=\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s} \end{aligned}$$

(5.7)

as $n\rightarrow \infty$.

Inspired by [12, 17, 33], we are going to estimate (5.4). In view of (5.5), for $0<\delta <\beta _0$ and $n\in {\mathbb {N}}$, we set

$$\begin{aligned} U_{n,\delta }:=\{x\in B_{R_0}(0):t_nG_n(x)\ge t_\delta \}\ \ \mathrm{and}\ \ V_{n,\delta }:=B_{R_0}(0)\setminus U_{n,\delta }. \end{aligned}$$

Splitting the integral (5.4) on $U_{n,\delta }$ and $V_{n,\delta }$ and using (5.5), we deduce

$$\begin{aligned}&M(t_n^{N/s}\Vert G_n\Vert ^{N/s})t_n^{N/s}\Vert G_n\Vert ^{N/s}\nonumber \\&\quad \ge (\beta _0-\delta )\int _{B_{R_0}(0)}\frac{1}{|x|^\beta }\exp \left( \frac{\alpha _0\alpha _{N,s}^*}{\alpha _{N,s}}(t_nG_n)^{N/(N-s)}\right) dx \nonumber \\&\qquad - (\beta _0-\delta )\int _{V_{n,\delta }}\frac{1}{|x|^\beta } \exp \left( \frac{\alpha _0\alpha _{N,s}^*}{\alpha _{N,s}}(t_nG_n)^{N/(N-s)}\right) dx\nonumber \\&\qquad +\int _{V_{n,\delta }} \frac{1}{|x|^\beta }f(x,t_nG_n)t_nG_ndx. \end{aligned}$$

(5.8)

Since $G_n(x)\rightarrow 0$ a.e. in $B_{R_0}(0)$, we deduce that the characteristic functions $\chi _{V_{n,\delta }}$ satisfies

$$\begin{aligned} \chi _{V_{n,\delta }}\rightarrow 1\ \ \mathrm{a.e.\ in}\ B_{R_0}(0)\ \ \mathrm{as}\ n\rightarrow \infty . \end{aligned}$$

By $t_nG_n<t_\delta$ and the Lebesgue dominated convergence theorem, we have

$$\begin{aligned}&\int _{V_{n,\delta }}\frac{1}{|x|^\beta }\exp \left( \frac{\alpha _0\alpha _{N,s}^*}{\alpha _{N,s}}(t_nG_n)^{N/(N-s)}\right) dx\rightarrow \frac{\omega _{N-1}}{N-\beta } R_0^{N-\beta } \ \ \mathrm{and}\nonumber \\&\quad \int _{V_{n,\delta }}\frac{1}{|x|^\beta }f(x,t_nG_n)t_nG_ndx\rightarrow 0. \end{aligned}$$

(5.9)

The key point is to estimate the first term on the right hand of (5.8). By (5.3) and the definition of $G_n$, we have

$$\begin{aligned}&\int _{B_{R_0}(0)}\frac{1}{|x|^\beta }\exp \left( \frac{\alpha _0\alpha _{N,s}^*}{\alpha _{N,s}}(t_nG_n)^{N/(N-s)}\right) dx\nonumber \\&\quad \ge R_0^{N-\beta }\int _{B_{1/n}(0)}\frac{1}{|x|^\beta }\exp ((N-\beta )\ln n)dx\nonumber \\&\qquad +R_0^{N-\beta }\int _{1/n<|x|<1}\frac{1}{|x|^\beta }\exp \left[ (N-\beta )\frac{|\ln |x||^{N/(N-s)}}{(\ln n)^{s/(N-s)}}\right] dx\nonumber \\&\quad =\frac{\omega _{N-1}R_0^{N-\beta }}{N-\beta }+R_0^{N-\beta }\int _{1/n<|x|<1}\frac{1}{|x|^\beta }\exp \left[ (N-\beta )\frac{|\ln |x||^{N/(N-s)}}{(\ln n)^{s/(N-s)}}\right] dx\nonumber \\&\quad \ge \frac{\omega _{N-1}R_0^{N-\beta }}{N-\beta }+\frac{\omega _{N-1}R_0^{N-\beta }}{N-\beta }\left( 1-\frac{1}{n^{N-\beta }}\right) . \end{aligned}$$

(5.10)

Inserting (5.9) and (5.10) in (5.8) and using (5.7), we arrive at

$$\begin{aligned} M\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s} \ge (\beta _0-\delta )\frac{\omega _{N-1}R_0^{N-\beta }}{N-\beta },\ \ \forall \delta \in (0,\beta _0). \end{aligned}$$

Letting $\delta \rightarrow 0^+$, we obtain

$$\begin{aligned} \beta _0\le \frac{M\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}}{\frac{\omega _{N-1}R_0^{N-\beta }}{N-\beta }}, \end{aligned}$$

which contradicts $(f_5)$. Therefore, the lemma is proved. $\square$

By Lemma 5.3, we obtain the desired estimate for the level $c_*$.

Lemma 5.4

Assume $(M_1), (M_3), (M_4)$ and $(f_3)$ hold. Then

$$\begin{aligned} c_*<\frac{s}{N}{\mathscr {M}}\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) . \end{aligned}$$

Proof

Since $G_n\ge 0$ in $\Omega$ and $\Vert G_n\Vert \rightarrow 1$, as in the proof of Lemma 5.2, we deduce that $I(tG_n)\rightarrow -\infty$ as $t\rightarrow \infty$. Consequently,

$$\begin{aligned} c_*\le \max _{t\ge 0} I(tG_n),\ \ \forall n\in {\mathbb {N}}. \end{aligned}$$

Thus, the desired result follows by using Lemma 5.3. $\square$

Consider the Nehari manifold associated to the functional I, that is,

$$\begin{aligned} {\mathcal {N}}=\left\{ u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\setminus \{0\}:\langle I^\prime (u),u\rangle =0\right\} \end{aligned}$$

and define $c^*:=\inf _{u\in {\mathcal {N}}}I(u)$.

The next result is crucial in our arguments to get the existence of a ground state solution for problem (1.8).

Lemma 5.5

Assume that $(M_3)$ and $(f_5)$ are satisfied. Then $c_*\le c^*$, where $c_*$ is given by (5.1).

Proof

The proof is similar to [17] and [33], so we omit the proof. $\square$

Lemma 5.6

(The $(PS)_{c}$ condition) Let $(M_1),(M_3), (M_4)$ and $(f_1^\prime )$, $(f_3)$, $(f_5)$ and $(f_6)$ hold. Then the functional I satisfies the $(PS)_{c_*}$ condition.

Proof

The proof is similar to Lemma 4.1 of [33]. Let $(u_n)_n$ be a Cerami sequence at level $c_*$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Then

$$\begin{aligned} I(u_n)\rightarrow c_*\ \ \mathrm{and}\ (1+\Vert u_n\Vert ) \Vert I^\prime (u_n)\Vert \rightarrow 0. \end{aligned}$$

If $d:=\inf _{n\ge 1}\Vert u_n\Vert =0$, we can discuss as Lemma 3.3. Thus, we need only consider the case $d:=\inf _{n\ge 1}\Vert u_n\Vert >0$.

In the following, we assume that $d:=\inf _{n\ge 1}\Vert u_n\Vert >0$. We first show that $(u_n)_n$ is bounded in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Arguing by contradiction, we assume that

$$\begin{aligned} \Vert u_n\Vert \ge 1\ \ \mathrm{and}\ \ \lim _{n\rightarrow \infty } \Vert u_n\Vert =\infty . \end{aligned}$$

Set

$$\begin{aligned} v_n=\frac{u_n}{\Vert u_n\Vert }. \end{aligned}$$

Then $\Vert v_n\Vert =1$. Going if necessary to a subsequence, we can assume that $v_n\rightharpoonup v$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Further, one can show that

$$\begin{aligned} {\left\{ \begin{array}{ll} v_n^+\rightharpoonup v^+\ \ \mathrm{in}\ \ W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ),\\ v_n^+\rightarrow v^+\ \ \mathrm{a.e.\ in}\ \Omega ,\\ v_n^+\rightarrow v^+ \ \ \mathrm{in}\ L^q(\Omega ,|x|^{-\beta })(\forall 1\le q<\infty ). \end{array}\right. } \end{aligned}$$

Now we prove that $v^+=0$ a.e. in $\Omega$. If the Lebesgue measure of set $U^+:=\{x\in \Omega :v^+(x)>0\}$ is positive, then we have

$$\begin{aligned} \lim _{n\rightarrow \infty } u_n^+(x)=\lim _{n\rightarrow \infty } v_n^+(x)\Vert u_n\Vert =\infty \ \ \mathrm{in }\ U^+. \end{aligned}$$

Thus, by (1.10), we deduce

$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{F(x,u_n^+(x))}{|x|^\beta |u_n^+|^{N\theta /s}}= \infty \ \ \mathrm{a.e.\ in}\ U^+, \end{aligned}$$

which implies that

$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{F(x,u_n^+(x))}{|x|^\beta |u_n^+|^{N\theta /s}}= \infty \ \ \mathrm{a.e.\ in}\ U^+. \end{aligned}$$

It follows that

$$\begin{aligned} \int _\Omega \liminf _{n\rightarrow \infty } \frac{F(x,u_n^+(x))}{|x|^\beta |u_n^+|^{N\theta /s}}dx= \infty . \end{aligned}$$

(5.11)

Note that $(u_n)_n$ is a Cerami sequence at level $c_*$. Then

$$\begin{aligned} {\mathscr {M}}(\Vert u_n\Vert ^{N/s}) =\frac{N}{s}c_* +\frac{N}{s}\int _\Omega \frac{1}{|x|^\beta }F(x,u_n^+)dx+o(1), \end{aligned}$$

which together with $\lim _{t\rightarrow \infty }{\mathscr {M}}(t)=\infty$ yields that

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _\Omega \frac{1}{|x|^\beta }F(x,u_n^+)dx=\infty . \end{aligned}$$

Hence,

$$\begin{aligned} \liminf _{n\rightarrow \infty } \int _\Omega \frac{F(x,u_n^+)}{|x|^\beta |u_n^+|^{N\theta /s}} |v_n^+|^{N\theta /s}dx&=\liminf _{n\rightarrow \infty } \int _\Omega \frac{F(x,u_n^+)}{|x|^\beta \Vert u_n\Vert ^{N\theta /s}} dx\\&\le \liminf _{n\rightarrow \infty } \frac{{\mathscr {M}}(1)\int _\Omega \frac{1}{|x|^\beta }F(x.u_n^+)dx}{\frac{N}{s} \left( c_*+\int _\Omega \frac{F(x,u_n^+)}{|x|^\beta }dx\right) +o(1)}\\&=\frac{s{\mathscr {M}}(1)}{N}. \end{aligned}$$

Here we have used the fact that

$$\begin{aligned} t^{\theta }\ge \frac{{\mathscr {M}}(t)}{{\mathscr {M}}(1)}\ \ \forall t\ge 1, \end{aligned}$$

thanks to (1.9). Note that $F(x,t)\ge 0$. By Fatou’s lemma and (5.11) , we get a contradiction. Thus, $v\le 0$ a.e. in $\Omega$ and $v_n^+\rightharpoonup 0$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$.

Clearly, there exist $t_n\in [0,1]$ such that

$$\begin{aligned} I(t_nu_n)=\max _{t\in [0,1]}I(tu_n). \end{aligned}$$

For any $R\in (0,(\frac{N-\beta }{N} \frac{\alpha _{N,s}}{\alpha _0})^{\frac{N-s}{N}})$, since f satisfies $(f_1^\prime )$, we choose $\varepsilon =\frac{N-\beta }{N} \frac{\alpha _{N,s}}{R^{N/(N-s)}}-\alpha _0$ and $\alpha _0<\alpha <\alpha _0+\varepsilon$ such that

$$\begin{aligned} F(x,t)\le C(R)|t|^{N\theta /s} +\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{R^{N/(N-s)}}-\alpha _0\right) \exp (\alpha |t|^{N/(N-s)}),\ \ \forall (x,t)\in \Omega \times {\mathbb {R}}. \end{aligned}$$

It follows that

$$\begin{aligned} I(Rv_n)&\ge \frac{s}{N}{\mathscr {M}}(R^{N/s}) -C(R)R^{N\theta /s}\int _\Omega \frac{|v_n^+|^{\frac{N\theta }{s}}}{|x|^\beta }dx \\&\quad -\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{R^{N/(N-s)}}-\alpha _0\right) \int _\Omega \frac{1}{|x|^\beta }\exp (\alpha R^{N/(N-s)}|v_n^+|^{N/(N-s)})dx. \end{aligned}$$

Since $v_n^+\rightharpoonup 0$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ and $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\hookrightarrow L^q(\Omega ,|x|^{-\beta })$ is compact for any $q\ge 1$, we have

$$\begin{aligned} \int _\Omega \frac{1}{|x|^\beta }|v_n^+|^{\frac{N\theta }{s}}dx\rightarrow 0. \end{aligned}$$

By Theorem 2.3 and $\alpha R^{N/(N-s)}< \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}$, we know that

$$\begin{aligned} \int _\Omega \frac{1}{|x|^\beta }\exp (\alpha R^{N/(N-s)}|v_n^+|^{N/(N-s)})dx \end{aligned}$$

is bounded. Thus,

$$\begin{aligned} I(Rv_n)&\ge \frac{s}{N}{\mathscr {M}}(R^{N/s}) -C(R)R^{N\theta /s}\int _\Omega \frac{|v_n^+|^{\frac{N\theta }{s}}}{|x|^\beta }dx \\&\quad -C\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{R^{N/(N-s)}}-\alpha _0\right) . \end{aligned}$$

On the other hand, by $\Vert u_n\Vert \rightarrow \infty$, we deduce

$$\begin{aligned} I(t_nu_n)\ge I\left( \frac{R}{\Vert u_n\Vert }u_n\right) =I(Rv_n). \end{aligned}$$

Thus, letting $n\rightarrow \infty$ and then letting $R\rightarrow \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/N}$ , we obtain

$$\begin{aligned} \liminf _{n\rightarrow \infty }I(t_nu_n) \ge \frac{s}{N}{\mathscr {M}}\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) >c_*. \end{aligned}$$

(5.12)

Since $I(0)=0$ and $I(u_n)\rightarrow c_*$, we can assume that $t_n\in (0,1)$. Then $\frac{d}{dt}I(tu_n)|_{t_n}=0$. Thus, we get $\langle I^\prime (t_nu_n),t_nu_n\rangle =0$, that is,

$$\begin{aligned} M(t_n^{N/s}\Vert u_n\Vert ^{N/s})t_n^{N/s}\Vert u_n\Vert ^{N/s} =\int _\Omega \frac{1}{|x|^\beta }f(x,t_nu_n)t_nu_ndx. \end{aligned}$$

From (1.11), it yields that

$$\begin{aligned} I(t_nu_n)&=\frac{s}{N}{\mathscr {M}}(t_n^{N/s}\Vert u_n\Vert ^{N/s}) -\int _\Omega \frac{1}{|x|^\beta }F(x,t_nu_n)dx\\&=\frac{s}{N}{\mathscr {M}}(t_n^{N/s}\Vert u_n\Vert ^{N/s})-\frac{s}{N\theta }M(t_n^{N/s}\Vert u_n\Vert ^{N/s})t_n^{N/s}\Vert u_n\Vert ^{N/s} \\&\qquad +\int _\Omega \frac{1}{|x|^\beta }\left[ \frac{s}{N\theta }f(x,t_nu_n)t_nu_n- F(x,t_nu_n)\right] dx\\&\le \frac{s}{N}{\mathscr {M}}(\Vert u_n\Vert ^{N/s})-\frac{s}{N\theta }M(\Vert u_n\Vert ^{N/s})\Vert u_n\Vert ^{N/s} \\&\qquad +\int _\Omega \frac{1}{|x|^\beta }\left[ \frac{s}{N\theta }f(x,u_n)u_n- F(x,u_n)\right] dx. \end{aligned}$$

Moreover, it follows from $(u_n)_n$ is a Cerami sequence that

$$\begin{aligned}&\frac{s}{N}{\mathscr {M}}(\Vert u_n\Vert ^{N/s})-\frac{s}{N\theta }M(\Vert u_n\Vert ^{N/s})\Vert u_n\Vert ^{N/s} \\&\qquad +\int _\Omega \frac{1}{|x|^\beta }\left[ \frac{s}{N\theta }f(x,u_n)u_n- F(x,u_n)\right] d =c_*+o(1). \end{aligned}$$

Thus,

$$\begin{aligned} \limsup _{n\rightarrow \infty }I(t_nu_n)\le c_*, \end{aligned}$$

which contradicts (5.12). Therefore, $(u_n)_n$ is bounded in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$.

Next we show that $(u_n)_n$ has a convergence subsequence in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Going if necessary to a subsequence, there exist a function $u\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ and $\xi >0$ such that

$$\begin{aligned}&u_n\rightharpoonup u\ \ \mathrm{weakly\ in}\ W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ),\nonumber \\&u_n\rightarrow u\ \ \mathrm{strongly\ in}\ L^{\nu }(\Omega )( \nu \ge 1),\nonumber \\&u_n\rightarrow u\ \ \mathrm{a.e.\ in}\ \Omega \nonumber \\&\Vert u_n\Vert \rightarrow \xi . \end{aligned}$$

(5.13)

Here we have used the compact embedding from $W_0^{s,N/s}(\Omega )$ to $L^\nu (\Omega )$ for any $\nu \ge 1$ (see Theorem 2.2) and the embedding $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )\hookrightarrow W_0^{s,N/s}(\Omega )$ is continuous. Using a similar discussion as [33], we can deduce that

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{\Omega }\frac{1}{|x|^\beta }f(x,u_n)dx =&\int _{\Omega }\frac{1}{|x|^\beta }f(x,u)dx \ \ \mathrm{and}\ \lim _{n\rightarrow \infty }\int _\Omega \frac{1}{|x|^\beta }F(x,u_n)dx\nonumber \\ =&\int _\Omega \frac{1}{|x|^\beta }F(x,u)dx. \end{aligned}$$

(5.14)

Now, we assert that $u\ne 0$. Arguing by contradiction, we assume that $u=0$. Then, $\int _\Omega \frac{1}{|x|^\beta }F(x,u_n)dx\rightarrow 0$ and $I(u_n)\rightarrow c$ gives that

$$\begin{aligned} \frac{s}{N}{\mathscr {M}}(\Vert u_n\Vert ^{N/s}) \rightarrow c<\frac{s}{N}{\mathscr {M}}\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) \end{aligned}$$

as $n\rightarrow \infty$. Thus, there exist $n_0\in {\mathbb {N}}$ and $\delta >0$ such that $\Vert u_n\Vert ^{N/(N-s)}<\delta <\frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}.$ Choosing $\nu >1$ close to 1 and $\alpha >\alpha _0$ close to $\alpha _0$ such that we still have $\nu \alpha \Vert u_n\Vert ^{N/(N-s)}<\delta <\frac{N-\beta }{N}\alpha _{N,s}$. Thus, it follows from (2.2) with $q=1$ that

$$\begin{aligned}&\left| \int _\Omega \frac{1}{|x|^\beta }f(x,u_n)u_ndx\right| \\&\quad \le C\left( \int _\Omega \frac{1}{|x|^\beta }|u_n|^{\theta N/s}dx+\int _\Omega \frac{1}{|x|^\beta }|u_n|\exp (\alpha |u_n|^{N/(N-s)})dx\right) \\&\quad \le C\left( \int _\Omega \frac{1}{|x|^\beta }|u_n|^{\theta N/s}dx+\left( \int _\Omega \frac{1}{|x|^\beta } |u_n|^{\frac{\nu }{\nu -1}}dx\right) ^{\frac{\nu -1}{\nu }}\right. \\&\qquad \left. \left( \int _{\Omega }\frac{1}{|x|^\beta }\exp [\nu \alpha \Vert u_n\Vert ^{\frac{N}{N-s}}(\frac{u_n}{\Vert u_n\Vert })^{\frac{N}{N-s}}]dx\right) ^{\frac{1}{\nu }}\right) \\&\quad \le C\left( \int _\Omega \frac{1}{|x|^\beta }|u_n|^{\theta N/s}dx+\left( \int _\Omega \frac{1}{|x|^\beta } |u_n|^{\frac{\nu }{\nu -1}}dx\right) ^{\frac{\nu -1}{\nu }}\right) \rightarrow 0 \end{aligned}$$

as $n\rightarrow \infty$. Since $(u_n)_n$ is a bounded Cerami sequence, we get

$$\begin{aligned} \langle I^\prime (u_n),u_n\rangle =M(\Vert u_n\Vert ^{N/s})\Vert u_n\Vert ^{N/s}-\int _{\Omega }\frac{1}{|x|^\beta }f(x,u_n)u_ndx \rightarrow 0, \end{aligned}$$

which implies that

$$\begin{aligned} M(\Vert u_n\Vert ^{N/s})\Vert u_n\Vert ^{N/s}\rightarrow 0. \end{aligned}$$

From this and assumption $(M_1)$, we deduce $\Vert u_n\Vert \rightarrow 0$, which contradicts the assumption that $\inf _{n\ge 1}\Vert u_n\Vert >0$. Therefore, we must have $u\ne 0$.

We claim that $I(u)\ge 0$. Arguing by contradiction, we assume that $I(u)<0$. Set $\zeta (t):=I(tu)$ for all $t\ge 0$. Then $\zeta (0)=0$ and $\zeta (1)<0$. Arguing as in the proof of Lemma 3.1, we can see that $\zeta (t)>0$ for $t>0$ small enough. Hence there exists $t_0\in (0,1)$ such that

$$\begin{aligned} \zeta (t_0)=\max _{t\in [0,1]}\zeta (t),\ \ \zeta ^\prime (t_0) =\langle I^\prime (t_0u),u\rangle =0, \end{aligned}$$

which means that $t_0u\in {\mathcal {N}}$. Therefore, by Remarks 1.1 and 1.2, the semicontinuity of norm and Fatou’s lemma, we get

$$\begin{aligned} c_*\le c^*\le I(t_0u)&=I(t_0u)-\frac{s}{N\theta }\langle I^\prime (t_0u),t_0u\rangle \\&=\frac{s}{N}{\mathscr {M}}(\Vert t_0u\Vert ^{N/s})-\frac{ s}{N\theta } M(\Vert t_0u\Vert ^{N/s})\Vert t_0u\Vert ^{N/s}\\&\quad +\frac{s}{N\theta } \int _\Omega \frac{1}{|x|^\beta }\left[ f(x,t_0u)t_0u-\frac{\theta N}{s}F(x,t_0u)\right] dx\\&<\frac{s}{N}{\mathscr {M}}(\Vert u\Vert ^{N/s})-\frac{ s}{N\theta } M(\Vert u\Vert ^{N/s})\Vert u\Vert ^{N/s}\\&\quad +\frac{ s}{N\theta } \int _\Omega \frac{1}{|x|^\beta }\left[ f(x,u)u-\frac{\theta N}{s}F(x,u)\right] dx. \end{aligned}$$

By the weak lower semicontinuity of convex functional, we have

$$\begin{aligned} \Vert u\Vert ^{N/s}\le \liminf _{n\rightarrow \infty }\Vert u_n\Vert ^{N/s}=\xi ^{N/s}. \end{aligned}$$

In view of Remark 1.1 and the continuity of M, we deduce that

$$\begin{aligned}&\frac{s}{N}{\mathscr {M}}(\Vert u\Vert ^{N/s})-\frac{ s}{N\theta } M(\Vert u\Vert ^{N/s})\Vert u\Vert ^{N/s}\\&\quad \le \frac{s}{N}{\mathscr {M}}(\xi ^{N/s})-\frac{ s}{N\theta } M(\xi ^{N/s})\xi ^{N/s}\\&\quad =\lim _{n\rightarrow \infty }\left[ \frac{s}{N}{\mathscr {M}}(\Vert u_n\Vert ^{N/s})-\frac{ s}{N\theta } M(\Vert u_n\Vert ^{N/s})\Vert u_n\Vert ^{N/s}\right] . \end{aligned}$$

By Fatou’s lemma, we get

$$\begin{aligned} \int _\Omega \frac{1}{|x|^\beta }f(x,u)udx\le \liminf _{n\rightarrow \infty }\int _\Omega \frac{1}{|x|^\beta }f(x,u_n)u_ndx. \end{aligned}$$

It follows from above results and (5.14) that

$$\begin{aligned} c_*\le c^*&<\lim _{n\rightarrow \infty }\left[ \frac{s}{N}{\mathscr {M}}(\Vert u_n\Vert ^{N/s}) -\frac{ s}{N\theta } M(\Vert u_n\Vert ^{N/s})\Vert u_n\Vert ^{N/s}\right] \\&\ \ +\frac{s}{N\theta }\liminf _{n\rightarrow \infty } \int _\Omega \frac{1}{|x|^\beta }\left[ f(x,u_n)u_n-\frac{N\theta }{s}F(x,u_n)\right] dx\\&\le \lim _{n\rightarrow \infty }\left[ I(u_n)-\frac{ s}{N\theta }\langle I^\prime (u_n),u_n\rangle \right] =c_* \end{aligned}$$

which is absurd. Thus the claim holds true.

Now we claim that

$$\begin{aligned} I(u)=c_*. \end{aligned}$$

(5.15)

Obviously, by (5.14) and semicontinuity of norm, we have $I(u)\le c_*$. Next we prove that $I(u_0)<c_*$ can not occur. Actually, if $I(u)<c_*$, then

$$\begin{aligned} \Vert u\Vert <\xi . \end{aligned}$$

Note that (5.14) yields that

$$\begin{aligned} \frac{s}{N}{\mathscr {M}}(\xi ^{N/s}) =\lim _{n\rightarrow \infty }\frac{s}{N}{\mathscr {M}}(\Vert u_n\Vert ^{N/s}) =c_*+\int _\Omega \frac{1}{|x|^\beta } F(x,u)dx. \end{aligned}$$

(5.16)

This gives that

$$\begin{aligned} \xi ^{\frac{N}{s}}={\mathscr {M}}^{-1}\left( \frac{N}{s}c_*+\frac{N}{s}\int _\Omega \frac{1}{|x|^\beta }F(x,u)dx\right) . \end{aligned}$$

Set $w_n=u_n/\Vert u_n\Vert$. Then $w_n\rightharpoonup w=u/\xi$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$ and $\Vert w\Vert <1$. Thus, it follows from Theorem 2.4 that

$$\begin{aligned} \sup _n\int _\Omega \frac{\exp \left( \alpha ^\prime |w_n|^{N/(N-s)}\right) }{|x|^\beta }dx<\infty , \ \ \forall \ \alpha ^\prime <\frac{(1-\frac{\beta }{N})\alpha _{N,s}}{(1-\Vert w\Vert ^{N/s})^{s/(N-s)}}. \end{aligned}$$

(5.17)

On the other hand, by (5.16), we have

$$\begin{aligned} \frac{N}{s}c_*-\frac{N}{s}I(u)={\mathscr {M}}(\xi ^{N/s}) -{\mathscr {M}}(\Vert u\Vert ^{N/s}). \end{aligned}$$

Thus, it follows from $I(u)\ge 0$ that

$$\begin{aligned} {\mathscr {M}}(\xi ^{N/s})\le \frac{N}{s}c_*+{\mathscr {M}}(\Vert u\Vert ^{N/s}) <{\mathscr {M}}\left( \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) +{\mathscr {M}}(\Vert u\Vert ^{N/s}). \end{aligned}$$

Furthermore, by $(M_1)$, we get

$$\begin{aligned} \xi ^{N/s}<&{\mathscr {M}}^{-1}\left[ {\mathscr {M}}\left( \left( \frac{N-\beta }{N} \frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}\right) +{\mathscr {M}}(\Vert u\Vert ^{N/s})\right] \nonumber \\&\quad \le \left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}+\Vert u\Vert ^{N/s}. \end{aligned}$$

(5.18)

Note that

$$\begin{aligned} \xi ^{N/s}=\frac{\xi ^{N/s}-\Vert u\Vert ^{N/s}}{1-\Vert w\Vert ^{N/s}}. \end{aligned}$$

Hence, it follows from (5.18) that

$$\begin{aligned} \xi ^{N/s}<\frac{\left( \frac{N-\beta }{N}\frac{\alpha _{N,s}}{\alpha _0}\right) ^{(N-s)/s}}{1-\Vert w\Vert ^{N/s}}. \end{aligned}$$

Thus, there exist $n_0\in {\mathbb {N}}$ and $\alpha ^{\prime \prime }>0$ such that

$$\begin{aligned} \alpha _0\Vert u_n\Vert ^{N/(N-s)}<\alpha ^{\prime \prime }<\frac{\frac{(N-\beta )\alpha _{N,s}}{N}}{(1-\Vert w\Vert ^{N/s})^{s/(N-s)}} \end{aligned}$$

for all $n\ge n_0$. We choose $\nu >1$ close to 1 and $\alpha >\alpha _0$ close to $\alpha _0$ such that

$$\begin{aligned} \nu \alpha \Vert u_n\Vert ^{N/(N-s)}\le \alpha ^{\prime \prime }<\frac{\frac{(N-\beta )\alpha _{N,s}}{N}}{(1-\Vert v\Vert ^{N/s})^{s/(N-s)}}. \end{aligned}$$

In view of (5.17), for some $C>0$ and n large enough, we obtain

$$\begin{aligned} \int _\Omega \frac{1}{|x|^\beta } \exp (\nu \alpha |u_n|^{N/(N-s)})dx \le \int _\Omega \frac{1}{|x|^\beta }\exp (\alpha ^{\prime \prime } |w_n|^{N/(N-s)})dx\le C. \end{aligned}$$

Therefore, we deduce from (2.2) that

$$\begin{aligned}&\left| \int _\Omega \frac{1}{|x|^\beta } f(x,u_n)(u_n-u)dx\right| \\&\le C\left( \int _\Omega \frac{1}{|x|^\beta }|u_n-u|^{ N\theta /s}dx+\int _\Omega \frac{1}{|x|^\beta }|u_n-u|\exp (\alpha |u_n|^{N/(N-s)})dx\right) \\&\le C\int _\Omega \frac{1}{|x|^\beta }|u_n-u|^{ N\theta /s}dx+C\left( \int _\Omega \frac{1}{|x|^\beta }|u_n-u|^{\frac{\nu }{\nu -1}}dx\right) ^{\frac{\nu -1}{\nu }} \rightarrow 0 \end{aligned}$$

as $n\rightarrow \infty$.

Since $(u_n)_n$ is a bounded Cerami sequence in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$, we have

$$\begin{aligned} o(1)&=\langle I^\prime (u_n),u_n-u\rangle \nonumber \\&=M(\Vert u_n\Vert ^{N/s})\langle u_n,u_n-u\rangle _{s,N/s}-\int _{\Omega }\frac{1}{|x|^\beta }f(x,u_n)(u_n-u)dx. \end{aligned}$$

(5.19)

Define a functional L as follows:

$$\begin{aligned} \langle L(v),w\rangle =\langle v,w\rangle _{s,N/s} \end{aligned}$$

for all $v,w\in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. By the Hölder inequality, one can see that

$$\begin{aligned} |\langle L(v),w\rangle |\le \Vert v\Vert ^{\frac{N}{s}-1}\Vert w\Vert , \end{aligned}$$

which together with the definition of L implies that for each v, L(v) is a bounded linear functional on $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Thus, $\langle L(u),u_n-u\rangle =o(1)$, that is,

$$\begin{aligned} \langle u,u_n-u\rangle _{s,N/s}=o(1). \end{aligned}$$

In conclusion, we can deduce from (5.19) that

$$\begin{aligned} M(\Vert u_n\Vert ^{N/s})\left[ \langle u_n,u_n-u\rangle _{s,N/s}-\langle u,u_n-u\rangle _{s,N/s}\right] =o(1). \end{aligned}$$

In view of the fact that $\Vert u_n\Vert \rightarrow \xi$ and $\xi >0$, by using $(M_1)$ and a similar discussion as in [33], we obtain that $u_n\rightarrow u$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Furthermore, using (5.14) and the continuity of ${\mathscr {M}}$, we have $I(u)=c_*$, which is a contradiction. Thus, the assertion (5.15) holds true.

Combining $I(u)=c_*$ with $I(u_n)\rightarrow c_*$ and $\Vert u_n\Vert \rightarrow \xi$, we conclude that

$$\begin{aligned} {\mathscr {M}}(\xi ^{N/s})={\mathscr {M}}(\Vert u\Vert ^{N/s}), \end{aligned}$$

which implies that $\xi =\Vert u\Vert$. By the uniform convexity of norm, we obtain that $u_n\rightarrow u$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. This finishes the proof. $\square$

Proof of Theorem 1.3

By Lemmas 5.1 and 5.2, we know that I satisfies all the assumptions of Theorem 5.1. Thus there exists a Cerami sequence $(u_n)_n\subset W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. Moreover, by Lemma 5.6, there exists a subsequence of $(u_n)_n$ (still labeled by $(u_n)_n$) such that $u_n\rightarrow u$ in $W_{0,{\mathcal {K}}}^{s,N/s}(\Omega )$. It follows from $I^\prime (u_n)\rightarrow 0$ that

$$\begin{aligned} M(\Vert u_n\Vert ^{N/s})\langle u_n,\varphi \rangle _{s,N/s} =\int _\Omega \frac{1}{|x|^\beta } f(x,u_n)\varphi dx,\ \ \forall \varphi \in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ). \end{aligned}$$

Furthermore, we have

$$\begin{aligned} M(\Vert u\Vert ^{N/s})\langle u,\varphi \rangle _{s,N/s} =\int _\Omega \frac{1}{|x|^\beta }f(x,u)\varphi dx\ \ \forall \varphi \in W_{0,{\mathcal {K}}}^{s,N/s}(\Omega ), \end{aligned}$$

which means that u is a nontrivial solution of (1.8) satisfying $I(u)=c_*$, that is, $I^\prime (u)=0$ and $I(u)=c_*$. Therefore, by the definition of $c^*$ and $c_*\le c^*$, we know that u is a ground state solution of problem (1.8). Moreover, Lemma 2.1 shows that u is nonnegative. $\square$