1 Introduction

In this article, we are concerned with the existence of positive solutions for a class of fractional Kirchhoff-type problems

$$ \textstyle\begin{cases} M (\int_{\mathbb{R}^{N}\times\mathbb{R}^{N}}\frac {|u(x)-u(y)|^{2}}{|x-y|^{N+2s}}\,dx\,dy )(-\Delta)^{s}u= f(x,u)& \text{in } \Omega, \\ u=0& \text{in } \mathbb{R}^{N}\setminus\Omega, \end{cases} $$
(1)

where \(N>2s\) with \(s\in(0,1)\), Ω is a bounded domain in \({\mathbb{R}}^{N}\) with smooth boundary Ω, M and f are two continuous functions, and \(-(-\Delta)^{s}\) is the fractional Laplace operator defined as

$$ -(-\Delta)^{s}u(x)= \int_{\mathbb{R}^{N}}\frac {u(x+y)+u(x-y)-2u(x)}{|y|^{N+2s}}\,dy,\quad x\in\mathbb{R}^{N}. $$
(2)

As \(s\rightarrow1^{-}\), problem (1) becomes the elliptic Kirchhoff equation

$$ -M \biggl( \int_{\Omega}|\nabla u|^{2}\,dx \biggr)\Delta u=f(x,u), \quad x\in\Omega, $$
(3)

where \(\Omega\subset\mathbb{R}^{N} \) is a smooth domain, and u satisfies some boundary conditions; see, for instance, [1, 2] for more information about Eq. (3). It is easy to find that Eq. (3) is related to the stationary analogue of the Kirchhoff equation

$$ u_{tt}-M \biggl( \int_{\Omega}|\nabla u|^{2}\,dx \biggr)\Delta u=f(x,u), \quad x\in\Omega, $$
(4)

where \(M(t)=a+bt\) for all \(t\geq0\) with \(a,b>0\); see, for instance, [3] for recent results. It was proposed by Kirchhoff in 1883 as a generalization of the well-known D’Alembert wave equation

$$\rho\frac{\partial^{2}u}{\partial t^{2}}- \biggl( \frac{\rho_{0}}{\lambda }+\frac{E}{2L} \int_{0}^{L}\biggl\vert \frac{\partial u}{\partial x}\biggr\vert ^{2}\,dx \biggr) \frac{\partial^{2}u}{\partial x^{2}}=f(x,u) $$

for free vibrations of elastic strings; see [4]. Here, ρ, \(\rho_{0}\), λ, E, L are all constants. Equation (4) received much attention only after Lions [5] proposed an abstract framework for this problem. Equation (4) models some physical and biological systems where u describes a process which depends on the average of itself.

When \(M=1\), problem (1) reduces to the fractional Laplacian equation

$$ \textstyle\begin{cases}(-\Delta)^{s}u= f(x,u)& \text{in } \Omega, \\ u=0& \text{in } \mathbb{R}^{N}\setminus\Omega. \end{cases} $$
(5)

In recent years, a great attention has been focused on the study of fractional and nonlocal operators of elliptic type, both for the pure mathematical research and for concrete real-world applications. The fractional and nonlocal operators appear in many fields such as, among the others, optimization, finance, phase transitions, stratified materials, anomalous diffusion, crystal dislocation, soft thin films, semipermeable membranes, flame propagation, conservation laws, ultra-relativistic limits of quantum mechanics, quasi-geostrophic flows, multiple scattering, minimal surfaces, materials science, and water waves. Just to mention a few, we recall, for instance, the following papers and the references therein: [6, 7] for regularity results, [815] for the existence of solutions, and [1618] for multiplicity of solutions.

In recent paper, Fiscella and Valdinoci [19] studied the following Kirchhoff-type problem involving an integro-differential operator:

$$ \textstyle\begin{cases}-M (\int_{\mathbb{R}^{N}\times\mathbb {R}^{N}}|u(x)-u(y)|^{2}K(x-y)\,dx\,dy )\mathcal{L}_{K}u= f(x,u)& \text{in } \Omega, \\ u=0& \text{in } \mathbb{R}^{N}\setminus\Omega, \end{cases} $$
(6)

where \(\mathcal{L}_{K}\) is the integro-differential operator with a singular symmetric kernel K defined by

$$\mathcal{L}_{K} u(x)= \int_{\mathbb{R}^{N}}\bigl(u(x+y)+u(x-y)-2u(x)\bigr) K(y)\,dy,\quad x\in \mathbb{R}^{N}, $$

where \(K:\mathbb{R}^{N}\setminus\{0\} \rightarrow(0,+\infty)\) is a singular symmetric kernel function satisfying the property

  1. (K)

    there exist \(\theta>0\) and \(s\in(0,1)\) such that

    $$\theta|x|^{-(N+2s)}\leq K(x)\leq\theta^{-1}|x|^{-(N+2s)} \quad \text{for any } x\in\mathbb{R}^{N}\setminus\{0\}. $$

Clearly, a typical model for K is given by the singular kernel \(K(x)=|x|^{-(N+2s)}\), which gives rise to the fractional Laplacian operator \(-(-\Delta)^{s}\). As a result, problem (6) reduces to our problem (1). For narrative convenience, in the following context, we always denote \(|x|^{-(N+2s)}\) by \(K(x)\).

Nyamoradi [20] studied problem (1) in a bounded domain Ω and obtained three solutions by using a three-critical-point theorem. Nyamoradi and Teng [21] also established the existence of nontrivial solutions for problem (1) by using the minimal principle and Morse theory. Xiang et al. [22] studied the existence of infinitely many solutions for problem (1) by using the fountain theorem. Xiang et al. [23] did similar work for the stationary Kirchhoff problems involving the fractional p-Laplacian. For study of this aspect, we also refer the interested readers to [24, 25].

Inspired by the articles mentioned, in this paper, we would like to generalize and correct Bensedik and Bouchekif’s work for a class of asymptotically linear elliptic Kirchhoff-type equations (see [26]) to our problem (1).

The paper is organized as follows. In Section 2, we give some preliminary facts and some basic properties, which are needed later, and present our main results. Section 3 is devoted to the proofs of our results.

2 Preliminaries and main results

In this section, we give some preliminary results. We briefly recall the related definition and notes for the functional space \(X_{0}\) introduced in [27].

The functional space X denotes the linear space of Lebesgue-measurable functions from \(\mathbb{R}^{N}\) to \(\mathbb{R}\) such that the restriction to Ω of any function g in X belongs to \(L^{2}(\Omega)\) and the map \((x,y)\longmapsto (g(x)-g(y))\sqrt{K(x-y)}\) is in \(L^{2}((\mathbb{R}^{N}\times \mathbb{R}^{N})\setminus(\mathcal{C}\Omega\times \mathcal{C}\Omega),dx\,dy)\) (here \(\mathcal{C}\Omega=\mathbb{R}^{N}\setminus\Omega\)). Also, we denote by \(X_{0}\) the following linear subspace of X:

$$ X_{0}:=\bigl\{ g\in X: g=0 \text{ a.e. in } \mathbb{R}^{N} \setminus\Omega\bigr\} . $$

Note that X and \(X_{0}\) are nonempty since \(C_{0}^{2}(\Omega)\subseteq X_{0}\) by [27]. Moreover, the space X is endowed with the norm defined as

$$ \|g\|_{X}=|g|_{L^{2}(\Omega)}+ \biggl( \int_{\textit {Q}}\bigl\vert g(x)-g(y)\bigr\vert ^{2}K(x-y)\,dx\,dy \biggr)^{\frac{1}{2}}, $$
(7)

where \(Q=(\mathbb{R}^{N}\times\mathbb{R}^{N})\setminus \mathcal{O}\) and \(\mathcal{O}=(\mathcal{C}\Omega)\times (\mathcal{C}\Omega)\subset\mathbb{R}^{N}\times\mathbb{R}^{N}\). We equip \(X_{0}\) with the norm

$$ \|g\|_{X_{0}}= \biggl( \int_{Q}\bigl\vert g(x)-g(y)\bigr\vert ^{2} K(x-y)\,dx\,dy \biggr)^{\frac{1}{2}}, $$
(8)

which is equivalent to the usual one defined in (7) (see [28]). It is easy to see that \((X_{0}, \|\cdot\|_{X_{0}})\) is a Hilbert space with scalar product

$$ \langle u, v\rangle_{X_{0}}= \int_{Q} \bigl(u(x)-u(y)\bigr) \bigl(v(x)-v(y)\bigr)K(x-y)\,dx \,dy. $$
(9)

Denote by \(H^{s}(\Omega)\) the usual fractional Sobolev space with respect to the Gagliardo norm

$$ \|g\|_{H^{s}(\Omega)}=|g|_{L^{2}(\Omega)}+ \biggl( \int_{\Omega\times \Omega}\frac{|g(x)-g(y)|^{2}}{|x-y|^{N+2s}}\,dx\,dy \biggr)^{\frac{1}{2}}. $$
(10)

Now, we give a basic fact to be used later.

Lemma 2.1

[28]

The embedding \(j: X_{0}\hookrightarrow L^{v}(\Omega)\) is continuous for any \(v\in[1,2^{*}]\) and compact for \(v\in[1,2^{*})\).

Next, we make the assumptions on M and the nonlinearity term \(f(x,u)\) as follows:

(M0):

M is a continuous function on \(\mathbb{R}^{+}\) such that, for some \(m_{0}>0\), we have

$$M(t)\geq m_{0}\quad \text{for all } t\in\mathbb{R}^{+}. $$
(M1):

There exists \(m_{1}>0\) such that \(M(t)=m_{1}\), \(t\geq t_{0}\), for some \(t_{0}>0\).

(f1):

\(f(x,t)\) is a continuous function on \(\bar{\Omega}\times \mathbb{R}\) such that

$$f(x,t)\geq0 \quad \forall t\geq0, x\in\Omega,\quad \text{and} \quad f(x,t)=0 \quad \forall t\leq0,x\in\bar{\Omega}; $$
(f2):

\(t\mapsto\frac{f(x,t)}{t}\) is a nondecreasing function for any fixed \(x\in\Omega\);

(f3):

\(\lim_{t\rightarrow0}\frac{f(x,t)}{t}=p(x)\); \(\lim_{t\rightarrow+\infty}\frac{f(x,t)}{t} =q(x)\neq0\) uniformly in \(x\in\Omega\), where \(0\leq p(x)\), \(q(x)\in L^{\infty}(\Omega)\) and \(|p|_{\infty}< m_{0}\lambda_{1}\), where \(\lambda_{1}\) is the first eigenvalue of \((-\Delta)^{s}\) with homogeneous Dirichlet boundary data.

We observe that problem (1) has a variational structure. Indeed, it is the Euler-Lagrange equation of the functional \(\mathcal{J}: X_{0}\rightarrow\mathbb{R}\) defined as follows:

$$\mathcal{J}(u)=\frac{1}{2} \int_{\mathbb{R}^{N}\times \mathbb{R}^{N}}\bigl\vert u(x)-u(y)\bigr\vert ^{2}K(x-y)\,dx\,dy- \int_{\Omega}F\bigl(x,u(x)\bigr)\,dx. $$

It is well known that the functional \(\mathcal{J}\) is Frechét differentiable in \(X_{0}\) and, for any \(\varphi\in X_{0}\),

$$\bigl\langle \mathcal{J}'(u),\varphi\bigr\rangle = \int_{\mathbb{R}^{N}\times \mathbb{R}^{N}} \bigl(u(x)-u(y)\bigr) \bigl(\varphi(x)-\varphi(y) \bigr)K(x-y)\,dx\,dy- \int_{\Omega}f\bigl(x,u(x)\bigr)\varphi(x)\,dx. $$

Thus, critical points of \(\mathcal{J}\) are solutions of problem (1).

Before stating our results, we need to introduce some notation and establish some important propositions and lemmas.

Notation 2.1

Throughout this paper, we denote by \(|\cdot|_{p}\) the \(L^{p}\) norm, \(1\leq p\leq\infty\), and use the notation \(u^{\pm}=\max\{\pm u,0\}\). The letter C will denote different constants in different conditions.

Lemma 2.2

Assume that \(0\leq q(x)\in L^{\infty}(\Omega)\) and \(q(x)\neq0\). Then the eigenvalue problem

$$ \textstyle\begin{cases} (-\Delta)^{s} u=\lambda q(x)u& \textit{in } \Omega, \\ u=0& \textit{in } \mathbb{R}^{N}\setminus\Omega \end{cases} $$

has a principal eigenvalue denoted by \(\lambda_{1}(q)\) and its associated eigenfunction \(\phi_{1}\).

Proof

Let \(\{u_{n}\}\subset X_{0}\) be a minimizing sequence with \(\int_{\Omega}q(x)u_{n}^{2}\,dx=1\), and thus

$$ \lambda=\lim_{n\rightarrow\infty}\|u_{n} \|_{X_{0}}^{2}. $$
(11)

Thus, \(\{u_{n}\}\) is bounded in \(X_{0}\), and by Lemma 2.1 its subsequence, again denoted by \(\{u_{n}\}\), converges to some limit u in \(L^{2}(\Omega;q(x))\) that also satisfies \(\int _{\Omega}q(x)u^{2}\,dx=1\). In fact, since

$$\|u_{n}-u_{m}\|_{X_{0}}^{2}+ \|u_{n}+u_{m}\|_{X_{0}}^{2}=2 \|u_{n}\|_{X_{0}}^{2}+2\|u_{m} \|_{X_{0}}^{2} $$

for all \(n,m\in N\) and

$$\|u_{n}+u_{m}\|_{X_{0}}^{2}\geq\lambda \int_{\Omega}q(x) (u_{n}+u_{m})^{2} \,dx, $$

we get

$$ \|u_{n}-u_{m}\|_{X_{0}}^{2} \leq2\|u_{n}\|_{X_{0}}^{2}+2\|u_{m} \|_{X_{0}}^{2}-\lambda \int _{\Omega}q(x) (u_{n}+u_{m})^{2} \,dx. $$
(12)

Since by the choice of the sequence \(\{u_{n}\}\), \(\|u_{n}\|_{X_{0}}^{2}\) and \(\|u_{m}\|_{X_{0}}^{2}\) converge to λ, and \(\int_{\Omega}q(x)(u_{n}+u_{m})^{2}\,dx\) converges to 4, the right-hand side of (12) converges to 0, and so does the left-hand side. Hence, \(\{u_{n}\}\) is a Cauchy sequence in \(X_{0}\), and so it also converges to u in \(X_{0}\). It is easy to verify that \(\||u\||_{X_{0}}^{2}=\lambda\). Therefore, the considered eigenvalue problem has a principal eigenvalue denoted by \(\lambda_{1}(q)\) and its eigenfunction denoted by \(\phi_{1}\). □

Lemma 2.3

Assume that (M0) holds. If \(0\leq q(x)\in L^{\infty}(\Omega)\) and \(q(x)\neq0\), then the eigenvalue problem

$$ \textstyle\begin{cases} M(\|u\|_{X_{0}}^{2})(-\Delta)^{s} u=\mu q(x)u& \textit{in } \Omega, \\ u=0& \textit{in } \mathbb{R}^{N}\setminus\Omega \end{cases} $$

has a principal eigenvalue denoted by \(\mu_{1}\) and its associated positive eigenfunction \(\psi_{1}\).

Proof

Let

$$\mu:= \inf_{\int_{\Omega}q(x)u^{2}\,dx=1}\widehat{M}\bigl(\|u\|_{X_{0}}^{2} \bigr). $$

Since M satisfies \(M_{0}\), according to the proof of Lemma 2.2, we can find a positive \(\psi_{1}\in X_{0}\) that realizes this infimum denoted by \(\mu_{1}\). □

Now, we give our main results.

Theorem 2.1

Assume that (f1) and (f3) hold and M satisfies (M0) and (M1). Then if \(m_{1} \lambda_{1}(q)<1\), then problem (1) has a positive solution.

Remark 2.1

Here, we have revised the second result of Theorem 1 in [26] since their proof is not clear under their assumptions.

Theorem 2.2

Assume that (f1) to (f3) hold and M satisfies (M0) and (M1). Then:

  1. (i)

    If \(\mu_{1}>1\), then problem (1) has no solution.

  2. (ii)

    If \(\mu_{1}=1\), \(m_{0}\lambda_{1}(q)\geq1\), and there is a positive solution \(u\in X_{0}\) of problem (1), then

    $$f(x,u)=\lambda_{1}(q)q(x)M\bigl(\|u\|_{X_{0}}^{2} \bigr) \quad \textit{a.e. in } \Omega. $$

Remark 2.2

This our result is an analogue and some generalization of the first and third results in Theorem 1 (see [26]).

For proving our main results, the following version of the mountain pass theorem is our main tool, which can be found in [29].

Lemma 2.4

Let E be a real Banach space and suppose that \(I\in C^{1}(E,\mathbb{R})\) satisfies the condition

$$\max\bigl\{ I(0),I(u_{1})\bigr\} \leq\alpha< \beta\leq\inf _{\|u\|_{E}=\rho}I(u) $$

for some \(\alpha<\beta\), \(\rho>0\) and \(u_{1}\in E\) with \(\|u_{1}\|_{E}>\rho \). Let \(c\geq\beta\) be characterized by

$$c=\inf_{\gamma\in\Gamma}\max_{s\in[0,1]}I\bigl(\gamma(s)\bigr), \quad \textit{where } \Gamma=\bigl\{ \gamma\in C\bigl([0,1],E\bigr);\gamma(0)=0, \gamma(1)=u_{1}\bigr\} $$

is the set of continuous paths joining 0 and \(u_{1}\). Then there exists a sequence \(\{u_{n}\}\subset E\) such that

$$I(u_{n})\rightarrow c\geq\beta \quad \textit{and} \quad \bigl(1+ \Vert u_{n}\Vert _{E}\bigr) \bigl\Vert I'(u_{n})\bigr\Vert _{E'}\rightarrow0 \quad \textit{as } n\rightarrow\infty, $$

where \(E'\) is the dual of E.

Proposition 2.1

Under the assumptions of Theorem  2.1, we have:

  1. (a)

    There exist \(\rho,\beta>0\) such that \(\mathcal{J}(u)\geq\beta\) for all \(u\in X_{0}\) with \(\|u\|_{X_{0}}=\rho\);

  2. (b)

    \(\mathcal{J}(t\phi_{1})=-\infty\) as \(t\rightarrow+\infty\).

Proof

It follows from (f1) and (f3) that, for any \(\varepsilon >0\), there exists \(A=A(\varepsilon)\geq0\) such that, for all \((x,s)\in\Omega\times{\mathbb{R}} \),

$$ F(x,s)\leq \frac{1}{2}\bigl(\vert p\vert _{\infty}+\varepsilon\bigr)s^{2}+As^{\gamma+1}, $$
(13)

where \(\gamma\in(1, \frac{N+s}{N-s})\).

Choose \(\varepsilon>0\) such that \(|p|_{\infty}+\varepsilon <\lambda_{1}\). By (13), Lemma 2.1, and the Sobolev inequality we obtain

$$\begin{aligned} \mathcal{J}(u) =&\frac{1}{2}\widehat{M}\bigl(\|u\|_{X_{0}}^{2} \bigr)- \int_{\Omega}{F(x,u)}\,dx \\ \geq&\frac{1}{2}\widehat{M}\bigl(\|u\|_{X_{0}}^{2} \bigr)-\frac{1}{2} \int_{\Omega}{\bigl[\bigl(|p|_{\infty}+ \varepsilon\bigr)u^{2}+A|u|^{\gamma+1} \bigr]\,dx} \\ \geq&\frac{1}{2}\biggl(m_{0}-\frac{|p|_{\infty}+\varepsilon }{\lambda _{1}}\biggr)\|u \|_{X_{0}}^{2}-c\|u\|_{X_{0}}^{\gamma+1}. \end{aligned}$$

So, part (a) holds if we choose \(\|u\|=\rho>0\) small enough.

To prove (b), we can write, for t sufficiently large,

$$\begin{aligned} \begin{aligned} \widehat{M}(t)&= \int_{0}^{t}M(s)\,ds= \int_{0}^{t_{0}}M(s)\,ds+ \int _{t_{0}}^{t}m_{1}\,ds \\ &= \widehat{M}(t_{0})-m_{1}t_{0}+m_{1}t \leq m_{2}+m_{1}t, \quad \mbox{with } m_{2}= \bigl\vert \widehat{M}(t_{0})-m_{1}t_{0}\bigr\vert . \end{aligned} \end{aligned}$$

Using Fatou’s lemma, for \(\varepsilon >0\) small enough, we get

$$\begin{aligned} \lim_{t\rightarrow+\infty}\frac{\mathcal{J}(t\phi_{1})}{t^{2}} \leq &\lim_{t\rightarrow+\infty} \frac{1}{2}\biggl(\frac{m_{2}}{t^{2}}+m_{1}\|\phi_{1} \|_{X_{0}}^{2}\biggr)-\lim_{t\rightarrow+\infty} \int_{\Omega}\frac{F(x,t\phi_{1})}{t^{2}}\,dx \\ \leq& \frac{1}{2}m_{1}\|\phi_{1} \|_{X_{0}}^{2}- \int_{\Omega}\lim_{t\rightarrow +\infty}\frac{F(x,t\phi_{1})}{t^{2}\phi_{1}^{2}} \phi_{1}^{2}\,dx \\ \leq& \frac{1}{2}(m_{1}+\varepsilon )\|\phi_{1} \|_{X_{0}}^{2}-\frac{1}{2\lambda _{1}(q)}\|\phi_{1} \|_{X_{0}}^{2}. \end{aligned}$$

Then

$$\lim_{t\rightarrow+\infty} \frac{\mathcal{J}(t\phi_{1})}{t^{2}}\leq \frac{1}{2} \biggl(m_{1}+\varepsilon -\frac{1}{\lambda_{1}(q)} \biggr)\|\phi _{1} \|_{X_{0}}^{2}< 0\quad \mbox{since } m_{1} \lambda_{1}(q)< 1, $$

and part (b) is proved. □

Finally, we recall a definition of the compactness condition and a version of the mountain pass theorem.

Definition 2.1

Let \((X_{0},\|\cdot\|_{X_{0}})\) be a real Banach space with its dual space \((X_{0}^{*},\|\cdot\|_{X_{0}^{*}})\) and \(\mathcal{J}\in C^{1}(X_{0},\mathbb{R})\). For \(c\in\mathbb{R}\), we say that \(\mathcal{J}\) satisfies the \((C)_{c}\) condition stated in [30] if for any sequence \(\{x_{n}\}\subset X_{0}\) with

$$ \mathcal{J}(x_{n})\rightarrow c,\qquad \bigl\Vert D \mathcal{J}(x_{n})\bigr\Vert _{X_{0}^{*}}\bigl(1+\Vert x_{n}\Vert _{X_{0}}\bigr)\rightarrow0, $$

there is a subsequence \(\{ x_{n_{k}}\}\) such that \(\{ x_{n_{k}}\}\) converges strongly in \(X_{0}\).

3 Proof of the main results

Proof of Theorem 2.1

Since \(m_{1}\lambda_{1}(q)<1\), by Proposition 2.1(a), (b) we can find \(t_{1}\) large enough such that \(\mathcal{J}(t_{1}\phi_{1})<0\), where \(\phi_{1}>0\) is given in Lemma 2.2. Define

$$\Gamma=\bigl\{ \gamma\in C\bigl([0,1], X_{0}\bigr); \gamma(0)=0, \gamma(1)=t_{1}\phi _{1}\bigr\} \quad \text{and}\quad c=\inf _{\gamma\in\Gamma}\max_{s\in[0,1]}\mathcal{J}\bigl(\gamma(s) \bigr). $$

Then, \(c\geq\beta>0\), and by Lemma 2.4 there exists a sequence \(\{ u_{n}\}\) such that

$$ \mathcal{J}(u_{n})=\frac{1}{2}\widehat{M}\bigl( \|u_{n}\|_{X_{0}}^{2}\bigr)- \int _{\Omega}F(x,u_{n})\,dx=c+o(1) $$
(14)

and

$$ \bigl(1+\Vert u_{n}\Vert _{X_{0}}\bigr) \bigl\Vert \mathcal{J}'(u_{n})\bigr\Vert _{X_{0}}\rightarrow0\quad \mbox{as } n\rightarrow\infty, $$
(15)

which implies that

$$ \bigl\langle \mathcal{J}'(u_{n}),u_{n} \bigr\rangle =M\bigl(\Vert u_{n}\Vert _{X_{0}}^{2} \bigr) \Vert u_{n}\Vert _{X_{0}}^{2}- \int_{\Omega}f(x,u_{n})u_{n} \,dx=o(1). $$
(16)

Now, we need to show that \(\{\|u_{n}\|_{X_{0}}\}\) is bounded. Suppose by contradiction that \(\|u_{n}\|_{X_{0}}\rightarrow\infty\) as \(n\rightarrow\infty\) and let

$$ w_{n}=\frac{\sqrt{t_{0}}}{\|u_{n}\|_{X_{0}}}u_{n}. $$
(17)

Notice that \(\|w_{n}\|_{X_{0}}=\sqrt{t_{0}}\). Then there exists a subsequence \(\{w_{n}\}\) such that

$$\begin{aligned}& w_{n}\rightharpoonup w \quad \text{weakly in } X_{0}, \\& w_{n}\rightarrow w\quad \text{in } L^{2}(\Omega), \\& w_{n}\rightarrow w\quad \text{a.e. in } \Omega. \end{aligned}$$

We have

$$w\neq0. $$

Indeed, suppose \(w=0\). By (f1) and (f3) there exist \(C>0\) such that \(|f(x,t)|\leq C|t|\) for \(x\in\Omega\) and \(t\geq0\). Then from (16) and (17) we get

$$t_{0}m_{0}\leq t_{0}M\bigl(\|u_{n} \|_{X_{0}}^{2}\bigr)= \int_{\Omega}\frac {f(x,u_{n})}{u_{n}}w_{n}^{2}\,dx +o(1)\leq\theta \int_{\Omega}w_{n}^{2}\,dx +o(1)\rightarrow0, $$

which is a contradiction. So \(w\neq0\).

We claim that the identity

$$\int_{\mathbb{R}^{N}\times\mathbb{R}^{N}} m_{1}\bigl(w(x)-w(y)\bigr) \bigl(\varphi (x)-\varphi(y)\bigr) K(x-y)\,dx\,dy- \int_{\Omega}q(x)w\varphi \,dx=0 $$

is true for any \(\varphi\in X_{0}\). Set \(p_{n}(x)=f(x,u_{n})u_{n}^{-1}(M(\|u_{n}\|_{X_{0}}^{2}))^{-1}\) if \(u_{n}(x)>0\); otherwise, \(p_{n}(x)=0\). As before, \(0\leq p_{n}(x)\leq\frac{C}{m_{0}}\) for all \(x\in\Omega\). Then there exists a subsequence \(\{p_{n}\}\) such that

$$p_{n}\rightharpoonup h\quad \text{in } L^{2}(\Omega) \text{ with } 0\leq h\leq \frac{C}{m_{0}}. $$

Since \(\|u_{n}\|_{X_{0}}\rightarrow+\infty\) and \(w_{n}\rightarrow w\) a.e. in Ω, it follows from (17) that

$$u_{n}\rightarrow+\infty \quad \text{in } \Omega \text{ if } w(x)>0 \text{ a.e. in } \Omega. $$

Then by (f3) and (M1) we obtain

$$ h(x)=q(x) (m_{1})^{-1}\quad \text{if } w(x)>0. $$
(18)

Since \(w_{n}\rightarrow w\) in \(L^{2}(\Omega)\), we have

$$\int_{\Omega}p_{n}(x) w_{n}(x)\varphi(x)\,dx \rightarrow \int_{\Omega}h(x)w^{+}\varphi \,dx $$

for all \(\varphi\in L^{2}(\Omega)\); then

$$ p_{n}w_{n}\rightharpoonup hw^{+}\quad \mbox{in } L^{2}(\Omega). $$
(19)

Using (15) and the fact that \(\|u_{n}\|_{X_{0}}\rightarrow\infty\), we get that, for any \(\varphi\in X_{0}\),

$$\begin{aligned}& \biggl\vert \int_{\mathbb{R}^{N}\times\mathbb{R}^{N}} \bigl(w_{n}(x)-w_{n}(y)\bigr) \bigl(\varphi(x)-\varphi(y)\bigr) K(x-y)\,dx\,dy- \int_{\Omega}p_{n}(x)w_{n}(x)\varphi \,dx \biggr\vert \\& \quad =\frac{\sqrt{t_{0}}}{\|u_{n}\|_{X_{0}}M(\|u_{n}\|_{X_{0}}^{2})}\rightarrow0\quad \text{as } n\rightarrow\infty. \end{aligned}$$

From \(w_{n}\rightharpoonup w\) in \(X_{0}\) and (19) we get

$$\int_{\mathbb{R}^{N}\times\mathbb{R}^{N}} \bigl(w(x)-w(y)\bigr) \bigl(\varphi (x)-\varphi(y) \bigr) K(x-y)\,dx\,dy- \int_{\Omega}h(x) w\varphi \,dx=0,\quad \forall \varphi\in X_{0}. $$

Taking \(\varphi=w^{-}\), it follows that \(\|w^{-}\|_{X_{0}}=0\), and so \(w=w^{+}\geq0\) on Ω. Then by the strong maximum principle (see [31]) we get \(w(x)>0\) on Ω. Thus, by (18) we have

$$\int_{\mathbb{R}^{N}\times\mathbb{R}^{N}} m_{1}\bigl(w(x)-w(y)\bigr) \bigl(\varphi (x)-\varphi(y)\bigr) K(x-y)\,dx\,dy- \int_{\Omega}q(x) w\varphi \,dx=0,\quad \forall \varphi\in X_{0}. $$

By Lemma 2.2, since \(m_{1}\lambda_{1}(q)<1\), this is a contradiction. So \(\{u_{n}\}\) is bounded in \(X_{0}\).

Finally, we show that \(u_{n}\rightarrow u\) in \(X_{0}\). Indeed, since \(\{ u_{n}\}\) is bounded in \(X_{0}\), we may assume that there exists \(u\in X_{0}\) such that

$$\begin{aligned}& u_{n}\rightharpoonup u \quad \text{weakly in } X_{0}, \\& u_{n}\rightarrow u \quad \text{in } L^{2}(\Omega), \\& u_{n}\rightarrow u\quad \text{a.e. in } \Omega. \end{aligned}$$

Hence, by (15) we have

$$\begin{aligned}& M\bigl(\Vert u_{n}\Vert _{X_{0}}^{2} \bigr) \int_{\mathbb{R}^{N}\times\mathbb {R}^{N}}\bigl(u_{n}(x)-u_{n}(y)\bigr) \bigl[\bigl(u(x)-u_{n}(x)\bigr)-\bigl(u(y)-u_{n}(y)\bigr) \bigr] K(x-y)\,dx\,dy \\& \quad {} - \int_{\Omega}f(x,u_{n}) (u_{n}-u)\,dx \rightarrow0. \end{aligned}$$
(20)

By (f1) and (f3) there exists a constant \(C>0\) such that

$$\bigl\vert f(x,t)\bigr\vert \leq C\vert t\vert ,\quad \forall(x,t)\in \Omega\times\mathbb{R}. $$

Consequently,

$$\begin{aligned} \int_{\Omega}\bigl\vert f(x,u_{n}) (u-u_{n})\bigr\vert \,dx \leq&C \biggl( \int_{\Omega} \vert u_{n}\vert ^{2}\,dx \biggr)^{\frac{1}{2}} \biggl( \int_{\Omega} \vert u_{n}-u\vert ^{2}\,dx \biggr)^{\frac{1}{2}} \\ \leq& C\vert u_{n}-u\vert _{2}\rightarrow0 \end{aligned}$$
(21)

as \(n\rightarrow\infty\). From (20) and (21) we know

$$\|u_{n}\|_{X_{0}}\rightarrow\|u\|_{X_{0}}\quad \text{as } n\rightarrow\infty. $$

Hence, \(u_{n}\rightarrow u\) in \(X_{0}\) as \(n\rightarrow\infty\). □

Proof of Theorem 2.2

If \(\mu_{1}>1\), then problem (1) has no solution. Indeed, if \(u\in X_{0}\) is a solution of problem (1), then from (f1), (f2), and (f3) we have

$$M\bigl(\|u\|_{X_{0}}^{2}\bigr) \|u\|_{X_{0}}^{2}= \int_{\Omega}f(x,u)u\, dx\leq \int_{\Omega}q(x)u^{2}\,dx. $$

Then

$$\mu_{1}\leq1. $$

Thus, we have proved (i). Next, we prove (ii). Suppose that \(\mu_{1}=1\). From Lemma 2.3 we have

$$\begin{aligned}& M\bigl(\|\psi_{1}\|_{X_{0}}^{2}\bigr) \int_{\mathbb{R}^{N}\times\mathbb{R}^{N}}\bigl(\psi _{1}(x)-\psi_{1}(y) \bigr) \bigl(\varphi(x)-\varphi(y)\bigr) K(x-y)\,dx\,dy \\& \quad = \int_{\Omega }q(x) \psi_{1} \varphi \,dx \end{aligned}$$
(22)

for all \(\varphi\in X_{0}\). If u is a positive solution of problem (1), for \(\psi_{1}\), we have

$$\begin{aligned}& M\bigl(\|u\|_{X_{0}}^{2}\bigr) \int_{\mathbb{R}^{N}\times\mathbb{R}^{N}}\bigl(\psi _{1}(x)-\psi_{1}(y) \bigr) \bigl(u(x)-u(y)\bigr) K(x-y)\,dx\,dy \\& \quad = \int_{\Omega}q(x)f(x,u) \psi _{1} \,dx. \end{aligned}$$
(23)

From (22) and (23) we get

$$ \int_{\Omega}\biggl(\frac{f(x,u)}{M(\|u\|_{X_{0}}^{2})}-\frac {q(x)u}{M(\|\psi_{1}\|_{X_{0}}^{2})} \biggr) \psi_{1}\,dx=0. $$
(24)

By condition (M0), \(\mu_{1}=1\), and \(m_{0}\lambda_{1}(q)\geq1\). Similarly to the proof of case 3 in Theorem 1 (see [26]), we have

$$f(x,u)=\lambda_{1}(q)q(x) M\bigl(\|u\|_{X_{0}}^{2} \bigr)u\quad \text{a.e. in } \Omega. $$

 □