1 Introduction

In this paper, we investigate the multiplicity and concentration phenomenon of solutions for the following fractional (pq)-Laplacian Kirchhoff type problem:

$$\begin{aligned} \left\{ \begin{array}{ll} \left( 1+ [u]_{s,p}^{p}\right) (-\Delta )_{p}^{s}u+\left( 1+ [u]^{q}_{s, q}\right) (-\Delta )_{q}^{s}u + V(\varepsilon x) (|u|^{p-2}u + |u|^{q-2}u)= f(u) &{} \text{ in } \mathbb {R}^{N}, \\ u\in W^{s, p}(\mathbb {R}^{N})\cap W^{s,q}(\mathbb {R}^{N}), \quad u>0 \text{ in } \mathbb {R}^{N}, \end{array} \right. \end{aligned}$$
(1.1)

where \(\varepsilon >0\) is a small parameter, \(s\in (0, 1)\), \(1< p<q<\frac{N}{s}<2q\), \(V: \mathbb {R}^{N}\rightarrow \mathbb {R}\) is a bounded and continuous potential fulfilling the following conditions [18]:

\((V_1)\):

there exists \(V_{0}>0\) such that \(V_{0}=\inf _{x\in \mathbb {R}^{N}} V(x)\),

\((V_2)\):

there exists a bounded open set \(\Lambda \subset \mathbb {R}^{N}\) such that

$$\begin{aligned} V_{0}< \min _{\partial \Lambda } V \quad \text{ and } \quad 0\in M=\{x\in \Lambda : V(x)=V_{0}\}, \end{aligned}$$

and \(f:\mathbb {R}\rightarrow \mathbb {R}\) is a continuous nonlinearity such that \(f(t)=0\) for \(t\le 0\) and satisfying the following hypotheses:

\((f_{1})\):

\(\displaystyle {\lim _{|t|\rightarrow 0} \frac{|f(t)|}{|t|^{2p-1}}=0}\),

\((f_{2})\):

there exists \(\nu \in (2q, q^{*}_{s})\) such that \(\displaystyle {\lim _{|t|\rightarrow \infty } \frac{|f(t)|}{|t|^{\nu -1}}=0}\), where \(q^{*}_{s}= \frac{Nq}{N-sq}\),

\((f_{3})\):

there exists \(\vartheta \in (2q, \nu )\) such that \(\displaystyle {0<\vartheta F(t)= \vartheta \int _{0}^{t} f(\tau ) \, d\tau \le t f(t)}\) for all \(t>0\),

\((f_{4})\):

the map \(t\mapsto \displaystyle {\frac{f(t)}{t^{2q-1}}}\) is increasing in \((0, \infty )\).

The symbol \((-\Delta )^{s}_{t}\), with \(t\in \{p, q\}\), stands for the fractional t-Laplacian operator defined, up to a normalization constant depending on N, s and t, by setting

$$\begin{aligned} (-\Delta )_{t}^{s}u(x)= 2\lim _{r\rightarrow 0} \int _{\mathbb {R}^{N}\setminus B_{r}(x)} \frac{|u(x)- u(y)|^{t-2}(u(x)- u(y))}{|x-y|^{N+st}} dy \quad (x\in \mathbb {R}^{N}), \end{aligned}$$

for any function \(u: \mathbb {R}^{N}\rightarrow \mathbb {R}\) sufficiently smooth. We recall that the recent years have seen a surge of interest in nonlocal and fractional problems involving the fractional t-Laplacian operator because of the presence of two features: the nonlinearity of the operator and its nonlocal character. For this reason, several existence, multiplicity and regularity results have been established by many authors; see for instance [4, 8, 10, 20, 24, 28, 38].

When \(s=1\), the study of (1.1) is strictly related to the following (pq)-Laplacian equation

$$\begin{aligned} -\Delta _{p}u-\Delta _{q} u+|u|^{p-2}u+|u|^{q-2}u= f(x,u) \text{ in } \mathbb {R}^{N}, \end{aligned}$$

which comes from a general reaction–diffusion system

$$\begin{aligned} u_{t}={{\,\mathrm{div}\,}}(D(u)\nabla u)+c(x,u) \text{ where } D(u)=|\nabla u|^{p-2}+|\nabla u|^{q-2}. \end{aligned}$$

This system has a wide range of applications in physics and related sciences, such as biophysics, plasma physics, and chemical reaction design. In such applications, the function u describes a concentration, \({{\,\mathrm{div}\,}}(D(u) \nabla u)\) corresponds to the diffusion with diffusion coefficient D(u), and the reaction term c(xu) relates to source and loss processes. Typically, in chemical and biological applications, the reaction term c(xu) is a polynomial of u with variable coefficients; see [17]. Some classical results for (pq)-Laplacian problems in bounded or unbounded domains can be found in [2, 22, 26, 27, 31, 33, 34] and the references therein. We also mention [15, 30] in which the authors discussed Kirchhoff type problems with the (pq)-Laplacian operator \(-\Delta _{p}-\Delta _{q}\).

For what concerns the nonlocal framework, only few papers studied fractional (pq)-Laplacian problems. Such problems involve the sum of two nonlocal nonlinear operators with different scaling properties and so some nontrivial additional technical difficulties arise with respect to the local case \(s=1\) and \(p\ne q\), and the fractional case \(s\in (0, 1)\) and \(p=q\).

In [16], the authors obtained existence, nonexistence, and multiplicity of solutions for a subcritical fractional (pq)-Laplacian problem. In [5], the author proved an existence result for a critical fractional (pq)-Laplacian problem, by using a concentration-compactness lemma and the mountain pass theorem. Multiplicity results for a class of fractional (pq)-Laplacian problems in bounded domains and with critical nonlinearities have been established in [12]. The multiplicity of concentrating solutions for a fractional (pq)-Laplacian problem of Schrödinger type has been recently demonstrated in [11]. For other contributions devoted to this class of problems, we refer to [1, 7, 9, 12, 25, 29].

To our knowledge, no results for Kirchhoff type problems driven by the fractional (pq)-Laplacian operator \((-\Delta )^{s}_{p}+(-\Delta )^{s}_{q}\) appear in the current literature. Particularly motivated by this fact and the above-mentioned works, in this paper, we examine the multiplicity and concentration properties of solutions for (1.1). More precisely, our main result can be stated as follows:

Theorem 1.1

Assume that \((V_{1})\)-\((V_{2})\) and \((f_{1})\)-\((f_{4})\) hold. Then, for any \(\delta >0\) such that

$$\begin{aligned} M_{\delta }=\{x\in \mathbb {R}^{N}: \mathrm{dist}(x, M)\le \delta \}\subset \Lambda , \end{aligned}$$

there exists \(\varepsilon _{\delta }>0\) such that, for any \(\varepsilon \in (0, \varepsilon _{\delta })\), problem (1.1) has at least \(cat_{M_{\delta }}(M)\) positive solutions. Moreover, if \(u_{\varepsilon }\) denotes one of these solutions and \(x_{\varepsilon }\in \mathbb {R}^{N}\) is a global maximum point of \(u_{\varepsilon }\), then

$$\begin{aligned} \lim _{\varepsilon \rightarrow 0} V(\varepsilon x_{\varepsilon })=V_{0}. \end{aligned}$$

The proof of Theorem 1.1 is based on the generalized Nehari manifold method, a penalization technique, and the Lusternik–Schnirelman category theory. Firstly, inspired by [18], we modify the nonlinearity f in a suitable way and we consider an auxiliary problem whose advantage with respect to (1.1) is that the corresponding energy functional \(\mathcal {J}_{\varepsilon }\) possesses a mountain pass geometry [3]. Moreover, an accurate analysis allows us to verify that \(\mathcal {J}_{\varepsilon }\) satisfies the Palais–Smale condition at any level \(c\in \mathbb {R}\) (\((PS)_{c}\) condition for short). Secondly, since we are interested in providing a multiplicity result for (1.1), and our nonlinearity f is only continuous, we implement the barycenter machinery and adapt some abstract critical point results found in [36]. This kind of argument also appears in [23] to analyze a Schrödinger–Kirchhoff elliptic equation, in [6] to handle various fractional Laplacian elliptic problems, and in [11] to deal with a fractional (pq)-Schrödinger equation. However, with respect to [6, 11, 23], the mixture of Kirchhoff terms and two different nonhomogeneous nonlocal operators makes the study of (1.1) rather tough and an appropriate investigation will be done to circumvent some significant technical complications; see for instance the proofs of Lemmas 2.4, 2.5, 2.7 and Theorem 3.1. Finally, we show that the solutions of the modified problem are solutions to (1.1) for \(\varepsilon >0\) small enough, by using a Moser type iteration [32] and the Hölder regularity result in [11]. As far as we know, this is the first time that the penalization approach and the Lusternik–Schnirelman category theory are combined to treat fractional (pq)-Laplacian problems like (1.1).

The paper is organized as follows. In Sect. 2, we collect some basic results for fractional Sobolev spaces and we introduce the modified problem. In Sect. 3, we tackle the limiting Kirchhoff problem. In Sect. 4, we present a multiplicity result for the modified problem. The last section is dedicated to the proof of Theorem 1.1.

2 The Modified Problem

2.1 Notations and Some Useful Lemmas

Let \(p\in [1, \infty ]\) and \(A\subset \mathbb {R}^{N}\) be a measurable set. We will denote by \(|\cdot |_{L^{p}(A)}\) the norm in \(L^{p}(A)\), and we will simply use the notation \(|\cdot |_{p}\) when \(A=\mathbb {R}^{N}\).

Let \(s\in (0, 1)\), \(p\in (1, \infty )\) and \(N>sp\). The fractional Sobolev space \(W^{s, p}(\mathbb {R}^{N})\) is defined by

$$\begin{aligned} W^{s, p}(\mathbb {R}^{N})=\left\{ u\in L^{p}(\mathbb {R}^{N}): \iint _{\mathbb {R}^{2N}}\frac{|u(x)-u(y)|^{p}}{|x-y|^{N+sp}}\, dxdy<\infty \right\} , \end{aligned}$$

which is a Banach space with the norm

$$\begin{aligned} \Vert u\Vert _{W^{s,p}(\mathbb {R}^{N})}=(|u|^{p}_{p}+[u]^{p}_{s, p})^{\frac{1}{p}}, \text{ where } [u]_{s, p}=\left( \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{p}}{|x-y|^{N+sp}}\, dxdy\right) ^{\frac{1}{p}}. \end{aligned}$$

For \(u, v\in W^{s,p}(\mathbb {R}^{N})\), we put

$$\begin{aligned} \langle u, v\rangle _{s,p}=\iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{p-2}(u(x)-u(y))(v(x)-v(y))}{|x-y|^{N+sp}} \, dx dy. \end{aligned}$$

The following embeddings are well known in the literature.

Theorem 2.1

[19] Let \(s\in (0, 1)\), \(p\in (1, \infty )\) and \(N>sp\). Then, \(W^{s, p}(\mathbb {R}^{N})\) is continuously embedded in \(L^{t}(\mathbb {R}^{N})\) for any \(t\in [p, p^{*}_{s}]\) and compactly embedded in \(L^{t}_{loc}(\mathbb {R}^{N})\) for any \(t\in [1, p^{*}_{s})\).

For the reader’s convenience, we also recall some useful lemmas.

Lemma 2.1

[8] Let \(s\in (0, 1)\), \(p\in (1, \infty )\) and \(N>sp\). Let \(r\in [p, p^{*}_{s})\). If \(\{u_{n}\}_{n\in \mathbb {N}}\) is a bounded sequence in \(W^{s, p}(\mathbb {R}^{N})\) and if

$$\begin{aligned} \lim _{n\rightarrow \infty } \sup _{y\in \mathbb {R}^{N}} \int _{B_{R}(y)} |u_{n}|^{r} dx=0, \end{aligned}$$

where \(R>0\), then \(u_{n}\rightarrow 0\) in \(L^{t}(\mathbb {R}^{N})\) for all \(t\in (p, p^{*}_{s})\).

Lemma 2.2

[8] Let \(s\in (0, 1)\), \(t\in (1, \infty )\) and \(N>st\). Let \(\{u_{n}\}_{n\in \mathbb {N}}\subset W^{s, t}(\mathbb {R}^{N})\) be a bounded sequence in \(W^{s, t}(\mathbb {R}^{N})\), and let \(\phi \in C^{\infty }(\mathbb {R}^{N})\) be a function such that \(0\le \phi \le 1\) in \(\mathbb {R}^{N}\), \(\phi =0\) in \(B_{1}(0)\) and \(\phi =1\) in \(B^{c}_{2}(0)\). For each \(\rho >0\) let \(\phi _{\rho }(x)=\phi (\frac{x}{\rho })\). Then

$$\begin{aligned} \lim _{\rho \rightarrow \infty } \limsup _{n\rightarrow \infty } \iint _{\mathbb {R}^{2N}} \frac{|\phi _{\rho }(x)-\phi _{\rho }(y)|^{t}}{|x-y|^{N+st}} |u_{n}(x)|^{t}\, dxdy=0. \end{aligned}$$

Proof

The proof of this result can be found in [8], but here we give a more direct proof. Using the definition of \(\phi _{\rho }\), polar coordinates and the boundedness of \(\{u_{n}\}_{n\in \mathbb {N}}\) in \(W^{s, t}(\mathbb {R}^{N})\), we can see that

$$\begin{aligned}&\iint _{\mathbb {R}^{2N}} \frac{|\phi _{\rho }(x)-\phi _{\rho }(y)|^{t}}{|x-y|^{N+st}}|u_{n}(x)|^{t} \,dx dy \\&=\int _{\mathbb {R}^{N}} \int _{|y-x|>\rho } \frac{|\phi _{\rho }(x)-\phi _{\rho }(y)|^{t}}{|x-y|^{N+st}}|u_{n}(x)|^{t} \,dx dy \\&\quad +\int _{\mathbb {R}^{N}} \int _{|y-x|\le \rho } \frac{|\phi _{\rho }(x)-\phi _{\rho }(y)|^{t}}{|x-y|^{N+st}}|u_{n}(x)|^{t} \,dx dy \\&\le C \int _{\mathbb {R}^{N}} |u_{n}(x)|^{t} \left( \int _{|y-x|>\rho } \frac{dy}{|x-y|^{N+st}}\right) \,dx \\&\quad + \frac{C}{\rho ^{t}} \int _{\mathbb {R}^{N}} |u_{n}(x)|^{t} \left( \int _{|y-x|\le \rho } \frac{dy}{|x-y|^{N+st-t}}\right) \,dx \\&\le C \int _{\mathbb {R}^{N}} |u_{n}(x)|^{t} \left( \int _{|z|>\rho } \frac{dz}{|z|^{N+st}}\right) \,dx + \frac{C}{\rho ^{t}} \int _{\mathbb {R}^{N}} |u_{n}(x)|^{t} \left( \int _{|z|\le \rho } \frac{dz}{|z|^{N+st-t}}\right) \,dx \\&\le C \int _{\mathbb {R}^{N}} |u_{n}(x)|^{t} \,dx \left( \int _{\rho }^{\infty } \frac{dr}{r^{st+1}}\right) + \frac{C}{\rho ^{t}} \int _{\mathbb {R}^{N}} |u_{n}(x)|^{t} \,dx \left( \int _{0}^{\rho } \frac{dr}{r^{st-t+1}}\right) \\&\le \frac{C}{\rho ^{st}} \int _{\mathbb {R}^{N}} |u_{n}(x)|^{t} \,dx+\frac{C}{\rho ^{t}} \rho ^{-st+t} \int _{\mathbb {R}^{N}} |u_{n}(x)|^{t} \,dx \le \frac{C}{\rho ^{st}} \int _{\mathbb {R}^{N}} |u_{n}(x)|^{t} \,dx\le \frac{C}{\rho ^{st}}, \end{aligned}$$

and letting first \(n\rightarrow \infty \) and then \(\rho \rightarrow \infty \), we get the thesis. \(\square \)

Let \(s\in (0, 1)\) and \(p, q\in (1, \infty )\). Consider the space

$$\begin{aligned} \mathcal {W}=W^{s, p}(\mathbb {R}^{N})\cap W^{s, q}(\mathbb {R}^{N}) \end{aligned}$$

endowed with the norm

$$\begin{aligned} \Vert u\Vert _{\mathcal {W}}=\Vert u\Vert _{W^{s,p}(\mathbb {R}^{N})}+\Vert u\Vert _{W^{s,q}(\mathbb {R}^{N})}. \end{aligned}$$

Since \(W^{s,r}(\mathbb {R}^{N})\), with \(r\in (1, \infty )\), is a separable reflexive Banach space (this can be proved by using the operator \(T: W^{s, r}(\mathbb {R}^{N})\rightarrow L^{r}(\mathbb {R}^{N})\times L^{r}(\mathbb {R}^{2N})\) defined by \(Tu=(u, (u(x)-u(y))|x-y|^{-\frac{N}{r}-s})\) and arguing as in the proof of Proposition 8.1 in [13]), we obtain that \(\mathcal {W}\) is also a separable reflexive Banach space.

For any \(\varepsilon >0\), we introduce the space

$$\begin{aligned} \mathbb {X}_{\varepsilon }=\left\{ u\in \mathcal {W}: \int _{\mathbb {R}^{N}} V(\varepsilon x) \left( |u|^{p}+|u|^{q}\right) \,dx<\infty \right\} \end{aligned}$$

equipped with the norm

$$\begin{aligned} \Vert u\Vert _{\mathbb {X}_{\varepsilon }}= \Vert u\Vert _{V_{\varepsilon }, p} + \Vert u\Vert _{V_{\varepsilon }, q}, \end{aligned}$$

where

$$\begin{aligned} \Vert u\Vert _{V_{\varepsilon }, t}= \left( [u]_{s,t}^{t} + \int _{\mathbb {R}^{N}} V(\varepsilon x) |u|^{t}\, dx\right) ^{\frac{1}{t}} \quad \text{ for } t\in \{p, q\}. \end{aligned}$$

2.2 The Penalization Approach

We adapt in a suitable way the del Pino–Felmer penalization approach [18] to attack (1.1). First, we observe that the map \(t\mapsto \frac{f(t)}{t^{p-1}+t^{q-1}}\) is increasing in \((0, \infty )\). Indeed,

$$\begin{aligned} \frac{f(t)}{t^{p-1}+t^{q-1}}=\frac{f(t)}{t^{2q-1}} \frac{t^{2q-1}}{t^{p-1}+t^{q-1}} \end{aligned}$$

and noting that \(t\mapsto \frac{f(t)}{t^{2q-1}}\) is increasing in \((0, \infty )\) (by \((f_4)\)), and that \(t\mapsto \frac{t^{2q-1}}{t^{p-1}+t^{q-1}}\) is increasing in \((0, \infty )\) (because \(2q>p\)), we deduce the desired result.

Now, let us fix

$$\begin{aligned} K>\frac{q}{p}\left( \frac{\vartheta -p}{\vartheta -q}\right) >1, \end{aligned}$$

and let \(a>0\) be such that

$$\begin{aligned} f(a)=\frac{V_{0}}{K}(a^{p-1}+a^{q-1}). \end{aligned}$$

We define

$$\begin{aligned} {\tilde{f}}(t)=\left\{ \begin{array}{ll} f(t) &{} \text{ if } t\le a,\\ \displaystyle \frac{V_{0}}{K} (t^{p-1}+t^{q-1}) &{} \text{ if } t>a, \end{array} \right. \end{aligned}$$

and

$$\begin{aligned} g(x, t)= \left\{ \begin{array}{ll} \chi _{\Lambda }(x)f(t)+(1-\chi _{\Lambda }(x)){\tilde{f}}(t) &{} \text{ if } t>0, \\ 0 &{} \text{ if } t\le 0, \end{array} \right. \end{aligned}$$

where \(\chi _{A}\) denotes the characteristic function of \(A\subset \mathbb {R}^{N}\). By \((f_1)\)-\((f_4)\), we infer that \(g: \mathbb {R}^{N}\times \mathbb {R}\rightarrow \mathbb {R}\) is a Carathéodory function that fulfills the following assumptions:

\((g_1)\):

\(\displaystyle \lim _{t\rightarrow 0} \frac{g(x,t)}{t^{2p-1}}=0\) uniformly with respect to \(x\in \mathbb {R}^{N}\),

\((g_2)\):

\(g(x,t)\le f(t)\) for all \(x\in \mathbb {R}^{N}\) and \(t>0\),

\((g_3)\):

(i) \(0< \vartheta G(x,t)\le g(x,t)t\) for all \(x\in \Lambda \) and \(t>0\), (ii) \(0\le pG(x,t)\le g(x,t)t\le \frac{V_{0}}{K} (t^{p}+t^{q})\) for all \(x\in \Lambda ^{c}\) and \(t>0\),

\((g_4)\):

for each \(x\in \Lambda \), the function \(t\mapsto \displaystyle \frac{g(x,t)}{t^{p-1}+t^{q-1}}\) is increasing in \((0, \infty )\), and for each \(x\in \Lambda ^{c}\), the function \(t\mapsto \displaystyle \frac{g(x,t)}{t^{p-1}+t^{q-1}}\) is increasing in (0, a).

Let us introduce the auxiliary problem

$$\begin{aligned} \left\{ \begin{array}{ll} \left( 1+[u]_{s, p}^{p}\right) (-\Delta )^{s}_{p}u+\left( 1+[u]_{s, q}^{q}\right) (-\Delta )^{s}_{q}u+V(\varepsilon x)(|u|^{p-2}u+|u|^{q-2}u)=g(\varepsilon x, u) \quad &{} \text{ in } \mathbb {R}^{N}, \\ u\in W^{s, p}(\mathbb {R}^{N})\cap W^{s,q}(\mathbb {R}^{N}), \quad u>0 \text{ in } \mathbb {R}^{N}. \end{array} \right. \nonumber \\ \end{aligned}$$
(2.1)

We stress that if \(u_{\varepsilon }\) is a solution to (2.1) such that \(u_{\varepsilon }(x)\le a\) for all \(x\in \Lambda _{\varepsilon }^{c}\), where \(\Lambda _{\varepsilon }=\{x\in \mathbb {R}^{N} \, : \, \varepsilon x\in \Lambda \}\), then \(u_{\varepsilon }\) is also a solution to (1.1). Then we consider the functional \(\mathcal {J}_{\varepsilon }: \mathbb {X}_{\varepsilon }\rightarrow \mathbb {R}\) associated with (2.1), that is

$$\begin{aligned} \mathcal {J}_{\varepsilon }(u)= \frac{1}{p}\Vert u\Vert _{V_{\varepsilon },p}^{p}+\frac{1}{2p}[u]^{2p}_{s, p}+\frac{1}{q}\Vert u\Vert _{V_{\varepsilon },q}^{q}+\frac{1}{2q}[u]^{2q}_{s, q}-\int _{\mathbb {R}^{N}} G(\varepsilon x, u)\, dx. \end{aligned}$$

Clearly, \(\mathcal {J}_{\varepsilon }\in C^{1}(\mathbb {X}_{\varepsilon }, \mathbb {R})\) and it holds

$$\begin{aligned} \langle \mathcal {J}'_{\varepsilon }(u), \varphi \rangle&=(1+[u]_{s, p}^{p}) \langle u, \varphi \rangle _{s, p}+(1+[u]_{s, q}^{q})\langle u, \varphi \rangle _{s, q} \\&\quad +\int _{\mathbb {R}^{N}}V(\varepsilon x) |u|^{p-2}u\,\varphi \, dx+ \int _{\mathbb {R}^{N}} V(\varepsilon x)|u|^{q-2}u\,\varphi \, dx \\&\quad - \int _{\mathbb {R}^{N}} g(\varepsilon x, u)\varphi \, dx \end{aligned}$$

for any \(u, \varphi \in \mathbb {X}_{\varepsilon }\). We denote by \(\mathcal {N}_{\varepsilon }\) the Nehari manifold associated with \(\mathcal {J}_{\varepsilon }\), namely

$$\begin{aligned} \mathcal {N}_{\varepsilon }=\{u\in \mathbb {X}_{\varepsilon }: \langle \mathcal {J}'_{\varepsilon }(u), u\rangle =0\}, \end{aligned}$$

and we set

$$\begin{aligned} c_{\varepsilon }=\inf _{u\in \mathcal {N}_{\varepsilon }} \mathcal {J}_{\varepsilon }(u). \end{aligned}$$

Let \(\mathbb {X}_{\varepsilon }^{+}\) be the open set given by

$$\begin{aligned} \mathbb {X}_{\varepsilon }^{+}=\{u\in \mathbb {X}_{\varepsilon }: |\mathrm{supp}(u^{+})\cap \Lambda _{\varepsilon }|>0\}, \end{aligned}$$

and \(\mathbb {S}_{\varepsilon }^{+}=\mathbb {S}_{\varepsilon }\cap \mathbb {X}_{\varepsilon }^{+}\), where \(\mathbb {S}_{\varepsilon }=\{u\in \mathbb {X}_{\varepsilon }: \Vert u\Vert _{\mathbb {X}_{\varepsilon }}=1\}\) is the unit sphere in \(\mathbb {X}_{\varepsilon }\). Note that \(\mathbb {S}_{\varepsilon }^{+}\) is an incomplete \(C^{1,1}\)-manifold of codimension one. Hence, \(\mathbb {X}_{\varepsilon }=T_{u}\mathbb {S}_{\varepsilon }^{+}\oplus \mathbb {R}u\) for all \(u\in \mathbb {S}_{\varepsilon }^{+}\), where

$$\begin{aligned} T_{u}\mathbb {S}_{\varepsilon }^{+}&=\left\{ v\in \mathbb {X}_{\varepsilon }: (1+[u]_{s, p}^{p}) \langle u, v\rangle _{s, p}+(1+[u]_{s, q}^{q})\langle u, v\rangle _{s, q} \right. \\&\quad \left. +\int _{\mathbb {R}^{N}} V(\varepsilon x)(|u|^{p-2}uv+|u|^{q-2}uv) \, dx=0 \right\} . \end{aligned}$$

The next lemma ensures that \(\mathcal {J}_{\varepsilon }\) possesses a mountain pass geometry [3].

Lemma 2.3

The functional \(\mathcal {J}_{\varepsilon }\) satisfies the following properties:

(i):

There exist \(\alpha , \rho >0\) such that \(\mathcal {J}_{\varepsilon }(u) \ge \alpha \) for any \(u\in \mathbb {X}_{\varepsilon }\) with \(\Vert u\Vert _{\mathbb {X}_{\varepsilon }}= \rho \).

(ii):

There exists \(e\in \mathbb {X}_{\varepsilon }\) such that \(\Vert e\Vert _{\mathbb {X}_{\varepsilon }}>\rho \) and \(\mathcal {J}_{\varepsilon }(e)<0\).

Proof

(i) Pick \(\zeta \in (0, V_{0})\). From \((g_{1})\), \((g_{2})\), \((f_{1})\), and \((f_{2})\), we can find \(C_{\zeta }>0\) such that

$$\begin{aligned} |g(x, t)|\le \zeta |t|^{p-1}+ C_{\zeta }|t|^{\nu -1} \quad \text{ for } (x,t)\in \mathbb {R}^{N}\times \mathbb {R}. \end{aligned}$$

Taking into account the above estimate and applying Theorem 2.1, we have

$$\begin{aligned} \mathcal {J}_{\varepsilon }(u)&\ge \frac{1}{p}\Vert u\Vert _{V_{\varepsilon },p}^{p}+\frac{1}{q}\Vert u\Vert _{V_{\varepsilon },q}^{q} - \frac{\zeta }{p} |u|_{p}^{p}- \frac{C_{\zeta }}{\nu } |u|_{\nu }^{\nu } \\&\ge C_{1}\Vert u\Vert ^{p}_{V_{\varepsilon },p}+\frac{1}{q}\Vert u\Vert _{V_{\varepsilon },q}^{q}- \frac{C_{\zeta }}{\nu } |u|_{\nu }^{\nu }. \end{aligned}$$

Choosing \(\Vert u\Vert _{\mathbb {X}_{\varepsilon }}=\rho \in (0, 1)\) and recalling that \(1<p<q\), we get \(\Vert u\Vert _{V_{\varepsilon },p}<1\) and thus \(\Vert u\Vert ^{p}_{V_{\varepsilon },p}\ge \Vert u\Vert ^{q}_{V_{\varepsilon },p}\). Using

$$\begin{aligned} a^{t}+b^{t}\ge C_{t} (a+b)^{t} \quad \text{ for } \text{ all } a, b\ge 0 \text{ and } t>1, \end{aligned}$$

and Theorem 2.1, we can see that

$$\begin{aligned} \mathcal {J}_{\varepsilon }(u)\ge C_{2} \Vert u\Vert ^{q}_{\mathbb {X}_{\varepsilon }}-\frac{C_{\zeta }}{\nu } |u|_{\nu }^{\nu }\ge C_{2} \Vert u\Vert ^{q}_{\mathbb {X}_{\varepsilon }}-C_{3} \Vert u\Vert _{\mathbb {X}_{\varepsilon }}^{\nu }. \end{aligned}$$

Since \(\nu >q\), there exists \(\alpha >0\) such that \(\mathcal {J}_{\varepsilon }(u)\ge \alpha \) for any \(u\in \mathbb {X}_{\varepsilon }\) with \(\Vert u\Vert _{\mathbb {X}_{\varepsilon }}= \rho \).

(ii) It follows from \((f_{3})\) that, for some constants \(A, B>0\),

$$\begin{aligned} F(t)\ge A t^{\vartheta }-B \quad \text{ for } \text{ all } t>0. \end{aligned}$$

Then, for all \(u\in \mathbb {X}^{+}_{\varepsilon }\) and \(t>0\), we obtain

$$\begin{aligned}&\mathcal {J}_{\varepsilon }(t u) \le \frac{t^{p}}{p} \Vert u\Vert _{V_{\varepsilon },p}^{p} +\frac{t^{2p}}{2p}[u]_{s, p}^{2p}+\frac{t^{q}}{q} \Vert u\Vert _{V_{\varepsilon },q}^{q}+\frac{t^{2q}}{2q}[u]_{s, q}^{2q} \\&\qquad \qquad - A t^{\vartheta } \int _{\Lambda _{\varepsilon }} (u^{+})^{\vartheta } \, dx +B|\mathrm{supp}(u^{+})\cap \Lambda _{\varepsilon }| \end{aligned}$$

which combined with the fact that \(\vartheta>2q>2p\) implies that \(\mathcal {J}_{\varepsilon }(tu)\rightarrow -\infty \) as \(t\rightarrow \infty \). Hence, for large \(t>1\), we can take \(e=tu\) such that \(\Vert e\Vert _{\mathbb {X}_{\varepsilon }}>\rho \) and \(\mathcal {J}_{\varepsilon }(e)<0\). \(\square \)

In view of Lemma 2.3, we can define the minimax level

$$\begin{aligned} c'_{\varepsilon }=\inf _{\gamma \in \Gamma _{\varepsilon }} \max _{t\in [0, 1]} \mathcal {J}_{\varepsilon }(\gamma (t)) \quad \text{ where }&\quad \Gamma _{\varepsilon }=\{\gamma \in C([0, 1], \mathbb {X}_{\varepsilon }): \gamma (0)=0&\\ \text{ and } \mathcal {J}_{\varepsilon }(\gamma (1))<0\}. \end{aligned}$$

Exploiting a version of the mountain pass theorem without the Palais–Smale condition (see [37]), we can find a Palais–Smale sequence \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {X}_{\varepsilon }\) at the level \(c'_{\varepsilon }\) (\((PS)_{c'_{\varepsilon }}\) sequence for short).

Remark 2.1

We may always assume that any \((PS)_{c}\) sequence \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {X}_{\varepsilon }\) of \(\mathcal {J}_{\varepsilon }\) is nonnegative. Indeed, noting that \(\langle \mathcal {J}'_{\varepsilon }(u_{n}), u^{-}_{n}\rangle =o_{n}(1)\), where \(u_{n}^{-}=\min \{u_{n}, 0\}\), and using \(g(\varepsilon \cdot , t)=0\) for \(t\le 0\), we have

$$\begin{aligned}&(1+[u_{n}]_{s, p}^{p})\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p-2}(u_{n}(x)-u_{n}(y))}{|x-y|^{N+sp}} (u^{-}_{n}(x)-u^{-}_{n}(y))\,dxdy \\&\quad +(1+[u_{n}]_{s, q}^{q})\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q-2}(u_{n}(x)-u_{n}(y))}{|x-y|^{N+sq}} (u^{-}_{n}(x)-u^{-}_{n}(y))\,dxdy \\&\quad + \int _{\mathbb {R}^{N}} V(\varepsilon x) (|u_{n}|^{p-2} u_{n}\, +|u_{n}|^{q-2} u_{n})\, u_{n}^{-} \,dx=o_{n}(1). \end{aligned}$$

Recalling that

$$\begin{aligned} |x-y|^{t-2}(x-y)(x^{-}- y^{-})\ge |x^{-}- y^{-}|^{t} \quad {{ \text{ for } \text{ all } x,y\in \mathbb {R}\, \text{ and } t>1,}} \end{aligned}$$
(2.2)

we arrive at

$$\begin{aligned} \Vert u_{n}^{-}\Vert _{V_{\varepsilon }, p}^{p}+\Vert u_{n}^{-}\Vert _{V_{\varepsilon }, q}^{q}= o_{n}(1), \end{aligned}$$

that is \(u_{n}^{-}\rightarrow 0\) in \(\mathbb {X}_{\varepsilon }\). Moreover, \(\{u_{n}^{+}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {X}_{\varepsilon }\). Since \([u_{n}]_{s, t}^{t}=[u^{+}_{n}]_{s, t}^{t}+o_{n}(1)\) and \(\Vert u_{n}\Vert _{V_{\varepsilon }, t}=\Vert u_{n}^{+}\Vert _{V_{\varepsilon }, t}+o_{n}(1)\) for \(t\in \{p, q\}\), we can easily deduce that \(\mathcal {J}_{\varepsilon }(u_{n})=\mathcal {J}_{\varepsilon }(u^{+}_{n})+o_{n}(1)\) and \(\mathcal {J}'_{\varepsilon }(u_{n})=\mathcal {J}'_{\varepsilon }(u^{+}_{n})+o_{n}(1)\). Therefore, \(\mathcal {J}_{\varepsilon }(u^{+}_{n})\rightarrow c\) and \(\mathcal {J}'_{\varepsilon }(u^{+}_{n})\rightarrow 0\).

The next two results are very important because they allow us to overcome the nondifferentiability of \(\mathcal {N}_{\varepsilon }\) and the incompleteness of \(\mathbb {S}_{\varepsilon }^{+}\).

Lemma 2.4

Assume that \((V_1)\)-\((V_2)\) and \((f_1)\)-\((f_4)\) hold. Then we have the following properties:

(i):

For each \(u\in \mathbb {X}_{\varepsilon }^{+}\), let \(h_{u}:\mathbb {R}^{+}\rightarrow \mathbb {R}\) be defined by \(h_{u}(t)= \mathcal {J}_{\varepsilon }(tu)\). Then, there is a unique \(t_{u}>0\) such that

$$\begin{aligned}&h'_{u}(t)>0 \, \text{ for } \text{ all } t\in (0, t_{u}),\\&h'_{u}(t)<0 \, \text{ for } \text{ all } t\in (t_{u}, \infty ). \end{aligned}$$
(ii):

There exists \(\tau >0\), independent of u, such that \(t_{u}\ge \tau \) for any \(u\in \mathbb {S}_{\varepsilon }^{+}\). Moreover, for each compact set \(\mathbb {K}\subset \mathbb {S}_{\varepsilon }^{+}\), there is a constant \(C_{\mathbb {K}}>0\) such that \(t_{u}\le C_{\mathbb {K}}\) for any \(u\in \mathbb {K}\).

(iii):

The map \({\hat{m}}_{\varepsilon }: \mathbb {X}_{\varepsilon }^{+}\rightarrow \mathcal {N}_{\varepsilon }\) given by \({\hat{m}}_{\varepsilon }(u)= t_{u}u\) is continuous and \(m_{\varepsilon }= {\hat{m}}_{\varepsilon }|_{\mathbb {S}_{\varepsilon }^{+}}\) is a homeomorphism between \(\mathbb {S}_{\varepsilon }^{+}\) and \(\mathcal {N}_{\varepsilon }\). Moreover, \(m_{\varepsilon }^{-1}(u)=\frac{u}{\Vert u\Vert _{\mathbb {X}_{\varepsilon }}}\).

(iv):

If there is a sequence \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {S}_{\varepsilon }^{+}\) such that \(\mathrm{dist}(u_{n}, \partial \mathbb {S}_{\varepsilon }^{+})\rightarrow 0\), then \(\Vert m_{\varepsilon }(u_{n})\Vert _{\mathbb {X}_{\varepsilon }}\rightarrow \infty \) and \(\mathcal {J}_{\varepsilon }(m_{\varepsilon }(u_{n}))\rightarrow \infty \).

Proof

(i) From the proof of Lemma 2.3, we derive that \(h_{u}(0)=0\), \(h_{u}(t)>0\) for \(t>0\) small enough and \(h_{u}(t)<0\) for \(t>0\) sufficiently large. Then there exists a global maximum point \(t_{u}>0\) for \(h_{u}\) in \([0, \infty )\) such that \(h_{u}'(t_{u})=0\) and \(t_{u}u \in \mathcal {N}_{\varepsilon }\). We claim that \(t_{u}>0\) is the unique number such that \(h'_{u}(t_{u})=0\). Arguing by contradiction, we assume that there exists \(t_{1}> t_{2}>0\) such that \(h_{u}'(t_{1})=h_{u}'(t_{2})=0\), or equivalently

$$\begin{aligned}&t_{1}^{p-1}\Vert u\Vert _{V_{\varepsilon },p}^{p}+t_{1}^{2p-1}[u]_{s, p}^{2p}+t_{1}^{q-1}\Vert u\Vert _{V_{\varepsilon },q}^{q}+t_{1}^{2q-1}[u]_{s,q}^{2q}= \int _{\mathbb {R}^{N}} g(\varepsilon x, t_{1}u) u\, dx, \\&t_{2}^{p-1}\Vert u\Vert _{V_{\varepsilon },p}^{p}+t_{2}^{2p-1}[u]_{s,p}^{2p}+t_{2}^{q-1} \Vert u\Vert _{V_{\varepsilon },q}^{q}+t_{2}^{2q-1}[u]_{s, q}^{2q}= \int _{\mathbb {R}^{N}} g(\varepsilon x, t_{2}u) u\, dx. \end{aligned}$$

Hence,

$$\begin{aligned} \frac{\Vert u\Vert ^{p}_{V_{\varepsilon },p}}{t_{1}^{2q-p}}+\frac{\Vert u\Vert _{V_{\varepsilon },q}^{q}}{t_{1}^{q}}+\frac{[u]_{s, p}^{2p}}{t_{1}^{2q-2p}}+[u]_{s, q}^{2q}=\int _{\mathbb {R}^{N}} \frac{g(\varepsilon x, t_{1}u)}{(t_{1}u)^{2q-1}} u^{2q} dx \end{aligned}$$

and

$$\begin{aligned} \frac{\Vert u\Vert ^{p}_{V_{\varepsilon },p}}{t_{2}^{2q-p}}+\frac{\Vert u\Vert _{V_{\varepsilon },q}^{q}}{t_{2}^{q}}+\frac{[u]_{s, p}^{2p}}{t_{2}^{2q-2p}}+[u]_{s, q}^{2q}=\int _{\mathbb {R}^{N}} \frac{g(\varepsilon x, t_{2}u)}{(t_{2}u)^{2q-1}} u^{2q} dx. \end{aligned}$$

Using the definition of g, \((g_4)\) and \((f_4)\), we have

$$\begin{aligned}&\left( \frac{1}{t_{1}^{2q-p}}-\frac{1}{t_{2}^{2q-p}}\right) \Vert u\Vert _{V_{\varepsilon },p}^{p}+\left( \frac{1}{t_{1}^{q}}-\frac{1}{t_{2}^{q}} \right) \Vert u\Vert _{V_{\varepsilon },q}^{q}+\left( \frac{1}{t_{1}^{2q-2p}}-\frac{1}{t_{2}^{2q-2p}} \right) [u]_{s, p}^{2p}\\&\quad =\int _{\mathbb {R}^{N}} \left[ \frac{g(\varepsilon x, t_{1}u)}{(t_{1}u)^{2q-1}} - \frac{g(\varepsilon x, t_{2}u)}{(t_{2}u)^{2q-1}}\right] u^{2q} dx\\&\quad \ge \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} \left[ \frac{g(\varepsilon x, t_{1}u)}{(t_{1}u)^{2q-1}} - \frac{g(\varepsilon x, t_{2}u)}{(t_{2}u)^{2q-1}}\right] u^{2q} dx \\&q\quad + \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} \left[ \frac{g(\varepsilon x, t_{1}u)}{(t_{1}u)^{2q-1}} - \frac{g(\varepsilon x, t_{2}u)}{(t_{2}u)^{2q-1}}\right] u^{2q} dx\\&\qquad +\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{1}u<a\}} \left[ \frac{g(\varepsilon x, t_{1}u)}{(t_{1}u)^{2q-1}} - \frac{g(\varepsilon x, t_{2}u)}{(t_{2}u)^{2q-1}}\right] u^{2q} dx \\&\quad \ge \frac{V_{0}}{K} \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} \left[ \left( \frac{1}{(t_{1}u)^{2q-p}}-\frac{1}{(t_{2}u)^{2q-p}}\right) +\left( \frac{1}{(t_{1}u)^{q}}-\frac{1}{(t_{2}u)^{q}}\right) \right] u^{2q} dx\\&\qquad +\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a <t_{1}u\}} \left[ \frac{V_{0}}{K}\left( \frac{1}{(t_{1}u)^{2q-p}}+\frac{1}{(t_{1}u)^{q}}\right) - \frac{f(t_{2}u)}{(t_{2}u)^{2q-1}}\right] u^{2q} dx. \end{aligned}$$

Multiplying both sides by \(\frac{(t_{1}t_{2})^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}<0\) (recall that \(2q>p\) and \(t_{1}>t_{2}\)), we get

$$\begin{aligned}&\Vert u\Vert _{V_{\varepsilon },p}^{p}+ \frac{(t_{1}t_{2})^{q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} (t_{2}^{q}-t_{1}^{q}) \Vert u\Vert _{V_{\varepsilon },q}^{q}\\&\quad =\Vert u\Vert _{V_{\varepsilon },p}^{p}+ \frac{(t_{1}t_{2})^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} \frac{t_{2}^{q}-t_{1}^{q}}{(t_{1}t_{2})^{q}} \Vert u\Vert _{V_{\varepsilon },q}^{q}\\&\quad \le \frac{V_{0}}{K} \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} u^{p} dx +\frac{V_{0}}{K} \frac{(t_{1}t_{2})^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} \frac{t_{2}^{q}-t_{1}^{q}}{(t_{2}t_{1})^{q}} \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} u^{q} dx\\&\quad \quad +\frac{(t_{1}t_{2})^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} \left[ \frac{V_{0}}{K}\left( \frac{1}{(t_{1}u)^{2q-p}}+\frac{1}{(t_{1}u)^{q}}\right) - \frac{f(t_{2}u)}{(t_{2}u)^{2q-1}}\right] u^{2q} dx \\&\quad \le \frac{V_{0}}{K} \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} u^{p} dx +\frac{V_{0}}{K} \frac{(t_{1}t_{2})^{q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} (t_{2}^{q}-t_{1}^{q}) \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} u^{q} dx \\&\quad \quad + \frac{V_{0}}{K} \frac{t_{2}^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} u^{p}\, dx+\frac{V_{0}}{K}\frac{t^{q-p}_{1}t_{2}^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} u^{q}\, dx\\&\quad \quad -\frac{(t_{1}t_{2})^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} \frac{f(t_{2}u)}{(t_{2}u)^{2q-1}} u^{2q} dx\\&\quad \le \frac{V_{0}}{K} \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} u^{p} dx +\frac{V_{0}}{K} \frac{(t_{1}t_{2})^{q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} (t_{2}^{q}-t_{1}^{q}) \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} u^{q} dx \\&\quad \quad + \frac{V_{0}}{K}\frac{t_{2}^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} u^{p}\, dx+\frac{V_{0}}{K}\frac{t^{q-p}_{1}t_{2}^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} u^{q}\, dx\\&\quad \quad -\frac{V_{0}}{K}\frac{t_{1}^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} u^{p}\, dx- \frac{V_{0}}{K}\frac{t_{1}^{2q-p}t_{2}^{q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} u^{q}\, dx\\&\quad =\frac{V_{0}}{K} \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} u^{p} dx +\frac{V_{0}}{K} \frac{(t_{1}t_{2})^{q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} (t_{2}^{q}-t_{1}^{q}) \int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u>a\}} u^{q} dx \\&\quad \quad + \frac{V_{0}}{K}\frac{t_{2}^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} u^{p}\, dx+\frac{V_{0}}{K}\frac{t^{q-p}_{1}t_{2}^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} u^{q}\, dx\\&\quad \quad -\frac{V_{0}}{K}\frac{t_{1}^{2q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a<t_{1}u\}} u^{p}\, dx- \frac{V_{0}}{K}\frac{t_{1}^{2q-p}t_{2}^{q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}}\int _{\Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a <t_{1}u\}} u^{q}\, dx\\&\quad \le \frac{V_{0}}{K} \int _{\Lambda ^{c}_{\varepsilon }} u^{p}\, dx+\frac{V_{0}}{K} \frac{(t_{1}t_{2})^{q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} (t_{2}^{q}-t_{1}^{q}) \int _{\Lambda _{\varepsilon }^{c}} u^{q} dx\\&\quad \le \frac{1}{K}\Vert u\Vert ^{p}_{V_{\varepsilon },p}+\frac{1}{K} \frac{(t_{1}t_{2})^{q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} (t_{2}^{q}-t_{1}^{q}) \Vert u\Vert _{V_{\varepsilon },q}^{q}, \end{aligned}$$

where we used the fact that \((f_4)\) and our choice of the constant a produce

$$\begin{aligned} \frac{f(t_{2}u)}{(t_{2}u)^{2q-1}}&=\frac{f(t_{2}u)}{(t_{2}u)^{p-1}+(t_{2}u)^{q-1}} \, \frac{(t_{2}u)^{p-1}+(t_{2}u)^{q-1}}{(t_{2}u)^{2q-1}}\\&\le \frac{f(a)}{a^{p-1}+a^{q-1}} \frac{(t_{2}u)^{p-1}+(t_{2}u)^{q-1}}{(t_{2}u)^{2q-1}}\\&= \frac{V_{0}}{K}\left( \frac{1}{(t_{2}u)^{2q-p}}+\frac{1}{(t_{2}u)^{q}}\right) \quad \text{ in } \Lambda _{\varepsilon }^{c}\cap \{t_{2}u\le a <t_{1}u\}. \end{aligned}$$

Therefore,

$$\begin{aligned} \left( 1-\frac{1}{K}\right) \left[ \Vert u\Vert _{V_{\varepsilon },p}^{p}+\frac{(t_{1}t_{2})^{q-p}}{t_{2}^{2q-p}- t_{1}^{2q-p}} (t_{2}^{q}-t_{1}^{q}) \Vert u\Vert _{V_{\varepsilon },q}^{q}\right] \le 0, \end{aligned}$$

which is inconsistent with \(u\ne 0\) and \(K>1\).

(ii) Fix \(u\in \mathbb {S}_{\varepsilon }^{+}\). By (i), there exists \(t_{u}>0\) such that \(h_{u}'(t_{u})=0\), that is

$$\begin{aligned} t_{u}^{p-1} \Vert u\Vert _{V_{\varepsilon },p}^{p}+t_{u}^{q-1} \Vert u\Vert _{V_{\varepsilon },q}^{q}+t_{u}^{2p-1}[u]_{s, p}^{2p}+t_{u}^{2q-1}[u]_{s, q}^{2q}= \int _{\mathbb {R}^{N}} g(\varepsilon x, t_{u}u) \,u \,dx. \end{aligned}$$

Pick \(\xi >0\). From \((g_1)\)-\((g_2)\) and Theorem 2.1, we derive

$$\begin{aligned} t_{u}^{p-1} \Vert u\Vert _{V_{\varepsilon },p}^{p}+t_{u}^{q-1} \Vert u\Vert _{V_{\varepsilon },q}^{q}&\le \int _{\mathbb {R}^{3}} g(\varepsilon x, t_{u}u)\, u\, dx\le \xi t_{u}^{p-1}\Vert u\Vert ^{p}_{V_{\varepsilon },p} \\&\quad + C_{\xi } t_{u}^{\nu -1}\Vert u\Vert _{V_{\varepsilon },q}^{\nu }. \end{aligned}$$

Choosing \(\xi >0\) sufficiently small, we have

$$\begin{aligned} Ct^{p-1}_{u}\Vert u\Vert _{V_{\varepsilon },p}^{p}+t_{u}^{q-1} \Vert u\Vert _{V_{\varepsilon },q}^{q}\le Ct_{u}^{\nu -1}\Vert u\Vert _{V_{\varepsilon },q}^{\nu }\le Ct_{u}^{\nu -1}. \end{aligned}$$

Now, if \(t_{u}\le 1\), then \(t_{u}^{q-1}\le t_{u}^{p-1}\), and using the facts that \(1=\Vert u\Vert _{\mathbb {X}_{\varepsilon }}\ge \Vert u\Vert _{V_{\varepsilon },p}\) and that \(q>p\) imply that \(\Vert u\Vert _{V_{\varepsilon },p}^{p}\ge \Vert u\Vert ^{q}_{V_{\varepsilon },p}\), we get

$$\begin{aligned}&Ct_{u}^{q-1} =Ct_{u}^{q-1} \Vert u\Vert ^{q}_{\mathbb {X}_{\varepsilon }} \le t_{u}^{q-1} (C\Vert u\Vert _{V_{\varepsilon },p}^{q}+\Vert u\Vert _{V_{\varepsilon },q}^{q})\le t_{u}^{q-1}(C\Vert u\Vert _{V_{\varepsilon },p}^{p}+\Vert u\Vert _{V_{\varepsilon },q}^{q})\\&\qquad \le Ct_{u}^{\nu -1}. \end{aligned}$$

Since \(\nu >q\), there exists \(\tau >0\), independent of u, such that \(t_{u}\ge \tau \).

When \(t_{u}>1\), then \(t_{u}^{q-1}>t_{u}^{p-1}\), and observing that \(1=\Vert u\Vert _{\mathbb {X}_{\varepsilon }}\ge \Vert u\Vert _{V_{\varepsilon },p}\) and that \(q>p\) yield \(\Vert u\Vert _{V_{\varepsilon },p}^{p}\ge \Vert u\Vert ^{q}_{V_{\varepsilon },p}\), we obtain

$$\begin{aligned}&Ct_{u}^{p-1} =Ct_{u}^{p-1} \Vert u\Vert ^{q}_{\mathbb {X}_{\varepsilon }} \le t_{u}^{p-1} (C\Vert u\Vert _{V_{\varepsilon },p}^{q}+\Vert u\Vert _{V_{\varepsilon },q}^{q})\le t_{u}^{p-1}(C\Vert u\Vert _{V_{\varepsilon },p}^{p}+\Vert u\Vert _{V_{\varepsilon },q}^{q})\\&\qquad \le Ct_{u}^{\nu -1}. \end{aligned}$$

As \(\nu>q>p\), we can find \(\tau >0\), independent of u, such that \(t_{u}\ge \tau \).

Now, let \(\mathbb {K}\subset \mathbb {S}_{\varepsilon }^{+}\) be a compact set, and suppose, by contradiction, that there exists \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {K}\) such that \(t_{n}=t_{u_{n}}\rightarrow \infty \). Since \(\mathbb {K}\) is compact, there is \(u\in \mathbb {K}\) such that \(u_{n}\rightarrow u\) in \(\mathbb {X}_{\varepsilon }\). By the proof of (ii) of Lemma 2.3, we see that

$$\begin{aligned} \mathcal {J}_{\varepsilon }(t_{n}u_{n})\rightarrow -\infty . \end{aligned}$$
(2.3)

On the other hand, if \(v\in \mathcal {N}_{\varepsilon }\), by \(\langle \mathcal {J}_{\varepsilon }'(v), v \rangle =0\) and \((g_{3})\), we get

$$\begin{aligned} \mathcal {J}_{\varepsilon }(v)&= \mathcal {J}_{\varepsilon }(v)- \frac{1}{\vartheta } \langle \mathcal {J}_{\varepsilon }'(v), v \rangle \ge {\tilde{C}}( \Vert v\Vert _{V_{\varepsilon },p}^{p}+\Vert v\Vert _{V_{\varepsilon },q}^{q}). \end{aligned}$$

Taking \(v_{n}=t_{u_{n}}u_{n}\in \mathcal {N}_{\varepsilon }\) in the above inequality, we arrive at

$$\begin{aligned} \mathcal {J}_{\varepsilon }(t_{n}u_{n})\ge {\tilde{C}} (\Vert v_{n}\Vert _{V_{\varepsilon },p}^{p}+\Vert v_{n}\Vert _{V_{\varepsilon },q}^{q}). \end{aligned}$$

Since \(\Vert v_{n}\Vert _{\mathbb {X}_{\varepsilon }}=t_{n}\rightarrow \infty \) and \(\Vert v_{n}\Vert _{\mathbb {X}_{\varepsilon }}=\Vert v_{n}\Vert _{V_{\varepsilon },p}+\Vert v_{n}\Vert _{V_{\varepsilon },q}\), we can use (2.3) to reach a contradiction.

(iii) First we note that \({\hat{m}}_{\varepsilon }\), \(m_{\varepsilon }\) and \(m_{\varepsilon }^{-1}\) are well defined. Indeed, by (i), for each \(u\in \mathbb {X}_{\varepsilon }^{+}\), there is a unique \(m_{\varepsilon }(u)\in \mathcal {N}_{\varepsilon }\). On the other hand, if \(u\in \mathcal {N}_{\varepsilon }\) then \(u\in \mathbb {X}^{+}_{\varepsilon }\). Otherwise, we would have

$$\begin{aligned} |{{\,\mathrm{supp}\,}}(u^{+}) \cap \Lambda _{\varepsilon }|=0, \end{aligned}$$

and by \((g_3)\)-(ii) we infer that

$$\begin{aligned} \Vert u\Vert _{V_{\varepsilon },p}^{p}+\Vert u\Vert ^{q}_{V_{\varepsilon },q}&\le \int _{\mathbb {R}^{N}} g(\varepsilon x, u)\, u \, dx= \int _{\Lambda _{\varepsilon }^{c}} g(\varepsilon x, u)\, u \, dx + \int _{\Lambda _{\varepsilon }} g(\varepsilon x, u)\, u \, dx \\&= \int _{\Lambda _{\varepsilon }^{c}} g(\varepsilon x, u^{+})\, u^{+} \, dx \\&\le \frac{1}{K} \int _{\Lambda _{\varepsilon }^{c}} V(\varepsilon x) (|u|^{p}+|u|^{q}) dx \\&\le \frac{1}{K} (\Vert u\Vert _{V_{\varepsilon },p}^{p}+\Vert u\Vert ^{q}_{V_{\varepsilon },q}) \end{aligned}$$

which gives a contradiction because \(K>1\) and \(u\ne 0\). Consequently, \(m_{\varepsilon }^{-1}(u)= \frac{u}{\Vert u\Vert _{\mathbb {X}_{\varepsilon }}}\in \mathbb {S}_{\varepsilon }^{+}\) is well defined and continuous. Since

$$\begin{aligned} m_{\varepsilon }^{-1}(m_{\varepsilon }(u))= m_{\varepsilon }^{-1}(t_{u}u)= \frac{t_{u}u}{\Vert t_{u}u\Vert _{\mathbb {X}_{\varepsilon }}}= \frac{u}{\Vert u\Vert _{\mathbb {X}_{\varepsilon }}}=u \quad \text{ for } \text{ all } u\in \mathbb {S}_{\varepsilon }^{+}, \end{aligned}$$

we deduce that \(m_{\varepsilon }\) is a bijection. Now we prove that \({\hat{m}}_{\varepsilon }: \mathbb {X}_{\varepsilon }^{+}\rightarrow \mathcal {N}_{\varepsilon }\) is continuous. Let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {X}_{\varepsilon }^{+}\) and \(u\in \mathbb {X}^{+}_{\varepsilon }\) be such that \(u_{n}\rightarrow u\) in \(\mathbb {X}_{\varepsilon }\). By (ii), there exists \(t_{0}>0\) such that \(t_{n}=t_{\frac{u_{n}}{\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }}}}\rightarrow t_{0}\). Using \(t_{n}\frac{u_{n}}{\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }}}\in \mathcal {N}_{\varepsilon }\), that is

$$\begin{aligned} t_{n}^{p}&\frac{\Vert u_{n}\Vert ^{p}_{V_{\varepsilon },p}}{\Vert u_{n}\Vert ^{p}_{\mathbb {X}_{\varepsilon }}}+t_{n}^{q} \frac{\Vert u_{n}\Vert _{V_{\varepsilon },q}^{q}}{\Vert u_{n}\Vert ^{q}_{\mathbb {X}_{\varepsilon }}}+t^{2p}_{n}\frac{[u_{n}]_{s, p}^{2p}}{\Vert u_{n}\Vert ^{2p}_{\mathbb {X}_{\varepsilon }}}+t^{2q}_{n} \frac{[u_{n}]_{s, q}^{2q}}{\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }}^{2q}}= \int _{\mathbb {R}^{N}} g\left( \varepsilon x, t_{n} \frac{u_{n}}{\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }}}\right) \, \\&\qquad t_{n} \frac{u_{n}}{\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }}} \, dx, \end{aligned}$$

and letting \(n\rightarrow \infty \) we find

$$\begin{aligned}&t_{0}^{p} \frac{\Vert u\Vert ^{p}_{V_{\varepsilon },p}}{\Vert u\Vert ^{p}_{\mathbb {X}_{\varepsilon }}}+t_{0}^{q} \frac{\Vert u\Vert _{V_{\varepsilon },q}^{q}}{\Vert u\Vert ^{q}_{\mathbb {X}_{\varepsilon }}}+t^{2p}_{0}\frac{[u]_{s, p}^{2p}}{\Vert u\Vert ^{2p}_{\mathbb {X}_{\varepsilon }}}+t^{2q}_{0} \frac{[u]_{s, q}^{2q}}{\Vert u\Vert _{\mathbb {X}_{\varepsilon }}^{2q}}= \int _{\mathbb {R}^{N}} g\left( \varepsilon x, t_{0} \frac{u}{\Vert u\Vert _{\mathbb {X}_{\varepsilon }}}\right) \,\\&\qquad t_{0} \frac{u}{\Vert u\Vert _{\mathbb {X}_{\varepsilon }}} \, dx, \end{aligned}$$

which implies that \(t_{0}\frac{u}{\Vert u\Vert _{\mathbb {X}_{\varepsilon }}}\in \mathcal {N}_{\varepsilon }\). From (i), \(t_{\frac{u}{\Vert u\Vert _{\mathbb {X}_{\varepsilon }}}}= t_{0}\) and this assures that \({\hat{m}}_{\varepsilon }(u_{n})\rightarrow {\hat{m}}_{\varepsilon }(u)\) in \(\mathbb {X}_{\varepsilon }^{+}\). Therefore, \({\hat{m}}_{\varepsilon }\) and \(m_{\varepsilon }\) are continuous functions.

(iv) Let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {S}_{\varepsilon }^{+}\) be a sequence such that \(\mathrm{dist}(u_{n}, \partial \mathbb {S}_{\varepsilon }^{+})\rightarrow 0\). Then, for each \(v\in \partial \mathbb {S}_{\varepsilon }^{+}\) and \(n\in \mathbb {N}\), we have \(u_{n}^{+}\le |u_{n}-v|\) a.e. in \(\Lambda _{\varepsilon }\). Hence, by \((V_1)\), \((V_2)\) and Theorem 2.1, we can see that for each \(r\in [p, q^{*}_{s}]\), there exists \(C_{r}>0\) such that

$$\begin{aligned} |u_{n}^{+}|_{L^{r}(\Lambda _{\varepsilon })}&\le \inf _{v\in \partial \mathbb {S}_{\varepsilon }^{+}} |u_{n}- v|_{L^{r}(\Lambda _{\varepsilon })}\\&\le C_{r} \inf _{v\in \partial \mathbb {S}_{\varepsilon }^{+}} \Vert u_{n}- v\Vert _{\mathbb {X}_{\varepsilon }} \quad \text{ for } \text{ all } n\in \mathbb {N}. \end{aligned}$$

Combining \((g_{1})\), \((g_{2})\), \((g_{3})\)-(ii) and \(q>p\), we get, for all \(t>0\),

$$\begin{aligned} \int _{\mathbb {R}^{N}} G(\varepsilon x, tu_{n})\, dx&= \int _{\Lambda _{\varepsilon }^{c}} G(\varepsilon x, tu_{n})\, dx + \int _{\Lambda _{\varepsilon }} G(\varepsilon x, tu_{n})\, dx\\&\le \frac{V_{0}}{Kp} \int _{\Lambda _{\varepsilon }^{c}} (t^{p}|u_{n}|^{p}+t^{q}|u_{n}|^{q}) dx+ \int _{\Lambda _{\varepsilon }} F(tu_{n})\, dx\\&\le \frac{t^{p}}{Kp} \int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{p}\, dx+\frac{t^{q}}{Kp} \int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{q}\, dx\\&\quad + C_{1}t^{p} \int _{\Lambda _{\varepsilon }} (u_{n}^{+})^{p} dx+ C_{2} t^{\nu } \int _{\Lambda _{\varepsilon }} (u_{n}^{+})^{\nu } dx\\&\le \frac{t^{p}}{Kp} \int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{p}\, dx+\frac{t^{q}}{Kp} \int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{q}\, dx \\&\quad + C_{p}' t^{p} \mathrm{dist}(u_{n}, \partial \mathbb {S}_{\varepsilon }^{+})^{p} + C_{\nu }' t^{\nu }\mathrm{dist}(u_{n}, \partial \mathbb {S}_{\varepsilon }^{+})^{\nu }. \end{aligned}$$

Thus, for all \(t>0\),

$$\begin{aligned} \int _{\mathbb {R}^{N}} G(\varepsilon x, tu_{n})\, dx \le \frac{t^{p}}{Kp} \int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{p}\, dx+\frac{t^q}{Kp} \int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{q}\, dx+o_{n}(1). \end{aligned}$$
(2.4)

Now, we recall that \(K>\frac{q}{p}>1\), and that \(1=\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }}\ge \Vert u_{n}\Vert _{V_{\varepsilon },p}\) implies \(\Vert u_{n}\Vert ^{p}_{V_{\varepsilon },p}\ge \Vert u_{n}\Vert ^{q}_{V_{\varepsilon },p}\). Then, for all \(t>1\), we obtain

$$\begin{aligned} \begin{aligned}&\frac{t^{p}}{p} \Vert u_{n}\Vert ^{p}_{V_{\varepsilon },p}+\frac{t^{q}}{q} \Vert u_{n}\Vert ^{q}_{V_{\varepsilon },q}-\frac{t^{p}}{Kp} \int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{p}\, dx-\frac{t^{q}}{Kp} \int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{q}\, dx \\&= \frac{t^{p}}{p} [u_{n}]^{p}_{s, p}+t^{p} \left( \frac{1}{p}-\frac{1}{Kp}\right) \int _{\mathbb {R}^{N}} V(\varepsilon x) |u_{n}|^{p}\, dx \\&\quad +\frac{t^{q}}{q} [u_{n}]^{q}_{s, q}+t^{q} \left( \frac{1}{q}-\frac{1}{Kp}\right) \int _{\mathbb {R}^{N}} V(\varepsilon x) |u_{n}|^{q}\, dx \\&\ge C_{1} t^{p} \Vert u_{n}\Vert _{V_{\varepsilon },p}^{p}+C_{2} t^{q} \Vert u_{n}\Vert _{V_{\varepsilon },q}^{q} \\&\ge C_{1} t^{p} \Vert u_{n}\Vert _{V_{\varepsilon },p}^{q}+C_{2} t^{q} \Vert u_{n}\Vert _{V_{\varepsilon },q}^{q} \\&\ge C_{1} t^{p} \Vert u_{n}\Vert _{V_{\varepsilon },p}^{q}+C_{2} t^{p} \Vert u_{n}\Vert _{V_{\varepsilon },q}^{q} \\&\ge C_{3} t^{p} (\Vert u_{n}\Vert _{V_{\varepsilon },p}+\Vert u_{n}\Vert _{V_{\varepsilon },q})^{q}=C_{3}t^{p}. \end{aligned} \end{aligned}$$
(2.5)

By using the definition of \(m_{\varepsilon }(u_{n})\), (2.4) and (2.5), we have

$$\begin{aligned} \liminf _{n\rightarrow \infty } \mathcal {J}_{\varepsilon }(m_{\varepsilon }(u_{n}))&\ge \liminf _{n\rightarrow \infty } \mathcal {J}_{\varepsilon }(t u_{n})\\&\ge \liminf _{n\rightarrow \infty }\left[ \frac{t^{p}}{p} \Vert u_{n}\Vert ^{p}_{V_{\varepsilon },p}+\frac{t^{q}}{q} \Vert u_{n}\Vert ^{q}_{V_{\varepsilon },q} \right. \\&\quad \left. -\int _{\mathbb {R}^{N}} G(\varepsilon x, tu_{n})\, dx\right] \ge C_{3} t^{p} \quad {{ \text{ for } \text{ all } t>1.}} \end{aligned}$$

Letting \(t\rightarrow \infty \) we deduce that \(\mathcal {J}_{\varepsilon }(m_{\varepsilon }(u_{n}))\rightarrow \infty \) as \(n\rightarrow \infty \). Furthermore, by the definition of \(\mathcal {J}_{\varepsilon }\), we can see that for all \(n\in \mathbb {N}\)

$$\begin{aligned}&\frac{1}{p}\Vert m_{\varepsilon }(u_{n})\Vert _{V_{\varepsilon },p}^{p}(1+\Vert m_{\varepsilon }(u_{n})\Vert _{V_{\varepsilon },p}^{p})+\frac{1}{q}\Vert m_{\varepsilon }(u_{n})\Vert _{V_{\varepsilon },q}^{q}(1+\Vert m_{\varepsilon }(u_{n})\Vert ^{q}_{V_{\varepsilon , q}})\\&\quad \ge \frac{1}{p}\Vert m_{\varepsilon }(u_{n})\Vert _{V_{\varepsilon },p}^{p}+\frac{1}{2p}[m_{\varepsilon }(u_{n})]_{s, p}^{2p}+\frac{1}{q} \Vert m_{\varepsilon }(u_{n})\Vert _{V_{\varepsilon },q}^{q}+\frac{1}{2q}[m_{\varepsilon }(u_{n})]_{s, q}^{2q} \\&\quad \ge \mathcal {J}_{\varepsilon }(m_{\varepsilon }(u_{n})) \end{aligned}$$

and this yields \(\Vert m_{\varepsilon }(u_{n})\Vert _{\mathbb {X}_{\varepsilon }}\rightarrow \infty \) as \(n\rightarrow \infty \). \(\square \)

Remark 2.2

There exists \(\kappa >0\), independent of \(\varepsilon \), such that \(\Vert u\Vert _{\mathbb {X}_{\varepsilon }}\ge \kappa \) for all \(u\in \mathcal {N}_{\varepsilon }\). Indeed, if \(u\in \mathcal {N}_{\varepsilon }\), we can use \((g_1)\), \((g_2)\) and Theorem 2.1 to see that

$$\begin{aligned} \Vert u\Vert _{V_{\varepsilon },p}^{p}+\Vert u\Vert ^{q}_{V_{\varepsilon },q}&\le \int _{\mathbb {R}^{N}} g(\varepsilon x, u)\, u \, dx\\&\le \zeta |u|_{p}^{p}+C_{\zeta }|u|_{q^{*}_{s}}^{q^{*}_{s}}\\&\le \frac{\zeta }{V_{0}}\Vert u\Vert _{V_{\varepsilon },p}^{p}+C'_{\zeta } \Vert u\Vert ^{q^{*}_{s}}_{V_{\varepsilon },q}. \end{aligned}$$

Choosing \(\zeta \in (0, V_{0})\), we get \(\Vert u\Vert _{V_{\varepsilon },q}\ge \kappa =(C'_{\zeta })^{-\frac{1}{q^{*}_{s}-q}}\) and thus \(\Vert u\Vert _{\mathbb {X}_{\varepsilon }}\ge \Vert u\Vert _{V_{\varepsilon },q}\ge \kappa \).

Now we define the maps

$$\begin{aligned} {\hat{\psi }}_{\varepsilon }: \mathbb {X}_{\varepsilon }^{+} \rightarrow \mathbb {R}\quad \text{ and } \quad \psi _{\varepsilon }: \mathbb {S}_{\varepsilon }^{+}\rightarrow \mathbb {R}, \end{aligned}$$

by setting \({\hat{\psi }}_{\varepsilon }(u)= \mathcal {J}_{\varepsilon }({\hat{m}}_{\varepsilon }(u))\) and \(\psi _{\varepsilon }={\hat{\psi }}_{\varepsilon }|_{\mathbb {S}_{\varepsilon }^{+}}\). From Lemma 2.4 and arguing as in the proofs of Proposition 9 and Corollary 10 in [36], we may obtain the result below.

Proposition 2.1

Assume that \((V_{1})\)-\((V_{2})\) and \((f_{1})\)-\((f_{4})\) hold. Then we have the following properties:

(a):

\({\hat{\psi }}_{\varepsilon } \in C^{1}(\mathbb {X}_{\varepsilon }^{+}, \mathbb {R})\) and

$$\begin{aligned} \langle {\hat{\psi }}_{\varepsilon }'(u), v\rangle = \frac{\Vert {\hat{m}}_{\varepsilon }(u)\Vert _{\mathbb {X}_{\varepsilon }}}{\Vert u\Vert _{\mathbb {X}_{\varepsilon }}} \langle \mathcal {J}_{\varepsilon }'({\hat{m}}_{\varepsilon }(u)), v\rangle \quad {{ \text{ for } \text{ all } u\in \mathbb {X}_{\varepsilon }^{+} \text{ and } v\in \mathbb {X}_{\varepsilon }.}} \end{aligned}$$
(b):

\(\psi _{\varepsilon } \in C^{1}(\mathbb {S}_{\varepsilon }^{+}, \mathbb {R})\) and

$$\begin{aligned} \langle \psi _{\varepsilon }'(u), v \rangle = \Vert m_{\varepsilon }(u)\Vert _{\mathbb {X}_{\varepsilon }} \langle \mathcal {J}_{\varepsilon }'(m_{\varepsilon }(u)), v\rangle \quad {{ \text{ for } \text{ all } v\in T_{u}\mathbb {S}_{\varepsilon }^{+}.}} \end{aligned}$$
(c):

If \(\{u_{n}\}_{n\in \mathbb {N}}\) is a \((PS)_{c}\) sequence for \(\psi _{\varepsilon }\), then \(\{m_{\varepsilon }(u_{n})\}_{n\in \mathbb {N}}\) is a \((PS)_{c}\) sequence for \(\mathcal {J}_{\varepsilon }\). If \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathcal {N}_{\varepsilon }\) is a bounded \((PS)_{c}\) sequence for \(\mathcal {J}_{\varepsilon }\), then \(\{m_{\varepsilon }^{-1}(u_{n})\}_{n\in \mathbb {N}}\) is a \((PS)_{c}\) sequence for \(\psi _{\varepsilon }\).

(d):

u is a critical point of \(\psi _{\varepsilon }\) if and only if \(m_{\varepsilon }(u)\) is a critical point for \(\mathcal {J}_{\varepsilon }\). Moreover, the corresponding critical values coincide and

$$\begin{aligned} \inf _{u\in \mathbb {S}_{\varepsilon }^{+}} \psi _{\varepsilon }(u)= \inf _{u\in \mathcal {N}_{\varepsilon }} \mathcal {J}_{\varepsilon }(u). \end{aligned}$$

Remark 2.3

As in [36], we have the following minimax characterization of the infimum of \(\mathcal {J}_{\varepsilon }\) over \(\mathcal {N}_{\varepsilon }\):

$$\begin{aligned} c_{\varepsilon }=\inf _{u\in \mathcal {N}_{\varepsilon }} \mathcal {J}_{\varepsilon }(u)=\inf _{u\in \mathbb {X}_{\varepsilon }^{+}} \max _{t>0} \mathcal {J}_{\varepsilon }(tu)=\inf _{u\in \mathbb {S}_{\varepsilon }^{+}} \max _{t>0} \mathcal {J}_{\varepsilon }(tu). \end{aligned}$$

Moreover, arguing as in [37], we can prove that \(c_{\varepsilon }=c'_{\varepsilon }\).

In the remainder of this section, we check that the modified functional satisfies the Palais–Smale condition. We start by showing the boundedness of Palais–Smale sequences.

Lemma 2.5

Let \(c\in \mathbb {R}\) and let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {X}_{\varepsilon }\) be a \((PS)_{c}\) sequence for \(\mathcal {J}_{\varepsilon }\). Then \(\{u_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {X}_{\varepsilon }\).

Proof

Using \((g_{3})\), \(q>p\) and \(\vartheta >2q\), we see that

$$\begin{aligned} C_{0}(1+\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }})&\ge \mathcal {J}_{\varepsilon }(u_{n})- \frac{1}{\vartheta } \langle \mathcal {J}'_{\varepsilon }(u_{n}), u_{n}\rangle \nonumber \\&=\left( \frac{1}{p}- \frac{1}{\vartheta }\right) \Vert u_{n}\Vert _{V_{\varepsilon },p}^{p} +\left( \frac{1}{2p}- \frac{1}{\vartheta }\right) [u_{n}]_{s, p}^{2p} \nonumber \\&\quad +\left( \frac{1}{q}- \frac{1}{\vartheta }\right) \Vert u_{n}\Vert _{V_{\varepsilon },q}^{q}+\left( \frac{1}{2q}- \frac{1}{\vartheta }\right) [u_{n}]_{s, q}^{2q} \nonumber \\&\quad + \frac{1}{\vartheta } \int _{\Lambda _{\varepsilon }^{c}} [g(\varepsilon x, u_{n})u_{n}- \vartheta G(\varepsilon x, u_{n})] \, dx \nonumber \\&\quad + \frac{1}{\vartheta } \int _{\Lambda _{\varepsilon }} [g(\varepsilon x, u_{n})u_{n}- \vartheta G(\varepsilon x, u_{n})] \, dx \nonumber \\&\ge \left( \frac{1}{q}- \frac{1}{\vartheta }\right) [\Vert u_{n}\Vert _{V_{\varepsilon },p}^{p} + \Vert u_{n}\Vert _{V_{\varepsilon },q}^{q}] \nonumber \\&\quad -\left( \frac{1}{p}- \frac{1}{\vartheta }\right) \frac{1}{K} \int _{\Lambda _{\varepsilon }^{c}} V(\varepsilon x) (|u_{n}|^{p}+|u_{n}|^{q})\, dx \nonumber \\&\ge \left[ \left( \frac{1}{q}- \frac{1}{\vartheta }\right) - \left( \frac{1}{p}- \frac{1}{\vartheta }\right) \frac{1}{K} \right] (\Vert u_{n}\Vert _{V_{\varepsilon },p}^{p}+\Vert u_{n}\Vert _{V_{\varepsilon },q}^{q}) \nonumber \\&= {\tilde{C}} (\Vert u_{n}\Vert _{V_{\varepsilon },p}^{p}+\Vert u_{n}\Vert _{V_{\varepsilon },q}^{q}), \end{aligned}$$
(2.6)

where \({\tilde{C}}=\left[ \left( \frac{1}{q}- \frac{1}{\vartheta }\right) - \left( \frac{1}{p}- \frac{1}{\vartheta }\right) \frac{1}{K} \right] >0\) since \(K>\left( \frac{\vartheta -p}{\vartheta -q}\right) \frac{q}{p}\). Suppose, by contradiction, that \(\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }}\rightarrow \infty \). Then we discuss the following cases:

Case 1 \(\Vert u_{n}\Vert _{V_{\varepsilon },p}\rightarrow \infty \) and \(\Vert u_{n}\Vert _{V_{\varepsilon }, q}\rightarrow \infty \).

For n large, we get \(\Vert u_{n}\Vert _{V_{\varepsilon }, q}^{q-p}\ge 1\), that is \(\Vert u_{n}\Vert _{V_{\varepsilon }, q}^{q}\ge \Vert u_{n}\Vert _{V_{\varepsilon }, q}^{p}\). Therefore, from (2.6),

$$\begin{aligned} C_{0}(1+ \Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }}) \ge&{\tilde{C}} (\Vert u_{n}\Vert _{V_{\varepsilon },p}^{p}+\Vert u_{n}\Vert _{V_{\varepsilon }, q}^{p})\ge C_{1} (\Vert u_{n}\Vert _{V_{\varepsilon },p}+\Vert u_{n}\Vert _{V_{\varepsilon },q})^{p}&\\ \quad =C_{1}\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }}^{p} \end{aligned}$$

which is a contradiction.

Case 2 \(\Vert u_{n}\Vert _{V_{\varepsilon },p}\rightarrow \infty \) and \(\Vert u_{n}\Vert _{V_{\varepsilon }, q}\) is bounded.

We have

$$\begin{aligned} C_{0}(1+\Vert u_{n}\Vert _{V_{\varepsilon },p}+\Vert u_{n}\Vert _{V_{\varepsilon },q})=C_{0}(1+\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon }})\ge {\tilde{C}} \Vert u_{n}\Vert _{V_{\varepsilon },p}^{p} \end{aligned}$$

and thus

$$\begin{aligned} C_{0}\left( \frac{1}{\Vert u_{n}\Vert _{V_{\varepsilon },p}^{p}}+\frac{1}{\Vert u_{n}\Vert _{V_{\varepsilon },p}^{p-1}}+\frac{\Vert u_{n}\Vert _{V_{\varepsilon },q}}{\Vert u_{n}\Vert ^{p}_{V_{\varepsilon },p}}\right) \ge {\tilde{C}}. \end{aligned}$$

Since \(p>1\) and letting \(n\rightarrow \infty \), we find \(0< {\tilde{C}}\le 0\), that is a contradiction.

Case 3 \(\Vert u_{n}\Vert _{V_{\varepsilon },q}\rightarrow \infty \) and \(\Vert u_{n}\Vert _{V_{\varepsilon }, p}\) is bounded.

This case is similar to the case 2, so we skip the details.

In conclusion, \(\{u_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {X}_{\varepsilon }\). \(\square \)

Lemma 2.6

Let \(c\in \mathbb {R}\) and let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {X}_{\varepsilon }\) be a \((PS)_{c}\) sequence for \(\mathcal {J}_{\varepsilon }\). Then for any \(\eta >0\) there exists \(R=R(\eta )>0\) such that

$$\begin{aligned}&\limsup _{n\rightarrow \infty } \int _{B_{R}^{c}(0)} \left( \int _{\mathbb {R}^{N}} \frac{|u_{n}(x)- u_{n}(y)|^{p}}{|x-y|^{N+sp}}+\frac{|u_{n}(x)- u_{n}(y)|^{q}}{|x-y|^{N+sq}} \,dy \right. \nonumber \\&\quad \left. + V(\varepsilon x) (|u_{n}|^{p}+|u_{n}|^{q}) \right) \, dx <\eta . \end{aligned}$$
(2.7)

Proof

Let \(\psi \in C^{\infty }(\mathbb {R}^{N})\) be such that \(0\le \psi \le 1\), \(\psi =0\) in \(B_{\frac{1}{2}}(0)\), \(\psi _{R}=1\) in \(B_{1}^{c}(0)\), and \(|\nabla \psi |_{\infty }\le C\), for some \(C>0\). For \(R>0\), define \(\psi _{R}(x)=\psi (\frac{x}{R})\). Then, \(0\le \psi _{R}\le 1\), \(\psi _{R}=0\) in \(B_{\frac{R}{2}}(0)\), \(\psi _{R}=1\) in \(B_{R}^{c}(0)\), and \(|\nabla \psi _{R}|_{\infty }\le \frac{C}{R}\) with \(C>0\) independent of R. Since \(\{\psi _{R}u_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {X}_{\varepsilon }\), it holds \(\langle \mathcal {J}_{\varepsilon }'(u_{n}), \psi _{R}u_{n}\rangle =o_{n}(1)\), that is

$$\begin{aligned}&(1+[u_{n}]^{p}_{s, p})\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)- u_{n}(y)|^{p}}{|x-y|^{N+sp}}\psi _{R}(x)\, dxdy +(1+[u_{n}]^{q}_{s, q}) \\&\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)- u_{n}(y)|^{q}}{|x-y|^{N+sq}}\psi _{R}(x)\, dxdy \\&\quad +\int _{\mathbb {R}^{N}} V(\varepsilon x) |u_{n}|^{p} \psi _{R} \, dx+\int _{\mathbb {R}^{N}} V(\varepsilon x) |u_{n}|^{q} \psi _{R} \, dx\\&=o_{n}(1) + \int _{\mathbb {R}^{N}} g(\varepsilon x, u_{n}) \psi _{R} u_{n}\, dx \\&\quad - (1+[u_{n}]^{p}_{s, p})\iint _{\mathbb {R}^{2N}} \\&\qquad \frac{|u_{n}(x)- u_{n}(y)|^{p-2} (u_{n}(x)- u_{n}(y)) (\psi _{R}(x)- \psi _{R}(y))}{|x-y|^{N+sp}}u_{n}(y)\, dxdy\\&\quad -(1+[u_{n}]^{q}_{s, q})\iint _{\mathbb {R}^{2N}}\\&\qquad \frac{|u_{n}(x)- u_{n}(y)|^{q-2} (u_{n}(x)- u_{n}(y)) (\psi _{R}(x)- \psi _{R}(y))}{|x-y|^{N+sq}}u_{n}(y)\, dxdy. \end{aligned}$$

Pick \(R>0\) such that \(\Lambda _{\varepsilon }\subset B_{\frac{R}{2}}(0)\). By the definition of \(\psi _{R}\) and using \((g_{3})\)-(ii), we obtain that

$$\begin{aligned} \begin{aligned}&\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)- u_{n}(y)|^{p}}{|x-y|^{N+sp}}\psi _{R}(x)\, dxdy + \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)- u_{n}(y)|^{q}}{|x-y|^{N+sq}}\psi _{R}(x)\, dxdy \\&\quad + \left( 1- \frac{1}{K}\right) \int _{\mathbb {R}^{N}} V(\varepsilon x) (|u_{n}|^{p}+|u_{n}|^{q}) \psi _{R} \, dx \\&\le o_{n}(1) - (1+[u_{n}]^{p}_{s, p}) \\&\iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)- u_{n}(y)|^{p-2} (u_{n}(x)- u_{n}(y)) (\psi _{R}(x)- \psi _{R}(y))}{|x-y|^{N+sp}}u_{n}(y)\, dxdy \\&\quad -(1+[u_{n}]_{s, q}^{q})\\&\qquad \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)- u_{n}(y)|^{q-2} (u_{n}(x)- u_{n}(y)) (\psi _{R}(x)- \psi _{R}(y))}{|x-y|^{N+sq}}u_{n}(y)\, dxdy. \end{aligned}\end{aligned}$$
(2.8)

Now, from the Hölder inequality and the boundedness of \(\{u_{n}\}_{n\in \mathbb {N}}\) in \(\mathbb {X}_{\varepsilon }\), we get, for \(t\in \{p, q\}\),

$$\begin{aligned}&\left| \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)- u_{n}(y)|^{t-2} (u_{n}(x)- u_{n}(y)) (\psi _{R}(x)- \psi _{R}(y))}{|x-y|^{N+st}}u_{n}(y)\, dxdy\right| \nonumber \\&\qquad \le C \left( \iint _{\mathbb {R}^{2N}} \frac{|\psi _{R}(x)- \psi _{R}(y)|^{t}}{|x-y|^{N+st}}|u_{n}(y)|^{t}\, dxdy\right) ^{\frac{1}{t}}. \end{aligned}$$
(2.9)

An inspection of the proof of Lemma 2.2 shows that, for \(t\in \{p, q\}\),

$$\begin{aligned} \limsup _{n\rightarrow \infty } \iint _{\mathbb {R}^{2N}} \frac{|\psi _{R}(x)- \psi _{R}(y)|^{t}}{|x-y|^{N+st}}|u_{n}(y)|^{t}\, dxdy\le \frac{C}{R^{st}}. \end{aligned}$$
(2.10)

Combining (2.8), (2.9) and (2.10), and recalling the definition of \(\psi _{R}\), for some \(C>0\), we can take \(R=R(\eta )>(\frac{C}{\eta })^{\frac{1}{s}}\) so that (2.7) is satisfied. \(\square \)

Since we are working with a Kirchhoff type problem, the next lemma will be fundamental to obtain the strong convergence of bounded Palais–Smale sequences.

Lemma 2.7

Let \(c\in \mathbb {R}\) and let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {X}_{\varepsilon }\) be a \((PS)_{c}\) sequence for \(\mathcal {J}_{\varepsilon }\). Let \(R>0\). Then

$$\begin{aligned} \begin{aligned}&\lim _{n\rightarrow \infty } \int _{B_{R}(0)}\\&\quad \left\{ \int _{\mathbb {R}^{N}} \left[ \frac{|u_{n}(x)-u_{n}(y)|^{p}}{|x-y|^{N+sp}}+\frac{|u_{n}(x)-u_{n}(y)|^{q}}{|x-y|^{N+sq}}\right] \, dy+ V(\varepsilon x) (|u_{n}|^{p}+|u_{n}|^{q}) \right\} \, dx\\&\quad = \int _{B_{R}(0)}\\&\quad \left\{ \int _{\mathbb {R}^{N}} \left[ \frac{|u(x)-u(y)|^{p}}{|x-y|^{N+sp}}+\frac{|u(x)-u(y)|^{q}}{|x-y|^{N+sq}}\right] \, dy+ V(\varepsilon x) (|u|^{p}+|u|^{q}) \right\} \, dx. \end{aligned} \end{aligned}$$
(2.11)

Proof

Let \(\eta \in C^{\infty }(\mathbb {R}^{N})\) be such that \(0\le \eta \le 1\), \(\eta =1\) in \(B_{1}(0)\), \(\eta =0\) in \(B^{c}_{2}(0)\) and \(|\nabla \eta |_{\infty }\le 2\). For \(\rho >0\), put \(\eta _{\rho }(x)=\eta (\frac{x}{\rho })\). Then \(0\le \eta _{\rho }\le 1\), \(\eta =1\) in \(B_{\rho }(0)\), \(\eta =0\) in \(B^{c}_{2\rho }(0)\) and \(|\nabla \eta |_{\infty }\le \frac{2}{\rho }\). Since \(\{u_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {X}_{\varepsilon }\) (by Lemma 2.5), we may suppose that \([u_{n}]_{s, p}^{p}\rightarrow \ell _{p}\) and \([u_{n}]_{s, q}^{q}\rightarrow \ell _{q}\) as \(n\rightarrow \infty \).

Fix \(R>0\) and take \(\rho >R\). We recall the following well-known elementary inequalities [35]: for any \(\xi , \eta \in \mathbb {R}^{N}\) we have

$$\begin{aligned}&(|\xi |^{r-2}\xi - |\eta |^{r-2}\eta ) \cdot (\xi - \eta ) \ge c_{1} |\xi -\eta |^{r} \quad \text{ for } r\ge 2, \end{aligned}$$
(2.12)
$$\begin{aligned}&{{(|\xi |+ |\eta |)^{2-r} [(|\xi |^{r-2}\xi - |\eta |^{r-2}\eta )\cdot (\xi - \eta )] \ge c_{2} |\xi -\eta |^{2}}} \quad \text{ for } 1<r<2, \end{aligned}$$
(2.13)

for some constants \(c_{1}, c_{2}>0\). Note that, when \(1<r<2\), using (2.13) and the elementary inequality

$$\begin{aligned} (|\xi |+|\eta |)^{r}\le 2^{r-1}(|\xi |^{r}+|\eta |^{r}) \quad \text{ for } \text{ all } \xi , \eta \in \mathbb {R}^{N}, \end{aligned}$$

we deduce that there exists \(c_{3}>0\) such that, for any \(\xi , \eta \in \mathbb {R}^{N}\), the following relation is satisfied

$$\begin{aligned} {{(|\xi |^{r}+|\eta |^{r})^{\frac{2-r}{2}} \left[ (|\xi |^{r-2}\xi - |\eta |^{r-2}\eta ) \cdot (\xi - \eta )\right] ^{\frac{r}{2}}\ge c_{3} |\xi -\eta |^{r} }} \quad \text{ for } 1<r<2. \end{aligned}$$
(2.14)

For \(t\in \{p, q\}\) and \(n\in \mathbb {N}\), we set

$$\begin{aligned} A^{t}_{n}(x)&=(1+[u_{n}]^{t}_{s, t})\\&\quad \int _{\mathbb {R}^{N}} \left[ \frac{|u_{n}(x)-u_{n}(y)|^{t-2}(u_{n}(x)-u_{n}(y))}{|x-y|^{N+st}}-\frac{|u(x)-u(y)|^{t-2}(u(x)-u(y))}{|x-y|^{N+st}} \right] \times \\&\qquad \qquad \times [(u_{n}(x)-u_{n}(y))-(u(x)-u(y))]\, dy\\&\quad +V(\varepsilon x) (|u_{n}(x)|^{t-2}u_{n}(x)-|u(x)|^{t-2}u(x))(u_{n}(x)-u(x)). \end{aligned}$$

Note that, for \(t\in \{p, q\}\) and \(n\in \mathbb {N}\), we have

$$\begin{aligned} 0&\le \int _{B_{R}(0)} A^{t}_{n}(x)\, dx=\int _{B_{R}(0)} A^{t}_{n}(x)\eta _{\rho }(x)\, dx \\&\le (1+[u_{n}]_{s, t}^{t})\\&\iint _{\mathbb {R}^{2N}} \left[ \frac{|u_{n}(x)-u_{n}(y)|^{t-2}(u_{n}(x)-u_{n}(y))}{|x-y|^{N+st}}-\frac{|u(x)-u(y)|^{t-2}(u(x)-u(y))}{|x-y|^{N+st}} \right] \times \\&\qquad \times [(u_{n}(x)-u_{n}(y))-(u(x)-u(y))]\eta _{\rho }(x)\, dxdy\\&\quad +\int _{\mathbb {R}^{N}} V(\varepsilon x) (|u_{n}|^{t-2}u_{n}-|u|^{t-2}u)(u_{n}-u)\eta _{\rho }\, dx \\&=(1+[u_{n}]_{s, t}^{t}) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{t}}{|x-y|^{N+st}} \eta _{\rho }(x)\, dx dy+\int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{t}\eta _{\rho }\, dx\\&\quad +(1+[u_{n}]_{s, t}^{t}) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t}}{|x-y|^{N+st}} \eta _{\rho }(x)\, dx dy+\int _{\mathbb {R}^{N}} V(\varepsilon x)|u|^{t}\eta _{\rho }\, dx\\&\quad -\left[ (1+[u_{n}]_{s, t}^{t}) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{t-2}}{|x-y|^{N+st}} (u_{n}(x)-u_{n}(y))(u(x)\right. \\&\quad \left. -u(y))\eta _{\rho }(x) \, dx dy \right. \\&\quad \left. +\int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{t-2}u_{n}u\eta _{\rho }\, dx \right] \\&\quad -\left[ (1+[u_{n}]_{s, t}^{t}) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t-2}}{|x-y|^{N+st}} (u_{n}(x)\right. \\&\quad \left. -u_{n}(y))(u(x)-u(y))\eta _{\rho }(x) \, dx dy \right. \\&\quad \left. +\int _{\mathbb {R}^{N}} V(\varepsilon x)|u|^{t-2}uu_{n}\eta _{\rho }\, dx \right] . \end{aligned}$$

Define

$$\begin{aligned} I_{n, \rho }^{1}&=(1+[u_{n}]_{s, p}^{p}) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p}}{|x-y|^{N+sp}} \eta _{\rho }(x)\, dx dy+\int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{p}\eta _{\rho }\, dx\\&\quad +(1+[u_{n}]_{s, q}^{q}) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q}}{|x-y|^{N+sq}} \eta _{\rho }(x)\, dx dy+\int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{q}\eta _{\rho }\, dx\\&\quad -\int _{\mathbb {R}^{N}} g(\varepsilon x, u_{n})u_{n}\eta _{\rho }\, dx, \end{aligned}$$
$$\begin{aligned} I_{n, \rho }^{2}&=(1+[u_{n}]_{s, p}^{p}) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{p}}{|x-y|^{N+sp}} \eta _{\rho }(x)\, dx dy\\&\quad +\int _{\mathbb {R}^{N}} V(\varepsilon x)|u|^{p}\eta _{\rho }\, dx\\&\quad - (1+[u_{n}]_{s, p}^{p}) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{p-2}}{|x-y|^{N+sp}} (u(x)\\&\quad -u(y)) (u_{n}(x)-u_{n}(y))\eta _{\rho }(x) \, dxdy \\&\quad -\int _{\mathbb {R}^{N}} V(\varepsilon x)|u|^{p-2}uu_{n}\eta _{\rho }\, dx \\&\quad +(1+[u_{n}]_{s, q}^{q}) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{q}}{|x-y|^{N+sq}} \eta _{\rho }(x)\, dxdy \\&\quad +\int _{\mathbb {R}^{N}} V(\varepsilon x)|u|^{q}\eta _{\rho }\, dx\\&\quad - (1+[u_{n}]_{s, q}^{q}) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{q-2}}{|x-y|^{N+sq}} (u(x)-u(y)) (u_{n}(x)\\&\quad -u_{n}(y))\eta _{\rho }(x) \, dxdy \\&\quad -\int _{\mathbb {R}^{N}} V(\varepsilon x)|u|^{q-2}uu_{n}\eta _{\rho }\, dx, \end{aligned}$$
$$\begin{aligned} I_{n, \rho }^{3}&=(1+[u_{n}]_{s, p}^{p}) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p-2}}{|x-y|^{N+sp}} (u_{n}(x)\\&\quad -u_{n}(y)) (u(x)-u(y))\eta _{\rho }(x) \, dxdy \\&\quad +\int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{p-2}u_{n}u\eta _{\rho }\, dx \\&\quad +(1+[u_{n}]_{s, q}^{q}) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q-2}}{|x-y|^{N+sq}} (u_{n}(x)-u_{n}(y)) (u(x)\\&\quad -u(y))\eta _{\rho }(x) \, dxdy \\&\quad +\int _{\mathbb {R}^{N}} V(\varepsilon x)|u_{n}|^{q-2}u_{n}u\eta _{\rho }\, dx \\&\quad -\int _{\mathbb {R}^{N}} g(\varepsilon x, u_{n}) u\eta _{\rho }\, dx, \end{aligned}$$

and

$$\begin{aligned} I_{n, \rho }^{4}=\int _{\mathbb {R}^{N}} g(\varepsilon x, u_{n})(u_{n}-u)\eta _{\rho }\, dx. \end{aligned}$$

Then it holds

$$\begin{aligned} 0\le \int _{B_{R}(0)} (A^{p}_{n}(x)+A^{q}_{n}(x))\, dx\le |I_{n, \rho }^{1}|+ |I_{n, \rho }^{2}|+ |I_{n, \rho }^{3}|+ |I_{n, \rho }^{4}|. \end{aligned}$$
(2.15)

Since

$$\begin{aligned} I_{n, \rho }^{1}&= \langle \mathcal {J}'_{\varepsilon }(u_{n}), u_{n}\eta _{\rho }\rangle - \Bigl [ \left( 1+ [u_{n}]_{s, p}^{p} \right) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p-2}}{|x-y|^{N+sp}} \\&\qquad (u_{n}(x)-u_{n}(y))(\eta _{\rho }(x)-\eta _{\rho }(y))u_{n}(y) \, dxdy \\&\quad + \left( 1+ [u_{n}]_{s, q}^{q} \right) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q-2}}{|x-y|^{N+sq}} (u_{n}(x)\\&\quad -u_{n}(y))(\eta _{\rho }(x)-\eta _{\rho }(y))u_{n}(y) \, dxdy \Bigr ] \end{aligned}$$

and \(\{u_{n}\eta _{\rho }\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {X}_{\varepsilon }\), we see that \(\langle \mathcal {J}'_{\varepsilon }(u_{n}), u_{n}\eta _{\rho }\rangle =o_{n}(1)\). Using the Hölder inequality and the boundedness of \(\{u_{n}\}_{n\in \mathbb {N}}\) in \(\mathbb {X}_{\varepsilon }\), we have

$$\begin{aligned}&\left| \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{t-2}}{|x-y|^{N+st}} (u_{n}(x)-u_{n}(y))(\eta _{\rho }(x)-\eta _{\rho }(y))u_{n}(y) \, dxdy \right| \\&\le C \left( \iint _{\mathbb {R}^{2N}} \frac{|\eta _{\rho }(x)-\eta _{\rho }(y)|^{t}}{|x-y|^{N+st}}|u_{n}(y)|^{t}\, dxdy \right) ^{\frac{1}{t}} \quad \text{ for } t\in \{p, q\}, \end{aligned}$$

which combined with Lemma in 2.2 (applied with \(\phi _{\rho }=1-\eta _{\rho }\)) yields

$$\begin{aligned}&\lim _{\rho \rightarrow \infty } \limsup _{n\rightarrow \infty }\\&\quad \left| \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{t-2}}{|x-y|^{N+st}} (u_{n}(x)-u_{n}(y))(\eta _{\rho }(x)-\eta _{\rho }(y))u_{n}(y) \, dxdy \right| =0 \\&\quad \text{ for } t\in \{p, q\}. \end{aligned}$$

Consequently, recalling that \([u_{n}]_{s, t}^{t}\rightarrow \ell _{t}\) for \(t\in \{p, q\}\), we get

$$\begin{aligned} \lim _{\rho \rightarrow \infty } \left[ \limsup _{n\rightarrow \infty } \left| I_{n, \rho }^{1}\right| \right] =0. \end{aligned}$$
(2.16)

We also observe that

$$\begin{aligned} I_{n, \rho }^{3}&= \langle \mathcal {J}'_{\varepsilon }(u_{n}), u\eta _{\rho }\rangle - \Bigl [ \left( 1+ [u_{n}]_{s, p}^{p} \right) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{p-2}}{|x-y|^{N+sp}} \\&\qquad (u_{n}(x)-u_{n}(y))(\eta _{\rho }(x)-\eta _{\rho }(y))u(y) \, dxdy \\&\quad + \left( 1+ [u_{n}]_{s, q}^{q} \right) \iint _{\mathbb {R}^{2N}} \frac{|u_{n}(x)-u_{n}(y)|^{q-2}}{|x-y|^{N+sq}} (u_{n}(x)\\&\quad -u_{n}(y))(\eta _{\rho }(x)-\eta _{\rho }(y))u(y) \, dxdy \Bigr ], \end{aligned}$$

and using \(\langle \mathcal {J}'_{\varepsilon }(u_{n}), u\eta _{\rho }\rangle =o_{n}(1)\), we can argue as before to achieve that

$$\begin{aligned} \lim _{\rho \rightarrow \infty } \left[ \limsup _{n\rightarrow \infty } |I_{n, \rho }^{3}|\right] =0. \end{aligned}$$
(2.17)

Next we prove that

$$\begin{aligned} \lim _{\rho \rightarrow \infty } \left[ \limsup _{n\rightarrow \infty } |I_{n, \rho }^{2}|\right] =0. \end{aligned}$$
(2.18)

From the weak convergence, we have

$$\begin{aligned} \int _{\mathbb {R}^{N}} V(\varepsilon x)|u|^{t-2} u(u_{n}-u)\eta _{\rho }\, dx=o_{n}(1) \quad \text{ for } t\in \{p, q\}. \end{aligned}$$

Notice that, for \(t\in \{p, q\}\),

$$\begin{aligned}&\left( 1+ [u_{n}]_{s, t}^{t} \right) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t-2}}{|x-y|^{N+st}}(u(x)-u(y))\\&\quad [(u_{n}-u)(x)-(u_{n}-u)(y)] \eta _{\rho }(x)\, dx dy \\&=\left( 1+ [u_{n}]_{s, t}^{t} \right) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t-2}}{|x-y|^{N+st}} (u(x)-u(y)) \\&\quad [(u_{n}-u)(x)-(u_{n}-u)(y)](\eta _{\rho }(x)-1)\, dx dy \\&\quad +\left( 1+ [u_{n}]_{s, t}^{t} \right) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t-2}}{|x-y|^{N+st}}(u(x)\\&\quad -u(y)) [(u_{n}-u)(x)-(u_{n}-u)(y)] \, dx dy. \end{aligned}$$

By \(u_{n}\rightharpoonup u\) in \(\mathbb {X}_{\varepsilon }\) and \([u_{n}]^{t}_{s, t}\rightarrow \ell _{t}\) for \(t\in \{p, q\}\), we deduce that

$$\begin{aligned}&\lim _{n\rightarrow \infty } \left( 1+ [u_{n}]_{s, t}^{t} \right) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t-2}}{|x-y|^{N+st}}(u(x)-u(y))\\&[(u_{n}-u)(x)-(u_{n}-u)(y)] \, dx dy=0 \quad \text{ for } t\in \{p, q\}. \end{aligned}$$

On the other hand, using the boundedness of \(\{u_{n}\}_{n\in \mathbb {N}}\) in \(\mathbb {X}_{\varepsilon }\) and applying the Hölder inequality, we see that

$$\begin{aligned}&\left| \left( 1+ [u_{n}]_{s, t}^{t} \right) \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t-2}}{|x-y|^{N+st}} (u(x)-u(y)) \right. \\&\left. [(u_{n}-u)(x)-(u_{n}-u)(y)](\eta _{\rho }(x)-1)\, dx dy \right| \\&\le (1+C) [u_{n}-u]_{s, t} \left( \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t}}{|x-y|^{N+st}} |\eta _{\rho }(x)-1|^{\frac{t}{t-1}} \, dxdy\right) ^{\frac{t-1}{t}} \\&\le C \left( \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t}}{|x-y|^{N+st}} |\eta _{\rho }(x)-1|^{\frac{t}{t-1}} \, dxdy\right) ^{\frac{t-1}{t}} \quad \text{ for } t\in \{p, q\}. \end{aligned}$$

Since \(\eta _{\rho }\rightarrow 1\) a.e. in \(\mathbb {R}^N\) as \(\rho \rightarrow \infty \) and \(u\in W^{s, t}(\mathbb {R}^{N})\), it follows from the dominated convergence theorem that

$$\begin{aligned} \lim _{\rho \rightarrow \infty } \iint _{\mathbb {R}^{2N}} \frac{|u(x)-u(y)|^{t}}{|x-y|^{N+st}} |\eta _{\rho }(x)-1|^{\frac{t}{t-1}} \, dxdy=0 \quad \text{ for } t\in \{p, q\}. \end{aligned}$$

The validity of (2.18) is now an immediate consequence of the definition of \(I^{2}_{n, \rho }\) and of the above relations.

Finally, exploiting \(u_{n}\rightarrow u\) in \(L^{r}_{loc}(\mathbb {R}^{N})\) for all \(r\in [p, q^{*}_{s})\) and the growth assumptions on g, we obtain

$$\begin{aligned} \lim _{n\rightarrow \infty } |I_{n, \rho }^{4}|=0 \quad \text{ for } \text{ any } \rho >R. \end{aligned}$$
(2.19)

Combining (2.15) with (2.16)–(2.19), we find

$$\begin{aligned} \lim _{n\rightarrow \infty } \int _{B_{R}(0)} (A^{p}_{n}(x)+A^{q}_{n}(x))\, dx=0, \end{aligned}$$

whence

$$\begin{aligned}&\lim _{n\rightarrow \infty } \Bigl \{ (1+[u_{n}]^{t}_{s, t}) \nonumber \\&\quad \int _{B_{R}(0)} \Bigl [\int _{\mathbb {R}^{N}}\nonumber \\&\quad \left( \frac{|u_{n}(x)-u_{n}(y)|^{t-2}(u_{n}(x)-u_{n}(y))}{|x-y|^{N+st}}-\frac{|u(x)-u(y)|^{t-2}(u(x)-u(y))}{|x-y|^{N+st}} \right) \times \nonumber \\&\quad \times ((u_{n}(x)-u_{n}(y))-(u(x)-u(y)))\, dy\Bigr ] \, dx \nonumber \\&\qquad + \int _{B_{R}(0)} V(\varepsilon x) \left( |u_{n}|^{t-2}u_{n} - |u|^{t-2} u\right) \left( u_{n}- u\right) \, dx \Bigr \} =0 \quad \text{ for } t\in \{p, q\}. \end{aligned}$$
(2.20)

Assume first that \(t\ge 2\). Using (2.12), the boundedness of \(\{u_{n}\}_{n\in \mathbb {N}}\) in \(\mathbb {X}_{\varepsilon }\) and (2.20), we get

$$\begin{aligned} 0&\le \int _{B_{R}(0)} \left[ \int _{\mathbb {R}^{N}} \frac{|(u_{n}-u)(x)-(u_{n}-u)(y)|^{t}}{|x-y|^{N+st}}\, dy\right] \, dx \\&\le C \int _{B_{R}(0)} \Bigl [\int _{\mathbb {R}^{N}} \left( \frac{|u_{n}(x)-u_{n}(y)|^{t-2}(u_{n}(x)-u_{n}(y))}{|x-y|^{N+st}}\right. \\&\quad \left. -\frac{|u(x)-u(y)|^{t-2}(u(x)-u(y))}{|x-y|^{N+st}} \right) \times \\&\quad \times ((u_{n}(x)-u_{n}(y))-(u(x)-u(y)))\, dy\Bigr ]\, dx=o_{n}(1). \end{aligned}$$

In a similar fashion,

$$\begin{aligned}&0\le \int _{B_{R}(0)} V(\varepsilon x) |u_{n}-u|^{t}\, dx\le C\int _{B_{R}(0)}\\&\quad V(\varepsilon x) \left( |u_{n}|^{t-2}u_{n} - |u|^{t-2} u\right) \left( u_{n}- u\right) \, dx=o_{n}(1). \end{aligned}$$

Suppose now that \(1<t<2\). From (2.14), the boundedness of \(\{u_{n}\}_{n\in \mathbb {N}}\) in \(\mathbb {X}_{\varepsilon }\), Hölder’s inequality, and (2.20), we derive

$$\begin{aligned}&\int _{B_{R}(0)} \left[ \int _{\mathbb {R}^{N}} \frac{|(u_{n}-u)(x)-(u_{n}-u)(y)|^{t}}{|x-y|^{N+st}}\, dy\right] \, dx \\&\le C \Bigl ([u_{n}]_{s, t}^{t}+[u]_{s, t}^{t} \Bigr )^{\frac{2-t}{2}} \\&\quad \Bigl \{ \int _{B_{R}(0)} \Bigl [\int _{\mathbb {R}^{N}}\\&\quad \left( \frac{|u_{n}(x)-u_{n}(y)|^{t-2}(u_{n}(x)-u_{n}(y))}{|x-y|^{N+st}}-\frac{|u(x)-u(y)|^{t-2}(u(x)-u(y))}{|x-y|^{N+st}} \right) \times \\&\quad \times ((u_{n}(x)-u_{n}(y))-(u(x)-u(y)))\, dy\Bigr ]\, dx \Bigr \}^{\frac{t}{2}} \\&\le C\Bigl \{ \int _{B_{R}(0)} \Bigl [\int _{\mathbb {R}^{N}}\\&\quad \left( \frac{|u_{n}(x)-u_{n}(y)|^{t-2}(u_{n}(x)-u_{n}(y))}{|x-y|^{N+st}}-\frac{|u(x)-u(y)|^{t-2}(u(x)-u(y))}{|x-y|^{N+st}} \right) \times \\&\quad \times ((u_{n}(x)-u_{n}(y))-(u(x)-u(y)))\, dy\Bigr ] \, dx \Bigr \}^{\frac{t}{2}}=o_{n}(1). \end{aligned}$$

Analogously,

$$\begin{aligned}&0\le \int _{B_{R}(0)} V(\varepsilon x) |u_{n}-u|^{t}\, dx\\&\quad \le C\left[ \int _{B_{R}(0)} V(\varepsilon x) \left( |u_{n}|^{t-2}u_{n} - |u|^{t-2} u\right) \left( u_{n}- u\right) \, dx\right] ^{\frac{t}{2}} =o_{n}(1). \end{aligned}$$

Consequently, for \(t\in \{p, q\}\),

$$\begin{aligned}&\lim _{n\rightarrow \infty } \int _{B_{R}(0)} \left[ \int _{\mathbb {R}^{N}} \frac{|u_{n}(x)-u_{n}(y)|^{t}}{|x-y|^{N+st}}\, dy+ V(\varepsilon x) |u_{n}|^{t}\right] \, dx \\&\quad = \int _{B_{R}(0)} \left[ \int _{\mathbb {R}^{N}} \frac{|u(x)-u(y)|^{t}}{|x-y|^{N+st}}\, dy+ V(\varepsilon x) |u|^{t}\right] \, dx \end{aligned}$$

which implies (2.11). This completes the proof. \(\square \)

Now we are ready to prove the following compactness result.

Lemma 2.8

\(\mathcal {J}_{\varepsilon }\) satisfies the \((PS)_{c}\) condition at any level \(c\in \mathbb {R}\).

Proof

Let \(c\in \mathbb {R}\) and let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {X}_{\varepsilon }\) be a \((PS)_{c}\) sequence for \(\mathcal {J}_{\varepsilon }\). By Lemma 2.5, we know that \(\{u_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {X}_{\varepsilon }\). Up to a subsequence, we may suppose that \(u_{n}\rightharpoonup u\) in \(\mathbb {X}_{\varepsilon }\) and \(u_{n}\rightarrow u\) in \(L^{r}_{loc}(\mathbb {R}^{N})\) for all \(r\in [1, q^{*}_{s})\). In view of Lemma 2.6, for each \(\eta >0\), there exists \(R=R(\eta )>(\frac{C}{\eta })^{\frac{1}{s}}\), with \(C>0\) independent of \(\eta \), such that (2.11) holds. This fact combined with Lemma 2.7 yields

$$\begin{aligned} \Vert u\Vert _{V_{\varepsilon }, p}^{p}+\Vert u\Vert _{V_{\varepsilon }, q}^{q}&\le \liminf _{n\rightarrow \infty } (\Vert u_{n}\Vert _{V_{\varepsilon }, p}^{p}+\Vert u_{n}\Vert _{V_{\varepsilon }, q}^{q})\\&\le \limsup _{n\rightarrow \infty } (\Vert u_{n}\Vert _{V_{\varepsilon }, p}^{p}+\Vert u_{n}\Vert _{V_{\varepsilon }, q}^{q})\\&=\limsup _{n\rightarrow \infty } \Bigl \{ \int _{B_{R}(0)} \\&\quad \left[ \int _{\mathbb {R}^{N}} \left( \frac{|u_{n}(x)-u_{n}(y)|^{p}}{|x-y|^{N+sp}}+\frac{|u_{n}(x)-u_{n}(y)|^{q}}{|x-y|^{N+sq}}\right) \, dy \right. \\&\quad \left. + V(\varepsilon x) (|u_{n}|^{p}+|u_{n}|^{q}) \right] \, dx \\&\quad + \int _{B^{c}_{R}(0)} \left[ \int _{\mathbb {R}^{N}} \left( \frac{|u_{n}(x)-u_{n}(y)|^{p}}{|x-y|^{N+sp}}+\frac{|u_{n}(x)-u_{n}(y)|^{q}}{|x-y|^{N+sq}}\right) \, dy \right. \\&\quad \left. + V(\varepsilon x) (|u_{n}|^{p}+|u_{n}|^{q}) \right] \, dx \Bigr \}\\&= \int _{B_{R}(0)} \left[ \int _{\mathbb {R}^{N}} \left( \frac{|u(x)-u(y)|^{p}}{|x-y|^{N+sp}}+\frac{|u(x)-u(y)|^{q}}{|x-y|^{N+sq}}\right) \, dy \right. \\&\quad \left. + V(\varepsilon x) (|u|^{p}+|u|^{q}) \right] \, dx\\&\quad + \limsup _{n\rightarrow \infty } \Bigl \{ \int _{B^{c}_{R}(0)}\\&\quad \left[ \int _{\mathbb {R}^{N}} \left( \frac{|u_{n}(x)-u_{n}(y)|^{p}}{|x-y|^{N+sp}}+\frac{|u_{n}(x)-u_{n}(y)|^{q}}{|x-y|^{N+sq}}\right) \, dy \right. \\&\quad \left. + V(\varepsilon x) (|u_{n}|^{p}+|u_{n}|^{q}) \right] \, dx \Bigr \}\\&< \int _{B_{R}(0)} \left[ \int _{\mathbb {R}^{N}} \left( \frac{|u(x)-u(y)|^{p}}{|x-y|^{N+sp}}+\frac{|u(x)-u(y)|^{q}}{|x-y|^{N+sq}}\right) \, dy\right. \\&\quad \left. + V(\varepsilon x) (|u|^{p}+|u|^{q}) \right] \, dx+\eta . \end{aligned}$$

Letting \(\eta \rightarrow 0\), we have \(R\rightarrow \infty \) and then

$$\begin{aligned} \Vert u\Vert _{V_{\varepsilon }, p}^{p}+\Vert u\Vert _{V_{\varepsilon }, q}^{q}&\le \liminf _{n\rightarrow \infty } (\Vert u_{n}\Vert _{V_{\varepsilon }, p}^{p}+\Vert u_{n}\Vert _{V_{\varepsilon }, q}^{q})\\&\le \limsup _{n\rightarrow \infty } (\Vert u_{n}\Vert _{V_{\varepsilon }, p}^{p}+\Vert u_{n}\Vert _{V_{\varepsilon }, q}^{q})\\&\le \Vert u\Vert _{V_{\varepsilon }, p}^{p}+\Vert u\Vert _{V_{\varepsilon }, q}^{q}, \end{aligned}$$

whence

$$\begin{aligned} \Vert u_{n}\Vert _{V_{\varepsilon }, p}^{p}+\Vert u_{n}\Vert _{V_{\varepsilon }, q}^{q}=\Vert u\Vert _{V_{\varepsilon }, p}^{p}+\Vert u\Vert _{V_{\varepsilon }, q}^{q}+o_{n}(1). \end{aligned}$$
(2.21)

Since the Brezis–Lieb lemma [14] gives

$$\begin{aligned}&\Vert u_{n}-u\Vert ^{p}_{V_{\varepsilon },p}=\Vert u_{n}\Vert ^{p}_{V_{\varepsilon },p}-\Vert u\Vert _{V_{\varepsilon },p}^{p}+o_{n}(1) \text{ and } \Vert u_{n}-u\Vert ^{q}_{V_{\varepsilon },q}=\Vert u_{n}\Vert ^{q}_{V_{\varepsilon },q}\\&\quad -\Vert u\Vert _{V_{\varepsilon },q}^{q}+o_{n}(1), \end{aligned}$$

we infer that

$$\begin{aligned} \Vert u_{n}-u\Vert ^{p}_{V_{\varepsilon },p}+\Vert u_{n}-u\Vert ^{q}_{V_{\varepsilon },q}=o_{n}(1). \end{aligned}$$

This last fact implies that \(u_{n}\rightarrow u\) in \(\mathbb {X}_{\varepsilon }\) as \(n\rightarrow \infty \). \(\square \)

Corollary 2.1

The functional \(\psi _{\varepsilon }\) satisfies the \((PS)_{c}\) condition on \(\mathbb {S}_{\varepsilon }^{+}\) at any level \(c\in \mathbb {R}\).

Proof

Let \(c\in \mathbb {R}\) and let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {S}^{+}_{\varepsilon }\) be a \((PS)_{c}\) sequence for \(\psi _{\varepsilon }\). Hence,

$$\begin{aligned} \psi _{\varepsilon }(u_{n})\rightarrow c \quad \text{ and } \quad \psi _{\varepsilon }'(u_{n})\rightarrow 0 \text{ in } (T_{u_{n}} \mathbb {S}_{\varepsilon }^{+})'. \end{aligned}$$

By Proposition 2.1-(c), we know that \(\{m_{\varepsilon }(u_{n})\}_{n\in \mathbb {N}}\subset \mathbb {X}_{\varepsilon }\) is a \((PS)_{c}\) sequence for \(\mathcal {J}_{\varepsilon }\). Then, by Lemma 2.8, we deduce that \(\mathcal {J}_{\varepsilon }\) satisfies the \((PS)_{c}\) condition in \(\mathbb {X}_{\varepsilon }\), and thus there exists \(u\in \mathbb {S}_{\varepsilon }^{+}\) such that, up to a subsequence,

$$\begin{aligned} m_{\varepsilon }(u_{n})\rightarrow m_{\varepsilon }(u) \text{ in } \mathbb {X}_{\varepsilon }. \end{aligned}$$

By Lemma 2.4-(iii), we conclude that \(u_{n}\rightarrow u\) in \(\mathbb {S}_{\varepsilon }^{+}\). \(\square \)

We conclude this section by establishing an existence result for (2.1).

Theorem 2.2

Assume that \((V_1)\)\((V_2)\) and \((f_1)\)\((f_4)\) hold. Then, for all \(\varepsilon >0\), there exists a positive ground state solution to (2.1).

Proof

In light of Lemmas 2.3 and 2.8, we can apply the mountain pass theorem [3] to see that for all \(\varepsilon >0\) there exists a nontrivial critical point \(u_{\varepsilon }\in \mathbb {X}_{\varepsilon }\) of \(\mathcal {J}_{\varepsilon }\). By Remark 2.3, we deduce that \(u_{\varepsilon }\) is a ground state solution to (2.1). Using \(\langle \mathcal {J}'_{\varepsilon }(u_{\varepsilon }), u^{-}_{\varepsilon }\rangle =0\), where \(u^{-}=\min \{u, 0\}\), \((V_1)\), \(g(\cdot , t)=0\) for \(t\le 0\) and (2.2), we have

$$\begin{aligned} C(\Vert u^{-}_{\varepsilon }\Vert _{W^{s, p}(\mathbb {R}^{N})}^{p} + \Vert u^{-}_{\varepsilon }\Vert _{W^{s, q}(\mathbb {R}^{N})}^{q})\le 0, \end{aligned}$$

which gives \(u^{-}_{\varepsilon }=0\), that is \(u_{\varepsilon }\ge 0\) in \(\mathbb {R}^{N}\). Arguing as in the proof of Lemma 5.1 below (see also Lemma 4.1 and Theorem 2.2 in [11]), we obtain that \(u_{\varepsilon }\in L^{\infty }(\mathbb {R}^{N})\cap C(\mathbb {R}^{N})\), and applying the strong maximum principle [7] we infer that \(u_{\varepsilon }>0\) in \(\mathbb {R}^{N}\). \(\square \)

3 The Limiting Kirchhoff Problem

Since we are interested in providing a multiplicity result for the auxiliary problem (2.1), it is important to analyze the limiting problem associated with (1.1), namely

$$\begin{aligned} \left\{ \begin{array}{ll} \left( 1+ [u]_{s.p}^{p} \right) (-\Delta )^{s}_{p}u +\left( 1+ [u]_{s, q}^{q} \right) (-\Delta )^{s}_{q}u + V_{0} (|u|^{p-2}u+|u|^{q-2}u)=f(u) &{} \text{ in } \mathbb {R}^{N},\\ u\in W^{s, p}(\mathbb {R}^{N})\cap W^{s, q}(\mathbb {R}^{N}), \quad u>0 \text{ in } \mathbb {R}^{N}. \end{array} \right. \end{aligned}$$
(3.1)

Let \(\mathbb {Y}_{V_{0}}=W^{s, p}(\mathbb {R}^{N})\cap W^{s, q}(\mathbb {R}^{N})\) equipped with the norm

$$\begin{aligned} \Vert u\Vert _{\mathbb {Y}_{V_{0}}}= \Vert u\Vert _{s, p}+ \Vert u\Vert _{s, q}, \end{aligned}$$

where

$$\begin{aligned} \Vert u\Vert _{s, t}= \left( [u]_{s, t}^{t}+V_{0} |u|_{t}^{t} \,\right) ^{\frac{1}{t}} \quad \text{ for } t\in \{p, q\}. \end{aligned}$$

The energy functional \(\mathcal {L}_{V_{0}}: \mathbb {Y}_{V_{0}}\rightarrow \mathbb {R}\) associated with (3.1) is given by

$$\begin{aligned} \mathcal {L}_{V_{0}}(u)= \frac{1}{p}\Vert u\Vert _{s, p}^{p}+\frac{1}{q}\Vert u\Vert _{s, q}^{q}+\frac{1}{2p}[u]^{2p}_{s, p}+\frac{1}{2q}[u]^{2q}_{s, q}-\int _{\mathbb {R}^{N}} F(u) \, dx. \end{aligned}$$

Standard arguments show that \(\mathcal {L}_{V_{0}}\in C^{1}(\mathbb {Y}_{V_{0}}, \mathbb {R})\) and that

$$\begin{aligned} \langle \mathcal {L}'_{V_{0}}(u), \varphi \rangle&=(1+[u]_{s, p}^{p}) \langle u, \varphi \rangle _{s, p}+(1+[u]_{s, q}^{q})\langle u, \varphi \rangle _{s, q} \\&\quad +V_{0} \left[ \int _{\mathbb {R}^{N}} |u|^{p-2}u\,\varphi \, dx+ \int _{\mathbb {R}^{N}} |u|^{q-2}u\,\varphi \, dx \right] - \int _{\mathbb {R}^{N}} f(u)\varphi \, dx \end{aligned}$$

for any \(u, \varphi \in \mathbb {Y}_{V_{0}}\). We also consider the Nehari manifold \(\mathcal {M}_{V_{0}}\) associated with \(\mathcal {L}_{V_{0}}\), that is

$$\begin{aligned} \mathcal {M}_{V_{0}} = \{u\in \mathbb {Y}_{V_{0}}\setminus \{0\} : \langle \mathcal {L}'_{V_{0}}(u), u\rangle =0\}, \end{aligned}$$

and we set \(d_{V_{0}}=\inf _{u\in \mathcal {M}_{V_{0}}} \mathcal {L}_{V_{0}}(u)\). Now we define

$$\begin{aligned} \mathbb {Y}_{V_{0}}^{+}=\{u\in \mathbb {Y}_{V_{0}}: |{{\,\mathrm{supp}\,}}(u^{+})|>0\}, \end{aligned}$$

and \(\mathbb {S}_{V_{0}}^{+}= \mathbb {S}_{V_{0}}\cap \mathbb {Y}_{V_{0}}^{+}\), where \(\mathbb {S}_{V_{0}}\) is the unit sphere of \(\mathbb {Y}_{V_{0}}\). As in Sect. 2, \(\mathbb {S}_{V_{0}}^{+}\) is an incomplete \(C^{1,1}\)-manifold of codimension one and contained in \(\mathbb {Y}_{V_{0}}^{+}\). Thus, \(\mathbb {Y}_{V_{0}}= T_{u}\mathbb {S}_{V_{0}}^{+}\oplus \mathbb {R}u\) for each \(u\in \mathbb {S}_{V_{0}}^{+}\), where

$$\begin{aligned}&T_{u}\mathbb {S}_{V_{0}}^{+}=\Bigl \{v\in \mathbb {Y}_{V_{0}}: (1+[u]_{s, p}^{p}) \langle u, v\rangle _{s, p}+(1+[u]_{s, q}^{q})\langle u, v \rangle _{s, q} \\&\qquad \qquad +V_{0}\int _{\mathbb {R}^{N}} (|u|^{p-2}uv+|u|^{q-2}uv) \, dx=0 \Bigr \}. \end{aligned}$$

In the sequel, we state without proofs the following results which can be obtained arguing as in Sect. 2.

Lemma 3.1

Assume that \((f_1)\)\((f_4)\) hold. Then we have the following properties:

(i):

For each \(u\in \mathbb {Y}_{V_{0}}^{+}\), let \(h:\mathbb {R}^{+}\rightarrow \mathbb {R}\) be defined by \(h_{u}(t)= \mathcal {L}_{V_{0}}(tu)\). Then, there is a unique \(t_{u}>0\) such that

$$\begin{aligned}&h'_{u}(t)>0 \, \text{ for } \text{ all } t\in (0, t_{u}),\\&h'_{u}(t)<0 \, \text{ for } \text{ all } t\in (t_{u}, \infty ). \end{aligned}$$
(ii):

There exists \(\tau >0\), independent of u, such that \(t_{u}\ge \tau \) for any \(u\in \mathbb {S}_{V_{0}}^{+}\). Moreover, for each compact set \(\mathbb {K}\subset \mathbb {S}_{V_{0}}^{+}\), there is a constant \(C_{\mathbb {K}}>0\) such that \(t_{u}\le C_{\mathbb {K}}\) for any \(u\in \mathbb {K}\).

(iii):

The map \({\hat{m}}_{V_{0}}: \mathbb {Y}_{V_{0}}^{+}\rightarrow \mathcal {M}_{V_{0}}\) given by \({\hat{m}}_{V_{0}}(u)= t_{u}u\) is continuous and \(m_{V_{0}}= {\hat{m}}_{V_{0}}|_{\mathbb {S}_{V_{0}}^{+}}\) is a homeomorphism between \(\mathbb {S}_{V_{0}}^{+}\) and \(\mathcal {M}_{V_{0}}\). Moreover, \(m_{V_{0}}^{-1}(u)=\frac{u}{\Vert u\Vert _{\mathbb {Y}_{V_{0}}}}\).

(iv):

If there is a sequence \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {S}_{V_{0}}^{+}\) such that \(\mathrm{dist}(u_{n}, \partial \mathbb {S}_{V_{0}}^{+})\rightarrow 0\), then \(\Vert m_{V_{0}}(u_{n})\Vert _{\mathbb {Y}_{V_{0}}}\rightarrow \infty \) and \(\mathcal {L}_{V_{0}}(m_{V_{0}}(u_{n}))\rightarrow \infty \).

Let us consider the maps

$$\begin{aligned} {\hat{\psi }}_{V_{0}}: \mathbb {Y}_{V_{0}}^{+} \rightarrow \mathbb {R}\quad \text{ and } \quad \psi _{V_{0}}: \mathbb {S}_{V_{0}}^{+}\rightarrow \mathbb {R}, \end{aligned}$$

defined by \({\hat{\psi }}_{V_{0}}(u)= \mathcal {L}_{V_{0}}({\hat{m}}_{V_{0}}(u))\) and \(\psi _{V_{0}}={\hat{\psi }}_{V_{0}}|_{\mathbb {S}_{V_{0}}^{+}}\).

Proposition 3.1

Assume that \((f_{1})\)-\((f_{4})\) hold. Then we have the following properties:

(a):

\({\hat{\psi }}_{V_{0}} \in C^{1}(\mathbb {Y}_{V_{0}}^{+}, \mathbb {R})\) and

$$\begin{aligned} \langle {\hat{\psi }}_{V_{0}}'(u), v\rangle = \frac{\Vert {\hat{m}}_{V_{0}}(u)\Vert _{\mathbb {Y}_{V_{0}}}}{\Vert u\Vert _{\mathbb {Y}_{V_{0}}}} \langle \mathcal {L}_{V_{0}}'({\hat{m}}_{V_{0}}(u)), v\rangle \quad \text{ for } \text{ all } u\in \mathbb {Y}_{V_{0}}^{+} \text{ and } v\in \mathbb {Y}_{V_{0}}. \end{aligned}$$
(b):

\(\psi _{V_{0}} \in C^{1}(\mathbb {S}_{V_{0}}^{+}, \mathbb {R})\) and

$$\begin{aligned} \langle \psi _{V_{0}}'(u), v \rangle = \Vert m_{V_{0}}(u)\Vert _{\mathbb {Y}_{V_{0}}} \langle \mathcal {L}_{V_{0}}'(m_{V_{0}}(u)), v\rangle \quad \text{ for } \text{ all } v\in T_{u}\mathbb {S}_{V_{0}}^{+}. \end{aligned}$$
(c):

If \(\{u_{n}\}_{n\in \mathbb {N}}\) is a \((PS)_{d}\) sequence for \(\psi _{V_{0}}\), then \(\{m_{V_{0}}(u_{n})\}_{n\in \mathbb {N}}\) is a \((PS)_{d}\) sequence for \(\mathcal {L}_{V_{0}}\). If \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathcal {M}_{V_{0}}\) is a bounded \((PS)_{d}\) sequence for \(\mathcal {L}_{V_{0}}\), then \(\{m_{V_{0}}^{-1}(u_{n})\}_{n\in \mathbb {N}}\) is a \((PS)_{d}\) sequence for \(\psi _{V_{0}}\).

(d):

u is a critical point of \(\psi _{V_{0}}\) if and only if \(m_{V_{0}}(u)\) is a nontrivial critical point for \(\mathcal {L}_{V_{0}}\). Moreover, the corresponding critical values coincide and

$$\begin{aligned} \inf _{u\in \mathbb {S}_{V_{0}}^{+}} \psi _{V_{0}}(u)= \inf _{u\in \mathcal {M}_{V_{0}}} \mathcal {L}_{V_{0}}(u). \end{aligned}$$

Remark 3.1

As in Sect. 2, we have the following minimax characterization of the infimum of \(\mathcal {L}_{V_{0}}\) over \(\mathcal {M}_{V_{0}}\):

$$\begin{aligned} 0<d_{V_{0}}&=\inf _{u\in \mathcal {M}_{V_{0}}} \mathcal {L}_{V_{0}}(u)=\inf _{u\in \mathbb {Y}_{V_{0}}^{+}} \max _{t>0} \mathcal {L}_{V_{0}}(tu)=\inf _{u\in \mathbb {S}_{V_{0}}^{+}} \max _{t>0} \mathcal {L}_{V_{0}}(tu). \end{aligned}$$

The lemma below allows us to assume that the weak limit of a \((PS)_{d_{V_{0}}}\) sequence of \(\mathcal {L}_{V_{0}}\) is nontrivial.

Lemma 3.2

Let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {Y}_{V_{0}}\) be a \((PS)_{d_{V_{0}}}\) sequence for \(\mathcal {L}_{V_{0}}\) such that \(u_{n}\rightharpoonup 0\) in \(\mathbb {Y}_{V_{0}}\). Then we have either

(a):

\(u_{n}\rightarrow 0\) in \(\mathbb {Y}_{V_{0}}\), or

(b):

there exists a sequence \(\{y_{n}\}_{n\in \mathbb {N}}\subset \mathbb {R}^{N}\) and constants \(R, \beta >0\) such that

$$\begin{aligned} \liminf _{n\rightarrow \infty }\int _{B_{R}(y_{n})} |u_{n}|^{q} \, dx \ge \beta . \end{aligned}$$

Proof

Suppose that (b) is false. Since \(\{u_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {Y}_{V_{0}}\), we can use Lemma 2.1 to see that

$$\begin{aligned} u_{n}\rightarrow 0 \quad \text{ in } L^{r}(\mathbb {R}^{N}) \quad \text{ for } \text{ all } r \in (p, q^{*}_{s}). \end{aligned}$$

Moreover, by \((f_{1})\) and \((f_{2})\), we have that

$$\begin{aligned} \int _{\mathbb {R}^{N}} f(u_{n})u_{n}\, dx =o_{n}(1) \quad \text{ as } n\rightarrow \infty . \end{aligned}$$

Since \(\langle \mathcal {L}'_{V_{0}}(u_{n}), u_{n}\rangle =o_{n}(1)\), we get

$$\begin{aligned} \Vert u_{n}\Vert _{s,p}^{p}+\Vert u_{n}\Vert _{s,q}^{q}\le \int _{\mathbb {R}^{N}} f(u_{n}) u_{n}\, dx=o_{n}(1), \end{aligned}$$

that is \(\Vert u_{n}\Vert _{\mathbb {Y}_{V_{0}}}\rightarrow 0\) as \(n\rightarrow \infty \). Then, (a) is true. \(\square \)

Remark 3.2

As it has been mentioned earlier, if \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {Y}_{V_{0}}\) is a \((PS)_{d_{V_{0}}}\) sequence for \(\mathcal {L}_{V_{0}}\) such that \(u_{n}\rightharpoonup u\) in \(\mathbb {Y}_{V_{0}}\), then we may assume that \(u\ne 0\). Otherwise, if \(u_{n}\rightharpoonup 0\) in \(\mathbb {Y}_{V_{0}}\) and, if \(u_{n}\nrightarrow 0\) in \(\mathbb {Y}_{V_{0}}\), it follows from Lemma 3.2 that there are \(\{y_{n}\}_{n\in \mathbb {N}}\subset \mathbb {R}^{N}\) and \(R, \beta >0\) such that

$$\begin{aligned} \liminf _{n\rightarrow \infty }\int _{B_{R}(y_{n})} |u_{n}|^{q} \, dx \ge \beta . \end{aligned}$$

Define \(v_{n}(x)=u_{n}(x+y_{n})\). Then, using the invariance of \(\mathbb {R}^N\) by translation, we see that \(\{v_{n}\}_{n\in \mathbb {N}}\) is a bounded \((PS)_{d_{V_{0}}}\) sequence for \(\mathcal {L}_{V_{0}}\) such that \(v_{n}\rightharpoonup v\) in \(\mathbb {Y}_{V_{0}}\) with \(v\ne 0\).

In the following lemma, we obtain a positive ground state solution for the autonomous problem (3.1).

Theorem 3.1

Let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {Y}_{V_{0}}\) be a \((PS)_{d_{V_{0}}}\) sequence of \(\mathcal {L}_{V_{0}}\). Then there exists \(u\in \mathbb {Y}_{V_{0}}\setminus \{0\}\), with \(u\ge 0\), such that, up to a subsequence, \(u_{n}\rightarrow u\) in \(\mathbb {Y}_{V_{0}}\). Moreover, u is a positive ground state solution to (3.1).

Proof

Proceeding as in the proof of Lemma 2.5, we can verify that \(\{u_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {Y}_{V_{0}}\). By passing to a subsequence if necessary, we may assume that

$$\begin{aligned} \begin{aligned} \begin{array}{ll} u_{n}\rightharpoonup u &{} \text{ in } \mathbb {Y}_{V_{0}}, \\ u_{n}\rightarrow u &{} \text{ in } L^{r}_{loc}(\mathbb {R}^{N}) \quad \text{ for } \text{ all } r\in [1, p^{*}_{s}). \end{array} \end{aligned} \end{aligned}$$
(3.2)

From Remark 3.2, we may suppose that \(u\ne 0\). Moreover, we may assume that \([u_{n}]^{p}_{s, p}\rightarrow t_{1}\) and \([u_{n}]^{q}_{s, q}\rightarrow t_{2}\). Our aim is to prove that \([u_{n}]_{s, t}\rightarrow [u]_{s, t}\) for \(t\in \{p, q\}\). By Fatou’s lemma, we know that \([u]_{s, p}^{p}\le t_{1}\) and \([u]_{s, q}^{q}\le t_{2}\). Now we show that \([u]_{s, p}^{p}= t_{1}\) and \([u]_{s, q}^{q}= t_{2}\). Assume, by contradiction, that \([u]_{s, p}^{p}<t_{1}\) and \([u]_{s, q}^{q}\le t_{2}\). Since \(\langle \mathcal {L}'_{V_{0}}(u_{n}), \varphi \rangle \rightarrow 0\) for all \(\varphi \in C^{\infty }_{c}(\mathbb {R}^{N})\), and \(C^{\infty }_{c}(\mathbb {R}^{N})\) is dense in \(\mathbb {Y}_{V_{0}}\) (see [19]), we can deduce that

$$\begin{aligned} (1+t_{1})[u]_{s, p}^{p}+(1+t_{2})[u]_{s, q}^{q}+V_{0}(|u|_{p}^{p}+|u|_{q}^{q})=\int _{\mathbb {R}^{N}} f(u)u\, dx. \end{aligned}$$

Therefore,

$$\begin{aligned}&(1+[u]_{s, p}^{p})[u]_{s, p}^{p}+(1+[u]_{s, q}^{q})[u]_{s, q}^{q}+V_{0}(|u|_{p}^{p}+|u|_{q}^{q})-\int _{\mathbb {R}^{N}} f(u)u\, dx\\&\qquad <(1+t_{1})[u]_{s, p}^{p}+(1+t_{2})[u]_{s, q}^{q}+V_{0}(|u|_{p}^{p}+|u|_{q}^{q})-\int _{\mathbb {R}^{N}} f(u)u\, dx=0, \end{aligned}$$

that is \(\langle \mathcal {L}'_{V_{0}}(u), u\rangle <0\). From \((f_{1})\) and \((f_{2})\), we have \(\langle \mathcal {L}'_{V_{0}}(t_{0}u), t_{0}u\rangle >0\) for some \(0<t_{0}\ll 1\). Hence, there exists \(\tau \in (t_{0}, 1)\) such that \(\langle \mathcal {L}'_{V_{0}}(\tau u), \tau u\rangle =0\). Combining this fact with the characterization of \(d_{V_{0}}\) and using the fact that \(t\mapsto \frac{1}{2q} f(t)t-F(t)\) is increasing (thanks to \((f_3)\) and \((f_4)\)), by Fatou’s lemma, we get

$$\begin{aligned} d_{V_{0}}\le \mathcal {L}_{V_{0}}(\tau u)&= \mathcal {L}_{V_{0}}(\tau u)- \frac{1}{2q}\langle \mathcal {L}_{V_{0}}'(\tau u), \tau u\rangle \\&<\mathcal {L}_{V_{0}}(u)- \frac{1}{2q}\langle \mathcal {L}_{V_{0}}'(u), u\rangle \\&\le \liminf _{n\rightarrow \infty } \left[ \mathcal {L}_{V_{0}}(u_{n})- \frac{1}{2q}\langle \mathcal {L}_{V_{0}}'(u_{n}), u_{n}\rangle \right] = d_{V_{0}} \end{aligned}$$

and we arrive at a contradiction. Hence, \([u_{n}]_{s, t}\rightarrow [u]_{s, t}\) for \(t\in \{p, q\}\), and we obtain \(\mathcal {L}'_{V_{0}}(u)=0\). Finally, we prove that u is positive in \(\mathbb {R}^N\). Since \(\langle \mathcal {L}'_{V_{0}}(u), u^{-}\rangle =0\), where \(u^{-}=\min \{u, 0\}\), and \(f(t)=0\) for \(t\le 0\), we have

$$\begin{aligned} \Vert u^{-}\Vert _{s, p}^{p} + \Vert u^{-}\Vert _{s, q}^{q}\le 0 \end{aligned}$$

which implies that \(u^{-}=0\), that is \(u\ge 0\) in \(\mathbb {R}^{N}\). Thus, \(u\ge 0\) and \(u\not \equiv 0\) in \(\mathbb {R}^{N}\). Using a Moser iteration argument [32] (see the proof of Lemma 5.1 below), we obtain that \(u\in L^{\infty }(\mathbb {R}^{N})\). Since u solves

$$\begin{aligned} \alpha _{u} (-\Delta )^{s}_{p}u+\beta _{u} (-\Delta )^{s}_{q}u=-V_{0}(u^{p-1}+u^{q-1})+f(u)\in L^{\infty }(\mathbb {R}^{N}), \end{aligned}$$

where \(\alpha _{u}=1+[u]_{s,p}^{p}\) and \(\beta _{u}=1+[u]_{s, q}^{q}\) are bounded quantities, we can argue as in the proof of Theorem 2.2 in [11] to infer that \(u \in C^{0, \alpha }(\mathbb {R}^{N})\). In particular, \(u(x)\rightarrow 0\) as \(|x|\rightarrow \infty \). By using the strong maximum principle [7], we deduce that \(u>0\) in \(\mathbb {R}^{N}\). \(\square \)

The next lemma is a compactness result for the autonomous problem (3.1).

Lemma 3.3

Let \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathcal {M}_{V_{0}}\) be a sequence such that \(\mathcal {L}_{V_{0}}(u_{n})\rightarrow d_{V_{0}}\). Then, \(\{u_{n}\}_{n\in \mathbb {N}}\) has a convergent subsequence in \(\mathbb {Y}_{V_{0}}\).

Proof

Since \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathcal {M}_{V_{0}}\) and \(\mathcal {L}_{V_{0}}(u_{n})\rightarrow d_{V_{0}}\), it follows from Lemma 3.1-(iii), Proposition 3.1-(d) and the definition of \(d_{V_{0}}\) that

$$\begin{aligned} v_{n}=m^{-1}_{V_{0}}(u_{n})=\frac{u_{n}}{\Vert u_{n}\Vert _{\mathbb {Y}_{V_{0}}}}\in \mathbb {S}_{V_{0}}^{+} \quad \text{ for } \text{ all } n\in \mathbb {N}, \end{aligned}$$

and

$$\begin{aligned} \psi _{V_{0}}(v_{n})=\mathcal {L}_{V_{0}}(u_{n})\rightarrow d_{V_{0}}=\inf _{v\in \mathbb {S}_{V_{0}}^{+}}\psi _{V_{0}}(v). \end{aligned}$$

Let us define \(\mathcal {G}: \overline{\mathbb {S}}_{V_{0}}^{+}\rightarrow \mathbb {R}\cup \{\infty \}\) by

$$\begin{aligned} \mathcal {G}(u)= {\left\{ \begin{array}{ll} \psi _{V_{0}}(u)&{} \text { if} u\in \mathbb {S}_{V_{0}}^{+}, \\ \infty &{} \hbox { if}\ u\in \partial \mathbb {S}_{V_{0}}^{+}. \end{array}\right. } \end{aligned}$$

We observe that the following properties hold:

  • \((\overline{\mathbb {S}}_{V_{0}}^{+}, \delta _{V_{0}})\), where \(\delta _{V_{0}}(u, v)=\Vert u-v\Vert _{\mathbb {Y}_{V_{0}}}\), is a complete metric space.

  • \(\mathcal {G}\in C(\overline{\mathbb {S}}_{V_{0}}^{+}, \mathbb {R}\cup \{\infty \})\), by Lemma 3.1-(iv).

  • \(\mathcal {G}\) is bounded below, by Proposition 3.1-(d).

By using the Ekeland variational principle [21], there exists \(\{{\hat{v}}_{n}\}_{n\in \mathbb {N}}\subset \mathbb {S}_{V_{0}}^{+}\) such that \(\{{\hat{v}}_{n}\}_{n\in \mathbb {N}}\) is a \((PS)_{d_{V_{0}}}\) sequence for \(\psi _{V_{0}}\) and \(\Vert {\hat{v}}_{n}-v_{n}\Vert _{\mathbb {Y}_{V_{0}}}=o_{n}(1)\). Now the remainder of the proof follows from Proposition 3.1, Theorem 3.1, and arguing as in the proof of Corollary 2.1. \(\square \)

We conclude this section by showing a useful relation between the minimax levels \(c_{\varepsilon }\) and \(d_{V_{0}}\).

Lemma 3.4

It holds \(\lim _{\varepsilon \rightarrow 0} c_{\varepsilon }=d_{V_{0}}\).

Proof

For \(\varepsilon >0\), let \(\omega _{\varepsilon }(x)= \psi _{\varepsilon }(x)\omega (x)\), where \(\omega \) is a positive ground state of (3.1) (whose existence is guaranteed by Theorem 3.1), and \(\psi _{\varepsilon }(x)= \psi (\varepsilon x)\) with \( \psi \in C^{\infty }_{c}(\mathbb {R}^{N})\) such that \(0\le \psi \le 1\), \( \psi (x)=1\) if \(|x|\le 1\) and \( \psi (x)=0\) if \(|x|\ge 2\). For simplicity, we assume that \(\mathrm{supp}( \psi )\subset B_{2}\subset \Lambda \). Using the dominated convergence theorem, we see that

$$\begin{aligned} \omega _{\varepsilon }\rightarrow \omega \quad \text{ in } \mathcal {W} \quad \text{ and } \quad \mathcal {L}_{V_{0}}(\omega _{\varepsilon }) \rightarrow \mathcal {L}_{V_{0}}(\omega )= d_{V_{0}} \end{aligned}$$
(3.3)

as \(\varepsilon \rightarrow 0\). Now, for each \(\varepsilon >0\), there exists \(t_{\varepsilon }>0\) such that

$$\begin{aligned} \mathcal {J}_{\varepsilon }(t_{\varepsilon } \omega _{\varepsilon })=\max _{t\ge 0} \mathcal {J}_{\varepsilon }(t \omega _{\varepsilon }). \end{aligned}$$

Therefore, \(\langle \mathcal {J}_{\varepsilon }'(t_{\varepsilon } \omega _{\varepsilon }), \omega _{\varepsilon }\rangle =0\) and this implies that

$$\begin{aligned}&t^{p}_{\varepsilon }[\omega _{\varepsilon }]_{s, p}^{p}+t^{2p}_{\varepsilon }[\omega _{\varepsilon }]_{s, p}^{2p}+t^{q}_{\varepsilon }[\omega _{\varepsilon }]_{s, q}^{q}\\&\quad +t^{2q}_{\varepsilon }[\omega _{\varepsilon }]_{s, q}^{2q}+ t^{p}_{\varepsilon }\int _{\mathbb {R}^{N}} V(\varepsilon x)\omega ^{p}_{\varepsilon } \,dx+t^{q}_{\varepsilon }\int _{\mathbb {R}^{N}} V(\varepsilon x)\omega ^{q}_{\varepsilon } \,dx\\&\quad =\int _{\mathbb {R}^{N}} f(t_{\varepsilon } \omega _{\varepsilon })t_{\varepsilon }\omega _{\varepsilon }\, dx. \end{aligned}$$

If \(t_{\varepsilon }\rightarrow \infty \), then

$$\begin{aligned}&t^{p-2q}_{\varepsilon }[\omega _{\varepsilon }]_{s, p}^{p}+t^{2p-2q}_{\varepsilon }[\omega _{\varepsilon }]_{s, p}^{2p} \nonumber \\&+t^{-q}[\omega _{\varepsilon }]_{s, q}^{q}+ [\omega _{\varepsilon }]_{s, q}^{2q}+t^{p-2q}_{\varepsilon }\int _{\mathbb {R}^{N}} V(\varepsilon x)\omega ^{p}_{\varepsilon } \,dx+t^{-q}\int _{\mathbb {R}^{N}} V(\varepsilon x)\omega ^{q}_{\varepsilon } \,dx \nonumber \\&=\int _{\mathbb {R}^{N}} \frac{f(t_{\varepsilon } \omega _{\varepsilon })}{(t_{\varepsilon } \omega _{\varepsilon })^{2q-1}}\omega _{\varepsilon }^{2q}\, dx, \end{aligned}$$
(3.4)

and using (3.3), \(p<2q\) and \((f_3)\), we obtain that \([\omega ]^{2q}_{s, q}=\infty \), which is impossible. Then, \(t_{\varepsilon }\rightarrow t_{0}\in [0, \infty )\). If \(t_{0}=0\), using \((f_1)\) and \((f_2)\), we see that, for \(\zeta \in (0, V_{0})\), it holds

$$\begin{aligned} \left( 1-\frac{\zeta }{V_{0}}\right) \Vert \omega _{\varepsilon }\Vert ^{p}_{V_{\varepsilon , p}}+t^{q-p}_{\varepsilon } \Vert \omega _{\varepsilon }\Vert ^{q}_{V_{\varepsilon , q}}\le C_{\zeta }t^{q-p}_{\varepsilon } \Vert \omega _{\varepsilon }\Vert ^{q^{*}_{s}}_{V_{\varepsilon , q}}. \end{aligned}$$

This together with \(q>p\) yields \(\Vert \omega \Vert ^{p}_{s, p}=0\), that is a contradiction. Hence, \(t_{\varepsilon }\rightarrow t_{0}\in (0, \infty )\).

Taking the limit as \(\varepsilon \rightarrow 0\) in (3.4), we get

$$\begin{aligned}&t^{p-2q}_{0}[\omega ]^{p}_{s, p}+t_{0}^{2p-2q}[\omega ]_{s, p}^{2p}+t^{-q}_{0}[\omega ]^{q}_{s, q}+[\omega ]_{s, q}^{2q}\\&\quad +t^{p-2q}_{0}\int _{\mathbb {R}^{N}} V_{0} \omega ^{p} \,dx+t^{-q}_{0}\int _{\mathbb {R}^{N}} V_{0} \omega ^{q} \,dx =\int _{\mathbb {R}^{N}} \frac{f(t_{0}\omega )}{(t_{0}\omega )^{2q-1}}\omega ^{2q}\,dx, \end{aligned}$$

which combined with \(2q>q>p\), \((f_{4})\) and \(\omega \in \mathcal {M}_{V_{0}}\), implies that \(t_{0}=1\).

Now, we note that

$$\begin{aligned}&c_{\varepsilon }\le \max _{t\ge 0} \mathcal {J}_{\varepsilon }(t \omega _{\varepsilon })=\mathcal {J}_{\varepsilon }(t_{\varepsilon } \omega _{\varepsilon })= \mathcal {\mathcal {L}}_{V_{0}}(t_{\varepsilon } \omega _{\varepsilon }) + \frac{t_{\varepsilon }^{p}}{p} \int _{\mathbb {R}^{N}} (V(\varepsilon x) - V_{0} ) \omega _{\varepsilon }^{p} \, dx \\&\quad +\frac{t_{\varepsilon }^{q}}{q} \int _{\mathbb {R}^{N}} (V(\varepsilon x) - V_{0}) \omega _{\varepsilon }^{q} \, dx. \end{aligned}$$

Since \(V(\varepsilon \cdot )\) is bounded on the support of \(\omega _{\varepsilon }\), we can use the dominated convergence theorem, (3.3) and the above inequality to deduce that \(\limsup _{\varepsilon \rightarrow 0}c_{\varepsilon }\le d_{V_{0}}\). By \((V_1)\), we obtain that \(\liminf _{\varepsilon \rightarrow 0}c_{\varepsilon }\ge d_{V_{0}}\), and thus \(\lim _{\varepsilon \rightarrow 0}c_{\varepsilon }= d_{V_{0}}\). This completes the proof. \(\square \)

4 A Multiplicity Result for (2.1)

In this section, we deal with the multiplicity of solutions to (2.1). Let \(\delta >0\) be such that

$$\begin{aligned} M_{\delta }=\{x\in \mathbb {R}^{N}: \mathrm{dist}(x, M)\le \delta \}\subset \Lambda , \end{aligned}$$

and let \(w\in \mathbb {Y}_{V_{0}}\) be a positive ground state solution to (3.1) (by virtue of Theorem 3.1).

Consider a nonincreasing function \(\eta \in C^{\infty }([0, \infty ), [0, 1])\) such that \(\eta (t)=1\) if \(0\le t\le \frac{\delta }{2}\), \(\eta (t)=0\) if \(t\ge \delta \) and \(|\eta '(t)|\le c\) for some \(c>0\). For any \(y\in M\), we define

$$\begin{aligned} \Psi _{\varepsilon , y}(x)=\eta (|\varepsilon x-y|) w\left( \frac{\varepsilon x-y}{\varepsilon }\right) . \end{aligned}$$

Let \(\Phi _{\varepsilon }: M\rightarrow \mathcal {N}_{\varepsilon }\) be given by

$$\begin{aligned} \Phi _{\varepsilon }(y)= t_{\varepsilon } \Psi _{\varepsilon , y}, \end{aligned}$$

where \(t_{\varepsilon }>0\) satisfies

$$\begin{aligned} \max _{t\ge 0} \mathcal {J}_{\varepsilon }(t \Psi _{\varepsilon , y})=\mathcal {J}_{\varepsilon }(t_{\varepsilon } \Psi _{\varepsilon , y}). \end{aligned}$$

By construction, \(\Phi _{\varepsilon }(y)\) has compact support for any \(y\in M\).

Lemma 4.1

The function \(\Phi _{\varepsilon }\) has the following property:

$$\begin{aligned} \lim _{\varepsilon \rightarrow 0} \mathcal {J}_{\varepsilon }(\Phi _{\varepsilon }(y))=d_{V_0} \quad \text{ uniformly } \text{ in } y\in M. \end{aligned}$$

Proof

Assume, by contradiction, that there exist \(\delta _{0}>0\), \(\{y_{n}\}_{n\in \mathbb {N}}\subset M\) and \(\varepsilon _{n}\rightarrow 0\) such that

$$\begin{aligned} |\mathcal {J}_{\varepsilon _{n}}(\Phi _{\varepsilon _{n}}(y_{n}))-d_{V_0}|\ge \delta _{0}. \end{aligned}$$
(4.1)

For each \(n\in \mathbb {N}\) and for all \(z\in B_{\frac{\delta }{\varepsilon _{n}}}(0)\), we have \(\varepsilon _{n} z\in B_{\delta }(0)\), and thus

$$\begin{aligned} \varepsilon _{n} z+y_{n}\in B_{\delta }(y_{n})\subset M_{\delta }\subset \Lambda . \end{aligned}$$

Using the change of variable \(z=\frac{\varepsilon _{n}x-y_{n}}{\varepsilon _{n}}\) and the fact that \(G=F\) in \(\Lambda \times \mathbb {R}\), we can write

$$\begin{aligned} \mathcal {J}_{\varepsilon _{n}}(\Phi _{\varepsilon _{n}}(y_{n}))&=\frac{t^{p}_{\varepsilon _{n}}}{p} \Vert \Psi _{\varepsilon _{n}, y_{n}} \Vert ^{p}_{V_{\varepsilon _{n}},p}+\frac{t^{2p}_{\varepsilon _{n}}}{2p} [\Psi _{\varepsilon _{n}, y_{n}}]^{2p}_{s, p}+\frac{t^{q}_{\varepsilon _{n}}}{q} \Vert \Psi _{\varepsilon _{n}, y_{n}} \Vert ^{q}_{V_{\varepsilon _{n}},q}\nonumber \\&\quad +\frac{t^{2q}_{\varepsilon _{n}}}{2q} [\Psi _{\varepsilon _{n}, y_{n}}]^{2q}_{s, q} -\int _{\mathbb {R}^{N}} G(\varepsilon _{n} x, t_{\varepsilon _{n}} \Psi _{\varepsilon _{n}, y_{n}}) \, dx \nonumber \\&=\frac{t^{p}_{\varepsilon _{n}}}{p} \left( [\eta (|\varepsilon _{n}\cdot |) w]^{p}_{s, p}+\int _{\mathbb {R}^{N}} V(\varepsilon _{n} z+y_{n}) (\eta (|\varepsilon _{n} z|) w(z))^{p} \, dz\right) \nonumber \\&\quad + \frac{t^{2p}_{\varepsilon _{n}}}{2p}[\eta (|\varepsilon _{n}\cdot |) w]^{2p}_{s, p} \nonumber \\&\quad +\frac{t^{q}_{\varepsilon _{n}}}{q} \left( [\eta (|\varepsilon _{n}\cdot |) w]^{q}_{s, q}+\int _{\mathbb {R}^{N}} V(\varepsilon _{n} z+y_{n}) (\eta (|\varepsilon _{n} z|) w(z))^{q} \, dz\right) \nonumber \\&\quad + \frac{t^{2q}_{\varepsilon _{n}}}{2q}[\eta (|\varepsilon _{n}\cdot |) w]^{2q}_{s, q} \nonumber \\&\quad -\int _{\mathbb {R}^{N}} F(t_{\varepsilon _{n}} \eta (|\varepsilon _{n} z|) w(z)) \, dz. \end{aligned}$$
(4.2)

We claim that \(t_{\varepsilon _{n}}\rightarrow 1\) as \(n\rightarrow \infty \). We start by proving that \(t_{\varepsilon _{n}}\rightarrow t_{0}\in [0, \infty )\). Since \(\Phi _{\varepsilon _{n}}(y_{n})\in \mathcal {N}_{\varepsilon _{n}}\) and \(g=f\) on \(\Lambda \times \mathbb {R}\), we have

$$\begin{aligned}&\frac{1}{t_{\varepsilon _{n}}^{2q-p}} \Vert \Psi _{\varepsilon _{n}, y_{n}}\Vert _{V_{\varepsilon _{n}},p}^{p}+\frac{1}{t_{\varepsilon }^{q}}\Vert \Psi _{\varepsilon _{n}, y_{n}}\Vert _{V_{\varepsilon _{n}},q}^{q}+\frac{1}{t_{\varepsilon }^{2q-2p}} [\Psi _{\varepsilon _{n}, y_{n}}]^{2p}_{s, p}+[\Psi _{\varepsilon _{n}, y_{n}}]^{2q}_{s, q} \nonumber \\&\quad =\int _{\mathbb {R}^{N}} \Bigl [ \frac{f(t_{\varepsilon _{n}} \eta (|\varepsilon _{n} z|) w(z))}{(t_{\varepsilon _{n}} \eta (|\varepsilon _{n} z|) w(z))^{2q-1}}\Bigr ] (\eta (|\varepsilon _{n} z|) w(z))^{2q} \, dz. \end{aligned}$$
(4.3)

Observing that \(\eta (|x|)=1\) for \(x\in B_{\frac{\delta }{2}}(0)\) and that \(B_{\frac{\delta }{2}}(0)\subset B_{\frac{\delta }{\varepsilon _{n}}}(0)\) for all n large enough, the identity (4.3) yields

$$\begin{aligned}&\frac{1}{t_{\varepsilon _{n}}^{2q-p}} \Vert \Psi _{\varepsilon _{n}, y_{n}}\Vert _{V_{\varepsilon _{n}},p}^{p}+\frac{1}{t_{\varepsilon }^{q}}\Vert \Psi _{\varepsilon _{n}, y_{n}}\Vert _{V_{\varepsilon _{n}},q}^{q}+\frac{1}{t_{\varepsilon }^{2q-2p}} [\Psi _{\varepsilon _{n}, y_{n}}]^{2p}_{s, p}+[\Psi _{\varepsilon _{n}, y_{n}}]^{2q}_{s, q} \\&\quad \ge \int _{B_{\frac{\delta }{2}}(0)} \Bigl [ \frac{f(t_{\varepsilon _{n}} w(z))}{(t_{\varepsilon _{n}} w(z))^{2q-1}}\Bigr ] |w(z)|^{2q} \, dz, \end{aligned}$$

which together with \((f_4)\) gives

$$\begin{aligned}&\frac{1}{t_{\varepsilon _{n}}^{2q-p}} \Vert \Psi _{\varepsilon _{n}, y_{n}}\Vert _{V_{\varepsilon _{n}},p}^{p}+\frac{1}{t_{\varepsilon }^{q}}\Vert \Psi _{\varepsilon _{n}, y_{n}}\Vert _{V_{\varepsilon _{n}},q}^{q}+\frac{1}{t_{\varepsilon }^{2q-2p}} [\Psi _{\varepsilon _{n}, y_{n}}]^{2p}_{s, p}+[\Psi _{\varepsilon _{n}, y_{n}}]^{2q}_{s, q} \nonumber \\&\quad \ge \Bigl [ \frac{f(t_{\varepsilon _{n}} w({\hat{z}}))}{(t_{\varepsilon _{n}} w({\hat{z}}))^{2q-1}}\Bigr ] |w({\hat{z}})|^{2q} |B_{\frac{\delta }{2}}(0)|, \end{aligned}$$
(4.4)

where

$$\begin{aligned} w({\hat{z}})=\min _{z\in {\bar{B}}_{\frac{\delta }{2}}(0)} w(z)>0 \end{aligned}$$

(we recall that w is continuous and positive in \(\mathbb {R}^N\)). If \(t_{\varepsilon _{n}}\rightarrow \infty \), the dominated convergence theorem results in

$$\begin{aligned} \Vert \Psi _{\varepsilon _{n}, y_{n}} \Vert _{V_{\varepsilon _{n}},r}\rightarrow \Vert w\Vert _{s,r}\in (0, \infty ) \quad {{ \text{ for } \text{ all } r\in \{p,q\},}} \end{aligned}$$
(4.5)

and recalling that \(2q>q>p\), we also have

$$\begin{aligned}&\frac{1}{t_{\varepsilon _{n}}^{2q-p}} \Vert \Psi _{\varepsilon _{n}, y_{n}}\Vert _{V_{\varepsilon _{n}},p}^{p}+\frac{1}{t_{\varepsilon }^{q}}\Vert \Psi _{\varepsilon _{n}, y_{n}}\Vert _{V_{\varepsilon _{n}},q}^{q}+\frac{1}{t_{\varepsilon }^{2q-2p}} [\Psi _{\varepsilon _{n}, y_{n}}]^{2p}_{s, p}\nonumber \\&\quad +[\Psi _{\varepsilon _{n}, y_{n}}]^{2q}_{s, q}\rightarrow [\omega ]_{s, q}^{2q}. \end{aligned}$$
(4.6)

On the other hand, by \((f_3)\), we get

$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{f(t_{\varepsilon _{n}} w({\hat{z}}))}{(t_{\varepsilon _{n}} w({\hat{z}}))^{2q-1}}=\infty . \end{aligned}$$
(4.7)

Combining (4.4), (4.6) and (4.7), we achieve a contradiction. Consequently, \(\{t_{\varepsilon _{n}}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {R}\) and, up to a subsequence, we may assume that \(t_{\varepsilon _{n}}\rightarrow t_{0}\) for some \(t_{0}\in [0, \infty )\). From (4.3), (4.5), \((f_1)\), \((f_2)\), we can see that \(t_{0}\in (0, \infty )\). Now we prove that \(t_{0}=1\). Letting \(n\rightarrow \infty \) in (4.3), and using (4.5) and the dominated convergence theorem, we have that

$$\begin{aligned} t_{0}^{p-2q} \Vert w\Vert ^{p}_{s,p}+t_{0}^{2p-2q}[w]_{s, p}^{2p}+t_{0}^{-q}\Vert w\Vert _{s,q}^{q}+[w]_{s, q}^{2q}=\int _{\mathbb {R}^{N}} \frac{f(t_{0} w)}{(t_{0}w)^{2q-1}} \,w^{2q} \, dx. \end{aligned}$$

Since \(w\in \mathcal {M}_{V_0}\), it holds

$$\begin{aligned} \Vert w\Vert ^{p}_{s,p}+\Vert w\Vert _{s,q}^{q}+[w]_{s, p}^{2p}+[w]_{s, q}^{2q}=\int _{\mathbb {R}^{N}} f(w) w\, dx, \end{aligned}$$

Then we obtain

$$\begin{aligned}&(t_{0}^{p-2q}-1)\Vert w\Vert ^{p}_{s,p}+(t_{0}^{2p-2q}-1)[w]_{s, p}^{2p}+(t_{0}^{-q}-1)[w]_{s, q}^{2q}\\&\quad =\int _{\mathbb {R}^{N}} \left[ \frac{f(t_{0} w)}{(t_{0}w)^{2q-1}} -\frac{f(w)}{w^{2q-1}} \right] \,w^{2q} \, dx. \end{aligned}$$

Using \(2q>q>p\) and assumption \((f_4)\), we conclude that \(t_{0}=1\). Therefore, passing to the limit as \(n\rightarrow \infty \) in (4.2), we deduce that

$$\begin{aligned} \lim _{n\rightarrow \infty } \mathcal {J}_{\varepsilon _{n}}(\Phi _{\varepsilon _{n}, y_{n}})=\mathcal {L}_{V_0}(w)=d_{V_0}, \end{aligned}$$

which contradicts (4.1). \(\square \)

Let \(\rho =\rho (\delta )>0\) be such that \(M_{\delta }\subset B_{\rho }(0)\). Define \(\varUpsilon : \mathbb {R}^{N}\rightarrow \mathbb {R}^{N}\) by setting

$$\begin{aligned} \varUpsilon (x)= \left\{ \begin{array}{ll} x &{} \text{ if } |x|<\rho , \\ \frac{\rho x}{|x|} &{} \text{ if } |x|\ge \rho . \end{array} \right. \end{aligned}$$

Let us consider the barycenter map \(\beta _{\varepsilon }: \mathcal {N}_{\varepsilon }\rightarrow \mathbb {R}^{N}\) given by

$$\begin{aligned} \beta _{\varepsilon }(u)=\frac{\displaystyle {\int _{\mathbb {R}^{N}} \varUpsilon (\varepsilon x)(|u(x)|^{p}+|u(x)|^{q}) \,dx}}{\displaystyle {\int _{\mathbb {R}^{N}} (|u(x)|^{p}+|u(x)|^{q}) \,dx}}. \end{aligned}$$

Arguing as in the proof of Lemma 3.6 in [11], we can prove the following result.

Lemma 4.2

The function \(\beta _{\varepsilon }\) satisfies the following limit

$$\begin{aligned} \lim _{\varepsilon \rightarrow 0} \beta _{\varepsilon }(\Phi _{\varepsilon }(y))=y \quad \text{ uniformly } \text{ in } y\in M. \end{aligned}$$

The next compactness result plays an important role in showing that the solutions of the modified problem are also solutions of the original one.

Lemma 4.3

Let \(\varepsilon _{n}\rightarrow 0\) and \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathcal {N}_{\varepsilon _{n}}\) be such that \(\mathcal {J}_{\varepsilon _{n}}(u_{n})\rightarrow d_{V_0}\). Then there exists \(\{{\tilde{y}}_{n}\}_{n\in \mathbb {N}}\subset \mathbb {R}^{N}\) such that \(v_{n}(x)=u_{n}(x+ {\tilde{y}}_{n})\) has a convergent subsequence in \(\mathbb {Y}_{V_0}\). Moreover, up to a subsequence, \(\{y_{n}\}_{n\in \mathbb {N}}=\{\varepsilon _{n}{\tilde{y}}_{n}\}_{n\in \mathbb {N}}\) is such that \(y_{n}\rightarrow y_{0}\in M\).

Proof

Since \(\langle \mathcal {J}'_{\varepsilon _{n}}(u_{n}), u_{n} \rangle =0\) and \(\mathcal {J}_{\varepsilon _{n}}(u_{n})\rightarrow d_{V_0}\), we can argue as in the proof of Lemma 2.5 to verify that \(\{u_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {Y}_{V_{0}}\). According to \(d_{V_0}>0\), \(\Vert u_{n}\Vert _{\mathbb {X}_{\varepsilon _{n}}}\nrightarrow 0\). Then, proceeding as in the proof of Lemma 3.2, we obtain a sequence \(\{{\tilde{y}}_{n}\}_{n\in \mathbb {N}}\subset \mathbb {R}^{N}\) and constants \(R, \beta >0\) such that

$$\begin{aligned} \liminf _{n\rightarrow \infty }\int _{B_{R}({\tilde{y}}_{n})} |u_{n}|^{q} dx\ge \beta . \end{aligned}$$

Set \(v_{n}(x)=u_{n}(x+ {\tilde{y}}_{n})\). Thus, \(\{v_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {Y}_{V_0}\), and, up to a subsequence, we may assume that \(v_{n}\rightharpoonup v\not \equiv 0\) in \(\mathbb {Y}_{V_0}\). Let \(t_{n}\in (0, \infty )\) be such that \({\tilde{v}}_{n}=t_{n}v_{n} \in \mathcal {M}_{V_0}\), and set \(y_{n}=\varepsilon _{n}{\tilde{y}}_{n}\). From the definition of \(d_{V_{0}}\), \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathcal {N}_{\varepsilon _{n}}\), \((g_2)\) and \(\mathcal {J}_{\varepsilon _{n}}(u_{n})\rightarrow d_{V_{0}}\), we have

$$\begin{aligned} d_{V_0}&\le \mathcal {L}_{V_0}({\tilde{v}}_{n})\\&\le \frac{1}{p} [{\tilde{v}}_{n}]_{s, p}^{p}+\frac{1}{2p} [{\tilde{v}}_{n}]_{s, p}^{2p}+\frac{1}{q}[{\tilde{v}}_{n}]_{s, q}^{q}+\frac{1}{2q} [{\tilde{v}}_{n}]_{s, q}^{2q}\\&\quad +\int _{\mathbb {R}^{N}}V(\varepsilon _{n} x+y_{n}) \left( \frac{1}{p}|{\tilde{v}}_{n}|^{p}+\frac{1}{q}|{\tilde{v}}_{n}|^{q}\right) \, dx-\int _{\mathbb {R}^{N}} F({\tilde{v}}_{n})\, dx \\&\le \frac{t_{n}^{p}}{p} [u_{n}]_{s, p}^{p}+\frac{t_{n}^{2p}}{2p} [u_{n}]_{s, p}^{2p}+\frac{t_{n}^{q}}{q} [u_{n}]_{s, q}^{q}+\frac{t_{n}^{2q}}{2q} [u_{n}]_{s, q}^{2q}\\&\quad +\int _{\mathbb {R}^{N}}V(\varepsilon _{n} x) \left( \frac{t_{n}^{p}}{p}|u_{n}|^{p}+\frac{t_{n}^{q}}{q}|u_{n}|^{q}\right) \, dx \\&\quad -\int _{\mathbb {R}^{N}} G(\varepsilon _{n} x, t_{n}u_{n})\, dx \\&=\mathcal {J}_{\varepsilon _{n}}(t_{n}u_{n}) \le \mathcal {J}_{\varepsilon _{n}}(u_{n})= d_{V_0}+ o_{n}(1), \end{aligned}$$

which implies that

$$\begin{aligned} \mathcal {L}_{V_0}({\tilde{v}}_{n})\rightarrow d_{V_0} \, \text{ and } \,\{{\tilde{v}}_{n}\}_{n\in \mathbb {N}}\subset \mathcal {M}_{V_0}. \end{aligned}$$
(4.8)

In particular, \(\{{\tilde{v}}_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {Y}_{V_0}\) and, by extracting a subsequence if necessary, we may assume that \({\tilde{v}}_{n}\rightharpoonup {\tilde{v}}\) in \(\mathbb {Y}_{V_0}\). Since \(\{v_{n}\}_{n\in \mathbb {N}}\) and \(\{{\tilde{v}}_{n}\}_{n\in \mathbb {N}}\) are bounded in \(\mathbb {Y}_{V_0}\), and \(v_{n}\nrightarrow 0\) in \(\mathbb {Y}_{V_0}\), we deduce that \(\{t_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {R}\) and, up to a subsequence, we may assume that \(t_{n}\rightarrow t_{0}\ge 0\). If \(t_{0}=0\), then \({\tilde{v}}_{n}\rightarrow 0\) in \(\mathbb {Y}_{V_0}\) (because \(\{v_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {Y}_{V_0}\)), and thus \(\mathcal {L}_{V_0}({\tilde{v}}_{n})\rightarrow 0\), which contradicts \(d_{V_{0}}>0\). Hence, \(t_{0}\in (0, \infty )\). From the uniqueness of the weak limit, we see that \({\tilde{v}}=t_{0} v\not \equiv 0\). This fact combined with Lemma 3.3 yields \({\tilde{v}}_{n}\rightarrow {\tilde{v}}\) in \(\mathbb {Y}_{V_0}\), and so \(\displaystyle {v_{n}\rightarrow v}\) in \(\mathbb {Y}_{V_0}\). Furthermore,

$$\begin{aligned} \mathcal {L}_{V_0}({\tilde{v}})=d_{V_0} \, \text{ and } \, \langle \mathcal {L}'_{V_0}({\tilde{v}}), {\tilde{v}}\rangle =0. \end{aligned}$$

In what follows, we show that \(\{y_{n}\}_{n\in \mathbb {N}}\) admits a subsequence, still denoted by itself, such that \(y_{n}\rightarrow y_{0}\in M\). We begin by proving that \(\{y_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {R}^{N}\). Suppose, by contradiction, that there exists a subsequence of \(\{y_{n}\}_{n\in \mathbb {N}}\), still denoted by itself, such that \(|y_{n}|\rightarrow \infty \). Choose \(R>0\) such that \(\Lambda \subset B_{R}(0)\). For n large enough, we may assume that \(|y_{n}|>2R\). Then, for each \(x\in B_{R/\varepsilon _{n}}(0)\),

$$\begin{aligned} |\varepsilon _{n} x+y_{n}|\ge |y_{n}|-|\varepsilon _{n} x|>R. \end{aligned}$$

Using \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathcal {N}_{\varepsilon _{n}}\), a change of variable, the definition of g and the above relation, we have

$$\begin{aligned} \Vert v_{n}\Vert _{s,p}^{p}+\Vert v_{n}\Vert _{s,q}^{q}&\le \int _{\mathbb {R}^{N}} g(\varepsilon _{n}x+y_{n}, v_{n}) v_{n} \, dx \\&\le \int _{B_{R/\varepsilon _{n}}(0)} {\tilde{f}}(v_{n}) v_{n} \,dx+\int _{B_{R/\varepsilon _{n}}^{c}(0)} f(v_{n}) v_{n} \, dx. \end{aligned}$$

Since \(v_{n}\rightarrow v\) in \(\mathbb {Y}_{V_0}\) and \(|B_{R/\varepsilon _{n}}^{c}(0)|\rightarrow 0\), it follows from the dominated convergence theorem that

$$\begin{aligned} \int _{B_{R/\varepsilon _{n}}^{c}(0)} f(v_{n}) v_{n} \, dx=o_{n}(1). \end{aligned}$$

On the other hand, \({\tilde{f}}(v_{n})v_{n}\le \frac{V_{0}}{K} (|v_{n}|^{p}+|v_{n}|^{q})\), and so

$$\begin{aligned} \Vert v_{n}\Vert _{s, p}^{p}+\Vert v_{n}\Vert _{s, q}^{q}\le \frac{V_{0}}{K} \int _{B_{R/\varepsilon _{n}}(0)} (|v_{n}|^{p}+|v_{n}|^{q}) \,dx+o_{n}(1). \end{aligned}$$

Consequently,

$$\begin{aligned} \left( 1-\frac{1}{K}\right) (\Vert v_{n}\Vert ^{p}_{s,p}+\Vert v_{n}\Vert ^{q}_{s,q})\le o_{n}(1), \end{aligned}$$

and we reach a contradiction because \(v_{n}\rightarrow v\not \equiv 0\) in \(\mathbb {Y}_{V_0}\). Thus, \(\{y_{n}\}_{n\in \mathbb {N}}\) is bounded in \(\mathbb {R}^{N}\) and, up to a subsequence, we may assume that \(y_{n}\rightarrow y_{0}\in \mathbb {R}^{N}\). If \(y_{0}\notin {\overline{\Lambda }}\), then we can argue as before to get \(v_{n}\rightarrow 0\) in \(\mathbb {Y}_{V_0}\), that is a contradiction. Hence, \(y_{0}\in {\overline{\Lambda }}\). Let us note that if \(V(y_{0})=V_{0}\), then \(y_{0}\notin \partial \Lambda \) in view of (V2). Therefore, it suffices to prove that \(V(y_{0})=V_{0}\) to deduce that \(y_{0}\in M\). To accomplish this, we assume, by contradiction, that \(V(y_{0})>V_{0}\). Using this fact, \({\tilde{v}}_{n}\rightarrow {\tilde{v}}\) in \(\mathbb {Y}_{V_0}\), Fatou’s lemma and the invariance of \(\mathbb {R}^{N}\) by translation, we see that

$$\begin{aligned} d_{V_0}&=\mathcal {L}_{V_{0}}({\tilde{v}})\\&< \liminf _{n\rightarrow \infty } \Bigl [ \frac{1}{p} [{\tilde{v}}_{n}]_{s, p}^{p}+\frac{1}{2p} [{\tilde{v}}_{n}]_{s, p}^{2p}+\frac{1}{q} [{\tilde{v}}_{n}]_{s, q}^{q}+\frac{1}{2q} [{\tilde{v}}_{n}]_{s, q}^{2q}\\&\quad +\int _{\mathbb {R}^{N}}V(\varepsilon _{n} x+y_{n}) \left( \frac{1}{p}|{\tilde{v}}_{n}|^{p}+\frac{1}{q}|{\tilde{v}}_{n}|^{q}\right) \, dx - \int _{\mathbb {R}^{N}} F({\tilde{v}}_{n})\, dx \Bigr ] \\&\le \liminf _{n\rightarrow \infty } \mathcal {J}_{\varepsilon _{n}}(t_{n}u_{n}) \le \liminf _{n\rightarrow \infty } \mathcal {J}_{\varepsilon _{n}} (u_{n})=d_{V_0}, \end{aligned}$$

which is a contradiction. The proof is now complete. \(\square \)

Let us define

$$\begin{aligned} {\widetilde{\mathcal {N}}}_{\varepsilon }=\left\{ u\in \mathcal {N}_{\varepsilon }: \mathcal {J}_{\varepsilon }(u)\le d_{V_0}+\pi (\varepsilon )\right\} , \end{aligned}$$

where \(\pi (\varepsilon )=\sup _{y\in M}|\mathcal {J}_{\varepsilon }(\Phi _{\varepsilon }(y))-d_{V_0}|\rightarrow 0\) as \(\varepsilon \rightarrow 0\), according to Lemma 4.1. By the definition of \(\pi (\varepsilon )\), we have that, for all \(y\in M\) and \(\varepsilon >0\), \(\Phi _{\varepsilon }(y)\in {\widetilde{\mathcal {N}}}_{\varepsilon }\) and thus \({\widetilde{\mathcal {N}}}_{\varepsilon }\ne \emptyset \). Arguing as in the proof of Lemma 3.7 in [11], we deduce the following result.

Lemma 4.4

For any \(\delta >0\), we have

$$\begin{aligned} \lim _{\varepsilon \rightarrow 0} \sup _{u\in \widetilde{\mathcal {N}}_{\varepsilon }} \mathrm{dist}(\beta _{\varepsilon }(u), M_{\delta })=0. \end{aligned}$$

We conclude the section by presenting a relation between the topology of M and the number of solutions of the modified problem (2.1). Since \(\mathbb {S}^{+}_{\varepsilon }\) is not a complete metric space, we invoke the abstract category result in [36] to achieve our purpose.

Theorem 4.1

Assume that \((V_1)\)\((V_2)\) and \((f_1)\)\((f_4)\) hold. Then, for any \(\delta >0\) such that \(M_{\delta }\subset \Lambda \), there exists \({\bar{\varepsilon }}_\delta >0\) such that, for any \(\varepsilon \in (0, {\bar{\varepsilon }}_\delta )\), problem (2.1) has at least \(cat_{M_{\delta }}(M)\) positive solutions.

Proof

For each \(\varepsilon >0\), we define the map \(\alpha _{\varepsilon } : M \rightarrow \mathbb {S}_{\varepsilon }^{+}\) by setting \(\alpha _{\varepsilon }(y)= m_{\varepsilon }^{-1}(\Phi _{\varepsilon }(y))\). By Lemma 4.1, we see that

$$\begin{aligned} \lim _{\varepsilon \rightarrow 0} \psi _{\varepsilon }(\alpha _{\varepsilon }(y)) = \lim _{\varepsilon \rightarrow 0} \mathcal {J}_{\varepsilon }(\Phi _{\varepsilon }(y))= d_{V_0} \text{ uniformly } \text{ in } y\in M. \end{aligned}$$

Hence, there is a number \({\hat{\varepsilon }}>0\) such that the set \( \widetilde{\mathcal {S}}^{+}_{\varepsilon }=\{ w\in \mathbb {S}_{\varepsilon }^{+} : \psi _{\varepsilon }(w) \le d_{V_0} + \pi (\varepsilon )\} \) is nonempty for all \(\varepsilon \in (0, {\hat{\varepsilon }})\), since \(\psi _{\varepsilon }(M)\subset \widetilde{\mathcal {S}}^{+}_{\varepsilon }\). Here \(\pi (\varepsilon )=\sup _{y\in M}|\psi _{\varepsilon }(\alpha _{\varepsilon }(y))-d_{V_0}|\rightarrow 0\) as \(\varepsilon \rightarrow 0\). From the above considerations, and taking into account Lemma 4.1, Lemma 2.4-(iii), Lemmas 4.4 and 4.2, we see that there exists \({\bar{\varepsilon }}= {\bar{\varepsilon }}_{\delta }>0\) such that, for any \(\varepsilon \in (0, {\bar{\varepsilon }})\), the diagram

$$\begin{aligned} M{\mathop {\rightarrow }\limits ^{\Phi _{\varepsilon }}} \Phi _{\varepsilon }(M) {\mathop {\rightarrow }\limits ^{m_{\varepsilon }^{-1}}} \alpha _{\varepsilon }(M){\mathop {\rightarrow }\limits ^{m_{\varepsilon }}} \Phi _{\varepsilon }(M) {\mathop {\rightarrow }\limits ^{\beta _{\varepsilon }}} M_{\delta } \end{aligned}$$

is well defined. According to Lemma 4.2, for \(\varepsilon >0\) small, we can write \(\beta _{\varepsilon }(\Phi _{\varepsilon }(y))= y+ \theta (\varepsilon , y)\) for \(y\in M\), where \(|\theta (\varepsilon , y)|<\frac{\delta }{2}\) uniformly in \(y\in M\). Define \(H(t, y)= y+ (1-t)\theta (\varepsilon , y)\) for \((t, y)\in [0,1]\times M\). Clearly, \(H: [0,1]\times M\rightarrow M_{\delta }\) is continuous, \(H(0, y)=\beta _{\varepsilon }(\Phi _{\varepsilon }(y))\) and \(H(1, y)=y\) for all \(y\in M\). Then H(ty) is a homotopy between \(\beta _{\varepsilon } \circ \Phi _{\varepsilon } = (\beta _{\varepsilon } \circ m_{\varepsilon }) \circ (m_{\varepsilon }^{-1}\circ \Phi _{\varepsilon })\) and the inclusion map \(id: M \rightarrow M_{\delta }\). This fact implies that

$$\begin{aligned} cat_{\alpha _{\varepsilon }(M)} \alpha _{\varepsilon }(M)\ge cat_{M_{\delta }}(M). \end{aligned}$$
(4.9)

It follows from Corollary 2.1, Lemma 3.4, and Theorem 27 in [36], with \(c= c_{\varepsilon }\le d_{V_0}+\pi (\varepsilon ) =d\) and \(K= \alpha _{\varepsilon }(M)\), that \(\Psi _{\varepsilon }\) has at least \(cat_{\alpha _{\varepsilon }(M)} \alpha _{\varepsilon }(M)\) critical points on \(\widetilde{\mathcal {S}}^{+}_{\varepsilon }\). Therefore, by Proposition 2.1-(d) and (4.9), we conclude that \(\mathcal {J}_{\varepsilon }\) admits at least \(cat_{M_{\delta }}(M)\) critical points in \(\widetilde{\mathcal {N}}_{\varepsilon }\). \(\square \)

5 Proof of Theorem 1.1

This section is devoted to the proof of the main result of this paper. The idea is to show that the solutions obtained in Theorem 4.1 satisfy, for \(\varepsilon >0\) small enough, the estimate \(u_{\varepsilon }(x)\le a\) for any \(x\in \Lambda ^{c}_{\varepsilon }\). This fact implies that these solutions are indeed solutions of the original problem (1.1). We start with the following lemma which plays a key role in studying the behavior of the maximum points of solutions to (1.1), whose proof is related to the Moser iteration method [32].

Lemma 5.1

Let \(\varepsilon _{n}\rightarrow 0\) and \(\{u_{n}\}_{n\in \mathbb {N}}\subset \widetilde{\mathcal {N}}_{\varepsilon _{n}}\) be a sequence of solutions to (2.1). Then \(\mathcal {J}_{\varepsilon _{n}}(u_{n})\rightarrow d_{V_{0}}\), and there exists \(\{{\tilde{y}}_{n}\}_{n\in \mathbb {N}}\subset \mathbb {R}^{N}\) such that \(v_{n}=u_{n}(\cdot +{\tilde{y}}_{n})\in L^{\infty }(\mathbb {R}^{N})\) and for some \(C>0\) it holds

$$\begin{aligned} |v_{n}|_{\infty }\le C \quad \text{ for } \text{ all } n\in \mathbb {N}. \end{aligned}$$

Moreover,

$$\begin{aligned} v_{n}(x)\rightarrow 0 \, \text{ as } |x|\rightarrow \infty \text{ uniformly } \text{ in } n\in \mathbb {N}. \end{aligned}$$
(5.1)

Proof

Since \(\mathcal {J}_{\varepsilon _{n}}(u_{n})\le d_{V_{0}}+\pi (\varepsilon _{n})\), with \(\pi (\varepsilon _{n})\rightarrow 0\) as \(n\rightarrow \infty \), we can argue as at the beginning of the proof of Lemma 4.3 to deduce that \(\mathcal {J}_{\varepsilon _{n}}(u_{n})\rightarrow d_{V_{0}}\). Then, using Lemma 4.3, we can find \(\{{\tilde{y}}_{n}\}_{n\in \mathbb {N}}\subset \mathbb {R}^{N}\) such that \(v_{n}=u_{n}(\cdot +{\tilde{y}}_{n})\rightarrow v\) in \(\mathbb {Y}_{V_{0}}\) for some \(v\in \mathbb {Y}_{V_{0}}\setminus \{0\}\) and \(\varepsilon _{n}{\tilde{y}}_{n}\rightarrow y_{0}\in M\).

Now we examine the boundedness of \(\{v_{n}\}_{n\in \mathbb {N}}\) in \(L^{\infty }(\mathbb {R}^{N})\). For each \(n\in \mathbb {N}\) and \(L>0\), we define

$$\begin{aligned} \gamma (v_{n})=v_{n}v_{n,L}^{q(\beta -1)} \in \mathbb {X}_{\varepsilon }, \end{aligned}$$

where \(v_{n,L}= \min \{v_{n}, L\}\), and \(\beta >1\) will be chosen later. Taking \(\gamma (v_{n})\) as test function in the problem solved by \(v_{n}\), we have

$$\begin{aligned}&(1+[v_{n}]_{s, p}^{p})\\&\iint _{\mathbb {R}^{2N}} \frac{|v_{n}(x)- v_{n}(y)|^{p-2} (v_{n}(x)- v_{n}(y)) ((v_{n}v_{n, L}^{q(\beta -1)}) (x) - (v_{n} v_{n,L}^{q(\beta -1)})(y))}{|x- y|^{N+sp}} \, dxdy\\&\quad +(1+[v_{n}]_{s, q}^{q})\\&\iint _{\mathbb {R}^{2N}} \frac{|v_{n}(x)- v_{n}(y)|^{q-2} (v_{n}(x)- v_{n}(y)) ((v_{n}v_{n,L}^{q(\beta -1)}) (x) - (v_{n}v_{n,L}^{q(\beta -1)})(y))}{|x- y|^{N+sq}} \, dxdy\\&\quad + \int _{\mathbb {R}^{N}} V(\varepsilon _{n} x+\varepsilon _{n}{\tilde{y}}_{n}) |v_{n}|^{p} v_{n,L}^{q(\beta -1)}\, dx+ \int _{\mathbb {R}^{N}} V(\varepsilon _{n} x+\varepsilon _{n}{\tilde{y}}_{n}) |v_{n}|^{q} v_{n,L}^{q(\beta -1)}\, dx \\&= \int _{\mathbb {R}^{N}} g(\varepsilon _{n} x+\varepsilon _{n}{\tilde{y}}_{n}, v_{n}) v_{n}v_{n,L}^{q(\beta -1)} \, dx. \end{aligned}$$

In light of the growth assumptions on g, we know that for all \(\xi \in (0, V_{0})\), there exists \(C_{\xi }>0\) such that

$$\begin{aligned} |g(x, t)|\le \xi |t|^{p-1}+C_{\xi } |t|^{q^{*}_{s}-1} \quad {{ \text{ for } (x,t)\in \mathbb {R}^{N}\times \mathbb {R}.}} \end{aligned}$$

From the above facts and \((V_{1})\), we obtain

$$\begin{aligned}&(1+[v_{n}]_{s, p}^{p})\nonumber \\&\iint _{\mathbb {R}^{2N}} \frac{|v_{n}(x)- v_{n}(y)|^{p-2} (v_{n}(x)- v_{n}(y)) ((v_{n}v_{n,L}^{q(\beta -1)}) (x) - (v_{n}v_{n,L}^{q(\beta -1)})(y))}{|x- y|^{N+sp}} \, dxdy\nonumber \\&\quad +(1+[v_{n}]_{s, q}^{q})\nonumber \\&\iint _{\mathbb {R}^{2N}} \frac{|v_{n}(x)- v_{n}(y)|^{q-2} (v_{n}(x)- v_{n}(y)) ((v_{n}v_{n,L}^{q(\beta -1)}) (x) - (v_{n}v_{n,L}^{q(\beta -1)})(y))}{|x- y|^{N+sq}} \, dxdy\nonumber \\&\le C \int _{\mathbb {R}^{N}} |v_{n}|^{q^{*}_{s}} v_{n,L}^{q(\beta -1)} \, dx. \end{aligned}$$
(5.2)

Observing that, for \(t\in \{p, q\}\), \(1\le 1+[v_{n}]_{s, t}^{t}\le C\) for all \(n\in \mathbb {N}\), we can reproduce the Moser iteration argument carried out in the proof of Lemma 4.1 in [11] to derive that \(|v_{n}|_{\infty }\le C\) for all \(n\in \mathbb {N}\). Since \(\{v_{n}\}_{n\in \mathbb {N}}\) is uniformly bounded in \(L^{\infty }(\mathbb {R}^{N})\cap \mathbb {Y}_{V_{0}}\), we can argue as in the proof of Theorem 2.2 in [11] to deduce that \(\Vert v_{n}\Vert _{C^{0, \alpha }(\mathbb {R}^{N})}\le C\) for all \(n\in \mathbb {N}\). This fact combined with \(v_{n}\rightarrow v\) in \(\mathbb {Y}_{V_{0}}\) implies that \(v_{n}(x)\rightarrow 0\) as \(|x|\rightarrow \infty \) uniformly in \(n\in \mathbb {N}\). The proof of Lemma 5.1 is complete. \(\square \)

We now have all ingredients to prove Theorem 1.1.

Proof of Theorem 1.1

Let \(\delta >0\) be a number satisfying \(M_{\delta } \subset \Lambda \). We first show that there exists \({\tilde{\varepsilon }}_{\delta }>0\) such that, for any \(\varepsilon \in (0, {\tilde{\varepsilon }}_{\delta })\) and any solution \(u_{\varepsilon } \in \widetilde{\mathcal {N}}_{\varepsilon }\) of (2.1), it holds

$$\begin{aligned} |u_{\varepsilon }|_{L^{\infty }(\Lambda _{\varepsilon }^{c})}<a. \end{aligned}$$
(5.3)

Assume, by contradiction, that there exists a subsequence \(\varepsilon _{n}\rightarrow 0\), \(u_{n}=u_{\varepsilon _{n}}\in \widetilde{\mathcal {N}}_{\varepsilon _{n}}\) such that \(\mathcal {J}'_{\varepsilon _{n}}(u_{\varepsilon _{n}})=0\) and

$$\begin{aligned} |u_{n}|_{L^{\infty }(\Lambda _{\varepsilon _{n}}^{c})}\ge a. \end{aligned}$$
(5.4)

As in the proof of Lemma 5.1, we can verify that \(\mathcal {J}_{\varepsilon _{n}}(u_{n}) \rightarrow d_{V_0}\). Then, applying Lemma 4.3, we obtain a sequence \(\{{\tilde{y}}_{n}\}_{n\in \mathbb {N}}\subset \mathbb {R}^{N}\) such that \(v_{n}=u_{n}(\cdot +{\tilde{y}}_{n})\rightarrow v\) in \(\mathbb {Y}_{V_{0}}\) and \(\varepsilon _{n}{\tilde{y}}_{n}\rightarrow y_{0} \in M\).

Pick \(r>0\) such that \(B_{r}(y_{0})\subset B_{2r}(y_{0})\subset \Lambda \). Thus, \(B_{\frac{r}{\varepsilon _{n}}}(\frac{y_{0}}{\varepsilon _{n}})\subset \Lambda _{\varepsilon _{n}}\) for all \(n\in \mathbb {N}\). Moreover, for any \(y\in B_{\frac{r}{\varepsilon _{n}}}({\tilde{y}}_{n})\), we see that

$$\begin{aligned} \left| y - \frac{y_{0}}{\varepsilon _{n}}\right| \le |y- {\tilde{y}}_{n}|+ \left| {\tilde{y}}_{n} - \frac{y_{0}}{\varepsilon _{n}}\right|<\frac{1}{\varepsilon _{n}}(r+o_{n}(1))<\frac{2r}{\varepsilon _{n}} \end{aligned}$$

for n large enough. For these values of n, we have

$$\begin{aligned} \Lambda _{\varepsilon _{n}}^{c}\subset B_{\frac{r}{\varepsilon _{n}}}^{c}({\tilde{y}}_{n}). \end{aligned}$$

Using (5.1), we can find \(R>0\) such that \(v_{n}(x)<a\) for any \(|x|\ge R\) and \(n\in \mathbb {N}\), and so \(u_{n}(x)<a\) for any \(x\in B_{R}^{c}({\tilde{y}}_{n})\) and \(n\in \mathbb {N}\). On the other hand, there exists \(n_{0} \in \mathbb {N}\) such that, for any \(n\ge n_{0}\),

$$\begin{aligned} \Lambda ^{c}_{\varepsilon _{n}}\subset B^{c}_{\frac{r}{\varepsilon _{n}}}({\tilde{y}}_{n})\subset B^{c}_{R}({\tilde{y}}_{n}). \end{aligned}$$

Hence, \(u_{n}(x)<a\) for any \(x\in \Lambda ^{c}_{\varepsilon _{n}}\) and \(n\ge n_{0}\), which is in contrast with (5.4). This proves our claim.

Let \({\bar{\varepsilon }}_{\delta }>0\) be given by Theorem 4.1 and set \(\varepsilon _{\delta }= \min \{{\tilde{\varepsilon }}_{\delta }, {\bar{\varepsilon }}_{\delta }\}\). Fix \(\varepsilon \in (0, \varepsilon _{\delta })\). Applying Theorem 4.1, we get at least \(cat_{M_{\delta }}(M)\) positive solutions to (2.1). If \(u_{\varepsilon }\) denotes one of these solutions, we have that \(u_{\varepsilon }\in \widetilde{\mathcal {N}}_{\varepsilon }\), and using (5.3) and the definition of g, we deduce that \(u_{\varepsilon }\) is also a solution to (1.1). Consequently, (1.1) admits at least \(cat_{M_{\delta }}(M)\) positive solutions.

Now we investigate the behavior of the maximum points of solutions to (1.1). Take \(\varepsilon _{n}\rightarrow 0\) and consider a sequence \(\{u_{n}\}_{n\in \mathbb {N}}\subset \mathbb {X}_{\varepsilon _{n}}\) of solutions to (1.1) as above. Let us observe that \((g_{1})\) implies that there exists \(\sigma \in (0, a)\) such that

$$\begin{aligned} g(\varepsilon x, t)t\le \frac{V_{0}}{K} (t^{p}+t^{q}) \quad \text{ for } (x, t)\in \mathbb {R}^{N}\times [0, \sigma ]. \end{aligned}$$
(5.5)

Arguing as before, we can choose \(R>0\) such that

$$\begin{aligned} |u_{n}|_{L^{\infty }(B^{c}_{R}({\tilde{y}}_{n}))}<\sigma . \end{aligned}$$
(5.6)

Moreover, up to a subsequence, we may assume that

$$\begin{aligned} |u_{n}|_{L^{\infty }(B_{R}({\tilde{y}}_{n}))}\ge \sigma . \end{aligned}$$
(5.7)

Indeed, if (5.7) does not hold, then, in view of (5.6), we have that \(|u_{n}|_{\infty }<\sigma \). Hence, using \(\langle \mathcal {J}'_{\varepsilon _{n}}(u_{n}), u_{n}\rangle =0\) and (5.5), we get

$$\begin{aligned} \Vert u_{n}\Vert _{V_{\varepsilon _{n}},p}^{p}+\Vert u_{n}\Vert _{V_{\varepsilon _{n}},q}^{q}\le \int _{\mathbb {R}^{N}} g(\varepsilon _{n} x, u_{n}) u_{n} \,dx\le \frac{V_{0}}{K} \int _{\mathbb {R}^{N}} (|u_{n}|^{p}+|u_{n}|^{q}) \, dx \end{aligned}$$

which leads to a contradiction. Therefore, (5.7) is satisfied.

Let \(p_{n}\in \mathbb {R}^{N}\) be a global maximum point of \(u_{n}\). Combining (5.6) and (5.7), we infer that \(p_{n}={\tilde{y}}_{n}+q_{n}\), for some \(q_{n}\in B_{R}(0)\). Since \(\varepsilon _{n}{\tilde{y}}_{n}\rightarrow y_{0}\in M\) and \(|q_{n}|<R\) for all \(n\in \mathbb {N}\), we have that \(\varepsilon _{n}p_{n}\rightarrow y_{0}\), and using the continuity of V we obtain

$$\begin{aligned} \lim _{n\rightarrow \infty }V(\varepsilon _{n}p_{n})= V(y_{0})=V_{0}. \end{aligned}$$

The proof of Theorem 1.1 is now complete. \(\square \)