Existence and multiplicity of solutions for Schrödinger–Kirchhoff type problems involving the fractional $p(\cdot )$-Laplacian in $\mathbb{R}^{N}$

Abstract

We are concerned with the following elliptic equations with variable exponents:

$$ M \bigl([u]_{s,p(\cdot,\cdot)} \bigr)\mathcal{L}u(x) +\mathcal {V}(x) \vert u \vert ^{p(x)-2}u =\lambda\rho(x) \vert u \vert ^{r(x)-2}u + h(x,u) \quad \text{in } \mathbb {R}^{N}, $$

where $[u]_{s,p(\cdot,\cdot)}:=\int_{\mathbb {R}^{N}}\int_{\mathbb {R}^{N}} \frac{|u(x)-u(y)|^{p(x,y)}}{p(x,y)|x-y|^{N+sp(x,y)}} \,dx \,dy$, the operator $\mathcal{L}$ is the fractional $p(\cdot)$-Laplacian, $p, r: {\mathbb {R}^{N}} \to(1,\infty)$ are continuous functions, $M \in C(\mathbb {R}^{+})$ is a Kirchhoff-type function, the potential function $\mathcal {V}:\mathbb {R}^{N} \to(0,\infty)$ is continuous, and $h:\mathbb {R}^{N}\times\mathbb {R} \to\mathbb {R}$ satisfies a Carathéodory condition. Under suitable assumptions on h, the purpose of this paper is to show the existence of at least two non-trivial distinct solutions for the problem above for the case of a combined effect of concave–convex nonlinearities. To do this, we use the mountain pass theorem and variant of the Ekeland variational principle as the main tools.

1 Introduction

In the last two decades an increasing deal of attention has been paid to the investigation on problems of differential equations and variational problems with nonstandard growth conditions because they can be corroborated as a model for many physical phenomena which arise in the research of elastic mechanics, electro-rheological fluid (“smart fluids”) and image processing, etc. We refer the reader to [6, 11, 17, 24, 34, 42, 44] and the references therein.

On the other hand, in recent years the study of fractional Sobolev spaces and the corresponding nonlocal equations has received a great amount of attention because of their occurrence in many different applications such as optimization, fractional quantum mechanics, the thin obstacle problem, phase transition phenomena, image process, game theory and Lévy processes; see [14, 23, 28, 36, 39, 45] and the references therein for more details.

In this direction it is a natural question to see which results can be recovered when we replace the local $p(\cdot)$-Laplacian, defined as $-\operatorname{div}(|\nabla u|^{p(x)-2}\nabla u)$, with the nonlocal fractional $p(\cdot)$-Laplacian. Very recently, many authors in [1, 5, 7, 8, 18, 19, 29, 33] have investigated elliptic problems involving the fractional $p(\cdot )$-Laplacian. A new class of fractional Sobolev spaces with variable exponents that takes into account a fractional variable exponent operator has been introduced by Kaumann et al. [33]. The authors in [8] particular presented further primary properties both on this function space and the related nonlocal operator. As applications, they gave the existence of at least one solution for equations involving the fractional $p(\cdot)$-Laplacian. Based on this recent work, Ho and Kim [29] provided fundamental embeddings for the fractional Sobolev space with variable exponent and their applications such as a priori bounds and multiplicity of solutions of the fractional $p(\cdot)$-Laplacian problems.

In this paper, we are concerned with a Schrödinger–Kirchhoff type problem driven by the non-local fractional $p(\cdot)$-Laplacian as follows:

where $[u]_{s,p(\cdot,\cdot)}:=\int_{\mathbb {R}^{N}}\int_{\mathbb {R}^{N}} \frac{|u(x)-u(y)|^{p(x,y)}}{p(x,y)|x-y|^{N+sp(x,y)}} \,dx \,dy$, $p, r: {\mathbb {R}^{N}} \to(1,\infty)$ are continuous functions with $1<\inf_{x\in\mathbb{R}^{N}}{r}(x)\leq\sup_{x\in\mathbb{R}^{N}}{p}(x)$, $M \in C(\mathbb {R}^{+})$ is a Kirchhoff type function, the potential function $\mathcal{V}:\mathbb {R}^{N} \to(0,\infty)$ is continuous, and $h:{\mathbb {R}^{N}}\times\mathbb {R}\to\mathbb {R}$ satisfies a Carathéodory condition satisfying the subcritical and $p(\cdot)$-superlinear nonlinearity, and $\mathcal{L}$ is the fractional $p(\cdot)$-Laplacian operator defined as

$$ \mathcal{L}u(x) = 2\lim_{\varepsilon\searrow0} \int_{\mathbb{R}^{N}\backslash B_{\varepsilon}(x)} \frac{ \vert u(x) - u(y) \vert ^{{p}(x,y)-2} (u(x) - u(y))}{ \vert x - y \vert ^{N+s{p}(x,y)}}\, dy, \quad x \in\mathbb {R}^{N}, $$

where $s \in(0,1)$ and $B_{\varepsilon}(x):=\{y\in\mathbb {R}^{N}: |y-x|\leq\varepsilon\}$. Here, $p(x)={p}(x,x)$ for all $x\in\mathbb {R}^{N}$ with ${p}\in C(\mathbb{R}^{N}\times\mathbb{R}^{N})$ satisfying ${p}(x,y)={p}(y,x)$ for all $x,y\in\mathbb{R}^{N}$ and $1<\inf_{(x,y)\in\mathbb{R}^{N}\times\mathbb{R}^{N}}{p}(x,y)\leq\sup_{(x,y)\in\mathbb{R}^{N}\times\mathbb{R}^{N}}{p}(x,y)<\frac{N}{s}$.

Let us first assume that a Kirchhoff function $M:\mathbb{R}_{0}^{+} \to\mathbb{R}^{+}$ satisfies the following conditions:

• (M1) $M\in C(\mathbb{R}_{0}^{+}, \mathbb{R}^{+})$ satisfies $\inf_{t\in\mathbb{R}_{0}^{+}} M(t) \geq m_{0} > 0$, where $m_{0}$ is a constant,

• (M2) There exists $\vartheta\in[1,\frac{N}{N-sp_{+}})$ such that $\vartheta\mathcal{M}(t)=\vartheta\int_{0}^{t} M(\tau)\,d \tau \geq M(t)t$ for any $t\geq0$.

A typical example for M is given by $M(t)=b_{0} +b_{1}t^{n}$ with $n>0$, $b_{0}>0$ and $b_{1}\geq0$. The Kirchhoff type problem was primarily introduced in [37] as a generalization of the classical D’Alembert wave equation for free vibrations of elastic strings. Some interesting researches by variational methods can be found in [20, 21, 40, 41, 43, 49] for Kirchhoff type problems. Recently, Pucci et al. [43] studied the existence of nontrivial solutions for the Schrödinger–Kirchhoff type equations involving the fractional p-Laplacian in $\mathbb {R}^{N}$; see [49] for problems with Dirichlet boundary data. In [40], the authors showed that a $p(x)$-Kirchhoff type problem admits at least two nontrivial different solutions by employing abstract critical point theorems which is based on a generalization of Ekeland’s variational principle [25]. The primary strategy for obtaining this is to observe the relationship between the mountain pass geometry and the existence of a local minima for an appropriate functional.

The main aim of the present paper is to establish the existence of at least two nontrivial distinct solutions for Schrödinger–Kirchhoff type problems in the case where the nonlinear term is concave–convex, by employing the mountain pass theorem (see [4]) and a variant of the variational principle of Ekeland (see [6]). This type of nonlinearity has been extensively investigated since the seminal work of Ambrosetti, Brezis and Cerami [3] for the Laplacian problem

$$ \textstyle\begin{cases} -\triangle u = \lambda \vert u \vert ^{q-2}u+ \vert u \vert ^{h-2}u &\text{in } \varOmega, \\ u = 0 &\text{on } \partial\varOmega, \end{cases} $$

where

$$1< q< 2< h< 2^{*}:= \textstyle\begin{cases} \frac{2N}{N-2} &\text{if } N>2,\\ +\infty& \text{if } N =1,2. \end{cases} $$

For elliptic equations with the concave–convex nonlinearity, we also infer the reader to [12, 15, 16, 22, 30, 47, 48, 50] and the references therein. Especially, the existence of multiple solutions for an elliptic problem of a nonhomogeneous fractional p-Kirchhoff type involving concave–convex nonlinearities has been established in [50]. In [30], the authors built the existence of two nontrivial nonnegative solutions and infinitely many solutions for the following $p(x)$-Laplacian equations involving concave–convex type nonlinearities with two parameters:

$$ \textstyle\begin{cases} -\operatorname{div}(w(x) \vert \nabla v \vert ^{p(x)-2} \nabla v)=\lambda a(x) \vert u \vert ^{q(x)-2}u+\mu b(x) \vert u \vert ^{h(x)-2}u &\text{in } \varOmega, \\ u = 0 &\text{on } \partial\varOmega, \end{cases} $$

where $\varOmega\subset\mathbb {R}^{N}$ is a bounded domain with a smooth boundary ∂Ω, $p, q, h \in C(\overline{\varOmega}, (1,\infty))$ with $q(x)< p(x)< h(x)$ for all $x\in\varOmega$, w, a, b are measurable functions on Ω that are positives a.e. in Ω, and λ, μ are real parameters. Very recently, Biswas and Tiwari [10] studied problem ($P_{\lambda }$) in a bounded domain, which is subject to Dirichlet boundary conditions with $M\equiv 1$, $\mathcal{V}\equiv0$ and $\rho\equiv1$. In order to obtain the multiplicity result, they considered two aspects: one is to assume the condition by Ambrosetti and Rabinowitz [4] (see [2, 26] for elliptic equations with variable exponents), and the other is to apply the mountain pass theorem and Ekeland’s variational principle. In that sense, the first purpose of this article is to show the existence of two nontrivial distinct solutions for the problem ($P_{\lambda }$) for the case of a combined effect of concave–convex nonlinearities when h fulfils the condition of Ambrosetti–Rabinowitz type, that is, there exists a constant $\theta>0$ such that $\theta> \vartheta\sup_{x\in\mathbb{R}^{N}}{p}(x)$ and

$$ \begin{gathered}[b] 0< \theta H(x,t)\le h(x,t)t, \quad\text{for all } t \in\mathbb {R} \setminus\{0\} \text{ and } x \in\mathbb {R}^{N},\\ \quad\text{where } H(x,t)= \int _{0}^{t}h(x,s)\,ds.\end{gathered} $$

(1.1)

As is well known, this condition is essential in ensuring the boundedness of a Palais–Smale sequence of the Euler–Lagrange functional corresponding to the problem ($P_{\lambda }$). However, in comparison with [2, 10, 11, 26], it is not easy to obtain this compactness condition because the Kirchhoff function M is not convex and the given problem has the concave–convex nonlinearity. The second one is to establish the existence of multiple solutions to ($P_{\lambda }$), provided that the nonlinear growth h fulfils a weaker condition than condition (1.1), which will be specified later. Roughly speaking, by utilizing analogous arguments to the first main aim we attempt to establish this multiplicity result when for h one has mild and different assumptions from condition (1.1) which is originally given in [31]. In order to obtain this, we give a sufficient condition for the modified function M which is not a Kirchhoff function as in the original work due to Kirchhoff [37]. As seen before, the main tools are the mountain pass theorem and a variant of the Ekeland variational principle for an energy functional with the compactness condition of the Palais–Smale type, namely the Cerami condition; see Lemma 2.9 in [32] and Corollary 3.2 in [6], respectively. To the best of our knowledge, the present paper is the first to study the existence of at least two nontrivial distinct solutions for Schrödinger–Kirchhoff type problems with the concave–convex nonlinearity in these situations even in the case of $M\equiv1$ or constant exponents.

This paper is designed as follows. In Sect. 2, we briefly review the definitions and collect some preliminary results for the Lebesgue spaces with variable exponents and the variable exponent Lebesgue–Sobolev space of fractional type. In Sect. 3, we give the existence results of multiple solutions to the problem ($P_{\lambda }$) by employing as the main tools the variational principle.

2 Preliminaries

In this section, we briefly introduce some useful definitions and well-known properties of the variable exponent Lebesgue–Sobolev space of fractional type $W^{s, p(\cdot,\cdot)}$ which will be treated in the next sections.

Set

$$C_{+}\bigl(\mathbb {R}^{N}\bigr)= \Bigl\{ f\in C\bigl(\mathbb {R}^{N} \bigr): \inf_{x\in\mathbb {R}^{N}}f(x)>1 \Bigr\} . $$

For any $f\in C_{+}(\mathbb {R}^{N})$, we define

$$f_{+}=\sup_{x\in\mathbb {R}^{N}}f(x)\quad\hbox{and}\quad f_{-}=\inf _{x\in\mathbb {R}^{N}}f(x). $$

For any $p\in C_{+}(\mathbb {R}^{N})$, we introduce the variable exponent Lebesgue space

$$L^{p(\cdot)}\bigl(\mathbb {R}^{N}\bigr) := \biggl\{ u : u \hbox{ is a measurable real-valued function}, \int_{\mathbb {R}^{N}} \bigl\vert u(x) \bigr\vert ^{p(x)} \,dx < \infty \biggr\} , $$

endowed with the Luxemburg norm

$$\Vert u \Vert _{L^{p(\cdot)}(\mathbb {R}^{N})}=\inf \biggl\{ \lambda>0: \int_{\mathbb {R}^{N}} \biggl\vert \frac{u(x)}{\lambda} \biggr\vert ^{p(x)}\,dx\le1 \biggr\} . $$

The dual space of $L^{p(\cdot)}(\mathbb {R}^{N})$ is $L^{p^{\prime}(\cdot)}(\mathbb {R}^{N})$, where $1/p(x) + 1/p^{\prime}(x)=1$.

Let $0< s<1$ and let $p\in C(\mathbb{R}^{N}\times\mathbb{R}^{N}, (1,\infty))$ be such that p is symmetric, i.e., $p(x,y) = p(y, x)$ for all $x,y\in\mathbb{R}^{N}$ and

$$1 < p^{-}:=\inf_{(x,y)\in\mathbb{R}^{N}\times\mathbb{R}^{N}}p(x,y) \le p^{+}:=\sup _{(x,y)\in\mathbb{R}^{N}\times\mathbb{R}^{N}}p(x,y) < +\infty. $$

For $p \in C_{+}(\mathbb{R}^{N})$, define

$$W^{s, p(\cdot), p(\cdot,\cdot)}\bigl(\mathbb{R}^{N}\bigr):= \biggl\{ u \in L^{p(\cdot)}\bigl(\mathbb{R}^{N}\bigr): \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx\,dy < + \infty \biggr\} , $$

and we set

$$\lvert u \rvert_{s,p(\cdot,\cdot)}:=\inf \biggl\{ \lambda >0: \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{\lambda^{p(x,y)} \vert x-y \vert ^{N+sp(x,y)}} \,dx\,dy < 1 \biggr\} . $$

Then $W^{s, p(\cdot), p(\cdot,\cdot)}(\mathbb{R}^{N})$ endowed with the norm

$$\Vert u \Vert _{s,p,\mathbb{R}^{N}}:= \Vert u \Vert _{L^{p(\cdot)}(\mathbb{R}^{N})}+ \lvert u \rvert_{s,p(\cdot,\cdot)} $$

is a separable reflexive Banach space (see [7, 8, 33]). It is immediate that

$$\lvert u \rvert_{s,p,\mathbb{R}^{N}}:=\inf \biggl\{ \lambda>0: \int_{\mathbb{R}^{N}} \biggl\vert \frac{u}{\lambda} \biggr\vert ^{p(x)}\,dx+ \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{\lambda^{p(x,y)} \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy < 1 \biggr\} $$

is an equivalent norm of $\|\cdot\|_{s,p,\mathbb{R}^{N}}$ with the relation

$$\frac{1}{2} \Vert u \Vert _{s, p,\mathbb{R}^{N}}\leq \lvert u \rvert_{s, p,\mathbb{R}^{N}}\leq2 \Vert u \Vert _{s, p,\mathbb{R}^{N}}. $$

Throughout this paper, for brevity, we write $p(x)$ instead of $p(x,x)$ for some cases and hence, $p\in C_{+}(\mathbb{R}^{N})$. Furthermore, we write $W^{s,p(\cdot,\cdot)}(\mathbb{R}^{N})$ instead of $W^{s,p(\cdot ),p(\cdot,\cdot)}(\mathbb{R}^{N})$.

Lemma 2.1

([27, 38])

The space$L^{p(\cdot)}(\mathbb {R}^{N})$is a separable, uniformly convex Banach space, and its conjugate space is$L^{p^{\prime}(\cdot)}(\mathbb {R}^{N})$where$1/p(x)+1/p^{\prime }(x)=1$. For any$u\in L^{p(\cdot)}(\mathbb {R}^{N})$and$v\in L^{p^{\prime}(\cdot)}(\mathbb {R}^{N})$, we have

$$\biggl\vert \int_{\mathbb {R}^{N}} uv\,dx \biggr\vert \le \biggl( \frac{1}{p_{-}}+\frac{1}{(p^{\prime})_{-}} \biggr) \Vert u \Vert _{L^{p(\cdot)}(\mathbb {R}^{N})} \Vert v \Vert _{L^{p'(\cdot)}(\mathbb {R}^{N})}\le 2 \Vert u \Vert _{L^{p(\cdot)}(\mathbb {R}^{N})} \Vert v \Vert _{L^{p'(\cdot)}(\mathbb {R}^{N})}. $$

Lemma 2.2

([27])

Denote

$$\rho(u)= \int_{\mathbb {R}^{N}} \vert u \vert ^{p(x)}\,dx, \quad\textit{for all } u\in L^{p(\cdot)}\bigl(\mathbb {R}^{N}\bigr). $$

Then

• (1) $\rho(u)>1$ (=1; <1) if and only if$\|u\|_{L^{p(\cdot )}(\mathbb {R}^{N})}>1$ (=1; <1), respectively;

• (2) if$\|u\|_{L^{p(\cdot)}(\mathbb {R}^{N})}>1$, then$\|u\|_{L^{p(\cdot)}(\mathbb {R}^{N})}^{p_{-}}\le\rho(u)\le \|u\|_{L^{p(\cdot)}(\mathbb {R}^{N})}^{p_{+}}$;

• (3) if$\|u\|_{L^{p(\cdot)}(\mathbb {R}^{N})}<1$, then$\|u\|_{L^{p(\cdot)}(\mathbb {R}^{N})}^{p_{+}}\le\rho(u)\le \|u\|_{L^{p(\cdot)}(\mathbb {R}^{N})}^{p_{-}}$.

Proposition 2.3

([29])

Denote

$$\widetilde{\rho}(u):= \int_{\mathbb {R}^{N}} \vert u \vert ^{p(x)}\,dx+ \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx\,dy. $$

On$W^{s,p(\cdot,\cdot)}(\mathbb {R}^{N})$we have:

• (i) for$u\in W^{s,p(\cdot,\cdot)}(\mathbb {R}^{N})\setminus \{0\}$, $\lambda=\|u\|_{s,p,\mathbb {R}^{N}}$if and only if$\widetilde {\rho}(\frac{u}{\lambda})=1$;

• (ii) $\widetilde{\rho}(u)>1$ (=1; <1) if and only if$\|u\|_{s,p}>1$ (=1; <1), respectively;

• (iii) if$\|u\|_{s,p,\mathbb {R}^{N}}\geq1$, then$\|u\|_{s,p,\mathbb {R}^{N}}^{p^{-}}\le\widetilde{\rho}(u)\le \|u\|_{s,p,\mathbb {R}^{N}}^{p^{+}}$;

• (iv) if$\|u\|_{s,p,\mathbb {R}^{N}}<1$, then$\|u\|_{s,p,\mathbb {R}^{N}}^{p^{+}}\le\widetilde{\rho}(u)\le \|u\|_{s,p,\mathbb {R}^{N}}^{p^{-}}$.

We recall the embedding theorem for the fractional Sobolev space with variable exponent as follows.

Lemma 2.4

(Subcritical embeddings, [29])

We have:

• (1) $W^{s,p(\cdot,\cdot)}(\varOmega)\hookrightarrow \hookrightarrow L^{r(\cdot)}(\varOmega)$, ifΩis a bounded Lipschitz domain and$r\in C_{+}(\overline {\varOmega})$such that$r(x)<\frac {Np(x)}{N-sp(x)}=:p^{*}_{s}(x)$for all$x\in \overline{\varOmega}$;

• (2) $W^{s,p(\cdot,\cdot)}(\mathbb {R}^{N})\hookrightarrow L^{r(\cdot)}(\mathbb{R}^{N})$for any uniformly continuous function$r\in C_{+}(\mathbb{R}^{N})$satisfying$p(x)\leq r(x)$for all$x\in\mathbb {R}^{N}$and$\inf_{x\in\mathbb{R}^{N}}(p_{s}^{\ast}(x)-r(x))>0$;

• (3) $W^{s,p(\cdot,\cdot)}(\mathbb {R}^{N})\hookrightarrow \hookrightarrow {L_{loc}^{r(\cdot)}}(\mathbb {R}^{N})$for any$r\in C_{+}(\mathbb {R}^{N})$satisfying$r(x)< p_{s}^{\ast}(x)$for all$x\in\mathbb {R}^{N}$.

Next, we consider the case that the potential function $\mathcal{V}$ satisfies

• (V) $\mathcal{V}\in C(\mathbb {R}^{N})$, $\inf_{x\in\mathbb {R}^{N}} \mathcal{V}(x)>0$, and meas$\{x\in\mathbb {R}^{N}:\mathcal{V}(x)\le \mathcal{V}_{0} \}<+\infty$ for all $\mathcal{V}_{0}\in\mathbb {R}$.

On the linear subspace

$$\begin{aligned} E&:= \biggl\{ u \in W^{s,p(\cdot,\cdot)}\bigl(\mathbb{R}^{N}\bigr):\\&\quad \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx\,dy + \int_{\mathbb {R}^{N}} \mathcal{V}(x) \bigl\vert u(x) \bigr\vert ^{p(x)} \,dx < + \infty \biggr\} ,\end{aligned} $$

we endow the norm

$$\Vert u \Vert _{E}:=\inf \biggl\{ \lambda>0: \int_{\mathbb{R}^{N}} \mathcal{V}(x) \biggl\vert \frac{u}{\lambda} \biggr\vert ^{p(x)}\,dx+ \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{\lambda^{p(x,y)} \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy < 1 \biggr\} . $$

Then $(E, \|\cdot\|_{E})$ is continuously embedded into $W^{s,p(\cdot ,\cdot)}$ as a closed subspace. Therefore, $(E, \|\cdot\|_{E})$ is also a separable reflexive Banach space. Let $E^{*}$ be a dual space of E. Furthermore, $\langle\cdot,\cdot \rangle$ denotes the pairing of E and its dual $E^{*}$.

With the aid of Lemma 2.4, the proof of the following assertion is essentially the same as in those of Lemma 2.6 in [2].

Lemma 2.5

If the potential function$\mathcal{V}$satisfies the assumption (V), then:

• (1) we have a compact embedding$E\hookrightarrow L^{p(\cdot )}(\mathbb {R}^{N})$;

• (2) for any measurable function$q:\mathbb {R}^{N}\to\mathbb {R}$with$p(x)< q(x)$for all$x\in \mathbb {R}^{N}$, there is a compact embedding$E\hookrightarrow L^{q(\cdot)}(\mathbb {R}^{N})$if$\inf_{x\in\mathbb {R}^{N}}(p^{*}(x)-q(x))>0$.

3 Existence of solutions

In this section, the existence of nontrivial weak solutions for ($P_{\lambda }$) is shown by applying the mountain pass theorem and a variant of Ekeland variational principle under suitable assumptions.

Definition 3.1

We say that $u\in E$ is a weak solution of ($P_{\lambda }$) if

$$\begin{aligned} &M \bigl([u]_{s,p(\cdot,\cdot)} \bigr) \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{{p(x,y)-2}}(u(x)-u(y))(w(x)-w(y))}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \\ &\qquad+ \int_{\mathbb {R}^{N}} \mathcal{V}(x) \lvert u \rvert ^{p(x)-2}uw \,dx\\&\quad=\lambda \int_{\mathbb {R}^{N}}\rho(x) \vert u \vert ^{r(x)-2}u w\, dx+ \int_{\mathbb {R}^{N}}h(x,u) w\,dx \end{aligned}$$

for any $w \in E$, where

$$[u]_{s,p(\cdot,\cdot)}:= \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{p(x,y) \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy. $$

Let us define the functional $\varPhi: E \to\mathbb {R}$ by

$$ \varPhi(u) = \mathcal{M} \bigl([u]_{s,p(\cdot,\cdot)} \bigr) + \int _{\mathbb {R}^{N}}\frac{\mathcal{V}(x)}{p(x)} \lvert u \rvert ^{p(x)}\,dx. $$

The following lemma can be proved by using arguments as in [43, Lemma 2].

Lemma 3.2

If (V) and (M1) hold, then the functional$\varPhi:E\to\mathbb {R}$is of class${C}^{1}(E,\mathbb {R})$and

$$ \begin{aligned}[b] \bigl\langle {\varPhi^{\prime}(u),w} \bigr\rangle &=M \bigl([u]_{s,p(\cdot,\cdot )} \bigr) \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{{p(x,y)-2}}(u(x)-u(y))(w(x)-w(y))}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy\hspace{-24pt} \\ &\quad+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert u \rvert ^{p(x)-2}uw \,dx, \end{aligned} $$

(3.1)

for any$u, w \in E$. Moreover, Φis weakly lower semi-continuous inE.

Proof

It is not difficult to prove that Φ has Fréchet derivative in E and (3.1) holds for any $u, w \in E$. Now, let $\{ z_{n} \}_{n} \subset E$ and $z \in E$ satisfy $z_{n} \to z$ strongly in E as $n \to\infty$. Without loss of generality, we assume that $z_{n} \to z$ a.e. in $\mathbb {R}^{N}$. Then the sequence

$$\biggl\{ \frac{ \vert z_{n}(x) - z_{n}(y) \vert ^{{p(x,y)-2}}(z_{n}(x)- z_{n}(y))}{ \vert x-y \vert ^{(N+sp(x,y))/p^{\prime}(x,y)}} \biggr\} _{n} $$

is bounded in $L^{p^{\prime}(\cdot,\cdot)}(\mathbb{R}^{N}\times \mathbb{R}^{N})$, as well as a.e. in $\mathbb{R}^{N}\times\mathbb{R}^{N}$

$$\begin{gathered} \mathcal{U}_{n} (x,y):=\frac{ \vert z_{n}(x) - z_{n}(y) \vert ^{{p(x,y)-2}}(z_{n}(x)- z_{n}(y))}{ \vert x-y \vert ^{(N+sp(x,y))/p^{\prime}(x,y)}} \\ \quad\longrightarrow\quad\mathcal{U}(x,y):=\frac{ \vert z(x) - z(y) \vert ^{{p(x,y)-2}}(z(x)- z(y))}{ \vert x-y \vert ^{(N+sp(x,y))/p^{\prime}(x,y)}}.\end{gathered} $$

Thus, the Brezis–Lieb lemma (see [9]) implies

$$ \begin{aligned}[b] &\lim_{n\to\infty} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\vert \mathcal{U}_{n} (x,y) - \mathcal{U}(x,y) \bigr\vert ^{p^{\prime}(x,y)} \,dx \,dy \\ &\quad=\lim_{n\to\infty} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \biggl( \frac{ \vert z_{n} (x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} - \frac { \vert z(x)-z(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \biggr) \,dx \,dy. \end{aligned} $$

(3.2)

The fact that $z_{n} \rightarrow z$ strongly in E yields

$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \biggl( \frac{ \vert z_{n} (x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} - \frac { \vert z(x)-z(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \biggr) \,dx \,dy = 0. $$

Moreover, the continuity of M implies that

$$ \lim_{n\to\infty} M \bigl([ z_{n} ]_{s,p(\cdot,\cdot)} \bigr) = M \bigl([z]_{s,p(\cdot,\cdot)} \bigr). $$

(3.3)

From (3.2) it follows that

$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\vert \mathcal{U}_{n} (x,y) - \mathcal{U}(x,y) \bigr\vert ^{p^{\prime}(x,y)} \,dx \,dy = 0. $$

(3.4)

Similarly,

$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}} \mathcal{V}(x) \bigl\vert \bigl\vert z_{n}(x) \bigr\vert ^{{p(x)-2}}z_{n}(x)- \bigl\vert z(x) \bigr\vert ^{{p(x)-2}}z(x) \bigr\vert ^{p^{\prime}(x)} \,dx = 0. $$

(3.5)

Combining (3.3)–(3.5) with the Hölder inequality, we have

$$ \bigl\Vert \varPhi^{\prime}(z_{n} ) - \varPhi^{\prime}(z) \bigr\Vert _{E^{*}} = \sup_{w \in E, \Vert w \Vert _{E}=1}{ \bigl\vert \bigl\langle {\varPhi^{\prime}( z_{n} )- \varPhi^{\prime}(z),w} \bigr\rangle \bigr\vert } \longrightarrow0 $$

as $n \rightarrow\infty$. Hence, $\varPhi\in C^{1} (E, \mathbb {R} )$. Finally, notice that the map $w \mapsto[w]_{s,p(\cdot,\cdot)}$ is lower semi-continuous in the weak topology of $W^{s,p(\cdot,\cdot )}(\mathbb{R}^{N})$ and $\mathcal{M}$ is nondecreasing and continuous on $\mathbb{R}_{0}^{+}$, so that $w \mapsto\mathcal{M} ( [w]_{s,p(\cdot,\cdot)} )$ is lower semi-continuous in the weak topology of $W^{s,p(\cdot,\cdot)}(\mathbb{R}^{N})$. Indeed, we can define $\gamma: W^{s,p(\cdot,\cdot)}(\mathbb{R}^{N}) \rightarrow\mathbb {R}$ as follows:

$$ \gamma(w)= \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\vert w(x)-w(y) \bigr\vert ^{{p(x,y)}} \vert x-y \vert ^{-N-sp(x,y)} \,dx \,dy. $$

It is easy to see that $\gamma\in C^{1} ( W^{s,p(\cdot,\cdot )}(\mathbb{R}^{N}) )$ and γ is a convex functional in $W^{s,p(\cdot,\cdot)}(\mathbb{R}^{N})$. By Corollary 3.8 in [13], we obtain $\gamma(w) \leq\lim\inf_{n \rightarrow\infty} \gamma( w_{n} )$. Hence, it is easy to see that Φ is weakly lower semi-continuous in E (see [46], Lemma 3.3 for more details). □

Let $H(x,t)=\int_{0}^{t}h(x,s)\,ds$. Assume that:

• (A1) $p, q, r\in C_{+}(\mathbb {R}^{N})$ and $1< r_{-}\leq r_{+}< p_{-}\le p_{+}< q_{-}\le q_{+}< p_{s}^{*}(x)$ for all $x\in\mathbb {R}^{N}$.

• (A2) $0\leq\rho\in L^{\frac{p(\cdot)}{p(\cdot)- r(\cdot )}}(\mathbb {R}^{N})\cap L^{\infty}(\mathbb {R}^{N})$ with meas$\{x\in \mathbb {R}^{N}:\rho(x)\neq0 \}>0$.

• (H1) $h: \mathbb {R}^{N}\times\mathbb {R} \to\mathbb {R}$ satisfies the Carathéodory condition.

• (H2) There exists nonnegative function $\sigma\in L^{\infty }(\mathbb {R}^{N})$ such that

  $$\bigl\lvert h(x,t) \bigr\rvert \le\sigma(x) \lvert t \rvert^{q(x)-1}, $$

  for all $(x,t)\in\mathbb {R}^{N}\times\mathbb {R}$.

• (H3) There exists a positive constant θ such that $\theta> \vartheta p^{+}$ and

  $$0< \theta H(x,t)\le h(x,t)t, \quad\text{for all } t \in\mathbb {R}\setminus\{0\} \text{ and } x \in\mathbb {R}^{N}, $$

  where ϑ is given in (M2).

• (H4) $H(x,t)\geq0$ for all $(x,t)\in\mathbb {R}^{N}\times\mathbb {R}$.

Let the functional $\varPsi_{\lambda}:E\to\mathbb {R}$ be defined by

$$\varPsi_{\lambda}(u) = \lambda \int_{\mathbb {R}^{N}}\frac{\rho (x)}{r(x)} \lvert u \rvert^{r(x)}\,dx+ \int_{\mathbb {R}^{N}}H(x,u)\,dx. $$

Then it is easy to check that $\varPsi_{\lambda} \in C^{1}(E,\mathbb {R})$, and its Fréchet derivative is

$$ \bigl\langle {\varPsi_{\lambda}^{\prime}}(u),w \bigr\rangle =\lambda \int_{\mathbb {R}^{N}}\rho \vert u \vert ^{r(x)-2}uw\,dx + \int_{\mathbb {R}^{N}}h(x,u)w\,dx, $$

for any $u,w \in E$. Subsequently, the functional $I_{\lambda}:E\to\mathbb {R}$ is defined by

$$ I_{\lambda}(u)=\varPhi(u)-\varPsi_{\lambda}(u). $$

(3.6)

Then according to Lemma 3.2, it follows that the functional $I_{\lambda}\in C^{1}(E,\mathbb {R})$, and its Fréchet derivative is

$$\begin{aligned} \bigl\langle I_{\lambda}^{\prime}(u),w \bigr\rangle ={}&M \bigl( [u]_{s,p(\cdot,\cdot)} \bigr) \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{{p(x,y)-2}}(u(x)-u(y))(w(x)-w(y))}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \\ &+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert u \rvert ^{p(x)-2}uw \,dx-\lambda \int_{\mathbb {R}^{N}}\rho(x) \vert u \vert ^{r(x)-2}u w\, dx- \int_{\mathbb {R}^{N}}h(x,u)w\,dx, \end{aligned}$$

for any $u,w\in E$.

Lemma 3.3

Assume that (A1)–(A2) and (H1)–(H2) hold. Then$\varPsi _{\lambda}$and$\varPsi_{\lambda}^{\prime}$are weakly strongly continuous onEfor any$\lambda>0$.

Proof

Let $\{z_{n}\}$ be a sequence in E such that $z_{n}\rightharpoonup z$ in E as $n\to\infty$. Since $\{z_{n}\}$ is bounded in E, Lemma 2.5 guarantees that there exists a subsequence such that

$$ \begin{gathered}z_{n_{k}}(x) \to z(x) \quad\text{a.e. in } \mathbb {R}^{N} \quad \text{and} \\ z_{n_{k}}\to z \quad\text{in } L^{p(\cdot)}\bigl(\mathbb {R}^{N}\bigr) \cap L^{q(\cdot)} \bigl(\mathbb {R}^{N}\bigr) \quad\text{as } k\to\infty.\end{gathered} $$

(3.7)

First we prove that $\varPsi_{\lambda}$ is weakly strongly continuous in E. By the convergence principle, there exists a function $g\in L^{p(\cdot)}(\mathbb {R}^{N}) \cap L^{q(\cdot)} (\mathbb {R}^{N})$ such that $|z_{n_{k}}(x)|\leq g(x)$ for all $k\in\mathbb {N}$ and for almost all $x\in\mathbb {R}^{N}$. Therefore from (H2) and Lemma 2.1, it follows from the Young inequality that

$$\begin{aligned} &\lambda \int_{\mathbb {R}^{N}}{\frac{\rho(x)}{r(x)} \lvert z_{n_{k}} \rvert^{r(x)}}\, dx+ \int_{\mathbb {R}^{N}} \bigl\vert H(x,z_{n_{k}}) \bigr\vert \, dx \\ &\quad\leq\frac{\lambda}{r_{-}} \int_{\mathbb {R}^{N}} \bigl\lvert \rho (x) \bigr\rvert \bigl\vert z_{n_{k}}(x) \bigr\vert ^{r(x)}\,dx+\frac{1}{q_{-}} \int_{\mathbb {R}^{N}}{\sigma(x)} \bigl\vert z_{n_{k}}(x) \bigr\vert ^{q(x)}\, dx \\ &\quad\leq\frac{\lambda}{r_{-}} \int_{\mathbb {R}^{N}}\frac {2(p(x)-r(x))}{p(x)} \bigl\lvert \rho(x) \bigr\rvert ^{\frac{p(x)}{p(x)-r(x) }}+\frac{r(x)}{p(x)} \bigl\lvert \bigl\lvert z_{n_{k}}(x) \bigr\rvert ^{{r(x)}} \bigr\rvert ^{\frac{p(x)}{r(x)}} \,dx\\&\qquad + \frac{ \Vert \sigma \Vert _{L^{\infty}(\mathbb {R}^{N})}}{q_{-}} \int_{\mathbb {R}^{N}} \bigl\vert z_{n_{k}}(x) \bigr\vert ^{q(x)}\, dx \\ &\quad\leq C_{0} \biggl[ \int_{\mathbb {R}^{N}} \bigl\lvert \rho(x) \bigr\rvert ^{\frac{p(x)}{p(x)-r(x) }}+ \bigl\lvert g(x) \bigr\rvert ^{p(x)}\,dx + \int_{\mathbb {R}^{N}} \bigl\vert g(x) \bigr\vert ^{q(x)} \,dx \biggr], \end{aligned}$$

for some positive constant $C_{0}$, and so the integral at the left-hand side is dominated by an integrable function. Since the function h satisfies the Carathéodory condition by (H1), it follows from (3.7) that

$$\frac{\rho(x)}{r(x)} \lvert z_{n_{k}} \rvert^{r(x)} \to \frac{\rho(x)}{r(x)} \lvert z \rvert^{r(x)} \quad\text{and} \quad H(x,z_{n_{k}})\to H(x,z) \quad\text{as } k\to\infty, $$

for almost all $x\in\mathbb {R}^{N}$. Therefore, Lebesgue’s dominated convergence theorem tells us that

$$\begin{gathered} \lambda \int_{\mathbb {R}^{N}}\frac{\rho(x)}{r(x)} \vert z_{n_{k}} \vert ^{r(x)}z_{n_{k}}\,dx+ \int_{\mathbb {R}^{N}} H(x,z_{n_{k}})\,dx \\\quad\to\lambda \int_{\mathbb {R}^{N}}\frac{\rho(x)}{r(x)} \vert z \vert ^{r(x)}z\,dx+ \int_{\mathbb {R}^{N}} H(x,z)\,dx \quad \text{as } k\to\infty,\end{gathered} $$

that is, $\varPsi_{\lambda}(z_{n_{k}}) \to\varPsi_{\lambda} (z)$ as $k\to \infty$. Thus $\varPsi_{\lambda}$ is weakly strongly continuous in E.

Next, we show that $\varPsi_{\lambda}^{\prime}$ is weakly strongly continuous in $E^{*}$. First of all, we note that

$$\begin{aligned} & \int_{\mathbb {R}^{N}} \bigl\lvert \rho(x) \vert z_{n_{k}} \vert ^{r(x)-2}z_{n_{k}}-\rho (x) \vert z \vert ^{r(x)-2}z \bigr\rvert ^{r^{\prime}(x)}\,dx \\ &\quad\leq C_{1} \int_{\mathbb {R}^{N}} \bigl\lvert \rho(x) \bigr\rvert ^{\frac {1}{r(x)-1}} \bigl\lvert \rho(x) \bigr\rvert \bigl( \vert z_{n_{k}} \vert ^{r(x)}+ \vert z \vert ^{r(x)}\bigr)\, dx \\ &\quad\leq C_{2} \int_{\mathbb {R}^{N}} \bigl\lvert \rho(x) \bigr\rvert \bigl( \vert z_{n_{k}} \vert ^{r(x)}+ \vert z \vert ^{r(x)}\bigr)\, dx \\ &\quad\leq C_{2} \int_{\mathbb {R}^{N}}\frac{2(p(x)-r(x))}{p(x)} \bigl\lvert \rho (x) \bigr\rvert ^{\frac{p(x)}{p(x)-r(x) }}+\frac{r(x)}{p(x)} \lvert z_{n_{k}} \rvert^{p(x)}+\frac {r(x)}{p(x)} \lvert z \rvert^{p(x)}\,dx, \end{aligned}$$

(3.8)

for some positive constants $C_{1}$, $C_{2}$. Due to (H2) and Lemma 2.1, we obtain

$$\begin{aligned} \int_{\mathbb {R}^{N}} \bigl\lvert h(x,z_{n_{k}})-h(x,z) \bigr\rvert ^{q^{\prime}(x)}\,dx &\leq C_{3} \int_{\mathbb {R}^{N}} \bigl\lvert h(x,z_{n_{k}}) \bigr\rvert ^{q^{\prime}(x)}+ \bigl\lvert h(x,z) \bigr\rvert ^{{q^{\prime}(x)}}\, dx \\ &\leq C_{4} \int_{\mathbb {R}^{N}} \lvert z_{n_{k}} \rvert^{q(x)} + \lvert z \rvert^{q(x)}\,dx, \end{aligned}$$

(3.9)

for some positive constants $C_{3}$, $C_{4}$. Invoking (3.7)–(3.9) and the convergence principle, one has

$$\bigl\lvert \rho(x) \lvert z_{n_{k}} \rvert^{r(x)-2}-\rho (x) \lvert z \rvert^{r(x)-2} \bigr\rvert ^{r^{\prime}(x)} \leq f_{1}(x) $$

and

$$\bigl\lvert h(x,z_{n_{k}})-h(x,z) \bigr\rvert ^{q^{\prime}(x)} \leq f_{2}(x), $$

for almost all $x\in\mathbb {R}^{N}$ and for some $f_{1}, f_{2} \in L^{1}(\mathbb {R}^{N})$, and thus $\rho(x)|z_{n_{k}}|^{r(x)-2}z_{n_{k}}\to \rho (x)|z|^{r(x)-2}z$ and $h(x,z_{n_{k}})\to h(x,z)$ as $k\to\infty$ for almost all $x\in\mathbb {R}^{N}$. This together with the Lebesgue dominated convergence theorem yields

$$\begin{aligned} & \bigl\Vert \varPsi_{\lambda}^{\prime}(z_{n_{k}})- \varPsi_{\lambda}^{\prime}(z) \bigr\Vert _{E^{*}}\\&\quad=\sup _{ \Vert w \Vert _{E}\leq1} \bigl\lvert \bigl\langle \varPsi_{\lambda }^{\prime}(z_{n_{k}})- \varPsi_{\lambda}^{\prime}(z),w\bigr\rangle \bigr\rvert \\ &\quad=\sup_{ \Vert w \Vert _{E}\leq1} \biggl\lvert \lambda \int_{\mathbb {R}^{N}}\bigl(\rho (x) \vert z_{n_{k}} \vert ^{r(x)-2}z_{n_{k}}- \rho(x) \vert z \vert ^{r(x)-2}z \bigr)w\,dx + \int_{\mathbb {R}^{N}}\bigl(h(x,z_{n_{k}})-h(x,z)\bigr){w}\,dx \biggr\rvert \\ &\quad\leq2 \bigl(\lambda \bigl\Vert \rho(x) \vert z_{n_{k}} \vert ^{r(x)-2}z_{n_{k}}- \rho (x) \vert z \vert ^{r(x)-2}z \bigr\Vert _{L^{{r^{\prime}(\cdot)}}(\mathbb {R}^{N})}+ \bigl\Vert {h(x,z_{n_{k}})-h(x,z)} \bigr\Vert _{L^{{q^{\prime}(\cdot)}}(\mathbb {R}^{N})} \bigr) \to0 \end{aligned}$$

as $k\to\infty$. Consequently, we derive that $\varPsi_{\lambda }^{\prime}(z_{n_{k}})\to\varPsi_{\lambda}^{\prime}(z)$ in ${E^{*}}$ as $k\to\infty$. This completes the proof. □

Combining Lemmas 3.2 and 3.3, we see that $I_{\lambda} \in{C}^{1}(E,\mathbb {R})$ and $I_{\lambda}$ is weakly semi-continuous in E. Before going to the proofs of our main results, we consider some useful lemmas and consequences presented below. The following assertion means that $I_{\lambda}$ satisfies the geometric condition in the mountain pass theorem.

Lemma 3.4

Assume that (V), (M1)–(M2), (A1)–(A2) and (H1)–(H4) hold. Let$I_{\lambda}$be defined as in (3.6). Then we have the followings:

• (1) There exists a positive constant$\lambda^{*}$such that for any$\lambda\in(0, \lambda^{*} )$we can choose$R>0$and$0<\delta <1$such that$I_{\lambda}(u) \geq R >0$for all$u \in E$with$\|u\| _{E} = \delta$;

• (2) there exists$\phi\in E$, $\phi>0$such that$I_{\lambda }(t\phi) \to- \infty$as$t \to+\infty$;

• (3) there exists$\psi\in E$, $\psi>0$such that$I_{\lambda }(t\psi) <0$as$t \to0^{+}$.

Proof

Let us prove condition (1). By Lemma 2.5, there exists a positive constant $C_{5}$ such that $\|u\|_{L^{\gamma(\cdot)}(\mathbb {R}^{N})}\le C_{5}\|u\|_{E}$ for $p(x) \leq\gamma(x) < p_{s}^{\ast}(x)$. Assume that $\|u\|_{E}<1$. Then it follows from (H2), Proposition 2.3, and Lemmas 2.1, 2.2(2) and 2.5 that

$$\begin{aligned} I_{\lambda}(u)&=\mathcal{M} \bigl( [u ]_{s,p(\cdot,\cdot)} \bigr) + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert u \rvert^{p(x)} \biggr)\,dx -\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert u \rvert^{r(x)} \biggr)\,dx- \int_{\mathbb {R}^{N}} H(x,u)\,dx \\ &\ge\frac{m_{0}}{\vartheta} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{p(x,y) \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy + \int _{\mathbb {R}^{N}}\frac{\mathcal{V}(x)}{p(x)} \lvert u \rvert ^{p(x)}\,dx \\ &\quad- 2\frac{\lambda}{r_{+}} \Vert \rho \Vert _{L^{\frac{p(\cdot)}{p(\cdot )-r(\cdot)}}(\mathbb {R}^{N})}C_{5} \max\bigl\{ \Vert u \Vert _{E}^{r_{+}}, \Vert u \Vert _{E}^{r _{-}}\bigr\} \\&\quad-\frac{ \Vert \sigma \Vert _{L^{\infty}(\mathbb {R}^{N})}}{q_{+}}\max\bigl\{ \Vert u \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q_{+}}, \Vert u \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q}\bigr\} \\ &\ge\min{ \biggl\{ \frac{m_{0}}{\vartheta}, \frac{1}{p_{+}} \biggr\} } \Vert u \Vert ^{p_{+}}_{E}- 2\frac{\lambda}{r_{+}} \Vert \rho \Vert _{L^{\frac {p(\cdot)}{p(\cdot)-r(\cdot)}}(\mathbb {R}^{N})}C_{5} \Vert u \Vert _{E}^{r _{-}}-\frac{C_{5}}{q_{+}} \Vert \sigma \Vert _{L^{\infty}(\mathbb {R}^{N})} \Vert u \Vert ^{q_{-}}_{E} \\ &\ge \biggl(\min{ \biggl\{ \frac{m_{0}}{\vartheta}, \frac{1}{p_{+}} \biggr\} }- 2\frac{\lambda}{r_{+}}C_{6} \Vert u \Vert _{E}^{r _{-}-p_{+}}-\frac{1}{q_{+}}C_{7} \Vert u \Vert ^{q_{-}-p_{+}}_{E} \biggr) \Vert u \Vert ^{p_{+}}_{E}, \end{aligned}$$

(3.10)

for positive constants $C_{6}$, $C_{7}$. Let us define the function $g_{\lambda}:(0,\infty) \to\mathbb {R}$ by

$$g_{\lambda}(t)=2C_{6}\frac{\lambda}{r_{+}}t^{r _{-}-p_{+}}+C_{7} \frac{1}{q_{+}}t^{q_{-}-p_{+}}. $$

Then it is clear that $g_{\lambda}$ has a local minimum at the point $t_{0}= ({\lambda\frac{p_{+}-r_{-}}{q_{-}-p_{+}}\cdot\frac {q_{+}}{r_{+}}\cdot\frac{2C_{6}}{C_{7}}} )^{\frac{1}{q_{+}-r_{+}}}$ and so

$$\lim_{\lambda\to0^{+}}g_{\lambda}(t_{0})=0. $$

Thus there is $\lambda^{\ast}>0$ such that for each $\lambda\in (0,\lambda^{\ast})$, there exist $R>0$ small enough and $\delta>0$ such that $I_{\lambda}(u)\ge\delta>0$ for any $u \in E$ with $\|u\| _{E} = R$.

Next we show condition (2). Note that condition (H3) implies

$$ H(x,s\eta)\ge s^{\theta}H(x,\eta), $$

(3.11)

for all $\eta\in\mathbb {R}$, $x \in\mathbb {R}^{N}$, and $s \ge1$.

Take $\phi\in E$ with $\phi>0$. Since $\mathcal{M}(\tau)\leq \mathcal{M}({1}) \tau^{\vartheta}$ for any $\tau\geq1$, it follows from (3.11) that

$$\begin{aligned} I_{\lambda}(t \phi)&=\mathcal{M} \bigl( [t \phi]_{s,p(\cdot,\cdot )} \bigr) + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert t \phi \rvert^{p(x)} \biggr)\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert t \phi \rvert^{r(x)} \biggr)\,dx- \int_{\mathbb {R}^{N}}H(x,t \phi)\,dx \\ &\le t^{\vartheta p^{+}} \biggl(\mathcal{M}(1){[ \phi]}^{\vartheta }_{s,p(\cdot,\cdot)} +\frac{1}{p^{-}} \int_{\mathbb {R}^{N}}\mathcal {V}(x) \lvert\phi \rvert^{p(x)} \,dx \biggr)\\&\quad-\lambda t^{r_{-}} \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert \phi \rvert^{r(x)} \biggr)\,dx-t^{\theta} \int_{\mathbb {R}^{N}}{H(x,\phi)}\,dx, \end{aligned}$$

for sufficiently large $t\ge1$. Since $\theta>\vartheta p^{+}>r_{+}$, we see that $I_{\lambda}(t \phi)\to-\infty$ as $t\to\infty$.

Finally it remains to prove condition (3). Choose $\psi\in E$ such that $\psi>0$. Let λ be fixed. For $t \in(0,1)$ small enough, from (H4) and Lemma 2.2, we obtain

$$\begin{aligned} I_{\lambda}(t\psi)&=\mathcal{M} \bigl( [t \psi]_{s,p(\cdot,\cdot )} \bigr) + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert t\psi \rvert^{p(x)} \biggr)\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert t\psi \rvert^{r(x)} \biggr)\,dx- \int_{\mathbb {R}^{N}}H(x,t\psi)\,dx \\ &\le t^{p^{-}} \biggl( \Bigl(\sup_{0 \leq\xi\leq\max\{\|u\| _{E}^{p^{-}}, \|u\|_{E}^{p ^{+}}\}} M(\xi) \Bigr) [ \psi]_{s,p(\cdot ,\cdot)} + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert\psi \rvert^{p(x)} \biggr)\,dx \biggr)\\&\quad-\lambda t^{r_{+}} \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert \psi \rvert^{r(x)} \biggr)\,dx. \end{aligned}$$

Since $p^{-}>r_{+}$, we see that $I_{\lambda}(t\psi)<0$ as $t\to 0^{+}$, as claimed. □

With the aid of Lemma 3.3, we will give that the energy functional $I_{\lambda}$ satisfies the Palais–Smale condition ($(PS)$-condition for short). This plays a key role in obtaining the existence of a nontrivial weak solution for the given problem. The basic idea of the proof of this assertion comes from [43].

Definition 3.5

We say that $I_{\lambda}$ satisfies the $(PS)$-condition in E, if any $(PS)$-sequence $\{z_{n}\} \subset E$, namely, $\{I_{\lambda }(z_{n})\}$ is bounded and $I^{\prime}_{\lambda}(z_{n})\to0$ as $n\to\infty$, admits a strongly convergent subsequence in E.

Lemma 3.6

If (V), (M1)–(M2), (A1)–(A2), and (H1)–(H4) hold, then the functional$I_{\lambda}$satisfies the$(PS)$-condition for any$\lambda>0$.

Proof

Let $\{z_{n}\}$ be a $(PS)$-sequence in E, i.e., there exists $K>0$ such that $| \langle I^{\prime}_{\lambda }(z_{n}),z_{n} \rangle| \leq K {\|z_{n}\|}_{E}$ and $| I_{\lambda}(z_{n})|\leq K$. It is first verified that the sequence $\{z_{n}\}$ is bounded in E. Suppose to the contrary that $\|z_{n}\|_{E} \to\infty$, in the subsequence sense, as $n \to\infty$. By assumption (M2), we deduce that

$$\begin{aligned} K + K { \Vert z_{n} \Vert }_{E} &\geq I_{\lambda}(z_{n})-\frac{1}{\theta} \bigl\langle I^{\prime}_{\lambda}(z_{n}),z_{n} \bigr\rangle \\ &=\mathcal{M} \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)- \frac {1}{\theta} M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int _{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr) \\ &\quad+ \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert z_{n} \rvert^{p(x)}-\frac{\mathcal{V}(x)}{\theta} \lvert z_{n} \rvert^{p(x)} \biggr)\,dx\\&\quad+\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{\theta} \lvert z_{n} \rvert^{r(x)}- \frac {\rho(x)}{r(x)} \lvert z_{n} \rvert^{r(x)} \biggr)\,dx \\ &\quad+ \int_{\mathbb {R}^{N}} \biggl(\frac{1}{\theta}h(x,z_{n})z_{n}- H(x,z_{n}) \biggr)\,dx \\ &\ge \biggl(\frac{1}{\vartheta}-\frac{1}{\theta} \biggr) M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \, dy \biggr) \\ &\quad+ \biggl(\frac{1}{p_{+}}-\frac{1}{\theta} \biggr) \int_{\mathbb {R}^{N}}{\mathcal{V}(x)} \lvert z_{n} \rvert^{p(x)}\, dx-\lambda \biggl(\frac{1}{r_{+}}- \frac{1}{\theta} \biggr) \int _{\mathbb {R}^{N}}{\rho(x)} \lvert z_{n} \rvert^{r(x)}\,dx \\ &\quad- \int_{\mathbb {R}^{N}} \biggl(H(x,z_{n})- \frac{1}{\theta }h(x,z_{n})z_{n} \biggr)\,dx, \end{aligned}$$

where θ is the positive constant from (H3). Combining this with conditions (M1) and (H3), we have

$$\begin{aligned} &\min{ \biggl\{ \biggl(\frac{1}{\vartheta}-\frac{1}{\theta} \biggr)m_{0}, \frac{1}{p_{+}}-\frac{1}{\theta} \biggr\} }\\&\qquad\times \frac {1}{p^{+}} \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy+ \int_{\mathbb {R}^{N}}{\mathcal {V}(x) \lvert z_{n} \rvert^{p(x)}\,dx} \biggr) \\ &\qquad-\lambda \biggl(\frac{1}{r_{+}}-\frac{1}{\theta} \biggr) \int _{\mathbb {R}^{N}}{\rho(x)} \lvert z_{n} \rvert^{r(x)}\,dx \\&\quad\le K + K { \Vert z_{n} \Vert }_{E}. \end{aligned}$$

For n large enough, we may assume that $\|z_{n}\|_{E}>1$. Then it follows from (H3), Proposition 2.3 and Lemma 2.4(1) that

$$\frac{1}{p^{+}}\min{ \biggl\{ \biggl(\frac{1}{\vartheta}- \frac {1}{\theta} \biggr)m_{0}, \frac{1}{p_{+}}- \frac{1}{\theta} \biggr\} } \Vert z_{n} \Vert ^{p_{-}}_{E}-\lambda \biggl(\frac{1}{r_{+}}- \frac{1}{\theta } \biggr)C_{8} \Vert z_{n} \Vert ^{r_{+}}_{E}\le K + K { \Vert z_{n} \Vert }_{E}. $$

Since $\theta> \vartheta p^{+} > p_{+}>1$ and $p_{-}>r_{+} >1$, this is a contradiction. Hence the sequence $\{z_{n}\}$ is bounded in E. Passing to the limit, if necessary, to a subsequence, by Lemma 2.4, we have

$$ \begin{gathered} z_{n} \rightharpoonup z \quad\text{in }E,\qquad z_{n} \to z \quad\text{a.e. in }\mathbb {R}^{N}\quad \text{and} \\ z_{n} \to z \quad\text{in }L^{p(\cdot)} \bigl(\mathbb {R}^{N}\bigr) \text{ and in } L^{q(\cdot )}\bigl(\mathbb {R}^{N}\bigr)\end{gathered} $$

(3.12)

as $n\to\infty$. To prove that $\{z_{n}\}$ converges strongly to z in E, let $\varphi \in E$ be fixed and let $\tilde{\varPhi}_{\varphi}$ denote the linear functional on E defined by

$$ \tilde{\varPhi}_{\varphi}(w)= \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert \varphi(x)- \varphi(y) \vert ^{{p(x,y)-2}}(\varphi(x)-\varphi (y))(w(x)-w(y))}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy, $$

for all $w\in E$. Obviously, by the Hölder inequality, $\tilde {\varPhi}_{\varphi}$ is also continuous, as

$$\begin{aligned} \bigl\lvert \tilde{\varPhi}_{\varphi}(w) \bigr\rvert &\leq 2\biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert \varphi(x)- \varphi(y) \vert ^{{p(x,y)}}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \biggr)^{\frac{\tilde{p}_{1}}{\tilde{p}_{2}}} \\&\quad\times\biggl( \int _{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert w(x)-w(y) \vert ^{{p(x,y)}}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \biggr)^{\frac{1}{\tilde{p}_{2}}} \\ &\leq2 \Vert \varphi \Vert ^{\tilde{p}_{1}}_{E} \Vert w \Vert _{E}, \end{aligned}$$

for any $w\in E$, where $\tilde{p}_{1}$ is either $p^{+} -1$ or $p^{-}-1$ and $\tilde{p}_{2}$ is either $p^{+}$ or $p^{-}$. Hence, Eq. (3.12) yields

$$ \lim_{n\to\infty} \bigl[M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)-M \bigl( [z]_{s,p(\cdot,\cdot)} \bigr) \bigr]\tilde{\varPhi}_{u}(z_{n}-z)=0, $$

(3.13)

because the sequence $\{M ( [z_{n} ]_{s,p(\cdot,\cdot )} )-M ( [z]_{s,p(\cdot,\cdot)} ) \}$ is bounded in $\mathbb {R}$. Using (H2) and Lemma 2.2, it follows that

$$\begin{aligned} & \int_{\mathbb {R}^{N}} \bigl\lvert \bigl(h(x,z_{n})-h(x,z) \bigr) (z_{n}-z) \bigr\rvert \, dx \\ &\quad\le \int_{\mathbb {R}^{N}}\sigma(x) \bigl( \lvert z_{n} \rvert ^{q(x)-1}+ \lvert z \rvert^{q(x)-1}\bigr) \lvert z_{n}-z \rvert\, dx \\ &\quad\le2 \Vert \sigma \Vert _{L^{\infty}(\mathbb {R}^{N})} \bigl( \Vert z_{n} \Vert _{L^{q(\cdot )}(\mathbb {R}^{N})}^{q_{+}-1}+ \Vert z_{n} \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q_{-}-1}+ \Vert z \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q_{+}-1}+ \Vert z \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q_{-}-1} \bigr)\\&\qquad\times \Vert z_{n}-z \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}. \end{aligned}$$

Then, due to (3.12), one has

$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}}\bigl(h(x,z_{n})-h(x,z)\bigr) (z_{n}-z)\, dx=0. $$

(3.14)

Because $z_{n}\rightharpoonup z$ in E and $I_{\lambda}'(z_{n})\to0$ in $E^{*}$ as $n\to\infty$, we have

$$\bigl\langle I_{\lambda}'(z_{n})-I_{\lambda}'(z), z_{n}-z\bigr\rangle \to0 \quad\text{as } n\to\infty. $$

Hence, Eqs. (3.12)–(3.14) yield as $n\to\infty$

$$\begin{aligned} o(1)&=\bigl\langle I_{\lambda}'(z_{n})-I_{\lambda}'(z), z_{n}-z\bigr\rangle \\ &=M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)\tilde{\varPhi }_{z_{n}}(z_{n}-z)-M \bigl( [z ]_{s,p(\cdot,\cdot)} \bigr) \tilde{\varPhi }_{z}(z_{n}-z) \\ &\quad+ \bigl(M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)-M \bigl( [z ]_{s,p(\cdot,\cdot)} \bigr) \bigr)\tilde{\varPhi}_{z}(z_{n}-z)\\&\quad+ \int _{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert ^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx \\ &\quad-\lambda \int_{\mathbb {R}^{N}}\rho(x) \bigl( \lvert z_{n} \rvert ^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}z\bigr) (z_{n}-z)\,dx\\&\quad- \int _{\mathbb {R}^{N}}\bigl(h(x,z_{n})-h(x,z)\bigr) (z_{n}-z)\,dx \\ &=M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \bigl[\tilde{\varPhi }_{z_{n}}(z_{n}-z)-\tilde{\varPhi}_{z}(z_{n}-z) \bigr]\\&\quad+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx \\ &\quad-\lambda \int_{\mathbb {R}^{N}}\rho(x) \bigl( \lvert z_{n} \rvert ^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}z\bigr) (z_{n}-z)\,dx\\&\quad- \int _{\mathbb {R}^{N}}\bigl(h(x,z_{n})-h(x,z)\bigr) (z_{n}-z)\,dx+o(1), \end{aligned}$$

that is,

$$ \begin{aligned}[b] &\lim_{n\to\infty} \biggl(M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \bigl[\tilde{ \varPhi}_{z_{n}}(z_{n}-z)-\tilde{\varPhi}_{z}(z_{n}-z) \bigr]\\&\quad+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert ^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx \\ &\quad-\lambda \int_{\mathbb {R}^{N}}\rho(x) \bigl( \lvert z_{n} \rvert ^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}z\bigr) (z_{n}-z)\,dx \biggr)=0. \end{aligned} $$

By convexity, (M1), (V), and (H2) we have in particular

$$\begin{gathered} M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)\bigl[\tilde{\varPhi }_{z_{n}}(z_{n}-z)-\tilde{\varPhi}_{z}(z_{n}-z) \bigr]\ge0, \\ \mathcal{V}(x) \bigl( \lvert z_{n} \rvert^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx\ge0,\end{gathered} $$

and

$$\rho(x) \bigl( \lvert z_{n} \rvert^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}z\bigr) (z_{n}-z)\,dx\ge0. $$

It follows that

$$\begin{aligned}& \lim_{n\to\infty}\tilde{\varPhi}_{z_{n}}(z_{n}-z)- \tilde{\varPhi}_{z}(z_{n}-z)=0, \end{aligned}$$

(3.15)

$$\begin{aligned}& \lim_{n\to\infty} \int_{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert^{p(x)-2}z_{n}- \lvert z \rvert ^{p(x)-2}u\bigr) (z_{n}-z)\,dx=0, \end{aligned}$$

(3.16)

and

$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}}\rho(x) \bigl( \lvert z_{n} \rvert^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}u\bigr) (z_{n}-z)\,dx=0. $$

(3.17)

It should be noted that we have the well-known useful inequalities

$$ \lvert\xi-\eta \rvert^{p(x,y)}\le \textstyle\begin{cases} C_{9}( \lvert\xi \rvert^{p(x,y)-2}\xi- \lvert\eta \rvert^{p(x,y)-2}\eta)\cdot(\xi-\eta) \quad\text{for } (x,y) \in\Delta_{1}, \\ C_{10}[( \lvert\xi \rvert^{p(x,y)-2}\xi- \lvert \eta \rvert^{p(x,y)-2}\eta)\cdot(\xi-\eta)]^{\frac {p(x,y)}{2}}\\ \quad{}\times( \lvert\xi \rvert^{p(x,y)}+ \lvert \eta \rvert^{p(x,y)})^{\frac{2-p(x,y)}{2}} \quad\text{for } (x,y) \in\Delta_{2} \text{ and } (\xi,\eta)\neq(0,0),\hspace{-12pt} \end{cases} $$

(3.18)

for all $\xi,\eta\in\mathbb {R}^{N}$, where $C_{9}$ and $C_{10}$ are positive constants depending only on $p(\cdot,\cdot)$, $\Delta_{1}=\{(x, y) \in\mathbb {R}^{N} \times\mathbb {R}^{N}: p(x,y)\ge 2\}$, and $\Delta_{2}=\{(x,y) \in\mathbb {R}^{N} \times\mathbb {R}^{N} : 1< p(x,y)<2\}$; see [35, Proposition 3.3].

It is now assumed that $(x,y) \in\Delta_{1}$. Then, by (3.15) and (3.18) as $n\to\infty$,

$$ \begin{aligned}[b] & \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert ( z_{n} - z)(x) - ( z_{n} - z)(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \, dx \,dy \\ &\quad= \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\vert z_{n} (x)- z_{n} (y)- z(x)+ z(y) \bigr\vert ^{{p(x,y)}} \vert x-y \vert ^{-(N+sp(x,y))} \,dx \,dy \\ &\quad\le C_{9} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl[ \bigl\vert z_{n} (x)-z_{n} (y) \bigr\vert ^{{p(x,y)-2}}\\&\qquad\times\bigl(z_{n} (x)-z_{n} (y)\bigr)- \bigl\vert z(x)-z(y) \bigr\vert ^{{p(x,y)-2}}\bigl(z(x)-z(y)\bigr)\bigr] \\ &\qquad\times\bigl(z_{n} (x)- z_{n} (y)- z(x)+ z(y)\bigr) \vert x-y \vert ^{-(N+sp(x,y))} \,dx \,dy \\ &\quad\le C_{9} \bigl(\tilde{\varPhi}_{z_{n}}(z_{n}-z)- \tilde{\varPhi }_{z}(z_{n}-z) \bigr)=o(1). \end{aligned} $$

(3.19)

Similarly, utilizing (V), (3.16) and (3.18) as $n\to \infty$,

$$ \int_{\Delta_{1}}\mathcal{V}(x) \lvert z_{n}-z \rvert ^{p(x)}\,dx\le C_{9} \int_{\Delta_{1}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert ^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx=o(1). $$

Subsequently, the case $(x, y) \in\Delta_{2}$ is considered. As $\{ z_{n}\}$ is bounded in E, there exists $K_{0} >0$ such that

$$\int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert z_{n} (x)-z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy\le K_{0} ,$$

for all $n\in\mathbb {N}$. By (3.15), (3.18) and Lemma 2.1, we have

$$ \begin{aligned}[b] & \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert ( z_{n} - z)(x) - ( z_{n} - z)(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \, dx \,dy \\ &\quad\le C_{10} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\{ \bigl[ \bigl\vert z_{n} (x)-z_{n} (y) \bigr\vert ^{{p(x,y)-2}}\\&\qquad\times\bigl(z_{n} (x)-z_{n} (y)\bigr)- \bigl\vert z(x)-z(y) \bigr\vert ^{{p(x,y)-2}}\bigl(z(x)-z(y)\bigr)\bigr] \\ &\qquad\times\bigl(z_{n} (x)- z_{n} (y)- z(x)+ z(y)\bigr)\bigr\} ^{\frac{p(x,y)}{2}}\bigl( \bigl\vert z_{n} (x)-z_{n} (y) \bigr\vert ^{p(x,y)}+ \bigl\vert z (x)-z (y) \bigr\vert ^{p(x,y)}\bigr)^{\frac{2-p(x,y)}{2}} \\ &\qquad\times \vert x-y \vert ^{-(N+sp(x,y))} \,dx \,dy \\ &\quad\le2 C_{11} \bigl(\tilde{\varPhi}_{z_{n}}(z_{n}-z)- \tilde{\varPhi }_{z}(z_{n}-z) \bigr)^{\alpha} \\ & \qquad\times \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy + \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z(x)- z(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \biggr)^{\beta}\\&\quad=o(1), \end{aligned} $$

where $C_{11} =2C_{10}(2K)^{\beta}$, α is either $p^{-} /2$ or $p^{+} /2$, and β is either $(2- p^{+} )/2$ or $(2- p^{-} )/2$. Similarly, by invoking (3.12), there is a positive constant L such that $\int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert z_{n} \rvert ^{p(x)}\,dx\le L$ for all $n\in\mathbb {N}$. Moreover, by Lemma 2.1, (3.16) and (3.18) as $n\to\infty$,

$$ \begin{aligned}[b] \int_{\Delta_{2}}\mathcal{V}(x) \lvert z_{n}-z \rvert ^{p(x)}\,dx&\le C_{12} \biggl( \int_{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2} u\bigr) (z_{n}-z)\,dx \biggr)^{\alpha}\\&=o(1),\end{aligned} $$

(3.20)

where $C_{12}=4C_{10}(2L)^{\beta}$. From (3.19) and (3.20), we obtain

$$\int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert z_{n}(x)-z_{n}(y) \vert ^{p(x,y)-2}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \, dy+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert z_{n}-z \rvert ^{p(x)}\,dx\to0 \quad\text{as } n\to\infty. $$

Therefore, $\|z_{n}-z\|_{E} \to0$ as $n\to\infty$. Hence, $I_{\lambda}$ satisfies the $(PS)$-condition. This completes the proof. □

The proof of the following theorem can be found in [10, 30, 43], however, we will give the proof for the reader’s convenience.

Theorem 3.7

Let (V), (M1)–(M2), (A1)–(A2), and (H1)–(H4) hold. Then there exists a positive constant$\lambda^{*}$such that for any$\lambda\in(0, \lambda^{*} )$, the functional$I_{\lambda}$admits at least two nontrivial different solutions inE.

Proof

Thanks to Lemmas 3.4 and 3.6, there exists $\lambda ^{*}>0$ such that for all $\lambda\in(0, \lambda^{*})$, $I_{\lambda }$ satisfies the mountain pass geometry and $(PS)$-condition. By employing the mountain pass theorem, we infer that there exists a critical point $u_{0} \in E$ of $I_{\lambda}$ with $I_{\lambda }(u_{0})= \overline{d}>0=I_{\lambda}(0)$. Hence $u_{0}$ is a nontrivial weak solution of the problem ($P_{\lambda }$). Let us denote $d:=\inf_{u \in\overline{B}_{r}}{I_{\lambda}(u)}$ where ${B}_{r} := \{ u \in E : \|u\|_{E} < r \}$ with a boundary $\partial{B}_{r}$. Then by (3.10) and Lemma 3.4(3), we have $-\infty< d < 0$. Putting $0< \epsilon< \inf_{u \in\partial{B}_{r}}{I_{\lambda }(u)}-d$, by Theorem 1.1 in [25] (see also [30]), we can find $u_{\epsilon} \in\overline{B}_{r}$ such that we have the well-known useful inequalities

$$ \textstyle\begin{cases} I_{\lambda}( u_{\epsilon} ) \leq d + \epsilon,\\ I_{\lambda}( u_{\epsilon} ) < I_{\lambda}(u) + \epsilon \Vert u-u_{\epsilon} \Vert _{E},\quad\text{for all } u \in\overline{B}_{r}, u \neq u_{\epsilon}. \end{cases} $$

(3.21)

This implies that $u_{\epsilon} \in{B}_{r}$ since $I_{\lambda}( u_{\epsilon} ) \leq d + \epsilon< \inf_{u \in\partial {B}_{r}}{I_{\lambda}(u)}$. From these facts we see that $u_{\epsilon}$ is a local minimum of $\widetilde{I}_{\lambda}( u ) = I_{\lambda}(u) + \epsilon\| u-u_{\epsilon} \|_{E}$. Now by taking $u = u_{\epsilon} +tw$ for $w \in{B}_{1}$ and sufficiently small $t>0$, from (3.21), we deduce

$$0 \leq\frac{\widetilde{I}_{\lambda}( u_{\epsilon} +tw )-\widetilde {I}_{\lambda}( u_{\epsilon} ) }{t} = \frac{{I_{\lambda}}( u_{\epsilon} +tw )-{I_{\lambda}}( u_{\epsilon} ) }{t} + \epsilon \Vert w \Vert _{E}. $$

Therefore, letting $t \rightarrow0+$, we get

$$\bigl\langle I_{\lambda}^{\prime}(u_{\epsilon}),w \bigr\rangle + \epsilon \Vert w \Vert _{E} \geq0. $$

Replacing w by −w in the argument above, we have

$$- \bigl\langle I_{\lambda}^{\prime}(u_{\epsilon}),w \bigr\rangle +\epsilon \Vert w \Vert _{E} \geq0. $$

Thus, one has

$$\bigl\vert \bigl\langle I_{\lambda}^{\prime}(u_{\epsilon}),w \bigr\rangle \bigr\vert \leq\epsilon \Vert w \Vert _{E}, $$

for any $w \in\overline{B}_{1}$. Hence

$$ \bigl\Vert I_{\lambda}^{\prime}(u_{\epsilon}) \bigr\Vert _{E^{*}} \leq\epsilon. $$

(3.22)

Using (3.21) and (3.22), we can choose a sequence $\{ z_{n} \} \subset B_{r}$ such that

$$ \textstyle\begin{cases} I_{\lambda}( z_{n} ) \rightarrow c \quad\text{as } n \to\infty, \\ \Vert I_{\lambda}^{\prime}(z_{n}) \Vert _{E^{*}} \to0 \quad\text{as } n \to \infty. \end{cases} $$

(3.23)

Thus, $\{ z_{n} \}$ is a bounded $(PS)$-sequence in the reflexive Banach space E. According to Lemma 3.6, $\{ z_{n} \}$ has a subsequence $\{ z_{n_{k}} \}$ such that $z_{n_{k}} \to u_{1}$ in E as $k \to\infty$. This together with (3.23) yields $I_{\lambda }( u_{1} ) =d$ and $I_{\lambda}^{\prime}( u_{1} )=0$. Hence $u_{1}$ is a nontrivial nonnegative solution of the given problem with $I_{\lambda }( u_{1} )<0$ which is different from $u_{0}$. This completes the proof. □

The existence of nontrivial solutions for the problem is now investigated if (H3) is replaced with the following condition:

• (H5) There exists a constant $\theta\ge1$ such that

  $$\theta{\mathcal {H}}(x,t) \ge{\mathcal {H}}(x,st) ,$$

  for $(x,t)\in\mathbb {R}^{N}\times\mathbb {R}$ and $s\in[0,1]$, where ${\mathcal {H}}(x,t)=h(x,t)t-p_{+}\vartheta H(x,t)$ and ϑ is given in (M2).

This condition originally comes from the work of Jeanjean [31]. As is well known, this is weaker condition than (1.1).

Definition 3.8

We say that $I_{\lambda}$ satisfies the Cerami condition ($(C)$-condition for short) in E, if any $(C)$-sequence $\{z_{n}\} _{n} \subset E$, i.e. $\{I_{\lambda}(z_{n})\}$ is bounded and $\| I_{\lambda}^{\prime}(z_{n})\|_{E^{*}}(1+\|z_{n}\|_{E})\to0$ as $n\to\infty$, has a convergent subsequence in E.

Lemma 3.9

It is assumed that (V), (M1)–(M2), (A1)–(A2), (H1)–(H2), and (H4)–(H5). hold. Furthermore, assume that

• (M3) $M:\mathbb{R}^{+} \to\mathbb{R}^{+}$is a differentiable and decreasing function.

Then, the functional$I_{\lambda}$satisfies the$(C)$-condition for any$\lambda>0$.

Proof

Let $\{z_{n}\}$ be a $(C)$-sequence in E, i.e., $\sup |I_{\lambda}(z_{n})|\leq K_{1}$ and $\langle I^{\prime}_{\lambda }(z_{n}),z_{n} \rangle=o(1) \to0$, as $n\to\infty$, and $K_{1}$ is a positive constant. In view of Lemma 3.6, it needs only to be proved that $\{z_{n}\}$ is bounded in E. To this end, arguing by contradiction, it is assumed that $\|z_{n}\|_{E}>1$ and $\|z_{n}\|_{E}\to\infty$ as $n\to\infty$, and a sequence $\{\omega_{n} \}$ is defined by $\omega_{n}={z_{n}}/{\|z_{n}\|_{E}}$. Then, up to a subsequence, still denoted by $\{\omega_{n} \}$, we obtain $\omega_{n}\rightharpoonup\omega$ in E as $n\to\infty$, and by Lemma 2.5,

$$\begin{gathered} \omega_{n}(x) \to\omega(x) \quad\text{a.e. in }\mathbb {R}^{N}, \qquad \omega_{n} \to\omega\quad\text{in }L^{r(\cdot)} \bigl(\mathbb {R}^{N}\bigr), \quad \text{and}\\ \omega_{n} \to\omega\quad\text{in }L^{p(\cdot)}\bigl(\mathbb {R}^{N}\bigr)\end{gathered} $$

as $n \to\infty$, where $p(x)< r(x)<{p_{s}}^{*}(x)$ for all $x\in \mathbb {R}^{N}$.

Let $\varOmega_{2}= \{ x\in\mathbb {R}^{N} : \omega(x)\neq0 \}$. By the same argument as in Lemma 3.6, $\lvert\varOmega_{2} \rvert=0$; thus, $\omega(x)=0$ for almost all $x\in \mathbb {R}^{N}$. As $I_{\lambda}(tz_{n})$ is continuous in $t\in[0,1]$, for each $n\in\mathbb {N}$, there exists $t_{n}\in[0,1]$ such that

$$ I_{\lambda}(t_{n}z_{n}):= \max_{t\in[0,1]} {I_{\lambda}(tz_{n})}. $$

Let $\{\ell_{k} \}$ be a positive sequence of real numbers such that $\lim_{k\to\infty}{\ell_{k}}=\infty$ and $\ell_{k}>1$ for any k. Then, it is clear that $\|\ell_{k}\omega_{n}\|_{E}=\ell_{k}>1$ for any k and n. Let k be fixed. Because $\omega_{n}\to0$ strongly in $L^{q(\cdot)}(\mathbb {R}^{N})$ as $n\to\infty$, it follows from the continuity of the Nemytskii operator that $H(x,\ell_{k}\omega_{n})\to0$ in $L^{1}(\mathbb {R}^{N})$ as $n\to\infty$. Hence,

$$ \lim_{n\to\infty}{ \int_{\mathbb {R}^{N}}{H(x,\ell_{k}\omega_{n})}} \,dx=0. $$

(3.24)

Because $\|z_{n}\|_{E}\to\infty$ as $n\to\infty$, we have $\|z_{n}\|_{E}>\ell_{k}$ for sufficiently large n. Thus, by (M2), (3.24), and Proposition 2.3 we have

$$\begin{aligned} I_{\lambda}(t_{n}z_{n}) &\geq I_{\lambda} \biggl(\frac{\ell_{k}}{ \Vert z_{n} \Vert _{E}}z_{n} \biggr)=I_{\lambda}(\ell_{k}\omega_{n}) \\ &=\mathcal{M} \bigl([\ell_{k}\omega_{n}]_{s,p(\cdot,\cdot)} \bigr)+ \int_{\mathbb {R}^{N}}\frac{\mathcal{V}(x)}{p(x)} \lvert\ell _{k} \omega_{n} \rvert^{p(x)}\,dx\\&\quad- \lambda \int_{\mathbb {R}^{N}}\frac{\rho(x)}{r(x)} \lvert\ell_{k} \omega_{n} \rvert^{r(x)}\,dx- \int_{\mathbb {R}^{N}}{H(x,\ell_{k}\omega_{n})} \,dx \\ &\ge\min{ \biggl\{ \frac{m_{0} }{\vartheta p^{+}}, \frac{1}{p_{+}} \biggr\} } \\&\quad\times\biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert \ell _{k}\omega_{n}(x)- \ell_{k}\omega_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy+ \int_{\mathbb {R}^{N}}{\mathcal {V}(x) \lvert\ell_{k} \omega_{n} \rvert^{p(x)}\,dx} \biggr) \\ &\quad- \lambda \int_{\mathbb {R}^{N}}\frac{\rho(x)}{r(x)} \lvert\ell _{k} \omega_{n} \rvert^{r(x)}\,dx- \int_{\mathbb {R}^{N}}{H(x,\ell _{k}\omega_{n})} \,dx \\ &\ge\min{ \biggl\{ \frac{m_{0} }{\vartheta p^{+}}, \frac{1}{p_{+}} \biggr\} } \Vert \ell_{k}\omega_{n} \Vert _{E}^{p^{-}}- 2\frac{\lambda}{r_{+}} \Vert \rho \Vert _{L^{\frac{p(\cdot)}{p(\cdot)-r(\cdot)}}(\mathbb {R}^{N})} \Vert \ell_{k}\omega_{n} \Vert _{E}^{r_{+}}\\&\quad- \int_{\mathbb {R}^{N}}{H(x,\ell_{k}\omega _{n})}\,dx \\ &\geq\min{ \biggl\{ \frac{m_{0} }{\vartheta p^{+}}, \frac{1}{p_{+}} \biggr\} } \ell_{k}^{p^{-}}-2C_{13}\frac{\lambda}{r_{+}} \ell_{k}^{r_{+}}, \end{aligned}$$

for sufficiently large n and $p^{-}>r_{+} >1$. Then, letting n and k tend to infinity, it follows that

$$ \lim_{n\to\infty}{I_{\lambda}(t_{n}z_{n})}= \infty. $$

(3.25)

Because $I_{\lambda}(0)=0$ and $|I_{\lambda}(z_{n})| \leq K_{1}$ as $n\to \infty$, it is obvious that $t_{n}\in(0,1)$ and $\langle I_{\lambda}^{\prime}(t_{n}z_{n}), t_{n}z_{n} \rangle=0$. Note that there is a positive constant $\mathcal{K}$ such that

$$ \int_{\mathbb {R}^{N}} \biggl(\frac{1}{r(x)}-\frac{1}{p_{+}\vartheta} \biggr)\rho (x) \lvert s z \rvert^{r(x)}\,dx\ge\theta \int_{\mathbb {R}^{N}} \biggl(\frac{1}{r(x)}-\frac{1}{p_{+}\vartheta} \biggr)\rho(x) \lvert z \rvert^{r(x)}\,dx-\mathcal{K}, $$

(3.26)

for any $z\in E$ and $s\in[0,1]$, where θ and ϑ come from (H5) and (M2), respectively. Therefore, by (M2), (M3), (H5) and (3.26), for all n large enough, we have

$$\begin{aligned} \frac{1}{\theta}I_{\lambda}(t_{n} z_{n})&=\frac{1}{\theta }I_{\lambda}(t_{n} z_{n})-\frac{1}{p_{+}\theta\vartheta} \bigl\langle I^{\prime}_{\lambda}(t_{n} z_{n}),t_{n} z_{n} \bigr\rangle +o(1) \\ &=\frac{1}{\theta}\mathcal{M} \bigl( [t_{n} z_{n} ]_{s,p(\cdot,\cdot )} \bigr)\\&\quad-\frac{1}{p_{+}\theta\vartheta} M \bigl( [t_{n} z_{n} ]_{s,p(\cdot ,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert t_{n} z_{n}(x)- t_{n} z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr) \\ &\quad+\frac{1}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal {V}(x)}{p(x)} \lvert t_{n} z_{n} \rvert^{p(x)}- \frac {\mathcal{V}(x)}{p_{+}\vartheta} \lvert t_{n} z_{n} \rvert ^{p(x)} \biggr)\,dx\\&\quad-\frac{\lambda}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert t_{n} z_{n} \rvert ^{r(x)}- \frac{\rho(x)}{p_{+}\vartheta} \lvert t_{n} z_{n} \rvert ^{r(x)} \biggr)\,dx \\ &\quad+\frac{1}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{1}{p_{+}\vartheta}h(x,t_{n} z_{n})t_{n} z_{n}- H(x,t_{n} z_{n}) \biggr)\,dx+o(1) \\ &\leq\frac{1}{\theta}\mathcal{M} \bigl( [t_{n} z_{n} ]_{s,p(\cdot ,\cdot)} \bigr)\\&\quad-\frac{1}{p_{+}\theta\vartheta} M \bigl( [t_{n} z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert t_{n} z_{n}(x)- t_{n} z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr) \\ &\quad+\frac{1}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal {V}(x)}{p(x)} \lvert t_{n} z_{n} \rvert^{p(x)}- \frac {\mathcal{V}(x)}{p_{+}\vartheta} \lvert t_{n} z_{n} \rvert ^{p(x)} \biggr)\,dx\\&\quad-\frac{\lambda}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert t_{n} z_{n} \rvert ^{r(x)}- \frac{\rho(x)}{p_{+}\vartheta} \lvert t_{n} z_{n} \rvert ^{r(x)} \biggr)\,dx \\ &\quad+\frac{1}{p_{+}\theta\vartheta} \int_{\mathbb {R}^{N}} {\mathcal {H}}(x,t_{n} z_{n})\, dx+o(1) \\ &\leq\frac{1}{\theta}\biggl[\mathcal{M} \bigl( [ z_{n} ]_{s,p(\cdot ,\cdot)} \bigr)\\&\quad-\frac{1}{p_{+}\vartheta} M \bigl( [ z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr)\biggr] \\ &\quad+\frac{1}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal {V}(x)}{p(x)} \lvert z_{n} \rvert^{p(x)}- \frac {\mathcal{V}(x)}{p_{+}\vartheta} \lvert z_{n} \rvert ^{p(x)} \biggr)\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert z_{n} \rvert ^{r(x)}- \frac{\rho(x)}{p_{+}\vartheta} \lvert z_{n} \rvert ^{r(x)} \biggr)\,dx \\ &\quad+\frac{1}{p_{+}\vartheta} \int_{\mathbb {R}^{N}} {\mathcal {H}}(x,z_{n})\, dx+\lambda\mathcal{K}+o(1) \\ &\leq\mathcal{M} \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)+ \int _{\mathbb {R}^{N}}\frac{\mathcal{V}(x)}{p(x)} \lvert z_{n} \rvert^{p(x)}\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}}\frac{\rho (x)}{r(x)} \lvert z_{n} \rvert^{r(x)}\,dx- \int_{\mathbb {R}^{N}}{H(x,z_{n})}\,dx \\ &\quad-\frac{1}{p_{+}\vartheta} M \bigl([z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr)\\&\quad- \frac{1}{p_{+}\vartheta} \int_{\mathbb {R}^{N}}{\mathcal{V}(x)} \lvert z_{n} \rvert ^{p(x)}\,dx \\ &\quad+\frac{\lambda}{p_{+}\vartheta} \int_{\mathbb {R}^{N}}{\rho(x)} \lvert z_{n} \rvert^{r(x)}\,dx+\frac{1}{p_{+}\vartheta} \int_{\mathbb {R}^{N}}h(x,z_{n})z_{n}\,dx+ \lambda\mathcal{K}+o(1) \\ &=I_{\lambda}(z_{n})-\frac{1}{p_{+}\vartheta} \bigl\langle I_{\lambda}^{\prime }(z_{n}),z_{n} \bigr\rangle +\lambda\mathcal{K}+o(1) \le \lambda\mathcal{K}+ K_{1}+o(1), \end{aligned}$$

which contradicts (3.25). This completes the proof. □

We give an example on the function M that fulfils the assumptions (M1)–(M3); see [40].

Example 3.10

Let us consider

$$M(t)=1+\frac{1}{e+t}, \quad t\ge0. $$

Then, it follows from direct calculations that this function M satisfies the assumptions (M1)–(M3).

In the rest of the present paper we establish the existence of at least two distinct nontrivial solutions to the problem ($P_{\lambda }$) under the condition on h which is weaker than (H3). In order to obtain this assertion we need to employ the following variational principle of Ekeland’s type in [6, 40], initially developed by Zhong [51].

Lemma 3.11

([6, 40])

LetEbe a Banach space and$x_{0}$be a fixed point ofE. Suppose that$h:{E}\rightarrow\mathbb{R}\cup\{+\infty\}$is a lower semi-continuous function, not identically +∞, bounded from below. Then, for every${\varepsilon }>0$and$y\in E$such that

$$h(y)< \inf_{{E}}{h}+{\varepsilon }, $$

and every$\lambda>0$, there exists some point$z\in E$such that

$$ h(z) \le h(y),\qquad \Vert z-x_{0} \Vert _{E}\le \bigl({1+ \Vert y \Vert _{E}}\bigr) \bigl(e^{{\lambda }}-1 \bigr), $$

and

$$ h(x)\ge h(z)-\frac{{\varepsilon }}{\lambda(1+ \Vert z \Vert _{E})}{ \Vert x-z \Vert _{E}}, \quad\textit{for all } x\in E. $$

Theorem 3.12

Let (V), (M1)–(M3), (A1)–(A2), (H1)–(H2), and (H4)–(H5) hold. In addition, assume that

• (H6) $\lim_{ \lvert t \rvert\to\infty}{\frac {H(x,t)}{ \lvert t \rvert^{\vartheta p_{+}}}}=\infty$uniformly for almost all$x\in\mathbb {R}^{N}$.

Then there exists a positive constant$\lambda^{*}$such that for any$\lambda\in(0, \lambda^{*} )$, the functional$I_{\lambda}$admits at least two nontrivial different solutions inE.

Proof

To apply Lemma 3.4, we first show condition (2) in this lemma. By the assumption (H6), for any $M_{0}>0$, there exists a constant $\delta>0$ such that

$$ H(x,t)\ge M_{0} \lvert t \rvert^{\vartheta p^{+}}, $$

(3.27)

for $\lvert t \rvert>\delta$ and for almost all $x \in \mathbb {R}^{N}$. Take $w\in E\setminus \{0 \}$. Then, for large enough $t>1$, Eq. (3.27) implies that

$$\begin{aligned} I_{\lambda}(tw)&=\mathcal{M} \bigl( [t w ]_{s,p(\cdot,\cdot)} \bigr) + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert tw \rvert^{p(x)} \biggr)\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert tw \rvert^{r(x)} \biggr)\, dx- \int_{\mathbb {R}^{N}}H(x,tw)\,dx \\ &\le\mathcal{M}(1) \bigl([t w ]_{s,p(\cdot,\cdot)} \bigr)^{\vartheta} + \frac {1}{p_{-}} \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert tw \rvert ^{p(x)} \,dx \\ &\quad-\frac{\lambda}{r_{-}} \int_{\mathbb {R}^{N}}\rho(x) \lvert tw \rvert^{r(x)}\,dx- \int_{\mathbb {R}^{N}}H(x,tw)\,dx \\ &\le \lvert t \rvert^{\vartheta p^{+}} \biggl({\mathcal{M}(1)} {[w ]}^{\vartheta}_{s,p(\cdot,\cdot)} +\frac{1}{p_{-}} \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert w \rvert^{p(x)} \,dx- M_{0} \int_{\mathbb {R}^{N}} \lvert w \rvert ^{\vartheta p^{+}}\,dx \biggr), \end{aligned}$$

where ϑ was given in (M2), because $\mathcal{M}(\tau )\le \mathcal{M}(1)\tau^{\vartheta}$ for $\tau\ge1$. If $M_{0}$ is large enough, then we deduce that $I_{\lambda}(tw)\to-\infty$ as $t \to \infty$, as required.

Thanks to Lemmas 3.4 and 3.9, there exists a positive number $\lambda^{*}$ such that for all $\lambda\in(0, \lambda ^{*})$, $I_{\lambda}$ satisfies the mountain pass geometry and $(C)$-condition. By employing the mountain pass theorem, we infer that there exists a critical point $z_{0} \in E$ of $I_{\lambda}$ with $I_{\lambda}(z_{0})= \overline{d}>0=I_{\lambda}(0)$. Hence $z_{0}$ is a nontrivial weak solution of the problem ($P_{\lambda }$). Let us denote $d:=\inf_{z \in\overline{B}_{r}}{I_{\lambda}(z)}$ where ${B}_{r} := \{ z \in E : \|z\|_{E} < r \}$ with a boundary $\partial{B}_{r}$. Then by (3.10) and Lemma 3.4(3), we have $-\infty< d < 0$. Putting $0< \epsilon< \inf_{z \in\partial{B}_{r}}{I_{\lambda }(z)}-d$, by Lemma 3.11, we can choose $z_{\epsilon} \in \overline{B}_{r}$ such that

$$ \textstyle\begin{cases} I_{\lambda}( z_{\epsilon} ) \leq d + \epsilon, \\ I_{\lambda}( z_{\epsilon} ) < I_{\lambda}(z) + \frac{\epsilon}{1+ \Vert z_{\epsilon} \Vert _{E} } \Vert z-z_{\epsilon} \Vert _{E},\quad\text{for all } z \in\overline{B}_{r}, z \neq z_{\epsilon}. \end{cases} $$

(3.28)

This implies that $z_{\epsilon} \in{B}_{r}$ since $I_{\lambda}( z_{\epsilon} ) \leq d + \epsilon< \inf_{z \in\partial {B}_{r}}{I_{\lambda}(z)}$. From these facts we see that $z_{\epsilon}$ is a local minimum of $\widetilde{I}_{\lambda}( z ) = I_{\lambda}(z) + \frac{\epsilon }{1+ \|z_{\epsilon}\|_{E} } \|z-z_{\epsilon} \|_{E}$. Now by taking $z = z_{\epsilon} +tw$ for $w \in{B}_{1}$ and sufficiently small $t>0$, from (3.28), we deduce

$$0 \leq\frac{\widetilde{I}_{\lambda}( z_{\epsilon} +tw )-\widetilde {I}_{\lambda}( z_{\epsilon} ) }{t} = \frac{{I_{\lambda}}( z_{\epsilon} +tw )-{I_{\lambda}}( z_{\epsilon} ) }{t} + \frac {\epsilon}{1+ \Vert z_{\epsilon} \Vert _{E} } \Vert w \Vert _{E}. $$

Therefore, letting $t \rightarrow0+$, we get

$$\bigl\langle I_{\lambda}^{\prime}(z_{\epsilon}),w \bigr\rangle + \frac{\epsilon}{1+ \Vert z_{\epsilon} \Vert _{E} } \Vert w \Vert _{E} \geq0. $$

Replacing w by −w in the argument above, we have

$$- \bigl\langle I_{\lambda}^{\prime}(z_{\epsilon}),w \bigr\rangle + \frac{\epsilon}{1+ \Vert z_{\epsilon} \Vert _{E} } \Vert w \Vert _{E} \geq0. $$

Thus, one has

$$\bigl(1+ \Vert z_{\epsilon} \Vert _{E} \bigr) \bigl\vert \bigl\langle I_{\lambda}^{\prime }(z_{\epsilon}),w \bigr\rangle \bigr\vert \leq\epsilon \Vert w \Vert _{E}, $$

for any $w \in\overline{B}_{1}$. Hence we have

$$ \bigl(1+ \Vert z_{\epsilon} \Vert _{E} \bigr) \bigl\Vert I_{\lambda}^{\prime}(z_{\epsilon}) \bigr\Vert _{E^{*}} \leq\epsilon. $$

(3.29)

Using (3.28) and (3.29), we can choose a sequence $\{ z_{n} \} \subset B_{r}$ such that

$$ \textstyle\begin{cases} I_{\lambda}( z_{n} ) \rightarrow d \quad\text{as } n \to\infty, \\ (1+ \Vert z_{n} \Vert _{E} ) \Vert I_{\lambda}^{\prime}(z_{n}) \Vert _{E^{*}} \to0 \quad \text{as } n \to\infty. \end{cases} $$

(3.30)

Thus, $\{ z_{n} \}$ is a bounded Cerami sequence in the reflexive Banach space E. According to Lemma 3.9, $\{ z_{n} \}$ has a subsequence $\{ z_{n_{k}} \}$ such that $z_{n_{k}} \to z_{1}$ in E as $k \to\infty$. This together with (3.30) shows that $I_{\lambda }( z_{1} ) =d$ and $I_{\lambda}^{\prime}(z_{1})=0$. Hence $z_{1}$ is a nontrivial nonnegative solution of the given problem with $I_{\lambda }( z_{1} )<0$ which is different from $z_{0}$. This completes the proof. □