Existence of solution for p-Laplacian boundary value problems with two singular and subcritical nonlinearities

Abstract

We consider a boundary value problem for p-Laplacian systems with two singular and subcritical nonlinearities. We obtain one theorem which shows that there exists at least one nontrivial weak solution for these problems under some conditions. We obtain this result by variational method and critical point theory.

1 Introduction

Let Ω be a bounded domain of $R^{n}$ with smooth boundary ∂Ω, $n\ge 2$. Let G be an open subset in $R^{2}$ with compact complement $C_{1}\cup C_{2}=R^{n}\setminus G$ containing $\theta =(0,0)$ and $e=(e_{1},e_{2})$, where $\theta =(0,0)\in C_{1}$ and $e=(e_{1},e_{2})\in C_{2}$, $n\ge 2$. In this paper we investigate existence and multiplicity of the solutions $(u,v)\in W^{1,p}(\varOmega , G)$ for the p-Laplacian system with two singular and subcritical nonlinearities under the Dirichlet boundary condition:

$$ \textstyle\begin{cases} -\operatorname{div}(|\nabla u|^{p-2}\nabla u)=a |u|^{p-2}u-\operatorname{grad}_{u} \frac{1}{(|u|^{2}+|v|^{2})^{q}}-\operatorname{grad}_{u}\frac{1}{(|u-e_{1}|^{2}+|v-e _{2}|^{2})^{r}} \\ \hphantom{-\operatorname{div}(|\nabla u|^{p-2}\nabla u)={}}{}+\frac{2\alpha }{\alpha +\beta }|u|^{\alpha -1}|v|^{ \beta }\quad \mbox{in } \varOmega , \\ -\operatorname{div}(|\nabla v|^{p-2}\nabla v)=b |v|^{p-2}v-\operatorname{grad}_{v} \frac{1}{(|u|^{2}+|v|^{2})^{q}}-\operatorname{grad}_{v}\frac{1}{(|u-e_{1}|^{2}+|v-e _{2}|^{2})^{r}} \\ \hphantom{-\operatorname{div}(|\nabla v|^{p-2}\nabla v)={}}{}+\frac{2\beta }{\alpha +\beta }|u|^{\alpha }|v|^{ \beta -1}\quad \mbox{in } \varOmega , \end{cases} $$

(1.1)

where a, b, p, q, r, α, and β are real constants, and $1< p<\infty$, $q, r>1$, and $p<\alpha +\beta <p ^{*}$, where $p^{*}$ is a critical exponent such that

$$ p^{*}= \textstyle\begin{cases} \frac{np}{n-p}& \mbox{if } n>p, \\ \infty& \mbox{if } n\le p. \end{cases} $$

Singular problems involving p-Laplacian arise in applications of non-Newtonian fluid theory or the turbulent flow of a gas in a porous medium (cf. [12, 19]). Our problems are characterized as a singular elliptic system with singular nonlinearities at $\{(u,v)=\theta \}$ and $\{(u,v)=e\}$. We recommend the book [12] for the singular elliptic problems. We also recommend Rǎdulescu’s paper [21] establishing the recent contributions in singular phenomena in nonlinear elliptic problems from blow-up boundary solutions to equations with singular nonlinearities in two types of stationary singular problems: the logistic equation

$$ \begin{aligned} &\Delta u=\varPhi (x,u,\nabla u) \quad \mbox{in } \varOmega , \\ &u>0\quad \mbox{in } \varOmega , \\ &u=+\infty\quad \mbox{on } \partial \varOmega \end{aligned} $$

(1.2)

and the Lane–Emden–Fowler equation

$$ \begin{aligned} &{-}\Delta u=\varPsi (x,u,\nabla u) \quad \mbox{in } \varOmega , \\ &u>0\quad \mbox{in } \varOmega , \\ &u=0\quad \mbox{on } \partial \varOmega , \end{aligned} $$

(1.3)

where Φ is a smooth nonlinear function, while Ψ has one or more singularities. The solutions of (1.2) are called large (or blow-up) solutions. More studies on blow-up boundary solutions of logistic type equation like (1.2) can be found in [2,3,4,5, 7, 8, 11, 13, 14, 16,17,18, 20, 22]. Singular Dirichlet boundary value problems for the Lane–Emden–Fowler equation like (1.3) involving singular nonlinearities have been intensively studied in the last decades. The first study in this direction is due to Fulks and Maybee [10], who proved existence and uniqueness of the problem

$$ \begin{aligned} &{-}\Delta u+u^{-\alpha }=u\quad \mbox{in } \varOmega , \\ &u>0\quad \mbox{in } \varOmega , \\ &u=0\quad \mbox{on } \partial \varOmega \end{aligned} $$

(1.4)

by using fixed point arguments and no solution of (1.4), provided that $0<\alpha <1$ and $\lambda _{1}\ge 1$ (that is, if Ω is small), where $\lambda _{1}$ denotes the first eigenvalue of −Δ in $H^{1}_{0}(\varOmega )$. Shi and Yau studied in [23] the existence of radial symmetric solutions of the problem

$$ \begin{aligned} &\Delta u+\lambda \bigl(u^{p}-u^{-\alpha } \bigr)=0 \quad \mbox{in } B_{1}, \\ &u>0\quad \mbox{in } B_{1}, \\ &u=0\quad \mbox{on } \partial B_{1}, \end{aligned} $$

(1.5)

where $\alpha >0$, $0< p<1$, $\lambda >0$ and $B_{1}$ is the unit ball in $R^{N}$. They showed in [23] that there exists $\lambda _{1}>\lambda _{0}>0$ such that (1.5) has no solution for $\lambda <\lambda _{0}$, exactly one solution for $\lambda =\lambda _{0}$ or $\lambda >\lambda _{1}$, and two solutions for $\lambda _{0}<\lambda \le \lambda _{1}$. Dupaigne, Ghergu, and Rǎdulescu [9] proved existence and multiplicity of the Lane–Emden–Fowler equation with convection and singular potential

$$\begin{aligned}& {-}\Delta u\pm \bigl(d(x)\bigr)g(u)=\lambda f(x,u)+\nu |\nabla u|^{\alpha } \quad \mbox{in } \varOmega, \\& u>0\quad \mbox{in } \varOmega , \\& u=0 \quad \mbox{on } \partial \varOmega . \end{aligned}$$

M. Trabelsi and N. Trabelsi [24] considered the semilinear elliptic system and proved existence of the singular limit solutions for a two-dimensional semilinear elliptic system of Liouville type.

We introduce the space

$$ L^{p}(\varOmega ,R)=\biggl\{ u\Bigm| u:\varOmega \to R \mbox{ is measurable}, \int _{\varOmega }|u|^{p}\,dx< \infty \biggr\} $$

endowed with the norm

$$ \|u\|^{p}_{L^{p}(\varOmega )}=\inf \biggl\{ \lambda >0\Bigm| \int _{\varOmega } \biggl\vert \frac{u(x)}{ \lambda } \biggr\vert ^{p}\le 1\biggr\} $$

and the Sobolev space

$$ W^{1,p}(\varOmega ,R)=\bigl\{ u\in L^{p}(\varOmega ,R)\mid \nabla u(x)\in L ^{p}(\varOmega ,R)\bigr\} $$

endowed with the norm

$$ \|u\|_{W^{1,p}(\varOmega ,R)}=\biggl[ \int _{\varOmega } \bigl\vert \nabla u(x) \bigr\vert ^{p} \,dx+ \int _{\varOmega } \bigl\vert u(x) \bigr\vert ^{p} \,dx \biggr]^{\frac{1}{p}}. $$

Let $L^{p}(\varOmega ,R^{2})=L^{p}(\varOmega ,R)\times L^{p}(\varOmega ,R)$ and $H=W^{1,p}(\varOmega ,R^{2})=W^{1,p}(\varOmega ,R)\times W^{1,p}(\varOmega ,R)$. Then $L^{p}(\varOmega ,R^{2})$ and H are Hilbert spaces with the norm

$$ \bigl\Vert (u,v) \bigr\Vert _{L^{p}(\varOmega ,R^{2})}=\|u\|_{L^{p}(\varOmega ,R)}+\|v \|_{L ^{p}(\varOmega ,R)} $$

and the norm

$$ \bigl\Vert (u,v) \bigr\Vert _{H}=\|u\|_{W^{1,p}(\varOmega ,R)}+\|v \|_{W^{1,p}(\varOmega ,R)}, $$

respectively. It was proved in [15] that, for $1< p<\infty$, the eigenvalue problem

$$\begin{aligned}& -\operatorname{div}\bigl(|\nabla u|^{p-2}\nabla u\bigr)=\lambda |u|^{p-2}u\quad \mbox{in } \varOmega , \\& u=0\quad \mbox{on } \partial \varOmega \end{aligned}$$

has a nondecreasing sequence of nonnegative eigenvalues $\lambda ^{(p)} _{j}$ obtained by the Ljusternik–Schnirelman principle tending to ∞ as $j\to \infty$, where the first eigenvalue $\lambda ^{(p)} _{1}$ is simple and only eigenfunctions associated with $\lambda ^{(p)} _{1}$ do not change sign, the set of eigenvalues is closed, the first eigenvalue $\lambda ^{(p)}_{1}$ is isolated. Thus there is a sequence of eigenfunctions $(\phi ^{(p)}_{j})_{j}$ corresponding to the eigenvalues $\lambda ^{(p)}_{j}$ such that the first eigenfunction $\phi ^{(p)}_{1}$ is positive or negative depending on p.

Let us set $-\Delta _{p} u=-\operatorname{div}(|\nabla u|^{p-2}\nabla u)$. Let us define an open subset of the Hilbert space $H=W^{1,p}(\varOmega ,R ^{2})$:

$$\begin{aligned} \varLambda G =&\bigl\{ (u,v)\in H\mid \nabla (u,v)\in L^{p}\bigl(\varOmega ,R^{2}\bigr), \bigl(u(x),v(x)\bigr)\in G=R^{2}\setminus (C_{1}\cup C_{2}), \\ &\theta \in C_{!}, (e_{1},e_{2})\in C_{2}, \forall \bigl(u(x),v(x)\bigr)\in G\subset R^{2}\bigr\} . \end{aligned}$$

Then ΛG is the loop space on G.

In this paper, we are looking for weak solutions $(u,v)$ of (1.1) in ΛG satisfying

$$\begin{aligned}& \int _{\varOmega }\biggl[-\Delta _{p} u\cdot z-\Delta _{p} v\cdot w-a |u|^{p-2}u \cdot z-b |v|^{p-2}v\cdot w \\& \quad {}+\operatorname{grad}_{u}\frac{1}{(|u|^{2}+|v|^{2})^{q}}\cdot z+ \operatorname{grad} _{v}\frac{1}{(|u|^{2}+|v|^{2})^{q}}\cdot w \\& \quad {}+\operatorname{grad}_{u}\frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}\cdot z+ \operatorname{grad}_{v}\frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}\cdot w \\& \quad {}-\frac{2\alpha }{\alpha +\beta }|u|^{\alpha -1}|v|^{\beta }\cdot z- \frac{2 \beta }{\alpha +\beta }|u|^{\alpha -1}|v|^{\beta }\cdot w\biggr]\,dx=0 \quad \forall (z,w)\in \varLambda G. \end{aligned}$$

Let A be $\left(\begin{array}{cc}a& 0\\ 0& b\end{array}\right)\in M_{2\times 2}(R)$. Let us set

$$ q_{\lambda ^{(p)}_{j}}(a,b)= \operatorname{Det}\bigl(\lambda ^{(p)}_{j} I-A\bigr)=\bigl( \lambda ^{(p)}_{j}-a\bigr) \bigl(\lambda ^{(p)}_{j}-b\bigr). $$

Let $\mu ^{1}_{\lambda ^{(p)}_{i}}$ and $\mu ^{2}_{\lambda ^{(p)}_{i}}$ be the eigenvalues of the matrix $\left(\begin{array}{cc}{\lambda }_i^{(p)}-a& 0\\ 0& {\lambda }_i^{(p)}-b\end{array}\right)\in M_{2\times 2}(R)$, i.e.,

$$\begin{aligned}& \mu ^{1}_{\lambda ^{(p)}_{i}}=\frac{1}{2}\Bigl\{ 2\lambda ^{(p)}_{i}-a-b-\sqrt{\bigl(2 \lambda ^{(p)}_{i}-a-b\bigr)^{2}-4q_{\lambda ^{(p)}_{i}}(a,b)} \Bigr\} , \\& \mu ^{2}_{\lambda ^{(p)}_{i}}=\frac{1}{2}\Bigl\{ 2\lambda ^{(p)}_{i}-a-b+\sqrt{\bigl(2 \lambda ^{(p)}_{i}-a-b\bigr)^{2}-4q_{\lambda ^{(p)}_{i}}(a,b)} \Bigr\} . \end{aligned}$$

We note that weak solutions of (1.1) correspond to critical points of the continuous and Fréchet differentiable functional $f(u,v)\in C^{1}(\varLambda G,R)$,

$$\begin{aligned} f(u,v) =&\frac{1}{p} \int _{\varOmega }\bigl[|\nabla u|^{p}+|\nabla v|^{p}-a |u|^{p}-b |v|^{p}\bigr]\,dx+ \int _{\varOmega }\frac{1}{(|u|^{2}+|v|^{2})^{q} }\,dx \\ &{}+ \int _{\varOmega }\frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}\,dx-\frac{2}{ \alpha +\beta } \int _{\varOmega }|u|^{\alpha }|v|^{\beta }\,dx \\ =&R_{a,b}(u,v)+ \int _{\varOmega }\frac{1}{(|u|^{2}+|v|^{2})^{q} }\,dx+ \int _{\varOmega }\frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}\,dx \\ &{}-\frac{2}{\alpha +\beta } \int _{\varOmega }|u|^{\alpha }|v|^{ \beta }\,dx, \end{aligned}$$

where $R_{a,b}(u,v)=\frac{1}{p}\int _{\varOmega }[|\nabla u|^{p}+|\nabla v|^{p}-a |u|^{p}-b |v|^{p}]\,dx$, which will be proved in Sect. 3. When $p<\alpha +\beta <p^{*}$, the embedding $W^{1,p}_{0}(\varOmega ,G)\hookrightarrow L^{\alpha +\beta }(\varOmega ,G)$ is continuous and compact, so we can assure that the functional $f(u,v)$ satisfies the $(\mathit{P.S.})$ condition, which will be also proved in Sect. 3.

Our main result is as follows.

Theorem 1.1

Assume that a, b, p, q, r, α, and β are real constants, and $1< p<\infty$, $q, r>1$, and $p<\alpha +\beta <p^{*}$, $n\ge 2$,

• (i) $2\lambda ^{(p)}_{i}>a+b$,

• (ii) $q_{{\lambda }_i^{(p)}}(a,b)=det\left(\begin{array}{cc}{\lambda }_i^{(p)}-a& 0\\ 0& {\lambda }_i^{(p)}-b\end{array}\right)>0$ for $i\ge 1$.

Then (1.1) has at least one nontrivial weak solution $(u(x),v(x))$ such that $(u(x),v(x))\neq (0,0)$ and $(u(x),v(x))\neq (e_{1},e_{2})$.

For the proof of Theorem 1.1, we approach variational method and use critical point theory on eigenspaces. In Sect. 2, we introduce the eigenspaces spanned by the eigenfunctions corresponding to the eigenvalues of the matrix $\left(\begin{array}{cc}{\lambda }_i^{(p)}-a& 0\\ 0& {\lambda }_i^{(p)}-b\end{array}\right)\in M_{2\times 2}(R)$ and obtain some variational results on the eigenspaces. In Sect. 3, we prove that the corresponding functional of (1.1) satisfies the $(\mathit{P.S.})$ condition and prove Theorem 1.1.

2 Variational results on eigenspaces

Let $q_{\lambda ^{(p)}_{i}}(a,b)$, $\mu ^{1}_{\lambda ^{(p)}_{i}}$, and $\mu ^{2}_{\lambda ^{(p)}_{i}}$ be the numbers introduced in Sect. 1. We note that

$$\begin{aligned}& \mbox{if } q_{\lambda ^{(p)}_{i}}(a,b)< 0, \mbox{then } \mu ^{1}_{\lambda ^{(p)}_{i}}< 0< \mu ^{2}_{\lambda ^{(p)}_{i}}, \\ & \mbox{if } 2\lambda ^{(p)}_{i}>a+b \mbox{ and } q_{ \lambda ^{(p)}_{i}}(a,b)>0, \mbox{then } 0< \mu ^{1}_{\lambda ^{(p)} _{i}}< \mu ^{2}_{\lambda ^{(p)}_{i}}, \\ & \mbox{if } 2\lambda ^{(p)}_{i}< a+b \mbox{ and } q_{ \lambda ^{(p)}_{i}}(a,b)>0, \mbox{then } \mu ^{1}_{\lambda ^{(p)} _{i}}< \mu ^{2}_{\lambda ^{(p)}_{i}}< 0, \\ & \mbox{if } 2\lambda ^{(p)}_{i}=a+b \mbox{ and } q_{ \lambda ^{(p)}_{i}}(a,b)>0, \mbox{then } \mu ^{1}_{\lambda ^{(p)} _{i}}=\mu ^{2}_{\lambda ^{(p)}_{i}}=0. \end{aligned}$$

Let $(c^{1}_{\lambda ^{(p)}_{i}},d^{1}_{\lambda ^{(p)}_{i}})$ and $(c^{2}_{\lambda ^{(p)}_{i}},d^{2}_{\lambda ^{(p)}_{i}})$ be the eigenvectors of $\left(\begin{array}{cc}{\lambda }_i^{(p)}-a& 0\\ 0& {\lambda }_i^{(p)}-b\end{array}\right)$ corresponding to $\mu ^{1}_{\lambda ^{(p)}_{i}}$ and $\mu ^{2}_{\lambda ^{(p)}_{i}}$, respectively. Let us set

$$\begin{aligned}& D_{\lambda ^{(p)}_{i}} = \bigl\{ (a,b)\in R^{2}\mid q_{\lambda ^{(p)}_{i}}(a,b)>0 \mbox{ for } i\ge 1\bigr\} , \\ & D^{\prime }_{\lambda ^{(p)}_{i}} = D_{\lambda ^{(p)}_{i}}\cap \bigl\{ 2 \lambda ^{(p)}_{i}\le a+b\bigr\} , \\ & D^{\prime \prime }_{\lambda ^{(p)}_{i}} = D_{\lambda ^{(p)}_{i}}\cap \bigl\{ 2 \lambda ^{(p)}_{i}>a+b\bigr\} , \\ & W_{\lambda ^{(p)}_{i}} = \bigl\{ \phi ^{(p)}_{i}\mid -\Delta _{p} \phi ^{(p)}_{i}=\lambda ^{(p)}_{i} \phi ^{(p)}_{i}\bigr\} \\ & H_{\lambda ^{(p)}_{i}} = \bigl\{ \bigl(c\phi ^{(p)},d\phi ^{(p)} \bigr)\in H\mid (c,d)\in R^{2}, \phi ^{(p)}\in W_{\lambda ^{(p)}_{i}} \bigr\} , \\ & H^{1}_{\lambda ^{(p)}_{i}} = \bigl\{ \bigl(c^{1}_{\lambda ^{(p)}_{i}} \phi ^{(p)},d^{1}_{\lambda ^{(p)}_{i}} \phi ^{(p)}\bigr)\in H\mid \phi ^{(p)} \in W_{\lambda ^{(p)}_{i}}\bigr\} , \\ & H^{2}_{\lambda ^{(p)}_{i}} = \bigl\{ \bigl(c^{2}_{\lambda ^{(p)}_{i}} \phi ^{(p)},d^{2}_{\lambda ^{(p)}_{i}} \phi ^{(p)}\bigr)\in H\mid \phi ^{(p)} \in W_{\lambda ^{(p)}_{i}}\bigr\} , \\ & H^{+}(a,b) = \biggl(\bigoplus _{\mu ^{1}_{\lambda ^{(p)}_{i}}>0} H^{1}_{ \lambda ^{(p)}_{i}} \biggr)\oplus \biggl(\bigoplus _{\mu ^{2}_{\lambda ^{(p)}_{i}}>0} E ^{2}_{\lambda ^{(p)}_{i}}\biggr), \\ & H^{-}(a,b) = \biggl(\bigoplus _{\mu ^{1}_{\lambda ^{(p)}_{i}}< 0} H^{1}_{ \lambda ^{(p)}_{i}} \biggr)\oplus \biggl(\bigoplus _{\mu ^{2}_{\lambda ^{(p)}_{i}}< 0} H ^{2}_{\lambda ^{(p)}_{i}}\biggr), \\ & H^{0}(a,b) = \biggl(\bigoplus _{\mu ^{1}_{\lambda ^{(p)}_{i}}=0} H^{1}_{ \lambda ^{(p)}_{i}} \biggr)\oplus \biggl(\bigoplus _{\mu ^{2}_{\lambda ^{(p)}_{i}}=0} H ^{2}_{\lambda ^{(p)}_{i}}\biggr). \end{aligned}$$

Then $H^{+}(a,b)$, $H^{-}(a,b)$, and $H^{0}(a,b)$ are the positive, negative, and null spaces relative to the quadratic form $R_{a,b}(u,v)$ in H and

$$ H=H^{+}\oplus H^{-}\oplus H^{0}. $$

From now on we shall assume that $2\lambda ^{(p)}_{i}>a+b$ and $q_{\lambda ^{(p)}_{i}}(a,b)>0$. Because $\mu ^{1}_{\lambda ^{(p)}_{i}}>0$ and $\mu ^{2}_{\lambda ^{(p)}_{i}}>0$, $\forall i\ge 1$,

$$ H^{0}=\emptyset,\qquad H^{-}=\emptyset $$

and

$$ H=H^{+}. $$

We note that H can be split by two subspaces $Y_{1}$ and $Y_{2}$ such that

$$\begin{aligned}& Y_{1}=\operatorname{span}\bigl\{ \mbox{eigenfunctions corresponding to eigenvalues } \mu ^{1}_{\lambda ^{(p)}_{i}} \mbox{ and } \mu ^{1}_{\lambda ^{(p)}_{i}}, \\& \hphantom{Y_{1}={}}\mbox{with } 1\le i\le m\bigr\} , \\& Y_{2}=\operatorname{span}\bigl\{ \mbox{eigenfunctions corresponding to eigenvalues } \mu ^{1}_{\lambda ^{(p)}_{i}} \mbox{ and } \mu ^{2}_{\lambda ^{(p)}_{i}}, \\& \hphantom{Y_{2}={}}\mbox{with } i\ge m+1\bigr\} , \end{aligned}$$

$\operatorname{dim}Y_{1}<\infty$ and

$$ H=Y_{1}\oplus Y_{2}. $$

Let us set

$$\begin{aligned}& X_{1}=Y_{1}\cap \varLambda G, \\& X_{2}=Y_{2}\cap \varLambda G. \end{aligned}$$

Then

$$ \varLambda G=X_{1}\oplus X_{2}. $$

Let us set

$$\begin{aligned}& B_{\rho }=\bigl\{ (u,v)\in \varLambda G\mid \bigl\Vert (u,v) \bigr\Vert _{H}\le \rho \bigr\} , \\& \partial B_{\rho }=\bigl\{ (u,v)\in \varLambda G\mid \bigl\Vert (u,v) \bigr\Vert _{H}=\rho \bigr\} , \\& Q=\bar{B_{R}}\cap X_{1}\oplus \bigl\{ \sigma (z_{1},z_{2})\mid (z_{1},z _{2})\in \partial B_{1}\cap \bigl(H^{1}_{\mu _{\lambda ^{(p)}_{m+1}}}\oplus H^{2}_{\mu _{\lambda ^{(p)}_{m+1}}}\bigr)\subset \partial B_{1}\cap X_{2}, 0< \sigma < R\bigr\} . \end{aligned}$$

Let us define

$$ \varGamma =\bigl\{ \gamma \in C(\bar{Q},\varLambda G)\mid \gamma =\mathrm{id} \mbox{ on } \partial Q\bigr\} . $$

We note that weak solutions of (1.1) correspond to critical points of the continuous and Fréchet differentiable functional $f(u,v)\in C^{1}(\varLambda G,R)$,

$$\begin{aligned} f(u,v) =&\frac{1}{p} \int _{\varOmega }\bigl[|\nabla u|^{p}+|\nabla v|^{p}-a |u|^{p}-b |v|^{p}\bigr]\,dx+ \int _{\varOmega }\frac{1}{(|u|^{2}+|v|^{2})^{q}} \,dx \\ &{}+ \int _{\varOmega }\frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}\,dx- \int _{\varOmega }\biggl[\frac{2}{\alpha +\beta }|u|^{\alpha }|v|^{\beta } \biggr]\,dx. \end{aligned}$$

Let us define

$$ C^{(p)}_{\alpha ,\beta }(\varOmega )=\inf_{(u,v)\in \varLambda G}\biggl\{ \tau >0\Bigm| \frac{(\int _{\varOmega }(|\nabla u|^{p}+|\nabla v|^{p})\,dx)^{ \frac{1}{p}}+(\int _{\varOmega }(|u|^{p}+|v|^{p})\,dx)^{\frac{1}{p}}}{( \int _{\varOmega }|u|^{\alpha }|v|^{\beta }\,dx)^{\frac{1}{\alpha +\beta }}} \le \tau \biggr\} . $$

(2.1)

Lemma 2.1

Assume that a, b, p, q, r, α, and β are real constants, and $1< p<\infty$, $q, r>1$, $p<\alpha +\beta <p^{*}$, $2\lambda ^{(p)}_{i}>a+b$, and $q_{\lambda ^{(p)}_{i}}(a,b)>0$. Let $i\in N$ and $(a_{0},b_{0})\in \partial D ^{\prime }_{\lambda ^{(p)}_{i}}$ and $(z_{1},z_{2})\in \partial B_{1} \cap (H^{1}_{\mu _{\lambda ^{(p)}_{m+1}}}\oplus H^{2}_{ \mu _{\lambda ^{(p)}_{m+1}}})\subset \partial B_{1}\cap X_{2}$. Then there exist a neighborhood W of $(a_{0},b_{0})$, a small number $\sigma >0$, and a large number $R>0$ such that, for any $(a,b)\in W \setminus D^{\prime }_{\lambda ^{(p)}_{i}}$, if $(u,v)\in \partial Q= \partial (\bar{B_{R}}\cap X_{1}\oplus \{\sigma (z_{1}z_{2})\mid 0< \sigma <R\})$, then

$$ \sup_{u\in \partial Q}f(u,v)< 0\quad \textit{and} \quad \sup _{u\in Q}f(u,v)< \infty . $$

Proof

Let $(a,b)\in W\setminus D^{\prime }_{\lambda ^{(p)}_{i}}$. Let us choose an element $(z_{1},z_{2})\in \partial B_{1}\cap X_{2}$ and $(u,v)\in X_{1}\oplus \{\sigma (z_{1},z_{2})\mid \sigma >0\}$. Then, by (2.1), we have

$$\begin{aligned} f(u,v) =&\frac{1}{p} \int _{\varOmega }\bigl[|\nabla u|^{p}+|\nabla v|^{p}-a |u|^{p}-b |v|^{p}\bigr]\,dx+ \int _{\varOmega }\frac{1}{(|u|^{2}+|v|^{2})^{q}} \,dx \\ &+ \int _{\varOmega }\frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}\,dx-\frac{2}{ \alpha +\beta } \int _{\varOmega }|u|^{\alpha }|v|^{\beta }\,dx \\ \le &\frac{1}{p}\mu ^{2}_{\lambda ^{(p)}_{m}} \bigl\Vert (u,v) \bigr\Vert ^{p}_{L ^{p}(\varOmega )}+\frac{1}{p}\sigma ^{p} \mu ^{2}_{\lambda ^{(p)}_{m}} \\ &{}+ \int _{\varOmega }\biggl[\frac{1}{(|u|^{2}+|v|^{2})^{q}}+\frac{1}{(|u-e_{1}|^{2}+|v-e _{2}|^{2})^{r}}\biggr] \,dx \\ &{} -\frac{2}{\alpha +\beta }\bigl(C^{(p)}_{\alpha ,\beta } \bigr)^{-(\alpha +\beta )}(\varOmega ) \bigl\Vert (u,v) \bigr\Vert ^{\alpha +\beta }_{H}. \end{aligned}$$

Then there exist a large number $R>0$ and a small number $\sigma >0$ with $0<\sigma <R$ such that if $(u,v)\in \partial Q$, then we have $0<\int _{\varOmega }[\frac{1}{(|u|^{2}+|v|^{2})^{q}}+\frac{1}{(|u-e_{1}|^{2}+|v-e _{2}|^{2})^{r}}]\,dx\le C_{1}$ for some constant $0< C_{1}<1$, and, by $p<\alpha +\beta <p^{*}$, we have

$$\begin{aligned} f(u,v) \le &\frac{1}{p}\mu ^{2}_{\lambda ^{(p)}_{m}} \bigl\Vert (u,v) \bigr\Vert ^{p}_{L ^{p}(\varOmega )} +\frac{1}{p}\sigma ^{p} \mu ^{2}_{\lambda ^{(p)}_{m}}+C _{1}- \frac{2}{\alpha +\beta } \int _{\varOmega }|u|^{\alpha }|v|^{\beta }\,dx \\ \le &\frac{1}{p}\mu ^{2}_{\lambda ^{(p)}_{m}} \bigl\Vert (u,v) \bigr\Vert ^{p}_{L ^{p}(\varOmega )} +\frac{1}{p}\sigma ^{p} \mu ^{2}_{\lambda ^{(p)}_{m}}+C _{1}- \frac{2}{\alpha +\beta }\bigl(C_{\alpha ,\beta }^{(p)}\bigr)^{-(\alpha + \beta )}( \varOmega ) \bigl\Vert (u,v) \bigr\Vert ^{\alpha +\beta }_{H}< 0. \end{aligned}$$

Thus we have $\sup_{(u,v)\in \partial Q} f(u,v)<0$. Moreover, if $(u,v)\in Q$, then we have $f(u,v)\le \frac{1}{p}\mu ^{2}_{\lambda ^{(p)} _{m}}\|(u,v)\|^{p}_{L^{p}(\varOmega )}+\frac{1}{p}\sigma ^{p} \mu ^{2}_{ \lambda ^{(p)}_{m}}+C_{1}<\infty$. □

Lemma 2.2

Assume that a, b, p, q, r, α, and β are real constants, and $1< p<\infty$, $q, r>1$, $p<\alpha +\beta <p^{*}$, $2\lambda ^{(p)}_{i}>a+b$, and $q_{\lambda ^{(p)}_{i}}(a,b)>0$. Let $i\in N$ and $(a_{0},b_{0})\in \partial D ^{\prime }_{\lambda ^{(p)}_{i}}$ and $(z_{1},z_{2})\in \partial B_{1} \cap (H^{1}_{\mu _{\lambda ^{(p)}_{m+1}}}\oplus H^{2}_{ \mu _{\lambda ^{(p)}_{m+1}}})\subset \partial B_{1}\cap X_{2}$. Then there exist a neighborhood W of $(a_{0},b_{0})$ and a small number $\rho >0$ such that, for any $(a,b)\in W\setminus D^{\prime }_{ \lambda ^{(p)}_{i}}$, if $(u,v)\in \partial B_{\rho }\cap X_{2}$, then we have

$$ \inf_{u\in \partial B_{\rho }\cap X_{2}}f(u,v)>0 \quad \textit{and} \quad \inf _{u\in B_{\rho }\cap X_{2}}f(u,v)>- \infty . $$

Proof

Let $(a,b)\in W\setminus D^{\prime }_{\lambda ^{(p)}_{i}}$ and $(u,v)\in X_{2}$. Then, by (2.1), we have

$$\begin{aligned} f(u,v) =&\frac{1}{p} \int _{\varOmega }\bigl[|\nabla u|^{p}+|\nabla v|^{p}-a |u|^{p}-b |v|^{p}\bigr]\,dx+ \int _{\varOmega }\frac{1}{(|u|^{2}+|v|^{2})^{q}} \,dx \\ &{}+ \int _{\varOmega }\frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}\,dx- \int _{\varOmega }\biggl[\frac{2}{\alpha +\beta }|u|^{\alpha }|v|^{\beta } \biggr]\,dx \\ \ge &\frac{1}{p}\mu ^{1}_{\lambda ^{(p)}_{m+1}} \bigl\Vert (u,v) \bigr\Vert ^{p} _{L^{p}(\varOmega )}+ \int _{\varOmega }\biggl[\frac{1}{(|u|^{2}+|v|^{2})^{q}}+\frac{1}{(|u-e _{1}|^{2}+|v-e_{2}|^{2})^{r}}\biggr] \,dx \\ &{}-\frac{2}{\alpha +\beta }\bigl(C_{\alpha ,\beta }^{(p)} \bigr)^{-(\alpha +\beta )}(\varOmega ) \bigl\Vert (u,v) \bigr\Vert ^{\alpha +\beta }_{H} \\ \ge &\frac{1}{p}\mu ^{1}_{\lambda ^{(p)}_{m+1}} \bigl\Vert (u,v) \bigr\Vert ^{p} _{L^{p}(\varOmega )}-\frac{2}{\alpha +\beta } \bigl(C_{\alpha ,\beta }^{(p)}\bigr)^{-( \alpha +\beta )}(\varOmega ) \bigl\Vert (u,v) \bigr\Vert ^{\alpha +\beta }_{H}. \end{aligned}$$

Since $p<\alpha +\beta <p^{*}$, there exists a small number $\rho >0$ such that if $(u,v)\in \partial B_{r}\cap X_{2}$, then $f(u,v)>0$. Thus $\inf_{(u,v)\in \partial B_{r}\cap X_{2}}f(u,v)>0$. Moreover, if $(u,v)\in B_{r}\cap X_{2}$, then $f(u,v)\ge -\frac{2}{\alpha +\beta }(C _{\alpha ,\beta }^{(p)})^{-(\alpha +\beta )}(\varOmega )\|(u,v)\|^{ \alpha +\beta }_{H}>-\infty$. Thus $\inf_{(u,v)\in B_{r}\cap X_{2}}f(u,v)>- \infty$. So the lemma is proved. □

Let us define

$$ \gamma =\inf_{h\in \varGamma }\sup_{u\in Q}f\bigl(h(u,v) \bigr). $$

Lemma 2.3

Assume that a, b, p, q, r, α, and β are real constants, and $1< p<\infty$, $q, r>1$, $p<\alpha +\beta <p^{*}$, $2\lambda ^{(p)}_{i}>a+b$, and $q_{\lambda ^{(p)}_{i}}(a,b)>0$. Let $i\in N$ and $(a_{0},b_{0})\in \partial D ^{\prime }_{\lambda ^{(p)}_{i}}$. Then there exist a neighborhood W of $(a_{0},b_{0})$, a small number σ, a small number $\rho >0$, and a large number $R>0$ such that, for any $(a,b)\in W\setminus D^{ \prime }_{\lambda ^{(p)}_{i}}$,

$$ 0< \inf_{u\in \partial B_{\rho }\cap X_{2}}f(u,v)\le \gamma = \inf_{h\in \varGamma } \sup_{u\in Q}f\bigl(h(u,v)\bigr)\le \sup_{u\in Q}f(u,v)< \infty . $$

Proof

By Lemma 2.1, we have

$$ \gamma =\inf_{h\in \varGamma }\sup_{u\in Q}f\bigl(h(u,v) \bigr)\le \sup_{u\in Q}f(u,v)< \infty . $$

By Lemma 2.2, we have

$$ 0< \inf_{u\in \partial B_{\rho }\cap X_{2}}f(u,v)\le \inf_{h\in \varGamma }\sup _{u\in Q}f\bigl(h(u,v)\bigr)=\gamma . $$

Thus the lemma is proved. □

3 $(\mathit{P.S.})$ condition and proof of Theorem 1.1

We need some lemma for the proof that $f(u,v)$ satisfies the $(\mathit{P.S.})$ condition.

Lemma 3.1

([1, 6])

Let $1< p<\infty$. Let $1<\tau \le p^{*}$. Then the embedding

$$ H=W^{1,p}\bigl(\varOmega ,R^{2}\bigr)\hookrightarrow L^{\tau }\bigl(\varOmega ,R^{2}\bigr) $$

is continuous and compact and, for every $(u,v)\in C^{\infty }_{0}( \varOmega ,R^{2})$, we have

$$ \bigl\Vert (u,v) \bigr\Vert _{L^{\tau }(\bar{\varOmega },R^{2})}\le C \bigl\Vert (u,v) \bigr\Vert _{H} $$

for a positive constant C independent of u.

By Lemma 3.1, we obtain the following.

Lemma 3.2

Assume that $1\le p<\infty$, a, b, p, q, r, α, and β are real constants and $q, r>1$ and $p<\alpha +\beta <p^{*}$. Then all the solutions of (1.1) belong to $\varLambda G\subset H$.

Proof

$$ \textstyle\begin{cases} u=-\Delta _{p}^{-1}(a |u|^{p-2}u-\operatorname{grad}_{u} \frac{1}{(|u|^{2}+|v|^{2})^{q}}-\operatorname{grad}_{u}\frac{1}{(|u-e_{1}|^{2}+|v-e _{2}|^{2})^{r}}+\frac{2\alpha }{\alpha +\beta }|u|^{\alpha -1}|v|^{ \beta }) \\ \hphantom{u={}} \mbox{in } \varOmega , \\ v=-\Delta _{p}^{-1}(b |v|^{p-2}v-\operatorname{grad}_{v} \frac{1}{(|u|^{2}+|v|^{2})^{q}}-\operatorname{grad}_{v} \frac{1}{(|u-d|^{2}+|v-d|^{2})^{r}}+\frac{2\beta }{\alpha +\beta }|u|^{ \alpha }|v|^{\beta -1}) \\ \hphantom{v={}} \mbox{in } \varOmega . \end{cases} $$

(3.1)

Since the right-hand side of (1.1) belongs to $L^{\alpha +\beta }( \varOmega ,G)$, where $p<\alpha +\beta <p^{*}$, and by Lemma 3.1, the embedding $W^{1,p}(\varOmega ,G)\hookrightarrow L^{\alpha +\beta }( \varOmega ,G)$ is continuous and compact, it follows that $-\Delta _{p} ^{-1}$ is a compact operator and the solutions of (3.1) are in $W^{1,p}(\varOmega ,G)=\varLambda G$. □

Lemma 3.3

Assume that $1\le p<\infty$, a, b, p, q, r, α, and β are real constants and $q, r>1$ and $p<\alpha +\beta <p^{*}$. Then the functional $f(u,v)$ is continuous, Fréchet differentiable with $Fr\acute{e}chet$ derivative in ΛG,

$$\begin{aligned}& Df(u,v)\cdot (z,w) \\& \quad = \int _{\varOmega }\biggl[-\Delta _{p} u\cdot z-\Delta _{p} v\cdot w-a |u|^{p-2}u \cdot z-b |v|^{p-2}v\cdot w \\& \qquad {}+\operatorname{grad}_{u}\frac{1}{(|u|^{2}+|v|^{2})^{q}}\cdot z+ \operatorname{grad} _{v}\frac{1}{(|u|^{2}+|v|^{2})^{q}}\cdot w \\& \qquad {}+\operatorname{grad}_{u}\frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}\cdot z+ \operatorname{grad}_{v}\frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}\cdot w \\& \qquad {}-\frac{2\alpha }{\alpha +\beta }|u|^{\alpha -1}|v|^{\beta }\cdot z- \frac{2 \beta }{\alpha +\beta }|u|^{\alpha }|v|^{\beta -1}\cdot w\biggr]\,dx \quad \forall (z,w)\in \varLambda G. \end{aligned}$$

Moreover, $Df\in C$. That is, $f\in C^{1}$.

Proof

Let us set $H(x,u,v)=\frac{1}{p}(a |u|^{p}+b |v|^{p}) - \frac{1}{(|u|^{2}+|v|^{2})^{q}}- \frac{1}{(|u-e_{1}|^{2}+|v-e_{2}|^{2})^{r}}+\frac{2}{\alpha +\beta }|u|^{ \alpha }|v|^{\beta }$. Then $H_{u}(x,u,v)=a |u|^{p-2}u-\operatorname{grad} _{u}\frac{1}{(|u|^{2}+|v|^{2})^{q}}-\operatorname{grad}_{u}\frac{1}{(|u-e_{1}|^{2}+|v-e _{2}|^{2})^{r}}+\frac{2\alpha }{\alpha +\beta }|u|^{\alpha -1}|v|^{ \beta }$ and $H_{v}(x,u,v)=b |v|^{p-2}v-\operatorname{grad}_{v} \frac{1}{(|u|^{2}+|v|^{2})^{q}}-\operatorname{grad}_{v}\frac{1}{(|u-e_{1}|^{2}+|v-e _{2}|^{2})^{r}}+\frac{2\beta }{\alpha +\beta }|u|^{\alpha }|v|^{ \beta -1}$. First we shall prove that $f(u,v)$ is continuous. For $u, v\in \varLambda G$,

$$\begin{aligned}& \bigl\vert f(u+z,v+w)-f(u,v) \bigr\vert \\& \quad = \biggl\vert \frac{1}{p} \int _{\varOmega }\bigl(-\Delta _{p} (u+z),-\Delta _{p} (v+w)\bigr) \cdot (u+z,v+w)\,dx- \int _{\varOmega }H(x,u+z,v+w)\,dx \\& \qquad {}-\frac{1}{p} \int _{\varOmega }(-\Delta _{p} u,-\Delta _{p} v) \cdot (u,v) \,dx + \int _{\varOmega }H(x,u,v)\,dx \biggr\vert \\& \quad = \biggl\vert \frac{1}{2} \int _{\varOmega }\bigl[\bigl(-\Delta _{p} (u+z),-\Delta _{p} (v+w)\bigr) \cdot (u+z,v+w)-(-\Delta _{p} u,-\Delta _{p} v) \cdot (u,v)\bigr]\,dx \\& \qquad {}- \int _{\varOmega }\bigl(H(x,u+z,v+w)-H(x,u,v)\bigr)\,dx \biggr\vert . \end{aligned}$$

We have

$$\begin{aligned}& \biggl\vert \frac{1}{2} \int _{\varOmega }\bigl[\bigl(-\Delta _{p} (u+z),-\Delta _{p} (v+w)\bigr) \cdot (u+z,v+w)-(-\Delta _{p} u,-\Delta _{p} v) \cdot (u,v)\bigr]\,dx \biggr\vert \\& \quad \le O\bigl( \bigl\Vert (z,w) \bigr\Vert _{H}\bigr) \end{aligned}$$

(3.2)

and

$$\begin{aligned}& \biggl\vert \int _{\varOmega }\bigl[H(x,u+z,v+w)-H(x,u,v)\bigr]\,dx \biggr\vert \\& \quad \le \biggl\vert \int _{\varOmega }\bigl[\bigl(H_{u}(x,u,v),H_{v}(x,u,v) \bigr)\cdot (z,w) +O\bigl( \bigl\Vert (z,w) \bigr\Vert _{H}\bigr) \bigr]\,dx \biggr\vert = O\bigl( \bigl\Vert (z,w) \bigr\Vert _{H} \bigr). \end{aligned}$$

(3.3)

Thus we have

$$ \bigl\vert f(u+z,v+w)-f(u,v) \bigr\vert = O\bigl( \bigl\Vert (z,w) \bigr\Vert _{H}\bigr). $$

Next we shall prove that $f(u,v)$ is Fréchet differentiable. For $u, v\in \varLambda G$,

$$\begin{aligned}& \bigl\vert f(u+z,v+w)-f(u,v)-D f(u,v)\cdot (z,w) \bigr\vert \\& \quad = \biggl\vert \frac{1}{p} \int _{\varOmega }\bigl(-\Delta _{p} (u+z),-\Delta _{p} (v+w)\bigr) \cdot (u+z,v+w)\,dx- \int _{\varOmega }H(x,u+z,v+w)\,dx \\& \qquad {}-\frac{1}{p} \int _{\varOmega }(-\Delta _{p} u,-\Delta _{p} v) \cdot (u,v) \,dx + \int _{\varOmega }H(x,u,v)\,dx \\& \qquad {}- \int _{\varOmega } \bigl(-\Delta _{p} u-H_{u}(x,u,v),- \Delta _{p} v-H_{v}(x,u,v)\bigr) \cdot (z,w)\,dx \biggr\vert \\& \quad = \biggl\vert \frac{1}{p} \int _{\varOmega }\bigl[\bigl(-\Delta _{p} (u+z),-\Delta _{p} (v+w)\bigr) \cdot (u+z,v+w) \\& \qquad {}-(-\Delta _{p} u,-\Delta _{p} v)\cdot (u,v)-(- \Delta _{p} u,-\Delta _{p} v) \cdot (z,w)\bigr]\,dx \\& \qquad {}- \int _{\varOmega }\bigl[H(x,u+z,v+w)-H(x,u,v)-\bigl(H_{u}(x,u,v),H_{v}(x,u,v) \bigr) \cdot (z,w)\bigr]\,dx \biggr\vert . \end{aligned}$$

By (3.2) and (3.3),

$$ \bigl\vert f(u+z,v+w)-f(u,v)-Df(u,v)\cdot (z,w) \bigr\vert = O\bigl( \bigl\Vert (z,w) \bigr\Vert ^{2}_{H}\bigr). $$

Thus $f\in C^{1}$. □

Lemma 3.4

(A priori estimate)

Assume that $1\le p<\infty$, a, b, p, q, r, α, and β are real constants, $q, r>1$, $p<\alpha +\beta <p^{*}$, $2\lambda ^{(p)}_{i}>a+b$, and $q_{\lambda ^{(p)}_{i}}(a,b)>0$. Let $(u_{n},v_{n})_{n}$ be any sequence in ΛG and $\gamma \in R$ be any positive real number. Then there exist constants $C_{i}=C_{i}( \gamma )$, $i=1, 2, 3$, such that if $(u_{n},v_{n})_{n}\in \varLambda G$ satisfies that $f(u_{n},v_{n})\to \gamma$ and $Df(u_{n},v_{n})\to \theta$, then

$$\begin{aligned}& \lim_{n\to \infty } \bigl\Vert (u_{n},v_{n}) \bigr\Vert _{L^{\alpha +\beta }(\varOmega )} \le C_{1}, \\& \lim_{n\to \infty } \int _{\varOmega }\frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q} }\,dx\le C_{2}, \\& \int _{\varOmega }\frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}\,dx \le C_{3}. \end{aligned}$$

Proof

Let $\gamma \in R$ be any positive real number. Let $(u_{n},v_{n})_{n}$ be any sequence in ΛG such that $f(u_{n},v_{n})\to \gamma$ and $Df(u_{n},v_{n})\to \theta$. Then there exists a small number $\epsilon >0$ such that

$$\begin{aligned}& \gamma +\epsilon \\& \quad \ge \lim_{n\to \infty }f(u_{n},v_{n})-\lim _{n\to \infty } \frac{1}{p}Df(u_{n},v_{n}) \cdot (u_{n},v_{n}) \\& \quad =\lim_{n\to \infty }\biggl[ \int _{\varOmega }\frac{1}{(|u_{n}|^{2}+|v _{n}|^{2})^{q} }\,dx+ \int _{\varOmega }\frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e _{2}|^{2})^{r}}\,dx \\& \qquad {}-\frac{2}{\alpha +\beta } \int _{\varOmega }|u_{n}|^{ \alpha }|v_{n}|^{\beta } \,dx\biggr] \\& \qquad {}-\lim_{n\to \infty }\frac{1}{p} \int _{\varOmega }\biggl[\operatorname{grad} _{u} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}\cdot u_{n}+\operatorname{grad} _{v} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}\cdot v_{n} \\& \qquad {}+\operatorname{grad}_{u}\frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \cdot u_{n}+\operatorname{grad}_{v} \frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}\cdot v_{n} \\& \qquad {}-\frac{2\alpha }{\alpha +\beta }|u_{n}|^{\alpha -1}|v_{n}|^{\beta } \cdot u_{n}-\frac{2\beta }{\alpha +\beta }|u_{n}|^{\alpha }|v_{n}|^{ \beta -1} \cdot v_{n}\biggr] \,dx \quad \forall (u_{n},v_{n}) \in \varLambda G. \end{aligned}$$

By $\lim_{n\to \infty }Df(u_{n},v_{n})=\theta$, we have

$$ \textstyle\begin{cases} \lim_{n\to \infty }u_{n}=\lim_{n\to \infty }(-\Delta _{p} -a g_{p})^{-1}(- \operatorname{grad}_{u} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}-\operatorname{grad} _{u}\frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \\ \hphantom{\lim_{n\to \infty }u_{n}={}}{}+\frac{2\alpha }{\alpha +\beta }|u_{n}|^{\alpha -1}|v_{n}|^{\beta }), \\ \lim_{n\to \infty }v_{n}=\lim_{n\to \infty } (-\Delta _{p} -b g_{p})^{-1}(- \operatorname{grad}_{v}\frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}-\operatorname{grad} _{v}\frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \\ \hphantom{\lim_{n\to \infty }v_{n}={}}{}+\frac{2\beta }{\alpha +\beta }|u_{n}|^{\alpha }|v_{n}|^{\beta -1}), \end{cases} $$

where $g_{p}(t)=|t|^{p-2}t$ for $t\neq 0$ and $g_{p}(0)=0$. By $2\lambda ^{(p)}_{i}>a+b$ and $q_{\lambda ^{(p)}_{i}}(a,b)= \operatorname{Det}( \lambda ^{(p)}_{j} I-A)=(\lambda ^{(p)}_{j}-a)(\lambda ^{(p)}_{j}-b)>0$, we have $\lambda ^{(p)}_{j}-a>0$ and $\lambda ^{(p)}_{j}-b>0$. Thus $(-\Delta _{p} -a g_{p})^{-1}$ and $(-\Delta _{p} -b g_{p})^{-1}$ are positive operators. Since

$$\begin{aligned}& \lim_{n\to \infty }\biggl({-}\operatorname{grad}_{u} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}-\operatorname{grad}_{u}\frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}+ \frac{2\alpha }{\alpha + \beta }|u_{n}|^{\alpha -1}|v_{n}|^{\beta } \biggr) \\& \quad >0, \\& \lim_{n\to \infty } \biggl({-}\operatorname{grad}_{v} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}-\operatorname{grad}_{v}\frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}+ \frac{2\beta }{\alpha +\beta }|u_{n}|^{\alpha }|v_{n}|^{\beta -1} \biggr) \\& \quad >0, \end{aligned}$$

and $(-\Delta _{p} -a g_{p})^{-1}$ and $(-\Delta _{p} -b g_{p})^{-1}$ are positive operators, it follows that $\lim_{n\to \infty }u_{n}>0$ and $\lim_{n\to \infty }v_{n}>0$. Then we have

$$\begin{aligned}& {-}\operatorname{grad}_{u} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}} u_{n}- \operatorname{grad}_{v}\frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}} v_{n}=2q \frac{1}{(|u _{n}|^{2}+|v_{n}|^{2})^{q}}>0, \\& {-}\operatorname{grad}_{u}\frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} u _{n}>0\quad \mbox{and}\quad {-}\operatorname{grad}_{v} \frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} v_{n}>0. \end{aligned}$$

Therefore we have

$$\begin{aligned}& \gamma +\epsilon \\& \quad \ge \lim_{n\to \infty }\biggl[ \int _{\varOmega }\frac{1}{(|u _{n}|^{2}+|v_{n}|^{2})^{q}}\,dx \\& \qquad {} + \int _{\varOmega } \frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}\,dx-\frac{2}{\alpha +\beta } \int _{\varOmega }|u_{n}|^{\alpha }|v_{n}|^{\beta } \,dx\biggr] \\& \qquad {}-\frac{1}{p}\lim_{n\to \infty } \int _{\varOmega }\biggl[\operatorname{grad} _{u} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}\cdot u_{n}+\operatorname{grad} _{v} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}\cdot v_{n} \\& \qquad {}+\operatorname{grad}_{u}\frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \cdot u_{n}+\operatorname{grad}_{v} \frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}\cdot v_{n} \\& \qquad {}-\frac{2\alpha }{\alpha +\beta }|u_{n}|^{\alpha -1}|v_{n}|^{\beta } \cdot u_{n}-\frac{2\beta }{\alpha +\beta }|u_{n}|^{\alpha }|v_{n}|^{ \beta -1} \cdot v_{n}\biggr]\,dx \\& \quad = \biggl(\frac{2q}{p}+1\biggr)\lim_{n\to \infty } \int _{\varOmega }\biggl[\frac{1}{(|u _{n}|^{2}+|v_{n}|^{2})^{q} } \biggr]\,dx+\lim _{n\to \infty } \int _{\varOmega }\frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}\,dx \\& \qquad {}-\frac{1}{p}\lim_{n\to \infty } \int _{\varOmega }\operatorname{grad}_{u} \frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \cdot u_{n} \,dx \\& \qquad {}-\frac{1}{p}\lim_{n\to \infty } \int _{\varOmega }\operatorname{grad}_{v}\frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \cdot v_{n}\,dx \\& \qquad {}+\biggl(\frac{2}{p}-\frac{2}{\alpha +\beta }\biggr) \lim _{n\to \infty } \int _{\varOmega }\bigl[|u_{n}|^{\alpha }|v_{n}|^{\beta } \bigr]\,dx \quad \forall (u_{n},v_{n})\in \varLambda G. \end{aligned}$$

Since $1< p<\infty$, $q, r>1$, $p<\alpha +\beta <p^{*}$, we have that $\frac{2q}{p}+1>0$ and $\frac{2}{p}- \frac{2}{\alpha +\beta }>0$. Since $\int _{\varOmega } \frac{1}{(|u_{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}\,dx>0$, $- \frac{1}{p}\lim_{n\to \infty }\int _{\varOmega }\operatorname{grad}_{u} \frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}\cdot u_{n} \,dx >0$, and $-\frac{1}{p}\lim_{n\to \infty }\int _{\varOmega }\operatorname{grad}_{v}\frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}\cdot v_{n}\,dx>0$, it follows that there exist constants $C_{i}=C_{i}(\gamma )$, $i=1, 2, 3$, such that $\lim_{n\to \infty }\|(u_{n},v_{n})\|_{L^{\alpha +\beta }(\varOmega )} \le C_{1}$, $\lim_{n\to \infty }\int _{\varOmega }(\frac{1}{|u_{n}|^{2}+|v _{n}|^{2})^{q} }\,dx\le C_{2}$, and $\int _{\varOmega }\frac{1}{(|u_{n}-e _{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}}\,dx\le C_{3}$. □

Lemma 3.5

If any sequence $(u_{n},v_{n})_{n}$ in ΛG satisfies

$$ (u_{n},v_{n})\to (z,w)\in \partial \varLambda G, $$

then

$$ f(u_{n},v_{n})\to \infty . $$

Proof

The proof can be checked easily. □

Now, we shall prove that $f(u,v)$ satisfies $(\mathit{P.S.})_{\gamma }$ with $\gamma >0$ as follows.

Lemma 3.6

(Palais–Smale condition)

Assume that $1\le p<\infty$, a, b, p, q, r, α, and β are real constants, and $q, r>1$ and $p<\alpha + \beta <p^{*}$. Let γ be any positive real number. Then $f(u,v)$ satisfies the Palais–Smale condition: if $(u_{n},v_{n})_{n}\in \varLambda G$ is any sequence such that $f(u_{n},v_{n})\to \gamma$ and $Df(u_{n},v_{n})\to \theta$, $\theta =(0,0)$, then $(u_{n},v_{n})$ has a convergent subsequence $(u_{n_{i}},v_{n_{i}})$ such that $(u_{n_{i}},v _{n_{i}})$ converges strongly to $(u_{0},v_{0})\in \varLambda G$.

Proof

Let $(u_{n},v_{n})_{n}$ be any sequence in ΛG such that $f(u_{n},v_{n})\to \gamma$, $\gamma >0$ and $Df(u_{n},v_{n})\to \theta$. By Lemma 3.4, $\lim_{n\to \infty }\|(u_{n},v_{n})\|_{L^{ \alpha +\beta }(\varOmega )}$ is finite. Thus $(u_{n},v_{n})_{n}$ is bounded in $L^{\alpha +\beta }(\varOmega )$. Then, up to subsequence, $(u_{n},v_{n})_{n}$ converges weakly to some $(u_{0},v_{0})$. Since $Df(u_{n},v_{n})\to \theta$, we have

$$ \textstyle\begin{cases} \lim_{n\to \infty } u_{n}=-\lim_{n\to \infty }\Delta ^{-1}_{p} (a |u _{n}|^{p-2}u_{n}-\operatorname{grad}_{u} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}-\operatorname{grad}_{u}\frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \\ \hphantom{\lim_{n\to \infty } u_{n}={}}{}+\frac{2\alpha }{\alpha +\beta }|u_{n}|^{\alpha -1}|v_{n}|^{\beta }), \\ \lim_{n\to \infty } v_{n}=-\lim_{n\to \infty }\Delta ^{-1}_{p} (b |v _{n}|^{p-2}v_{n}-\operatorname{grad}_{v} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}-\operatorname{grad}_{v}\frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \\ \hphantom{\lim_{n\to \infty } v_{n}={}}{} +\frac{2\beta }{\alpha +\beta }|u_{n}|^{\alpha }|v_{n}|^{\beta -1}). \end{cases} $$

By Lemma 3.4, $(u_{n},v_{n})_{n}$,

$$\begin{aligned}& \biggl(a |u_{n}|^{p-2}u_{n}-\operatorname{grad}_{u} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}-\operatorname{grad}_{u}\frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \\& \quad {}+ \frac{2\alpha }{\alpha + \beta }|u_{n}|^{\alpha -1}|v_{n}|^{\beta } \biggr)_{n} \end{aligned}$$

and

$$\begin{aligned}& \biggl(b |v_{n}|^{p-2}v_{n}-\operatorname{grad}_{v} \frac{1}{(|u_{n}|^{2}+|v_{n}|^{2})^{q}}-\operatorname{grad}_{v}\frac{1}{(|u _{n}-e_{1}|^{2}+|v_{n}-e_{2}|^{2})^{r}} \\& \quad {}+ \frac{2\beta }{\alpha +\beta }|u_{n}|^{\alpha -1}|v_{n}|^{\beta } \biggr)_{n} \end{aligned}$$

are bounded in $L^{\alpha +\beta }(\varOmega )$. Since the embedding ΛG into $L^{\alpha + \beta }(\varOmega )$, $p<\alpha +\beta <p^{*}$, is compact, $-\Delta ^{-1} _{p}$ is a compact operator, it follows that $(u_{n},v_{n})_{n}$ has a convergent subsequence $(u_{n_{i}},v_{n_{i}})$ converging strongly to some $(u_{0},v_{0})$ such that

$$ DF(u_{0},v_{0})=\lim_{n\to \infty }DF(u_{n_{i}},v_{n_{i}})=(0,0). $$

We claim that $(u_{0},v_{0})\neq \theta$ and $(u_{0},v_{0})\neq (e _{1},e_{2})$. By contradiction, we suppose that $(u_{0},v_{0})=\theta$ or $(u_{0},v_{0})=(e_{1},e_{2})$. Then $f(u_{0},v_{0})=\infty$, which is absurd. Thus $(u_{0},v_{0})\neq \theta$ and $(u_{0},v_{0})\neq (e _{1},e_{2})$. □

Proof of Theorem 1.1

Assume that a, b, p, q, r, α, and β are real constants, and $1< p<\infty$, $q, r>1$, $p<\alpha +\beta <p ^{*}$, $2\lambda ^{(p)}_{i}>a+b$, and $q_{\lambda ^{(p)}_{i}}(a,b)>0$. By Lemma 3.3, $f(u,v)$ is continuous and Fréchet differentiable in ΛG and $Df\in C$. By Lemma 3.6, $f(u,v)$ satisfies the Palais–Smale condition. We claim that $\gamma >0$ is a critical value of $f(u,v)$, that is, $f(u,v)$ has a critical point $(u_{0},v_{0})$ such that

$$\begin{aligned}& f(u_{0},v_{0})=\gamma , \\& Df(u_{0},v_{0})=0. \end{aligned}$$

In fact, by contradiction, we suppose that $\gamma >0$ is not a critical value of $f(u,v)$. Then by Theorem A.4 in [6], for any $\bar{\epsilon }\in (0,\gamma )>0$, there exist a constant $\epsilon \in (0,\bar{ \epsilon })$ and a deformation $\eta \in C([0,1]\times \varLambda G, \varLambda G)$ such that

• (i) $\eta (0,(u,v))=(u,v)$ for all $(u,v)\in \varLambda G$,

• (ii) $\eta (s,(u,v))=(u,v)$ for all $s\in [0,1]$ if $f(u,v)\notin [ \gamma -\bar{\epsilon },\gamma +\bar{\epsilon }]$,

• (iii) $f(\eta (1,(u,v)))\le \gamma -\epsilon$ if $f(u,v)\le \gamma +\epsilon$.

We can choose $h\in \varGamma$ such that

$$ \sup_{u\in Q} f\bigl(h(u,v)\bigr)\le \gamma +\epsilon $$

and

$$ f\bigl(h(u,v)\bigr)< \gamma -\bar{\epsilon } \quad \mbox{on } \partial Q. $$

This leads to $f(h(u,v))\notin [\gamma -\bar{\epsilon },\gamma +\bar{ \epsilon }]$. Thus by (ii),

$$ \eta \bigl(1,h(u,v)\bigr)=h(u,v) \quad \mbox{on } \partial Q. $$

Hence $\eta (1,h(u,v))\in \varGamma$. By (iii) and the definition of γ,

$$ \gamma \le \sup_{u\in Q}f\bigl(\eta \bigl(1,h(u,v)\bigr)\bigr)= \sup_{u\in Q}f\bigl(h(u,v)\bigr) \le \gamma -\epsilon , $$

which is a contradiction. Thus γ is a critical value of $f(u,v)$. Thus $f(u,v)$ has a critical point $(u_{0},v_{0})$ with a critical value

$$ \gamma =F(u_{0},v_{0}) $$

such that

$$ 0< \inf_{u\in \partial B_{\rho }\cap X_{2}}f(u,v)\le \gamma \le \sup_{u\in Q}f(u,v)< \infty . $$

By Lemma 3.4,

$$ (u_{0},v_{0})\neq \theta =(0,0), \qquad (u_{0},v_{0})\neq (e_{1},e_{2}). $$

Thus (1.1) has at least one nontrivial solution $(u_{0},v_{0})$ such that $(u_{0},v_{0})\neq \theta$ and $(u_{0},v_{0})\neq (e_{1},e_{2})$. Thus Theorem 1.1 is proved. □