Introduction

The aim of this paper is to establish the existence of nontrivial solutions to the following quasilinear elliptic system:

Δ p u μ | u | p 2 u | x | p = 1 p F u ( x , u , v ) + λ | u | q 2 u | x | s , x Ω , Δ p v μ | v | p 2 v | x | p = 1 p F v ( x , u , v ) + θ | v | q 2 v | x | s , x Ω , u = v = 0 , x ∂Ω ,
(1)

where 0 ∈ Ω is a bounded domain in R N (N≥3) with smooth boundary Ω, λ > 0, θ > 0, 0μ< μ ¯ N p p p , 0 ≤ s < p, 1 ≤ q < p, and p (t) p ( N t ) N p is the Hardy-Sobolev critical exponent. Note that p (0)= p = pN N p is the Sobolev critical exponent. We assume that F C 1 ( Ω ¯ × ( R + ) 2 , R + ) is positively homogeneous of degree p, that is, F(x,tu,tv)= t p F(x,u,v) (t > 0) holds for all (x,u,v) Ω ¯ × ( R + ) 2 , (F u (x,u,v),F v (x,u,v))=∇F(x,u,v).

Problem (1) is related to the well-known Caffarelli-Kohn-Nirenberg inequality in [1]:

Ω | u | r | x | t dx p r C r , t , p Ω |u | p dx,for allu D 0 1 , p (Ω),
(2)

where pr < p(t). If t =r =p, the above inequality becomes the well-known Hardy inequality [13]:

Ω | u | p | x | p dx 1 μ ¯ Ω |u | p dx,for allu D 0 1 , p (Ω).
(3)

In the space D 0 1 , p (Ω), we employ the following norm:

| | u | | = | | u | | D 0 1 , p ( Ω ) : = Ω | u | p μ | u | p | x | p dx 1 p , μ [ 0 , μ ¯ ) .

Using the Hardy inequality (3), this norm is equivalent to the usual norm Ω | u | p dx 1 p . The operator L:= | · | p 2 · μ | · | p 2 | x | p is positive in D1,p(Ω) if 0μ< μ ¯ .

Now, we define the space W= D 0 1 , p (Ω)× D 0 1 , p (Ω) with the norm

| | ( u , v ) | | p = | | u | | p + | | v | | p .

Also, by Hardy inequality and Hardy-Sobolev inequality, for 0μ< μ ¯ , 0 ≤ t < p and prp(t), we can define the best Hardy-Sobolev constant:

A μ , t , r ( Ω ) = inf u D 0 1 , p ( Ω ) { 0 } Ω | u | p μ | u | p | x | p dx Ω | u | r | x | t dx p r .

In the important case when r =p(t), we simply denote A μ , t , p ( t ) as Aμ,t. Note that Aμ,0 is the best constant in the Sobolev inequality, namely,

A μ , 0 ( Ω ) = inf u D 0 1 , p ( Ω ) { 0 } Ω | u | p μ | u | p | x | p dx Ω | u | p dx p p .

Also, we denote

à μ , F = inf ( u , v ) W { ( 0 , 0 ) } Ω | u | p + | v | p μ | u | p + | v | p | x | p dx Ω F ( x , u , v ) dx p p .
(4)

Throughout this paper, let R0 be the positive constant such that ΩB (0;R0), where B(0; R 0 )={x R N :|x|< R 0 }. By Hölder and Sobolev-Hardy inequalities, for all u D 0 1 , p (Ω), we obtain

Ω | u | q | x | s B ( 0 ; R 0 ) | x | s p ( s ) q p ( s ) Ω | u | p ( s ) | x | s q p ( s ) 0 R 0 r N s + 1 dr p ( s ) q p ( s ) A μ , s q p | | u | | q N ω N R 0 N s N s p ( s ) q p ( s ) A μ , s q p | | u | | q ,
(5)

where ω N = 2 Π N 2 ( N 2 ) is the volume of the unit ball in R N .

Existence of nontrivial nonnegative solutions for elliptic equations with singular potentials was recently studied by several authors, but, essentially, only with a solely critical exponent. We refer, e.g., in bounded domains and for p =2 in [36], and for general p > 1 in [711] and the references therein. For example, Kang [11] studied the following elliptic equation via the generalized Mountain Pass Theorem [12]:

Δ p u μ | u | p 2 u | x | p = | u | p ( t ) 2 u | x | t + λ | u | p 2 u | x | s , x Ω , u = 0 , x ∂Ω
(6)

where Ω R N is a bounded domain, 1 < p < N, 0 ≤ s,t < p and 0μ< μ ¯ N p p p . Also, the authors in [13] via the Mountain Pass Theorem of Ambrosetti and Rabinowitz [14] proved that

Δ p u μ u p 1 | x | p = | u | p 1 + u p ( s ) 1 | x | s , in R N

admits a positive solution in R N , whenever μ< μ ¯ N p p p .

Also, in recent years, several authors have used the Nehari manifold to solve semilinear and quasilinear problems (see [1522] and references therein). Brown and Zhang [23] have studied a subcritical semi-linear elliptic equation with a sign-changing weight function and a bifurcation real parameter in the case p =2 and Dirichlet boundary conditions. In [22], the author studied the Equation 6 via the Nehari manifold. Exploiting the relationship between the Nehari manifold and fibering maps (i.e., maps of the form tJ λ (t u), where J λ is the Euler function associated with the equation), they gave an interesting explanation of the well-known bifurcation result. In fact, the nature of the Nehari manifold changes as the parameter λ crosses the bifurcation value. In this work, we give a variational method which is similar to the fibering method (see [16, 23]) to prove the existence and multiplicity of nontrivial nonnegative solutions of problem (1).

Before stating our result, we need the following assumptions:

(H1) F: Ω ¯ × ( R + ) 2 R + is a C1 function such that F(x,tu,tv)= t p F(x,u,v) (t > 0) holds for all (x,u,v) Ω ¯ × ( R + ) 2 ;

(H2) F (x,,u,0) = F (x,,0,v) = F u (x,0,v) = F v (x,u,0) = 0 where u,v R + ;

(H3) F u (x,u,v) and F v (x,u,v) are strictly increasing functions about u > 0 and v > 0.

Moreover, using assumption (H1), we have the so-called Euler identity

(u,v)·F(x,u,v)= p F(x,u,v),
(7)

and

F(x,u,v)K | u | p + | v | p p p ,for some constantK>0.
(8)

This paper is divided into three sections organized as follows: In the ‘Notations and preliminaries,’ we establish some elementary results. Finally, in the ‘Main results and proof,’ we state our main result (Theorem 1) and prove it.

Notations and preliminaries

The corresponding energy functional of problem (1) is defined by

J λ , θ ( u , v ) = 1 p | | ( u , v ) | | p 1 p Ω F ( x , u , v ) dx 1 q K λ , θ ( u , v ) ,

for each (u,v) ∈ W, where K λ , θ (u,v)=λ Ω | u | q | x | s dx+θ Ω | v | q | x | s dx.

In order to verify J λ , θ C 1 (W,R), we need the following lemmas:

Lemma 1

Suppose that (H3) holds. Assume that F C 1 ( Ω ¯ × R 2 ,R) is positively homogenous of degree p, then F u , F v C( Ω ¯ × R + 2 , R + ) is positively homogenous of degree p−1.

Moreover by Lemma 1, we get the existence of positive constant M such that

| F u (x,u,v)|M | u | p 1 + | v | p 1 ,x Ω ¯ ,u,v R + ,
(9)
| F v (x,u,v)|M | u | p 1 + | v | p 1 ,x Ω ¯ ,u,v R + .
(10)

Now, we consider the functional ψ(u,v)= Ω F(x,u,v)dx, then by Lemma 1, (9), (10), and by similar computation as Lemma 2.2 in [24], we get the functional ψ of class C 1 (W, R + ) and ψ (u,v),(a,b)= Ω ( F u (x,u,v)a+ F u (x,u,v)b)dx, where (u,v),(a,b) ∈ W. Thus, we have J λ , θ C 1 (W,R)

Now, we consider the problem on the Nehari manifold. Define the Nehari manifold (cf.[25]):

N λ , θ = ( u , v ) W { ( 0 , 0 ) } | J λ , θ ( u , v ) , ( u , v ) = 0 ,

where

J λ , θ ( u , v ) , ( u , v ) = | | ( u , v ) | | p Ω F ( x , u , v ) dx K λ , θ ( u , v ) .

Note that Nλ,θ contains every nonzero solution of (1). Define

Φ λ , θ ( u , v ) = J λ , θ ( u , v ) , ( u , v ) ,

then for (u,v) ∈ Nλ,θ,

Φ λ , θ ( u , v ) , ( u , v ) = p | | ( u , v ) | | p p Ω F ( x , u , v ) dx q K λ , θ ( u , v )
(11)
= ( p q ) | | ( u , v ) | | p ( p q ) Ω F ( x , u , v ) dx
(12)
= ( p p ) | | ( u , v ) | | p ( q p ) K λ , θ ( u , v ) .
(13)

Now, we split Nλ,θ into three parts:

N λ , θ + = ( u , v ) N λ , θ : Φ λ , θ ( u , v ) , ( u , v ) > 0 , N λ , θ 0 = ( u , v ) N λ , θ : Φ λ , θ ( u , v ) , ( u , v ) = 0 , N λ , θ = ( u , v ) N λ , θ : Φ λ , θ ( u , v ) , ( u , v ) < 0 .

To state our main result, we now present some important properties of N λ , θ + , N λ , θ 0 and N λ,θ −.

Lemma 2.

There exists a positive number C = C(p,q,N,S) > 0 such that if 0< λ p p q + θ p p q <C, then N λ , θ 0 =.

Proof

Suppose otherwise, let

C = p q K ( p q ) p p p p p p q p p q × N ω N R 0 N s N s p ( p ( s ) q ) p ( s ) ( p q ) A μ , s q p q A μ , 0 p p p .

Then, there exists (λ,θ) with

0 < λ p p q + θ p p q < C ,

such that N λ , θ 0 . Then, for (u,v) N λ , θ 0 , by (12) and (13), one can get

| | ( u , v ) | | p = p q p q Ω F ( x , u , v ) dx.

By the Sobolev imbedding theorem, the Minkowski inequality and (8),

Ω F ( x , u , v ) dx K Ω | u | p + | v | p p p dx p p · p p K Ω | u | p dx p p + Ω | v | p dx p p p p = K | | u | | L p ( Ω ) p + | | v | | L p ( Ω ) p p p K A μ , 0 p p | | u | | p + | | v | | p p p = K A μ , 0 p p | | ( u , v ) | | p .
(14)

It follows that

| | ( u , v ) | | p q K ( p q ) A μ , 0 p p 1 p p ,

and

p p p q | | ( u , v ) | | p = K λ , θ ( u , v ) = λ Ω | u | q | x | s dx + θ Ω | v | q | x | s dx N ω N R 0 N s N s p ( s ) q p ( s ) × A μ , s q p λ p p q + θ p p q p q p | | ( u , v ) | | q .

Thus,

| | ( u , v ) | | p q p p 1 p q N ω N R 0 N s N s p ( s ) q p ( s ) ( p p ) × A μ , s q p ( p p ) λ p p q + θ p p q 1 p .

This implies

λ p p q + θ p p q C.

This is a contradiction! Here,

C = p q K ( p q ) p p p p p p q p p q × N ω N R 0 N s N s p ( p ( s ) q ) p ( s ) ( p q ) A μ , s q p q A μ , 0 p p p .

Lemma 3

The energy functional Jλ,θ is coercive and bounded below on Nλ,θ.

Proof

If (u,v) ∈ Nλ,θ, then by (5),

J λ , θ ( u , v ) = 1 p | | ( u , v ) | | p 1 p Ω F ( x , u , v ) dx 1 q K λ , θ ( u , v ) p p p p | | ( u , v ) | | p p q p q N ω N R 0 N s N s p ( s ) q p ( s ) × λ p p q + θ p p q p q p A μ , s q p | | ( u , v ) | | q .

Since 0 ≤ s < N, 1 < q < p < p, we see that Jλ,θ is coercive and bounded below on Nλ,θ. □

Furthermore, similar to the argument in Brown and Zhang (see[23], Theorem 2.3 or see Binding et al. [26]), we can conclude the following result:

Lemma 4

Assume that (u0,v0) is a local minimizer for Jλ,θ on Nλ,θ and that ( u 0 , v 0 ) N λ , θ 0 , then J λ,θ′(u0,v0)= 0 in W−1.

Now, by Lemma 2, we let

Θ C 0 = ( λ , θ ) R 2 { ( 0 , 0 ) } : 0 < λ p p q + θ p p q < C ,

where C 0 = q p p p q C<C. If (λ,θ) Λ C 0 , we have N λ , θ = N λ , θ + N λ , θ . Define

ξ λ , θ = inf ( u , v ) N λ , θ J λ , θ ( u , v ) ξ λ , θ + = inf ( u , v ) N λ , θ + J λ , θ ( u , v ) ξ λ , θ = inf ( u , v ) N λ , θ J λ , θ ( u , v )

Lemma 5

There exists a positive number C0 such that if (λ,θ) Λ C 0 , then

(i) ξλ,θξ λ,θ + < 0;

(ii) there exists d0= d0(p,q,N,K,S,λ,θ) > 0 such that ξ λ,θ − > d0.

Proof

(i) For (u,v) ∈ N λ,θ +, by (13), we have

K λ , θ ( u , v ) p p p q | | ( u , v ) | | p ,

and so

J λ , θ ( u , v ) = 1 p 1 p | | ( u , v ) | | p 1 q 1 p K λ , θ ( u , v ) 1 p 1 p | | ( u , v ) | | p 1 q 1 p p p p q | | ( u , v ) | | p = p p p 1 p 1 q | | ( u , v ) | | p < 0 .

Thus, from the definition of ξλ,θ and ξ λ,θ +, we can deduce that ξλ,θ < ξ λ,θ + < 0.

(ii) For (u,v) ∈ N λ,θ−, by Lemma 2,

| | ( u , v ) | | p q K ( p q ) A μ , 0 p p 1 p p .

Moreover, by Lemma 3,

J λ , θ ( u , v ) p p p p | | ( u , v ) | | p p q p q N ω N R 0 N s N s p ( s ) q p ( s ) × λ p p q + θ p p q p q p A μ , s q p | | ( u , v ) | | q
= | | ( u , v ) | | q p ( t ) p p p | | ( u , v ) | | p q p q p q N ω N R 0 N s N s p ( s ) q p ( s ) × A μ , s q p λ p p q + θ p p q p q p p q K ( p q ) A μ , 0 p p q p p p p p p | | ( u , v ) | | p q p q p q N ω N R 0 N s N s p ( s ) q p ( s ) × A μ , s q p λ p p q + θ p p q p q p .

Thus, if 0< λ p p q + θ p p q < C 0 , then for each (u,v) ∈ N λ,θ − one can get

J λ , θ ( u , v ) d 0 = d 0 ( p , q , N , K , S , λ , θ ) > 0 .

For each (u,v) ∈ W ∖{(0,0)} such that Ω F(x,u,v)dx>0, let

t max = ( p q ) | | ( u , v ) | | p ( p q ) Ω F ( x , u , v ) dx 1 p p .

Lemma 6

Assume that 0< λ p p q + θ p p q < C 0 . Then, for every (u,v) ∈ W with Ω F(x,u,v)dx>0, there exists tmax > 0 such that there are unique t+ and t with 0 < t+ < tmax < t such that ( t ± u, t ± v) N λ , θ ± and

J λ , θ ( t + u , t + v ) = inf 0 t t max J λ , θ ( tu , tv ) , J λ , θ ( t u , t v ) = sup t t max J λ , θ ( tu , tv ) .

Proof

The proof is similar to Lemma 2.6 in [17] and is omitted. □

Remark 1

If

0 < λ p p q + θ p p q < C 0 ,

then, by Lemmas 5 and 6 for every (u,v) ∈ W with Ω F(x,u,v)dx>0, we can easily deduce that there exists tmax > 0 such that there are unique t with tmax < t such that (tu,tv) ∈ N λ,θ− and

J λ , θ ( t u , t v ) = sup t 0 J λ , θ ( tu , tv ) ξ λ , θ > 0 .

Main results and proof

We are now ready to state our main result.

Theorem 1

Assume that 0 ≤ s < p, N≥3, 0μ< μ ¯ and 1 ≤ q < p. Then, we have the following results:

(i) If λ,θ > 0 satisfy λ p p q + θ p p q <C, then (1) has at least one positive solution in W.

(ii) If λ,θ > 0 satisfy 0< λ p p q + θ p p q < C 0 , then (1) has at least two positive solutions in W.

Now, we give an example to illustrate the result of Theorem 1.

Example 1

Consider the problem

Δ p u μ | u | p 2 u | x | p = α α + β | u | α 2 | v | β u + λ | u | q 2 u | x | s , x Ω , Δ p v μ | v | p 2 v | x | p = β α + β | u | α | v | β 2 v + θ | v | q 2 v | x | s , x Ω , u = v = 0 , x ∂Ω ,
(15)

where 1 < α,β < p − 1, and α+β = p. Then, all conditions of Theorem 1 hold. Hence, the system (15) has at least one positive solution if λ p p q + θ p p q <C and has at least two positive solutions if 0< λ p p q + θ p p q < C 0 .

First, we get the following result:

Lemma 7

(i) If 0< λ p p q + θ p p q <C, then there exists a ( PS ) ξ λ , θ -sequence {(u n ,v n )}⊂Nλ,θ in W for Jλ,θ;

(ii) If 0< λ p p q + θ p p q < C 0 , then there exists a ( PS ) ξ λ , θ -sequence {( u n , v n )} N λ , θ in W for Jλ,θ,

where C is the positive constant given in Lemma 2, and C 0 = q p p p q C.

Proof

The proof is similar to Proposition 9 in [19] and is omitted. □

Theorem 2

Assume that 0 ≤ s < p, N ≥ 3, 0μ< μ ¯ , and 1 ≤ q < p. If 0< λ p p q + θ p p q <C, then there exists (u 0+,v 0+)∈N λ,θ+ such that

(i) Jλ,θ(u 0+,v 0+) = ξλ,θ = ξ λ,θ+.

(ii) (u 0+,v 0+) is a positive solution of (1),

(iii) Jλ,θ(u 0+,v 0+)→0 as λ→0+, θ→0+.

Proof

By Lemma 7, there exists a minimizing sequence {(u n ,v n )} for Jλ,θ on Nλ,θ such that

J λ , θ ( u n , v n ) = ξ λ , θ + o ( 1 ) and J λ , θ ( u n , v n ) = o ( 1 ) in W 1 .
(16)

Since Jλ,θ is coercive on Nλ,θ (see Lemma 3), we get {(u n ,v n )} is bounded in W. Thus, there is a subsequence {(u n ,v n )} and (u 0+,v 0+))∈W such that

u n u 0 + , v n v 0 + , weakly in D 0 1 , p ( Ω ) , u n u 0 + , v n v 0 + , weakly in L p ( Ω ) , u n u 0 + , v n v 0 + , strongly in L q ( Ω , | x | s ) , for 1 q < p ( s ) , u n u 0 + , v n v 0 + , a.e. in Ω.
(17)

This implies that

K λ , θ ( u n , v n ) K λ , θ ( u 0 + , v 0 + ) , as n ∞.

By (16) and (17), it is easy to prove that (u 0+,v 0+) is a weak solution of problem (1). Since

J λ , θ ( u n , v n ) = p p p p | | ( u n , v n ) | | p p q q p K λ , θ ( u n , v n ) p q q p K λ , θ ( u n , v n ) ,

and by Lemma 5(i),

J λ , θ ( u n , v n ) ξ λ , θ < 0 as n ∞.

Letting n, we see that Kλ,θ(u 0+,v 0+)>0. Now, we prove that u n u 0+, v n v 0+ strongly in D 0 1 , p (Ω) and Jλ,θ(u 0+,v 0+)=ξλ,θ.

By applying Fatou’s lemma and (u 0+,v 0+)∈Nλ,θ, we get

ξ λ , θ J λ , θ ( u 0 + , v 0 + ) = p p p p | | ( u 0 + , v 0 + ) | | p p q q p K λ , θ ( u 0 + , v 0 + ) liminf n p p p p | | ( u n , v n ) | | p p q q p K λ , θ ( u n , v n ) liminf n J λ , θ ( u n , v n ) = ξ λ , θ .

This implies that

J λ , θ ( u 0 + , v 0 + ) = ξ λ , θ , lim n | | ( u n , v n ) | | p = | | ( u 0 + , v 0 + ) | | p .

Then, u n u 0+ and v n v 0+ strongly in D 0 1 , p (Ω).

Moreover, we have (u 0+,v 0+)∈N λ,θ+. In fact, if ( u 0 + , v 0 + ) N λ , θ , by Lemma 6, there are unique t 0+ and t 0 such that (t 0+u 0+,t 0+v 0+)∈N λ,θ+, (t 0 − u 0+,t 0 − v 0+)∈N λ,θ − and t 0+ < t 0 − =1. Since

d dt J λ , θ ( t 0 + u 0 + , t 0 + v 0 + ) = 0 and d 2 d t 2 J λ , θ ( t 0 + u 0 + , t 0 + v 0 + ) > 0 ,

there exist t 0 + < t ¯ t 0 such that J λ , θ ( t 0 + u 0 + , t 0 + v 0 + )< J λ , θ ( t ¯ 0 u 0 + , t ¯ 0 v 0 + ). By Lemma 6, we have

J λ , θ ( t 0 + u 0 + , t 0 + v 0 + ) < J λ , θ ( t ¯ 0 u 0 + , t ¯ 0 v 0 + ) J λ , θ ( t 0 u 0 + , t 0 u 0 + ) = J λ , θ ( u 0 + , v 0 + )

which contradicts Jλ,θ(u 0+,v 0+)=ξ λ,θ+.

Since Jλ,θ(u 0+,v 0+)=Jλ,θ(|u 0+|,|v 0+|) and (|u 0+|,|v 0+|)∈N λ,θ+, by Lemma 4, we may assume that (u 0+,v 0+) is a nonnegative solution of problem (1).

Moreover, by Lemmas 3 and 5, we have

0 > ξ λ , θ = J λ , θ ( u 0 + , v 0 + ) p q p q N ω N R 0 N s N s p ( s ) q p ( s ) × λ p p q + θ p p q p q p A μ , s q p | | ( u 0 + , v 0 + ) | | q .

This implies that Jλ,θ(u 0+,v 0+)→0 as λ→0+, θ→0+. □

Also, we need the following version of Brèzis-Lieb lemma [27].

Lemma 8

Consider F C 1 ( ( R + ) 2 , R + ) with F(0,0)=0 and |F u (x,u,v)|,|F v (x,u,v)| ≤ C1(|u|p − 1+|v|p − 1) for some 1 ≤ p < , C1 > 0. Let (u n ,v n ) be bounded sequence in L p ( Ω ¯ , ( R + ) 2 ), and such that ( u n , v n )(u,v) weakly in W k . Then, one has

Ω F ( u n , v n ) dx Ω F ( u n u , v n v ) dx + Ω F ( u , v ) dx as n ∞.

Lemma 9

Assume that 0 ≤ s < p, 1 ≤ q < p, and 0μ< μ ¯ . If {(u n ,v n )}⊂W is a (P S) c -sequence for Jλ,θ for all 0<c< c := 1 N ( A ~ μ , F ) N p , then there exists a subsequence of {(u n ,v n )} converging weakly to a nonzero solution of (1).

Proof

Suppose (u n ,v n )}⊂W satisfies Jλ,θ(u n ,v n )→c and J λ , θ ( u n , v n )0 with c < c. It is easy to show that {(u n ,v n )} is bounded in W and there exists (u,v) such that ( u n , v n )(u,v) up to a subsequence. Moreover, we may assume

u n u , v n v , weakly in D 0 1 , p ( Ω ) , u n u , v n v , weakly in L p ( Ω ) , u n u , v n v , strongly in L q ( Ω , | x | s ) , for all 1 q < p , u n u , v n v , a.e. on Ω.

Hence, we have J λ , θ (u)=0 by the weak continuity of Jλ,θ and

K λ , θ ( u n , v n ) K λ , θ (u,v).
(18)

Let u ~ n = u n u and v ~ n = v n v. Then, by Brèzis-Lieb lemma [27], we obtain

||( u ~ n , v ~ n )| | p ||( u n , v n )| | p ||(u,v)| | p ,asn,
(19)

and by Lemma 8,

Ω F ( x , u ~ n , v ~ n ) dx Ω F ( x , u n , v n ) dx Ω F ( x , u , v ) dx as n ∞.
(20)

Since Jλ,θ(u n ,v n )=c+o(1), J λ , θ ( u n , v n )=o(1) and (18) to (20), we can deduce that

1 p | | ( u ~ n , v ~ n ) | | p 1 p Ω F ( x , u ~ n , v ~ n ) dx = c J λ , θ ( u , v ) + o ( 1 ) ,

and

| | ( u ~ n , v ~ n ) | | p Ω F ( x , u ~ n , v ~ n ) dx = o ( 1 ) .

Now, we define

l:= lim n Ω F(x, u ~ n , v ~ n )dx,l:= lim n ||( u ~ n , v ~ n )| | p .
(21)

From the definition of A ~ μ , F and (21), one can get

A ~ μ , F l p p = A ~ μ , F lim n Ω F ( x , u ~ n , v ~ n ) dx p p lim n | | ( u ~ n , v ~ n ) | | p = l ,

which implies that either

l=0orl ( A ~ μ , F ) p p p = ( A ~ μ , F ) N t p t .
(22)

Note that J λ , θ (u,v),(u,v)=0 and

J λ , θ (u,v)= J λ , θ (u,v) 1 p J λ , θ (u,v),(u,v)0.
(23)

From (21) and (23), we get

c = J ( u n , v n ) + o ( 1 ) = J λ , θ ( u ~ n , v ~ n ) + J λ , θ ( u , v ) + o ( 1 ) 1 p | | ( u ~ n , v ~ n ) | | p 1 p Ω F ( x , u ~ n , v ~ n ) dx = p p p p l + o ( 1 ) = 1 N l + o ( 1 ) .
(24)

By (22) to (24) and the assumption c < c, we deduce that l=0. Up to a subsequence, (u n ,v n )→(u,v) strongly in W. □

Lemma 10

[28] Assume that 1 < p < N, 0 ≤ t < p, and 0μ< μ ¯ . Then, the limiting problem

Δ p u μ | u | p 1 | x | p = | u | p ( t ) 1 | x | t , in R N { 0 } , u W 1 , p ( R N ) , u > 0 , in R N { 0 } ,

has positive radial ground states

V ε (x) ε p N p U p , μ x ε = ε p N p U p , μ | x | ε ,ε>0,
(25)

that satisfy

Ω | V ε ( x ) | p μ | V ε ( x ) | p | x | p dx = Ω | V ε ( x ) | p ( t ) | x | t dx = ( A μ , t ) N t p t ,

where Up,μ(x)=Up,μ(|x|) is the unique radial solution of the limiting problem with

U p , μ ( 1 ) = ( N t ) ( μ ¯ μ ) N p 1 p ( t ) p .

Furthermore, Up,μ have the following properties:

lim r 0 r a ( μ ) U p , μ ( r ) = C 1 > 0 , lim r + r b ( μ ) U p , μ ( r ) = C 2 > 0 , lim r 0 r a ( μ ) + 1 | U p , μ ( r ) | = C 1 a ( μ ) 0 , lim r + r b ( μ ) + 1 | U p , μ ( r ) | = C 2 b ( μ ) > 0 ,

where C i (i=1,2) are positive constants and a(μ) and b(μ) are zeros of the function

f ( ζ ) = ( p 1 ) ζ p ( N p ) ζ p 1 + μ , ζ 0 , 0 μ < μ ¯ ,

that satisfy

0 a ( μ ) < N p p < b ( μ ) N p p 1 .

Now, we will give some estimates on the extremal function V ε (x) defined in (25). For mN large, choose φ(x) C 0 ( R N ), 0 ≤ φ(x) ≤ 1, φ(x)=1 for |x| 1 2 m , φ(x)=0 for |x| 1 m , ||φ(x)| | L p ( Ω ) 4m, set u ε (x)=φ(x)V ε (x). For ε→0, the behavior of u ε has to be the same as that of V ε , but we need precise estimates of the error terms. For 1 < p < N, 0 ≤ s,t < p and 1 < q < p(s), we have the following estimates [28]:

Ω | u ε | p μ | u ε | p | x | p dx= ( A μ , t ) N t p t +O ε b ( μ ) p + p N ,
(26)
Ω | u ε | p ( t ) | x | t dx= ( A μ , t ) N t p t +O ε b ( μ ) p ( t ) N + t ,
(27)
Ω | u ε | q | x | s dx C ε N s + ( 1 N p ) q , q > N s b ( μ ) , C ε N s + ( 1 N p ) q | ln ε | , q = N s b ( μ ) , C ε q ( b ( μ ) + 1 N p ) q , q < N s b ( μ ) .
(28)

Lemma 11

Assume that 0 ≤ s < p, 1 ≤ q < p, and 0μ< μ ¯ . There exists a nonnegative function (u,v)∈W∖{(0,0)} and δ1 > 0 such that for λ,θ > 0 satisfy 0< λ p p q + θ p p q < δ 1 , we have

sup τ 0 J(τu,τv)< c := 1 N ( A ~ μ , F ) N p .
(29)

In particular, ξ λ , θ < 1 N ( A ~ μ , F ) N p for all 0< λ p p q + θ p p q < δ 1 .

Proof

Set u=e1u ε , v=e2u ε , and (u,v)∈W, where ( e 1 , e 2 ) ( R + ) 2 , e 1 p + e 2 p =1 and inf x Ω ¯ F(x, e 1 , e 2 )K. Then, we consider the functions

g ( τ ) = J λ , θ ( τ e 1 u ε , τ e 2 u ε ) = τ p p | | ( e 1 u ε , e 2 u ε ) | | p τ q q K λ , θ ( τ e 1 u ε , τ e 2 u ε ) τ p p Ω F ( x , e 1 u ε , e 2 u ε ) dx , g 1 ( τ ) = τ p p | | ( e 1 u ε , e 2 u ε ) | | p τ p p Ω F ( x , e 1 u ε , e 2 u ε ) dx.

By (26), (27) for t=0, (4) and the fact that

sup τ 0 τ p p A τ p p B = 1 N A B p p N p ,A,B>0,
(30)

we conclude that

sup τ 0 g 1 ( τ ) 1 N ( e 1 p + e 2 p ) Ω ( | u ε | p μ | u ε | p | x | p ) dx ( Ω F ( x , e 1 u ε , e 2 u ε ) dx ) p p N p 1 N Ω ( | u ε | p μ | u ε | p | x | p ) dx K p p ( Ω | u ε | p dx ) p p N p 1 N 1 K p p N p ( A μ , 0 ) N p + O ( ε b ( μ ) p + p N ) ( ( A μ , 0 ) N p + O ( ε b ( μ ) p N ) ) p p N p 1 N 1 K p p N p A μ , 0 + O ( ε b ( μ ) p + p N ) N p = 1 N 1 K p p ( A μ , 0 ) N p + O ( ε b ( μ ) p + p N ) 1 N ( A ~ μ , F ) N p + O ( ε b ( μ ) p + p N ) .
(31)

On the other hand, using the definitions of g and u ε , we get

g ( τ ) = J λ , θ ( τ e 1 u ε , τ e 2 u ε ) τ p p | | ( e 1 u ε , e 2 u ε ) | | p , for all τ 0 and λ > 0 , θ > 0 .

Combining this with (26) and let ε∈(0,1), then there exists τ0∈(0,1) independent of ε such that

sup 0 τ τ 0 g(τ)< 1 N ( A ~ μ , F ) N p ,for all0< λ p p q + θ p p q < δ 1 .
(32)

Hence, as 0< λ p p q + θ p p q < δ 1 , 1 ≤ q<p, by (31), we have that

sup τ τ 0 g ( τ ) = sup τ τ 0 g 1 ( τ ) τ q q K λ , θ ( e 1 u ε , e 2 u ε ) 1 N ( A ~ μ , F ) N p + o ε b ( μ ) p + p N τ 0 q q e 1 q λ + e 2 q θ Ω | u ε | q | x | s dx.
(33)

(i) If 1q< N s b ( μ ) , then by (28), we have that

Ω | u ε | q | x | s dx C ε q ( b ( μ ) p + 1 N p ) ,

and since b(μ)> N p p , then

( b ( μ ) p + p N ) > q ( b ( μ ) p + 1 N p ) .

Combining this with (32) and (33), for any λ,θ > 0 which 0< λ p p q + θ p p q < δ 1 , we can choose ε small enough such that

sup τ 0 J ( τ e 1 u ε , τ e 2 u ε ) < 1 N ( A ~ μ , F ) N p .

(ii) If N s b ( μ ) q<p, then by (28) and b(μ)> N p p we have that

Ω | u ε | q | x | s dx C ε N s + ( 1 N p ) q , q > N s b ( μ ) , C ε N s + ( 1 N p ) q | ln ε | , q = N s b ( μ ) ,

and

( b ( μ ) p + p N ) > N s + ( 1 N p ) q.

Combining this with (32) and (33), for any λ,θ > 0 which 0< λ p p q + θ p p q < δ 1 , we can choose ε small enough such that

sup τ 0 J ( τ e 1 u ε , τ e 2 u ε ) < 1 N ( A ~ μ , F ) N p .

From (i) and (ii), (29) holds.

From Lemma 6, (29) and the definitions of ξ λ,θ − , for any λ,θ > 0 which 0< λ p p q + θ p p q < δ 1 , we obtain that there exists τ λ,θ − such that (τ λ,θe1u ε ,τ λ,θe2u ε )∈N λ,θ − and

ξ λ , θ J λ , θ ( τ λ , θ e 1 u ε , τ λ , θ e 2 u ε ) sup τ 0 J ( τ e 1 u ε , τ e 2 u ε ) < 1 N ( A ~ μ , F ) N p .

The proof is complete. □

Theorem 3

Assume that 0 ≤ s < p, 1 ≤ q < p, and 0μ< μ ¯ . There exists λ > 0 such that for any λ,θ > 0 satisfy 0< λ p p q + θ p p q <λ, the functional Jλ,θ has a minimizer (u,v) in N λ,θ − and satisfies the following:

(i) Jλ,θ(U,V)=ξ λ,θ −,

(ii) (u,v) is a positive solution of (1),

where λ = min{C0,δ1}

Proof

If 0< λ p p q + θ p p q < C 0 = q p p p q C, then by Lemmas 5(ii), 7, and 11, there exists a ( PS ) ξ λ , θ -sequence {( u n , v n )} N λ , θ in W for Jλ,θ with ξ λ , θ 0 , 1 N ( A ~ μ , F ) N p . By Lemma 3, {(u n ,v n )} is bounded in W. From Lemma 9, there exists a subsequence denoted by {(u n ,v n )} and nontrivial solution (u,v)∈W of (1) such that u n U, v n V weakly in D 0 1 , p (Ω).

First, we prove that (U,V) ∈ N λ,θ − . Arguing by contradiction, we assume (U,V) ∈ N λ,θ+. Since N λ,θ − is closed in W 0 1 , p (Ω), we have ||(U,V)||< liminfn||(u n ,v n )||. Thus, by Lemma 6, there exists a unique τ such that (τU,τV) ∈ N λ,θ −. If (U,V) ∈ N λ,θ −, then it is easy to see that

J λ , θ (u,v)= 1 N ||(u,v)| | p p q q p K λ , θ (u,v).
(34)

From Remark 1, (u n ,v n ) ∈ N λ,θ −, ||(U,V)||< liminfn||(u n ,v n )|| and (34), we can get

ξ λ , θ J λ , θ ( τ U , τ V ) lim n J λ , θ ( τ u n , τ v n ) < lim n J λ , θ ( u n , v n ) = ξ λ , θ .

This is a contradiction. Thus, (U,V) ∈ N λ,θ −, Next, by the same argument as that in Theorem 2, we get that (u n ,v n )→(u,v) strongly in W and Jλ,θ(U,V)=ξ λ,θ − > 0 for all 0< λ p p q + θ p p q < C 0 = q p p p q C. Since Jλ,θ(U,V)=Jλ,θ(|U|,|V|) and (|U|,|V|) ∈ N λ,θ −, by Lemma 4 we may assume that (u,v) is a nontrivial nonnegative solution of (1). Finally, by the maximum principle [29], we obtain that (U, V) is a positive solution of (1). The proof is complete. □

Proof of Theorem 1. The part (i) of Theorem 1 immediately follows from Theorem 2. When 0< λ p p q + θ p p q < C 0 = q p p p q C<C, by Theorems 2 and 3, we obtain (1) has at least two positive solutions (u0,v0) and (u,v) such that (u0,v0) ∈ N λ,θ+ and (U,V) ∈ N λ,θ−. Since N λ , θ + N λ , θ =, this implies that N λ,θ+ and N λ,θ − are distinct. This completes the proof of Theorem 1.