1 Introduction

The goal of this paper is to establish the existence of semi-classical solutions to the following perturbed elliptic system of Hamiltonian form:

{ ε 2 u + V ( x ) u = W v ( x , u , v ) , x R N , ε 2 v + V ( x ) v = W u ( x , u , v ) , x R N , u , v H 1 ( R N ) ,
(1.1)

where u,v: R N R, ε>0 is a small parameter, and V: R N R and W: R N × R 2 R satisfy the following basic assumptions, respectively:

(V0)VC( R N ) and there exists a b>0 such that the set V b :={x R N :V(x)<b} has finite measure;

(V1)V(x)minV=0;

(W1)W C 1 ( R N × R 2 ), and there exist constants p 0 (2, 2 ) and C 0 >0 such that

| W z (x,z)| C 0 ( 1 + | z | p 0 1 ) ,(x,z) R N × R 2 ,z=(u,v);

here and in the sequel 2 :=2N/(N2) if N3 and 2 :=+ if N=1 or 2;

(W2)| W z (x,z)|=o(|z|), as |z|0, uniformly in x R N .

For the case ε=1, the interest in the study of various qualitative properties of the solutions has steadily increased in recent years. In a bounded smooth domain Ω R N , similar systems have been extensively studied; see, for instance, [1]–[6] and the references therein. The problem set on the whole space R N was considered recently in some works. The first difficulty of such a type of problem is the lack of the compactness of the Sobolev embedding. A usual way to recover this difficulty is choosing suitable working space which has compact embedding property, for example, the radially symmetric functions space; see [7]–[9]. The second difficulty is that the energy functional is strongly indefinite different from the single equation case, and so the dual variational methods are involved to avoid this difficulty; see [10], [11]. Recently, with the aid of the linking arguments in [12]–[14], the existence of solutions or multiple solutions were obtained with periodic potential and nonlinearity, see [15]–[19] and the references therein.

For the problem with a small parameter ε>0, it is called the semi-classical problem, which describes the transition between of quantum mechanics and classical mechanics with the parameter ε goes to zero. There is much literature dealing with the existence of semi-classical solutions to the single particle equation

ε 2 u+V(x)u=g(x,u),x R N ,u H 1 ( R N ) ,
(1.2)

under various hypotheses on the potential V and the nonlinearity g; for example, see [20]–[27] and references therein. In a very recent paper [28], Lin and Tang developed a direct and simple approach to show the existence of semi-classical solutions for the single particle equation (1.2) with V satisfying (V0) and (V1). It is well known that the extension of these results of single equation to a system of equations presents some difficulties. One of the main difficulties is that the energy functional associated with (1.1) is strongly indefinite, so the approach used in the single equation is not applicable to system (1.1).

Inspired by the single equation, there are a few works considering the perturbed elliptic systems; see [11], [29]–[35] and references therein. To the best of our knowledge, the first approach to the singular perturbed system in a bounded domain, with Neumann boundary, and V(x)1 appeared in [11] by means of a dual variational formulation of the problem. Moreover, in [33], Sirakov and Soares considered the superquadratic case by using dual variational methods. In [31], Ramos and Tavares considered the following problem:

{ ε 2 u + V ( x ) u = g ( v ) , x Ω , ε 2 v + V ( x ) v = f ( u ) , x Ω , u ( x ) = 0 and v ( x ) = 0 on  Ω ,

where Ω is a domain of R N , f and g are power functions, superlinear but subcritical at infinity. The authors established the existence of positive solutions which concentrate, as ε0, on a prescribed finite number of local minimum points of the potential V; also see [30].

Since Kryszewski and Szulkin [6] proposed the generalized linking theorem for the strongly indefinite functionals in 1998, Li and Szulkin [13], Bartsch and Ding [12] gave several weaker versions, which provide another effective way to deal with such problems. With the aid of the generalized linking theorem, Xiao et al.[34] studied the asymptotically quadratic case and obtained the existence of multiple solutions. For the superquadratic case with magnetic potential, we refer the reader to [35] and references therein.

In the aforementioned references, it always was assumed that W satisfies a condition of the type of Ambrosetti-Rabinowitz, that is,

(AR) there is a μ>2 such that

0<μW(x,z) W z (x,z)z,(x,z) R N × R 2 ,z0;
(1.3)

together with a technical assumption that there is ν>2N/(N+2) such that

| W z (x,z) | ν c [ 1 + W z ( x , z ) z ] ,(x,z) R N × R 2 ;
(1.4)

or the superquadratic condition

lim | z | | W ( x , z ) | | z | 2 =,uniformly in x R N
(1.5)

and a condition of the type of Ding-Lee [36];

(DL) W ˜ (x,z):= 1 2 W z (x,z)zW(x,z)>0 for z0 and there exist c 0 >0 and κ>max{1,N/2} such that

| W z (x,z) | κ c 0 | z | κ W ˜ (x,z)for large |z|.
(1.6)

As is well known, conditions (1.3) and (1.4) have been successfully applied to Hamiltonian systems, to periodic Schrödinger systems, and to diffusion systems; see [37], [38] and so on. We refer the reader to [39]–[43] and the references therein where the condition (AR) was weakened by more general superlinear conditions. Condition (DL) was firstly given for a single Schrödinger equation by Ding and Lee [36]. Soon after, this condition was generalized by Zhang et al.[44].

Observe that conditions (1.5) and W(x,z)>0, z0 in (AR) or W ˜ (x,z)>0, z0, in (DL) play an important role in showing that any Palais-Smale sequence or Cerami sequence is bounded in the aforementioned works. However, there are many functions which do not satisfy these conditions, for example,

W(x,u,v)= | u + v | ϱ ,ϱ ( 2 , 2 )

or

W(x,u,v)= ( u + v ) 2 u 2 + v 2 .

Motivated by these works, in the present paper, we shall establish the existence of semi-classical solutions of system (1.1) with a weaker superlinear condition via the generalized linking theorem. To state our results, in addition to the basic hypotheses, we make the following assumptions:

(W3) there exist a 0 >0 and p(2, 2 ) such that

W(x,u,v) a 0 | u + v | p ,(x,u,v) R N × R 2 ;

(W4) W ˜ (x,z)0, (x,z) R N × R 2 , and there exist R 0 >0, a 1 >0, and κ>max{1,N/2} such that

| W u (x,z)+ W v (x,z)| b 3 |z|,(x,z) R N × R 2 ,|z| R 0

and

| W u (x,z)+ W v (x,z) | κ a 1 | z | κ W ˜ (x,z),(x,z) R N × R 2 ,|z| R 0 .

In the present paper, we make use of the techniques developed in [28], [45] to obtain the existence of semi-classical solutions for system (1.1) when ε ε 0 , where the bound ε 0 is formulated in terms of N, V, and W.

Since (p2)N2p<0, we can choose a h 0 1 such that

( p 2 ) ω N 4 N a 0 2 / ( p 2 ) [ N 3 + 2 ( N + 2 ) 2 ( N + 2 ) p ( 1 2 N ) 2 ] p / ( p 2 ) h 0 [ ( p 2 ) N 2 p ] / ( p 2 ) b ( 2 κ N ) / 2 3 κ 2 a 1 ( γ 2 γ 0 ) N .
(1.7)

Let x 0 R N be such that V( x 0 )=0. Then we can choose λ 0 >1 such that

sup λ 1 / 2 | x | 2 h 0 |V( x 0 +x)| h 0 2 ,λ λ 0 .
(1.8)

Let E be a Hilbert space as defined in Section 2 and λ= ε 2 . Under assumptions (V0), (V1), (W1), and (W2), the functional

Φ λ (z)= R N ( u v + λ V ( x ) u v ) dxλ R N W(x,u,v)dx,z=(u,v)E,
(1.9)

is well defined. Moreover, Φ λ C 1 (E,R) and for all z=(u,v),ζ=(φ,ψ)E

Φ λ ( z ) , ζ = R N [ u ψ + v φ + λ V ( x ) ( u ψ + v φ ) ] d x λ R N [ W u ( x , u , v ) φ + W v ( x , u , v ) ψ ] d x .
(1.10)

We are now in a position to state the main results of this paper.

Theorem 1.1

Assume that V and W satisfy (V0), (V1), (W1), (W2), (W3), and (W4). Then for0<ε λ 0 1 / 2 , (1.1) has a solution( u ε , v ε )such that0< Φ ε 1 / 2 ( u ε , v ε ) b ( 2 κ N ) / 2 3 κ 2 a 1 ( γ 2 γ 0 ) N ε N 2 , and

R N W ˜ (x, u ε , v ε )dx b ( 2 κ N ) / 2 3 κ 2 a 1 ( γ 2 γ 0 ) N ε N .

Theorem 1.2

Assume that V and W satisfy (V0), (V1), (W1), (W2), (W3), and (W4). Then forλ λ 0 , (2.1) has a solution( u λ , v λ )such that0< Φ λ ( u λ , v λ ) b ( 2 κ N ) / 2 3 κ 2 a 1 ( γ 2 γ 0 ) N λ 1 N / 2 , and

R N W ˜ (x, u λ , v λ )dx b ( 2 κ N ) / 2 3 κ 2 a 1 ( γ 2 γ 0 ) N λ N / 2 .

Before proceeding to the proofs of these theorems, we give two examples to illustrate the assumptions.

Example 1.3

W(x,u,v)=h(x) | u + v | ϱ satisfies (W1)-(W4), where ϱ(2, 2 ), hC( R N ) with inf R N h>0.

Example 1.4

W(x,u,v)=h(x) ( u + v ) 2 2 u 2 + v 2 satisfies (W1)-(W4), where hC( R N ) with inf R N h>0.

The rest of the paper is organized as follows. In Section 2, we provide a variational setting. In Section 3, we give the proofs of our theorems.

2 Variational setting

Letting ε 2 =λ, (1.1) is rewritten as

{ u + λ V ( x ) u = λ W v ( x , u , v ) , x R N , v + λ V ( x ) v = λ W u ( x , u , v ) , x R N , u , v H 1 ( R N ) .
(2.1)

Let

E V = { u H 1 ( R N ) : R N V ( x ) u 2 d x < + } , ( u , v ) λ V = R N [ u v + λ V ( x ) u v ] d x , u , v E V

and

u λ V = { R N [ | u | 2 + λ V ( x ) u 2 ] d x } 1 / 2 ,u E V .

Analogous to the proof of [46], Lemma 1], by using (V0), (V1), and the Sobolev inequality, one can demonstrate that there exists a constant γ 0 >0 independent of λ such that

u H 1 ( R N ) γ 0 u λ V ,u E V ,λ1.
(2.2)

This shows that ( E V , ( , ) λ V ) is a Hilbert space for λ1. Furthermore, by virtue of the Sobolev embedding theorem, we have

u s γ s u H 1 ( R N ) γ s γ 0 u λ V ,u E V ,λ1,2s 2 ;
(2.3)

here and in the sequel, by s we denote the usual norm in space L s ( R N ).

Set E= E V × E V , then E is a Hilbert space with the inner product

( z 1 , z 2 ) λ = ( u 1 , u 2 ) λ V + ( v 1 , v 2 ) λ V , z i =( u i , v i )E,i=1,2,

the corresponding norm is denoted by λ . Then we have

z λ 2 = u λ V 2 + v λ V 2 ,z=(u,v)E
(2.4)

and

z s s = R N ( u 2 + v 2 ) s / 2 d x 2 ( s 2 ) / 2 ( u s s + v s s ) 2 ( s 2 ) / 2 ( γ s γ 0 ) s ( u λ V s + v λ V s ) 2 ( s 2 ) / 2 ( γ s γ 0 ) s ( u λ V 2 + v λ V 2 ) s / 2 = 2 ( s 2 ) / 2 ( γ s γ 0 ) s z λ s , s ( 2 , 2 ] , z = ( u , v ) E .
(2.5)

Let

E = { ( u , u ) : u E V } , E + = { ( u , u ) : u E V } .

For any z=(u,v)E, set

z = ( u v 2 , v u 2 ) , z + = ( u + v 2 , u + v 2 ) .

It is obvious that z= z + z + , z and z + are orthogonal with respect to the inner products ( , ) L 2 and ( , ) λ . Thus we have E= E E + . By a simple calculation, one gets

1 2 ( z + λ 2 z λ 2 ) = R N [ u v + λ V ( x ) u v ] dx.

Therefore, the functional Φ λ defined in (1.9) can be rewritten in a standard way

Φ λ (z)= 1 2 ( z + λ 2 z λ 2 ) λ R N W(x,u,v)dx,z=(u,v)E.
(2.6)

Moreover,

Φ λ ( z ) , z = z + λ 2 z λ 2 λ R N [ W u ( x , u , v ) u + W v ( x , u , v ) v ] d x , z = ( u , v ) E .
(2.7)

The following generalized linking theorem provides a convenient approach to get a ( C ) c sequence.

Let X be a Hilbert space with X= X X + and X X + . For a functional φ C 1 (X,R), φ is said to be weakly sequentially lower semi-continuous if for any u n u in X one has φ(u) lim inf n φ( u n ), and φ is said to be weakly sequentially continuous if lim n φ ( u n ),v= φ (u),v for each vX.

Lemma 2.1

([37], Theorem 4.5], [13], Theorem 2.1])

Let X be a Hilbert space withX= X X + and X X + , and letφ C 1 (X,R)be of the form

φ(u)= 1 2 ( u + 2 u 2 ) ψ(u),u= u + + u X + X .

Suppose that the following assumptions are satisfied:

(I1)ψ C 1 (X,R)is bounded from below and weakly sequentially lower semi-continuous;

(I2) ψ is weakly sequentially continuous;

(I3)there existr>ρ>0ande X + withe=1such that

κ:=infφ( S ρ )>supφ(Q),

where

S ρ = { u X + : u = ρ } ,Q= { s e + v : v X , s 0 , s e + v r } .

Then for somecκ, there exists a sequence{ u n }Xsatisfying

φ( u n )c, φ ( u n ) ( 1 + u n ) 0.

Such a sequence is called a Cerami sequence on the level c, or a ( C ) c sequence.

3 Proofs of the theorems

In this section, we give the proofs of Theorems 1.1 and 1.2.

From now on we assume without loss of generality that x 0 =0 (see [47]), that is, V(0)=0, then λ 0 >1 defined by (1.8) satisfies

sup λ 1 / 2 | x | 2 h 0 |V(x)| h 0 2 ,λ λ 0 .
(3.1)

Let

ϑ(x):= { 1 h 0 , | x | h 0 , h 0 N 1 1 2 N [ | x | N ( 2 h 0 ) N ] , h 0 < | x | 2 h 0 , 0 , | x | > 2 h 0 .
(3.2)

Then ϑ H 1 ( R N ), moreover,

ϑ 2 2 = R N |ϑ(x) | 2 dx N 2 ω N ( N + 2 ) ( 1 2 N ) 2 h 0 N 4 ,
(3.3)
ϑ 2 2 = R N |ϑ(x) | 2 dx 2 ω N ( 1 2 N ) 2 N h 0 N 2
(3.4)

and

ϑ p p = R N |ϑ(x) | p dx ω N N h 0 N p .
(3.5)

Let e λ (x)=(ϑ( λ 1 / 2 x),0). Then we can prove the following lemma which is very important and crucial.

Lemma 3.1

Suppose that (V0), (V1), (W1), (W2), and (W3) are satisfied. Then

sup { Φ λ ( ζ + s e λ ) : ζ = ( w , w ) E , s 0 } b ( 2 κ N ) / 2 3 κ 2 a 1 ( γ 2 γ 0 ) N λ 1 N / 2 ,λ λ 0 .
(3.6)

Proof

Note that e λ + =(ϑ( λ 1 / 2 x)/2,ϑ( λ 1 / 2 x)/2), we have

e λ + λ 2 = 1 2 R N ( | ϑ ( λ 1 / 2 x ) | 2 + λ V ( x ) | ϑ ( λ 1 / 2 x ) | 2 ) d x = 1 2 λ 1 N / 2 R N ( | ϑ | 2 + V ( λ 1 / 2 x ) | ϑ | 2 ) d x 1 2 λ 1 N / 2 ( ϑ 2 2 + ϑ 2 2 sup λ 1 / 2 | x | 2 h 0 | V ( x ) | ) 1 2 λ 1 N / 2 ( ϑ 2 2 + h 0 2 ϑ 2 2 ) , λ λ 0 .
(3.7)

It follows from (W3), (1.7), (1.9), (3.3), (3.4), (3.5), and (3.7) that

Φ λ ( ζ + s e λ ) = 1 2 ( s 2 e λ + λ 2 ζ + s e λ λ 2 ) λ R N W ( x , w + s ϑ ( λ 1 / 2 x ) , w ) d x s 2 2 e λ + λ 2 a 0 λ s p R N | ϑ ( λ 1 / 2 x ) | p d x λ 1 N / 2 { s 2 4 ( ϑ 2 2 + h 0 2 ϑ 2 2 ) a 0 s p ϑ p p } λ 1 N / 2 [ s 2 4 ( N 3 + 2 ( N + 2 ) N ( N + 2 ) ( 1 2 N ) 2 ) ω N h 0 N 4 a 0 ω N N s p h 0 N p ] ( p 2 ) ω N 4 N a 0 2 / ( p 2 ) [ N 3 + 2 ( N + 2 ) 2 ( N + 2 ) p ( 1 2 N ) 2 ] p / ( p 2 ) h 0 [ ( p 2 ) N 2 p ] / ( p 2 ) λ 1 N / 2 b ( 2 κ N ) / 2 3 κ 2 a 1 ( γ 2 γ 0 ) N λ 1 N / 2 , s 0 , λ λ 0 , ζ = ( w , w ) E .
(3.8)

Now the conclusion of Lemma 3.1 follows by (3.8). □

Applying Lemma 2.1, by standard arguments (see, e.g., [45]), we can prove the following lemma.

Lemma 3.2

Suppose that (V0), (V1), (W1), (W2), and (W3) are satisfied. Then there exist a constant c λ (0,sup{ Φ λ (ζ+s e λ ):ζ=(w,w) E ,s0}]and a sequence{ z n }={( u n , v n )}Esatisfying

Φ λ ( z n ) c λ , Φ λ ( z n ) ( 1 + z n ) 0.
(3.9)

Lemma 3.3

Suppose that (V0), (V1), (W1), (W2), (W3), and (W4) are satisfied. Then any sequence{ z n }={( u n , v n )}Esatisfying (3.9) is bounded in E.

Proof

By virtue of (W3), (2.6), and (3.9), one gets

2 c λ + o ( 1 ) = z n + λ 2 z n λ 2 2 λ R N W ( x , u n , v n ) d x z n + λ 2 z n λ 2 2 a 0 λ R N | u n + v n | p d x .
(3.10)

To prove the boundedness of { z n }, arguing by contradiction, suppose that z n λ . Let ξ n = z n / z n λ =( φ n , ψ n ), then ξ n λ =1. If

δ:= lim sup n sup y R N B ( y , 1 ) | ξ n + | 2 dx=0,

then by Lions’ concentration compactness principle [48] or [49], Lemma 1.21], φ n + ψ n 0 in L s ( R N ) for 2<s< 2 . Hence, it follows from (W4), (2.4), (3.10), and the Hölder inequality that

λ 2 | z n | R 0 | W u ( x , u n , v n ) + W v ( x , u n , v n ) | | u n + v n | d x λ b 6 | z n | R 0 | z n | | u n + v n | d x λ b 6 R N V b | z n | | u n + v n | d x + λ b 6 V b | z n | | u n + v n | d x λ b 6 ( R N V b | z n | 2 d x ) 1 / 2 ( R N V b | u n + v n | 2 d x ) 1 / 2 + λ b [ meas ( V b ) ] 1 / ( N + 1 ) 6 ( V b | z n | 2 ( N + 1 ) / N d x ) N / 2 ( N + 1 ) × ( V b | u n + v n | 2 ( N + 1 ) / N d x ) N / 2 ( N + 1 ) 1 6 z n λ u n + v n λ V + λ b [ meas ( V b ) ] 1 / ( N + 1 ) 6 z n 2 ( N + 1 ) / N u n + v n 2 ( N + 1 ) / N = 1 6 z n λ u n + v n λ V + λ b [ meas ( V b ) ] 1 / ( N + 1 ) 6 ξ n 2 ( N + 1 ) / N × φ n + ψ n 2 ( N + 1 ) / N z n λ 2 [ 1 3 + o ( 1 ) ] z n λ 2 .
(3.11)

From (2.6), (2.7), and (3.9), one has

c λ +o(1)=λ R N W ˜ (x, u n , v n )dx.
(3.12)

Let κ =κ/(κ1), then 2<2 κ < 2 . Hence, by virtue of (W4), (3.12), and the Hölder inequality, one gets

λ 2 | z n | R 0 | W u ( x , u n , v n ) + W v ( x , u n , v n ) | | u n + v n | z n λ 2 d x = λ 2 | z n | R 0 | W u ( x , u n , v n ) + W v ( x , u n , v n ) | | ξ n | | φ n + ψ n | | z n | d x λ 2 ( | z n | R 0 | W u ( x , u n , v n ) + W v ( x , u n , v n ) z n | κ d x ) 1 / κ ( | z n | R 0 | ξ n | 2 κ d x ) 1 / 2 κ × ( | z n | R 0 | φ n + ψ n | 2 κ d x ) 1 / 2 κ λ 2 ( a 1 | z n | R 0 W ˜ ( x , u n , v n ) d x ) 1 / κ ξ n 2 κ φ n + ψ n 2 κ λ ( κ 1 ) / κ ( c λ a 1 ) 1 / κ ξ n 2 κ φ n + ψ n 2 κ = o ( 1 ) .
(3.13)

Combining (3.11) with (3.13) and using (1.10), (3.9), and (3.10), we have

1 2 + o ( 1 ) z n + λ 2 Φ λ ( z n ) , z n + z n λ 2 = λ 2 R N [ W u ( x , u n , v n ) + W v ( x , u n , v n ) ] ( u n + v n ) z n λ 2 d x = λ 2 | z n | R 0 [ W u ( x , u n , v n ) + W v ( x , u n , v n ) ] ( u n + v n ) z n λ 2 d x + λ 2 | z n | > R 0 [ W u ( x , u n , v n ) + W v ( x , u n , v n ) ] ( u n + v n ) z n λ 2 d x 1 3 + o ( 1 ) .
(3.14)

This contradiction shows that δ>0.

Going if necessary to a subsequence, we may assume the existence of k n Z N such that B 1 + N ( k n ) | ξ n + | 2 dx> δ 2 . Let ζ n (x)= ξ n (x+ k n ). Then

B 1 + N ( 0 ) | ζ n + | 2 dx> δ 2 .
(3.15)

Now we define z ˜ n (x)=( u ˜ n , v ˜ n )= z n (x+ k n ), then z ˜ n / z n λ = ζ n and ζ n H 1 ( R N ) 2 = ξ n H 1 ( R N ) 2 . Passing to a subsequence, we have ζ n ζ in E, ζ n ζ in L loc s ( R N ), 2s< 2 , and ζ n ζ a.e. on R N . Obviously, (3.15) implies that ζ + 0. For a.e. x{y R N : ζ + (y)0}:=Ω, we have lim n | u ˜ n (x)+ v ˜ n (x)|=. Hence, it follows from (2.6), (3.9), (W3), and Fatou’s lemma that

0 = lim n c + o ( 1 ) z n λ 2 = lim n Φ λ ( z n ) z n λ 2 = lim n [ 1 2 ( ξ n + λ 2 ξ n λ 2 ) λ R N W ( x , u n , v n ) z n λ 2 d x ] = lim n [ 1 2 ( ξ n + λ 2 ξ n λ 2 ) λ R N W ( x + k n , u ˜ n , v ˜ n ) | z ˜ n + | 2 | ζ n + | 2 d x ] = lim n [ 1 2 ( ξ n + λ 2 ξ n λ 2 ) 2 λ R N W ( x + k n , u ˜ n , v ˜ n ) | u ˜ n + v ˜ n | 2 | ζ n + | 2 d x ] 1 2 2 λ Ω lim inf n W ( x + k n , u ˜ n , v ˜ n ) | u ˜ n + v ˜ n | 2 | ζ n + | 2 d x = .

This contradiction shows that z n λ is bounded. □

Proof of Theorem 1.2

Applying Lemmas 3.1, 3.2, and 3.3, we deduce that there exists a bounded sequence { z n }={( u n , v n )}E satisfying (3.9) and (3.10) with

c λ b ( 2 κ N ) / 2 3 κ 2 a 1 ( γ 2 γ 0 ) N λ 1 N / 2 ,λ λ 0 .
(3.16)

Going if necessary to a subsequence, we can assume that z n z λ =( u λ , v λ ) in (E, λ ), and Φ λ ( z n )0. Next, we prove that z λ 0.

Arguing by contradiction, suppose that z λ =0, i.e. z n 0 in E, and so u n 0, v n 0 in L loc s ( R N ), 2s< 2 , and u n 0, v n 0 a.e. on R N . Since V b is a set of finite measure, we have

u n + v n 2 2 = R N V b | u n + v n | 2 dx+ V b | u n + v n | 2 dx 1 λ b u n + v n λ V 2 +o(1)
(3.17)

and

z n 2 2 = R N V b ( u n 2 + v n 2 ) dx+ V b ( u n 2 + v n 2 ) dx 1 λ b z n λ 2 +o(1).
(3.18)

For s(2, 2 ), it follows from (2.3), (2.5), (3.17), (3.18), and the Hölder inequality that

u n + v n s s u n + v n 2 2 ( 2 s ) / ( 2 2 ) u n + v n 2 2 ( s 2 ) / ( 2 2 ) ( γ 2 γ 0 ) 2 ( s 2 ) / ( 2 2 ) ( λ b ) ( 2 s ) / ( 2 2 ) u n + v n λ V s + o ( 1 )
(3.19)

and

z n s s z n 2 2 ( 2 s ) / ( 2 2 ) z n 2 2 ( s 2 ) / ( 2 2 ) 2 ( s 2 ) / 2 ( γ 2 γ 0 ) 2 ( s 2 ) / ( 2 2 ) ( λ b ) ( 2 s ) / ( 2 2 ) z n λ s + o ( 1 ) .
(3.20)

According to (W4), (3.10), (3.17), and (3.18), one gets

λ 2 | z n | R 0 | W u ( x , u n , v n ) + W v ( x , u n , v n ) | | u n + v n | d x λ b 6 | z n | R 0 | z n | | u n + v n | d x λ b 6 z n 2 u n + v n 2 1 6 z n λ u n + v n λ V 1 3 z n + λ 2 + o ( 1 ) .
(3.21)

By virtue of (1.9), (1.10), and (3.9), we have

Φ λ ( u n ) 1 2 Φ λ ( u n ) , u n =λ R N W ˜ (x, u n , v n )dx= c λ +o(1).
(3.22)

Using (W4), (3.16), (3.19), (3.20) with s=2κ/(κ1), and (3.22), we obtain

λ 2 | u n | > R 0 | W u ( x , u n , v n ) + W v ( x , u n , v n ) | | u n + v n | d x λ 2 ( | u n | > R 0 ( | W u ( x , u n , v n ) + W v ( x , u n , v n ) | | z n | ) κ d x ) 1 / κ z n s u n + v n s 2 ( s 2 ) / 2 s 2 ( γ 2 γ 0 ) 2 2 ( s 2 ) / s ( 2 2 ) λ ( a 1 | u n | > R 0 W ˜ ( x , u n , v n ) d x ) 1 / κ × ( λ b ) 2 ( 2 s ) / s ( 2 2 ) z n λ u n + v n λ V + o ( 1 ) 1 2 ( 2 a 1 ) 1 / κ ( γ 2 γ 0 ) N / κ λ 1 1 / κ c λ 1 / κ ( λ b ) ( N 2 κ ) / 2 κ z n λ u n + v n λ V + o ( 1 ) ( 2 a 1 ) 1 / κ ( γ 2 γ 0 ) N / κ b ( 2 κ N ) / 2 κ [ λ ( N 2 ) / 2 c λ ] 1 / κ z n + λ 2 + o ( 1 ) 1 3 z n + λ 2 + o ( 1 ) ,
(3.23)

which, together with (1.10), (3.9), (3.21), and (3.23), yields

o ( 1 ) = Φ λ ( z n ) , z n + = z n + λ 2 λ 2 R N [ W u ( x , u n , v n ) + W v ( x , u n , v n ) ] ( u n + v n ) d x 1 3 z n + λ 2 + o ( 1 ) ,
(3.24)

resulting in the fact that z n + λ 0, which contradicts (3.10). Thus z λ 0. By a standard argument, we easily verify that Φ λ ( z λ )=0 and Φ λ ( z λ ) c λ . Then z λ =( u λ , v λ ) is a nontrivial solution of (2.1), moreover,

c λ Φ λ ( z λ )= Φ λ ( z λ ) 1 2 Φ λ ( z λ ) , z λ =λ R N W ˜ (x, u λ , v λ )dx.
(3.25)

 □

Theorem 1.1 is a direct consequence of Theorem 1.2.