WANG Hao,LIU Xiao-chun
(School of Mathematics and Statistics,Wuhan University,Wuhan 430072,China)
In this paper,we will first consider the following system

whereN≥5,p=?λ1(B)<λ1,λ2<0,μ1,μ2>0 andβ/=0.Here B is[0,1)×XandX?RN?1is a smooth compact domain,λ1(B)is the first eigenvalue of?ΔBwith zero Dirichlet condition on?B,We will look for the positive least energy solutions for(1.1)in the cone Sobolev space(B),which was introduced in[13].In[2],Chen-Liu-Wei considered the following problem

and got a positive solution?.Recently,the authors in[8]also studied the positive least energy solutions forp-Laplacian system.Our study is in fact motivated by the study ofChen-Zou(see[1]),and we investigate the semi-linear equations with critical cone Sobolev exponent terms.
We call a solution(u,v)∈Hnontrivial ifu/≡0,v/≡0,whereH:=The weak solutions of(1.1)are the critical points of the functionalJ:H→R,which is given by

We say that a solution(u,v)of(1.1)is a least energy solution if(u,v)is nontrivial andJ(u,v)≤J(?,ψ)for any other nontrivial solution(?,ψ)of(1.1).If we define a “ Nehari”manifold(see[1,4–7,9])

then any nontrivial solutions of(1.1)belong toN,hereJ′(·,·)is the Fr′echet differentiation ofJ.We define the least energy of(1.3)as

If the equation

has a solution(d0,g0)with

then we prove the following theorem.
Theorem 1.1 Let(d0,g0)be a solution of(1.4)withd0in(1.5)and?λ1(B)<λ1=λ2=λ<0.Then for anyβ>0,is a positive solut ion of(1.1).Moreover,ifmax{μ1,μ2},then we have=A,that is,is a positive least energy solution of(1.1).
In the second part of this paper,we consider the existence of the least energy solution of the following problem


Analogously,we let

it is easy to see that any nontrivial solutions of(1.6)belong toM.Then we get the following theorem.
Theorem 1.2 (1)Ifβ <0,thenBis not attained.
(2)Ifβ>0,then there exists a positive least energy solution(U,V)of(1.6)withE(U,V)=B,which is partly radially symmetric decreasing.Furthermore,we have
(2-1)Let(d0,g0)be as in Theorem 1.1.Ifmax{μ1,μ2},then

(2-2)There exists 0<max{μ1,μ2}such that for any 0<β<β1,we have a solution(d(β),g(β))of(1.4)with

The terminology “partly radially symmetrization decreasing” in Theorem 1.2 will be explained in Section 3.Meanwhile,we will introduce “cone Schwartz symmetrization” in the same section.
The paper is organized as follows.In Section 2,we will give some preliminaries about cone Sobolev spaces and some auxiliary results.In Section 3,we will give the proofs of Theorems 1.1 and 1.2.
Here we first introduce the cone Sobolev spaces.LetXbe a closed,compactC∞manifold of dimensionN?1,and setwhich is the local model interpreted as a cone with the baseX.More details about the manifold with singularities can be found in[10].
Definition 2.1 For(x1,x′)∈R+×RN?1,we say thatu(x1,x′)∈

The weightedLp-spaces with weight dataγ∈R is denoted byand then with the norm

Definition 2.2 Form∈N,andγ∈R,we define the spaces

for arbitraryα ∈N,β ∈NN?1,and|α|+|β|≤m.In other words,ifu(x1,x′)thenIt’s easy to see thatis a Banach space with the norm

We will always denoteω(x1,x′)as a real-valued cut-offfunction which equals 1 near{0}×?B.
Definition 2.3 Let B be the stretched manifold to a manifoldBwith conical singularities.Then(B)form∈N,γ∈R denotes the subspace of allu∈(int B)such that

for any cut off functionω,supported by a collar neighbourhood of[0,1)×?B.Moreover,the subspace(B)of(B)is defined as follows

Next,we will recall the cone Sobolev inequality and Poincar′e inequality.For details we refer to[12].
Lemma 2.1 (Cone Sobolev inequality)Assume that 1<p<N,andγ∈R.The following estimate

holds for allu∈(B),whereγ?=γ?1Moreover,ifu∈we havewhere the constantc=c1+c2andc1,c2are given.
Lemma 2.2 (Poincar′e inequality).Let B=[0,1)×Xba a bounded subset inand 1<p<+∞,γ ∈R.Ifu(x1,x′)(B),then‖u(x1,x′)‖Lγp(B)≤c‖?Bu(x1,x′)‖Lγp(B),where the positive constantcdepending only on B andp.
Lemma 2.3 For 2<p<2?,the embeddingis compact.Then we set

Lemma 2.4 Suppose thatβ≥(p?1)max{μ1,μ2}.Then the following system has a unique solution(d0,g0).

Proof See[1,Lemmas 2.1,2.2,2.3,2.4].
Now we consider the solution of(1.2),we will prove that this solution is also a least energy solution.
Lemma 2.5 Assume that?λ1(B)<λ<0,and then(1.2)has a positive least energy solution?∈(B)with energy

Proof LetS(u;B)
λand the functional

From the result in[2],we know that(1.2)has a positive solution with energyC0.Furthermore,we will show thatC0is the least energy of(1.2).We set

Ifuis the solution of(1.2),thenu∈Nandfλ(u)=If we denote(u)as the least energy of(1.2),then

ThereforeC0=(u).Let?be a positive critical point offλ(u)with a critical valueA1.Then it is easy to getA1
In this section,we will prove Theorem 1.1 and Theorem 1.2.In particular,we will separate the proof of Theorem 1.2 into several steps.
Proof of Theorem 1.1 For?λ1(B)<λ1=λ2=λ<0,we can easily get thatJ(u,v)>0.β>0,so(1.3)has a solution(d0,g0).By Lemma 2.5,we obtain Z thatFor a direct computing,we can get thatis a positive solution of(1.1).Moreover,we have

Now ifβ≥(p?1)max{μ1,μ2},then we haveA=In fact,we can take a minimizing sequence{(un,vn)}n∈N?NforAsuch thatJ(un,vn)→A.Then we get

and


and then from(3.1),we have

Therefore,the sequences{cn}n∈N,{kn}n∈Nare uniformly bounded.Passing to a subsequence,we assume thatcn→candkn→kasn→∞for somec≥0,k≥0.By(3.2)and(3.3),
we haveμ1cp+≥NA>0.That meanscandkare not necessary to be all vanished.From(3.4)–(3.6),we get

Applying Lemma 2.4,we have


Next we start to prove Theorem 1.2.
Lemma 3.1 For?∞<β<0,ifBis attained by a couple(u,v)∈M,then this couple is a critical point ofE(u,v)in(1.7).The proof is analogous to that in[1,Lemma 2.5].So we omit it here.
By Lemma 2.1,letSbe the sharp constant of

Forε>0,let

ThenUεsatis fi es?ΔBu=(see[2,5]).Moreover,

Now we give the proof of first part of Theorem 1.2.
Proof of(1)in Theorem 1.2 Let?μi:=U1withU1being as in(3.9).Then?μisatisfies the equation?ΔBu=We sete2=(0,1,0,···,0)and(ur(x),vr(x))=(?μ1(x),?μ2(x+re2)).Thenvr?0 weakly in?0 weaklyinasr→∞.That is,

To complete this proof,we claim that:forr>0 suffciently large andβ<0,there exists(trur,srvr)∈Mwithtr>1,sr>1.
In fact,note thaturandvrsatisfy the equation?ΔBu=μi|u|2??2u.If(tur,svr)∈M,then we have

and

Sincevr(x)→0(r→∞),there existsδr>0 forrsuffciently large such thatvr(x)≤δrand=0.By cone Sobolev inequality,we obtain that for someC>0,

For simplicity,we denote


From the first equality of(3.11),we obtainsp=>0,and thereforet>1.Similarly,we haves>1.Note that(3.11)is equivalent tow(t)=0,where

For 1<p<2,we getw(1)=?Fr>0,andw(t)<0.So there existstr>1 such thatw(t)=0.
Note that(trur,srvr)∈M,and then we have

Up to a subsequence,iftr→∞asr→∞,then by the fact

we also gett→∞(r→∞).As 2?p<p,forrlarge enough,we have

Therefore,we obtain

For(trur,srvr)∈M,from(3.10)we have

Letr→∞,we get thatB
On the other hand,for any(u,v)∈M,by the factβ<0 and(3.8),we get that

Note thatB=and then we obtain thatB≥HenceB
Now ifBis attained by some(u,v)∈M,then(|u|,|v|)∈MandE(|u|,|v|)=B.From Lemma 3.1,we know that(|u|,|v|)is a nontrivial solution of(1.6).By the maximum principle,we may assume thatu>0,v>0,and soMoreover,we get

It is easy to see that

that is a contradiction.We complete the proof.
Now we begin to prove(2-1)in Theorem 1.2.
Proof of(2-1)in Theorem 1.2 Forβ>0,is a nontrivial solution of(1.6)and
We letβ≥(p?1)max{μ1,μ2}and{(un,vn)}n∈N?Mbe a minimizing sequence forB,that is,E(un,vn)→B.Definecn=and we have

which imply

Similarly as in the proof of Theorem 1.1,we have(n→∞).Moreover,we obtain


Next we continue the proof of(2-2)in Theorem 1.2.For this purpose we need to show that(1.6)has a positive least energy solution for any 0<β<(p?1)max{μ1,μ2}.Therefore,we assumeβ>0,and defineB′:=E(u,v),where

It is easy to see thatM?M′,and soB′≤B.By cone Sobolev inequality,we haveB′>0.We set ?R(1,0):=forx0=(1,0,···,0)∈Consider the system

and defineB′(RE(u,v),where

Lemma 3.2 For allR>0,we haveB′(R)≡B′.
Proof LetR1>R2,sinceM′(R2)?M′(R1),we getB(R1)≤B′(R2).For any(u,v)∈M′(R1),we define

It is easy to see that(u1,v1)∈M′(R2),and so

That is,B′(R2)≤B′(R1).Hence we haveB′(R1)=B′(R2).
Let{(un,vn)}n∈N?M′be a minimizing sequence ofB′.Moreover,we may assume
thatun,vn(?Rn(x0))for someRn>0.Then(un,vn)∈M′(Rn)and

Note thatB′≤B′(R)and consequently we haveB′(R)≡B′for anyR>0.
Let 0≤ε<p?1.Consider

and defineBε=Eε(u,v),where

Lemma 3.3 For 0<ε<p?1,there holds

The proof is analogous to that in[1,Lemma 2.7].So we omit it here.
Similarly as in Lemma 3.3,we have

where?μiis the same as in the proof of(1)in Theorem 1.2.
Now we introduce the “Cone Schwartz symmetrization”.Assume that ? is a bounded domain ofanduis a real measurable function defined on ?.We define the distribution function ofuas followsu#(t)=meas{x∈? :|u(x)|>t}fort∈R,where“meas”denotes the corresponding measure in cone Sobolev space.Then we can define the decreasing rearrangement ofuin the form~u(s)=inf{t∈R:u#(t)≤s}fors∈[0,|?|].We callu?(x)the cone Schwarz symmetrization ofuifu?(x)=forx∈~?,where~? is the sphere centred atx0with the same measure of ?,and|x?z|B=forx=(x1,x′),hereonis the measure of the unit ball inSince~uis decreasing,u?is partly radially symmetric decreasing in relation to|x|B.
Lemma 3.4 For any 0<ε<p?1,(3.14)has a classical least energy solution(uε,vε),anduε,vεare both partly radially symmetric decreasing.
ProofFix any 0<ε<p?1,and then it is easy to see thatBε>0.Let(u,v)∈withu≥0,v≥0,and(u?,v?)be its cone Schwartz symmetrization.Then we have

Similarly as in Lemma 3.3,there exists 0<t?≤1 such that(t?u?,t?v?)∈and then we get

We take a minimizing sequencewithun≥0,vn≥0 such thatEε(un,vn)→Bε.be its“cone Schwartz symmetrization”.Then there exists 0<∈.By(3.15),we get


which means(uε,vε)/=(0,0).Moreover,uε≥0,vε≥0 are partly radially symmetric.Meanwhile,since

we get

Therefore,there exists 0<tε≤1 such that(tεuε,tεvε)∈and then

That istε=1 and(uε,vε)∈withEε(uε,vε)=Bε.Therefore,→vεstrongly in(?1(x0))asn→∞.
By Lagrange multiplier theorem,we get that there exists a Lagrange multiplierτ∈R such that=0.Note that=Hε(uε,vε)=0 and

We get thatτ=0 and=0.By Lemma 3.3,we see thatuε/≡0,vε/≡0.This means that(uε,vε)is a least energy solution of(3.14).By regularity theory and the maximum principle,we see thatuε>0,vε>0 in ?1(x0),uε,vε∈C2(?1(x0)).This completes the proof.
Completion of the Proof of(2-2)in Theorem 1.2 For any(u,v)∈M′(1),it is easy to see that there existstε>0 such that(tεu,tεv)∈withtε→1(ε→0),then

By Lemma 3.2,we have

By Lemma 3.4,we know that there exists a positive least energy solution(uε,vε)of(3.14),which is partly radically symmetric decreasing.Recall that=0.By cone Sobolev inequality,we have

whereW0is a positive constant independent ofε.Thenuε,νεare uniformly bounded inPassing to a subsequence,we may assume thatu?u,v?vweakly inε0ε0asε→0.Then(u0,v0)is a solution of the following problem

Note thatuε(x0)=(x)and de fi neKε=max{uε(x0),vε(x0)}.We claim thatKε→∞asε→0.Suppose the contrary.IfKεis uniformly bounded,then by the dominated convergent theorem,we have that


which is a contradiction,hereυdenotes the outward unit normal vector on??1(x0).SoKε→+∞asε→0.De fi ne

Then we have

Then we have andUε,Vεsatisfy

Since

we get that{(Uε,Vε)}is bounded in=D.By elliptic estimates,up to a subsequence,we have(Uε,Vε)→(U,V)∈Duniformly in every compact subset ofasε→0,and(U,V)satisfies(1.6),that isE′(U,V)=0.Moreover,U,V≥0 are partly radially symmetric decreasing.Note that(3.18)we get(U,V)/=(0,0),and so(U,V)∈M′.Then we deduce from(3.16)that

SoE(U,V)=B′.Note thatB′<minand we haveU/≡0,V/≡0.By the strong maximum principle,U>0,V>0 are partly radially symmetric decreasing.We also have(U,V)∈M,and soE(U,V)≥B≥B′,that is,E(U,V)=B=B′.Moreover(U,V)is positive least energy solution of(1.6),which is partly radially symmetric decreasing.
Finally,with the help of(2.1)and[1,(2-2)in Theorem 1.6],we get that there existsd(β)andg(β)on(?β2,β2)for someβ2>0,andli(d(β),g(β))≡0 fori=1,2.This implies thatis a positive solution of(1.6).Therefore we have

that is,there exists 0< ?β1≤?β2such that

Recall that

and we have

[1]Chen Zhijie,Zou Wenming.Positive least energy solutions and phase separation for coupled Schr¨odinger equations with critical exponent:higher dimensional case[J].Calc.Var.,2015,52:423–467.
[2]Chen Hua,Liu Xiaochun,Wei Y.Existence theorem for a class of semilinear totally characteristic elliptic equations with critical cone Sobolev exponents[J].Ann.Glob.Ann.Geom.,2011,39(1):27–43.
[3]Chen Hua,Liu Xiaochun,Wei Y.Dirichlet problem for semilinear edge-degenerate elliptic equations with singular potential term[J].J.Diff.Equ.,2012,252(7):4289–4314.
[4]Fan Haining,Liu Xiaochun.Multiple positive solutions for a class of quasi-linear elliptic equations involving critical Sobolev exponent[J].Acta.Math.Sci.,2014,34(B)(4):1111–1126.
[5]Fan Haining,Liu Xiaochun.Multiple positive solutions for degenerate elliptic equations with critical cone Sobolev exponents on singular manifold[J].Elec.J.Diff.Equ.,2013,181:1–22.
[6]Brown K J,ZhangY.The Nehari manifold for a semilinear elliptic equation with a sign-changing weight function[J].J.Diff.Equ.,2003,193:481–499.
[7]Co Alves,AE Hamidi.Nehari manifold and existence of positive solutions to a class of quasillinear problems[J].Nonl.Anal.The.Meth.Appl.,2005,60(4):611–624.
[8]Guo Zhenyu,Perera Kainshka,Zou Wenming.On criticalp-Laplacian systems[J].Math.AP,2015.
[9]Chen Nanbo, Tu Qiang.The Nehari manifold for a quasillinear sub-elliptic equation with a sign-changing weight function on the Heisenberg group[J].J.Math.,2016,Doi:10.13548/j.sxzz.20170220.002.
[10]Schulze B W.Boundary value problems and singular pseudo-differential operators[M].Chichester:Wiley,1998.
[11]Chen Hua,Liu Xiaochun,Wei Y.Multiple solutions for semilinear totally characteristic elliptic equations with subcritical or critical cone Sobolev exponents[J].J.Diff.Equ.,2012,252:4200–4288.
[12]Chen Hua,Liu Xiaochun,Wei Y.Cone Sobolev inequality and dirichlet problem for nonlinear elliptic equations on mainfold with conical singularities[J].Cal.Var.Part.Diff.Equ.,2012,43:463–484.