Symmetry and monotonicity of positive solutions for a system involving fractional p&q-Laplacian in ${\mathbb {R}}^{n}$

Abstract

In this paper, we study a nonlinear system involving the fractional p&q-Laplacian in ${\mathbb {R}}^{n}$

$$\begin{aligned} \left\{ \begin{array}{ll} (-\Delta )_{p}^{s_{1}}u(x)+(-\Delta )_{q}^{s_{2}}u(x)=f(u(x),v(x)), &{}x\in {\mathbb {R}}^{n},\\ (-\Delta )_{p}^{s_{1}}v(x)+(-\Delta )_{q}^{s_{2}}v(x)=g(u(x),v(x)), &{}x\in {\mathbb {R}}^{n},\\ u,v>0,&{}x\in {\mathbb {R}}^{n}. \end{array} \right. \end{aligned}$$

where $0<s_{1}$, $s_{2}<1$, $p,q>2$. By using the direct method of moving planes, we prove that the positive solution (u, v) of system above must be radially symmetric and monotone decreasing in the whole space.