Classification of positive solutions to an integral system with the poly-harmonic extension operator

In this paper, we investigate the positive solutions to the following integral system with a poly-harmonic extension operator on \(\mathbb{R}_+^n\),

$$\left\{ {_{v(y) = {c_{n,a}}\int_{\mathbb{R}_ + ^n} {\frac{{x_n^{1 - a}{u^\theta }(x)}}{{{{\left| {x - y} \right|}^{n - a}}}}dx,y \in \partial \mathbb{R}_ + ^n,} }^{u(x) = {c_{n,a}}\int_{\partial \mathbb{R}_ + ^n} {\frac{{x_n^{1 - a}{v^\kappa }(y)}}{{{{\left| {x - y} \right|}^{n - a}}}}dy,x \in \mathbb{R}_ + ^n,} }} \right.$$

where \(n \geqslant 2,2 - n < a < 1,\kappa ,\theta > 0\). This integral system arises from the Euler-Lagrange equation corresponding to an integral inequality on the upper half space established by Chen (2014). The explicit formulations of positive solutions are obtained by the method of moving spheres for the critical case \(\kappa = \frac{{n - 2 + a}}{{n - a}},\theta = \frac{{n + 2 - a}}{{n - 2 + a}}\). Moreover, we also give the nonexistence of positive solutions in the subcritical case for the above system.