A uniqueness criterion of limit cycles for planar polynomial systems with homogeneous nonlinearities

This paper is devoted to study the planar polynomial system:

$\dot{x}=ax?y+P_n(x,y),\dot{y}=x+ay+Q_n(x,y),$

where $a\in \mathbb{R}$ and $P_n,Q_n$ are homogeneous polynomials of degree $n\ge 2$. Denote $\psi (\theta )=\cos ?(\theta )?Q_n(\cos ?(\theta ),\sin ?(\theta ))?\sin ?(\theta )?P_n(\cos ?(\theta ),\sin ?(\theta ))$. We prove that the system has at most 1 limit cycle surrounding the origin provided $(n?1)a\psi (\theta )+\dot{\psi }(\theta )\ne 0$. Furthermore, this upper bound is sharp. This is maybe the first uniqueness criterion, which only depends on a (linear) condition of ψ, for the limit cycles of this kind of systems. We show by examples that in many cases, the criterion is applicable while the classical ones are invalid. The tool that we mainly use is a new estimate for the number of limit cycles of Abel equation with coefficients of indefinite signs. Employing this tool, we also obtain another geometric criterion which allows the system to possess at most 2 limit cycles surrounding the origin.