球面上$\ell_1$正则优化的随机临近梯度方法

本文研究球面上的$\ell_1$正则优化问题，其目标函数由一般光滑函数项和非光滑$\ell_1$正则项构成，且假设光滑函数的随机梯度可由随机一阶oracle估计.这类优化问题被广泛应用在机器学习，图像、信号处理和统计等领域.根据流形临近梯度法和随机梯度估计技术，提出一种球面随机临近梯度算法.基于非光滑函数的全局隐函数定理，分析了子问题解关于参数的Lipschtiz连续性，进而证明了算法的全局收敛性.在基于随机数据集和实际数据集的球面$\ell_1$正则二次规划问题、有限和SPCA问题和球面$\ell_1$正则逻辑回归问题上数值实验结果显示所提出的算法与流形临近梯度法、黎曼随机临近梯度法相比CPU时间上具有一定的优越性.

STOCHASTIC PROXIMAL GRADIENT METHOD FOR l1 REGULARIZED OPTIMIZATION OVER A SPHERE

This paper presents a stochastic proximal gradient method for solving thel₁regularized optimization problem over a sphere.The objective function of the optimization problem is composed of a general smooth function and a non-smoothl₁regularization term,and it is.assumed that the noisy gradient of the smooth function can be estimated by some stochastic first-order oracles.These optimization problems are widely used in machine learning,image,signal processing,and statistics.We employ the manifold proximal gradient method and the stochastic technique for estimating the gradient information to present a sphere stochastic proximal gradient algorithm for solving thel₁regularized optimization over a sphere.Via establishing the global implicit function theorem of a certain non-smooth function,we analyze the Lipschtiz continuity of the solutions of the subproblems and prove the global convergence of the proposed algorithm under some assumptions.Numerical results on thel₁regularized quadratic programming problem,the finite-sum sparse PCA problem,and thel₁regularized logistic regression problem over the sphere with synthetic and real data sets illustrate that the proposed algorithm is competitive with the manifold proximal gradient algorithm and Riemannian stochastic proximal gradient method in terms of CPU time.