求解一般非线性互补问题的光滑化方法

在利用Fischer-Burmeister函数将非线性互补问题转化为非线性方程组的基础上,本文通过将信赖域方法与线性搜索方法结合起来,提出了求解一般非线性互补问题的光滑化方法.算法中我们给出了一个特定条件,条件满足时,采用信赖步,条件不满足时.采用梯度步.我们证明了算法具有全局收敛性.在解是R-正则的条件下,收敛速度是Q-超线性/Q-二阶收敛的.     

球面上$\ell_1$正则优化的随机临近梯度方法

本文研究球面上的$\ell_1$正则优化问题,其目标函数由一般光滑函数项和非光滑$\ell_1$正则项构成,且假设光滑函数的随机梯度可由随机一阶oracle估计.这类优化问题被广泛应用在机器学习,图像、信号处理和统计等领域.根据流形临近梯度法和随机梯度估计技术,提出一种球面随机临近梯度算法.基于非光滑函数的全局隐函数定理,分析了子问题解关于参数的Lipschtiz连续性,进而证明了算法的全局收敛性.在基于随机数据集和实际数据集的球面$\ell_1$正则二次规划问题、有限和SPCA问题和球面$\ell_1$正则逻辑回归问题上数值实验结果显示所提出的算法与流形临近梯度法、黎曼随机临近梯度法相比CPU时间上具有一定的优越性.     

凸约束非光滑方程组一个新的谱梯度投影算法

基于寻找分离超平面的三种经典线搜索技术,本文提出了一种自适应线搜索技术.结合谱梯度投影法,提出了凸约束非光滑单调方程组的一个谱梯度投影算法.该算法不需要计算和存储任何矩阵,因而适合求解大规模非光滑的非线性单调方程组.在较弱的条件下,证明了方法的全局收敛性,并分析了算法的收敛率.数值试验结果表明算法是有效的和鲁棒的.     

最小化三个凸函数之和的一个简单原始-对偶算法

提出一个简单的原始-对偶算法求解三个凸函数之和的最小化问题, 其中目标函数包含有梯度李普希兹连续的光滑函数, 非光滑函数和含有复合算子的非光滑函数. 在新方法中, 对偶变量迭代使用预估-矫正的方案. 分析了算法的收敛性和收敛速率. 最后, 数值实验说明了算法的有效性.     

带有P0函数的非线性互补问题的一个新的非内点连续算法

研究带有P₀函数的非线性互补问题. 基于一个新的光滑函数, 把问题近似成参数化的光滑方程组, 并且给出一个新的非内点连续算法. 所给算法在每步迭代只需要求解一个线性方程组和执行一次Armijo类型的线搜索. 在不需要严格互补条件的情况下, 证明了算法是全局收敛和超线性收敛的. 并且, 在一个较弱的条件下该算法具有局部二阶收敛性. 数值实验证实了算法的可行性和有效性.     

一个新的求解加权线性互补问题的非单调光滑牛顿法

本文研究了求解加权线性互补问题的光滑牛顿法.利用一类光滑函数将加权线性互补问题等价转化成一个光滑方程组,然后提出一个新的光滑牛顿法去求解它.在适当条件下,证明了算法具有全局和局部二次收敛性质.与现有的光滑牛顿法不同,我们的算法采用一个非单调无导数线搜索技术去产生步长,从而具有更好的收敛性质和实际计算效果.     

求解非光滑方程组的三次正则化方法

考虑求解非光滑方程组的三次正则化方法及其收敛性分析.利用信赖域方法的技巧,保证该方法是全局收敛的.在子问题非精确求解和BD正则性条件成立的前提下,分析了非光滑三次正则化方法的局部收敛速度.最后,数值实验结果验证了该算法的有效性.     

一个求解广义圆锥互补问题的无导数光滑算法

本文研究了一个求解广义圆锥互补问题的无导数光滑算法.利用光滑函数将广义圆锥互补问题等价转化成一个光滑方程组,然后再利用牛顿法求解此方程组.该算法采用了一种新的非单调无导数线搜索技术,并且在适当条件下具有全局和局部二次收敛性质.数值实验结果表明算法是非常有效的.     

基于新光滑函数的P0映射非线性互补问题的光滑牛顿法(英文)

非线性互补问题(NCP)可以重新表述为一个非光滑方程组的解.通过引入一个新的光滑函数,将问题近似为参数化光滑方程组.基于这个光滑函数,我们提出了一个求解P₀映射和R₀映射非线性互补问题的光滑牛顿法.该算法每次迭代只求解一个线性方程和一次线搜索.在适当的条件下,证明了该方法是全局和局部二次收敛的.数值结果表明,该算法是有效的.     

复合凸优化的快速邻近点算法

在大数据时代,随着数据采集手段的不断提升,大规模复合凸优化问题大量的出现在包括统计数据分析,机器与统计学习以及信号与图像处理等应用中.本文针对大规模复合凸优化问题介绍了一类快速邻近点算法.在易计算的近似准则和较弱的平稳性条件下,本文给出了该算法的全局收敛与局部渐近超线性收敛结果.同时,我们设计了基于对偶原理的半光滑牛顿法来高效稳定求解邻近点算法所涉及的重要子问题.最后,本文还讨论了如何通过深入挖掘并利用复合凸优化问题中由非光滑正则函数所诱导的非光滑二阶信息来极大减少半光滑牛顿算法中求解牛顿线性系统所需的工作量,从而进一步加速邻近点算法.     

一个解不等式约束非线性规划问题的有效方法

本文提出了一个解不等式约束非线性规划问题有效方法．在这个方法中,考虑解一个等价Ｋｕｈｎ－Ｔｕｃｋｅｒ条件的非线性方程组．这个方程组中ＮＣＰ函数的使用消去了对应于不等式约束的Ｌａｇｒａｎｇｅ乘子的非负性．截断牛顿方法被用来解这个非线性方程组．为了保证全局收敛性,一个强健的损失函数被选为寻查函数,同时方法中插入修正最速下降方向．本文证明了方法的分Ｑ－二阶收敛性,同时指出新方法可以有效地解稀疏大规模非线性规划问题。     

On certain optimization methods with finite-step inner algorithms for convex finite-dimensional problems with inequality constraints

Numerical methods are proposed for solving finite-dimensional convex problems with inequality constraints satisfying the Slater condition. A method based on solving the dual to the original regularized problem is proposed and justified for problems having a strictly uniformly convex sum of the objective function and the constraint functions. Conditions for the convergence of this method are derived, and convergence rate estimates are obtained for convergence with respect to the functional, convergence with respect to the argument to the set of optimizers, and convergence to the g-normal solution. For more general convex finite-dimensional minimization problems with inequality constraints, two methods with finite-step inner algorithms are proposed. The methods are based on the projected gradient and conditional gradient algorithms. The paper is focused on finite-dimensional problems obtained by approximating infinite-dimensional problems, in particular, optimal control problems for systems with lumped or distributed parameters.     

求可测函数的总体极小值方法的收敛性分析

[1]中提出了求解连续函数f(x)总体极小值的均值算法，并证明了算法的全局收敛性．若假设f(t)是定义在某可测集G上的可测函数，本证明了均值算法产生的迭代序列全局收敛到f(t)的本质极小值，若进一步假设函数f(t)满足测度Lipschitz条件，还证明了求可测函数的均值算法是线性收敛的．     

An implementable augmented Lagrange method for solving fixed point problems with coupled constraints

An augmented Lagrange function method for solving fixed point problems with coupled constraints is studied, and a theorem of its global convergence is demonstrated. The semismooth Newton method is used to solve the inner problems for obtaining approximate solutions, and numerical results are reported to verify the effectiveness of the augmented Lagrange function method for solving three examples with more than 1000 variables.     

从常步长梯度方法的视角看不可微凸优化增广Lagrange方法的收敛性

增广Lagrange方法是求解非线性规划的一种有效方法.从一新的角度证明不等式约束非线性非光滑凸优化问题的增广Lagrange方法的收敛性.用常步长梯度法的收敛性定理证明基于增广Lagrange函数的对偶问题的常步长梯度方法的收敛性,由此得到增广Lagrange方法乘子迭代的全局收敛性.     

A non-Euclidean gradient descent method with sketching for unconstrained matrix minimization

We propose a method that incorporates a non-Euclidean gradient descent step with a generic matrix sketching procedure, for solving unconstrained, nonconvex, matrix optimization problems, in which the decision variable cannot be saved in memory due to its size, and the objective function is the composition of a vector function on a linear operator. The method updates the sketch directly without updating or storing the decision variable. Subsequence convergence, global convergence under the Kurdyka–Lojasiewicz property, and rate of convergence, are established.     

Global convergence of a modified limited memory BFGS method for non-convex minimization

In this paper, a modified limited memory BFGS method for solving large-scale unconstrained optimization problems is proposed. A remarkable feature of the proposed method is that it possesses global convergence property without convexity assumption on the objective function. Under some suitable conditions, the global convergence of the proposed method is proved. Some numerical results are reported which illustrate that the proposed method is efficient.     

On the semilocal convergence behavior for Halley’s method

The present paper is concerned with the semilocal convergence problems of Halley’s method for solving nonlinear operator equation in Banach space. Under some so-called majorant conditions, a new semilocal convergence analysis for Halley’s method is presented. This analysis enables us to drop out the assumption of existence of a second root for the majorizing function, but still guarantee Q-cubic convergence rate. Moreover, a new error estimate based on a directional derivative of the twice derivative of the majorizing function is also obtained. This analysis also allows us to obtain two important special cases about the convergence results based on the premises of Kantorovich and Smale types.     

A variant of Steffensen’s method of fourth-order convergence and its applications

In this paper, a variant of Steffensen’s method of fourth-order convergence for solving nonlinear equations is suggested. Its error equation and asymptotic convergence constant are proven theoretically and demonstrated numerically. The derivative-free method only uses three evaluations of the function per iteration to achieve fourth-order convergence. Its applications on systems of nonlinear equations and boundary-value problems of nonlinear ODEs are showed as well in the numerical examples.     

An Algorithm for Continuous Type Optimal Location Problem

In this paper, we extend the ordinary discrete type facility location problems to continuous type ones. Unlike the discrete type facility location problem in which the objective function isn't everywhere differentiable, the objective function in the continuous type facility location problem is strictly convex and continuously differentiable. An algorithm without line search for solving the continuous type facility location problems is proposed and its global convergence, linear convergence rate is proved. Numerical experiments illustrate that the algorithm suggested in this paper have smaller amount of computation, quicker convergence rate than the gradient method and conjugate direction method in some sense.