Globally and Superlinearly Convergent QP-Free Algorithm for Nonlinear Constrained Optimization

A new, infeasible QP-free algorithm for nonlinear constrained optimization problems is proposed. The algorithm is based on a continuously differentiable exact penalty function and on active-set strategy. After a finite number of iterations, the algorithm requires only the solution of two linear systems at each iteration. We prove that the algorithm is globally convergent toward the KKT points and that, if the second-order sufficiency condition and the strict complementarity condition hold, then the rate of convergence is superlinear or even quadratic. Moreover, we incorporate two automatic adjustment rules for the choice of the penalty parameter and make use of an approximated direction as derivative of the merit function so that only first-order derivatives of the objective and constraint functions are used.     

求解不等式约束优化问题无严格互补松弛条件的QP-Free新算法

本文针对不等式约束优化问题,结合Facchinei-Fischer-Kanzow精确有效集识别技术,给出—个新的线性方程组与辅助方向相结合的可行下降算法.算法每步迭代只需求解一个降维的线性方程组或计算一次辅助方向,且获取辅助方向的投影矩阵只涉及近似有效约束集中的元素,问题规模大为减少,且当迭代次数充分大时,只需求解一个降维的线性方程组.无需严格互补松弛条件,算法全局且一步超线性收敛.     

A Truncated Nonmonotone Gauss-Newton Method for Large-Scale Nonlinear Least-Squares Problems

In this paper, a Gauss-Newton method is proposed for the solution of large-scale nonlinear least-squares problems, by introducing a truncation strategy in the method presented in [9]. First, sufficient conditions are established for ensuring the convergence of an iterative method employing a truncation scheme for computing the search direction, as approximate solution of a Gauss-Newton type equation. Then, a specific truncated Gauss-Newton algorithm is described, whose global convergence is ensured under standard assumptions, together with the superlinear convergence rate in the zero-residual case. The results of a computational experimentation on a set of standard test problems are reported.     

A Truncated Newton Method for the Solution of Large-Scale Inequality Constrained Minimization Problems

A new active set Newton-type algorithm for the solution of inequality constrained minimization problems is proposed. The algorithm possesses the following favorable characteristics: (i) global convergence under mild assumptions; (ii) superlinear convergence of primal variables without strict complementarity; (iii) a Newton-type direction computed by means of a truncated conjugate gradient method. Preliminary computational results are reported to show viability of the approach in large scale problems having only a limited number of constraints.     

两类解非线性约束问题的超记忆可行方向法

本文将无约束超记忆梯度法推广到非线性不等式约束优化问题上来,给出了两类形式很一般的超记忆可行方向法,并在非退化及连续可微等较弱的假设下证明了其全局收敛性.适当选取算法中的参量及记忆方向,不仅可得到一些已知的方法及新方法,而且还可能加快算法的收敛速度.     

Enriched Methods for Large-Scale Unconstrained Optimization

This paper describes a class of optimization methods that interlace iterations of the limited memory BFGS method (L-BFGS) and a Hessian-free Newton method (HFN) in such a way that the information collected by one type of iteration improves the performance of the other. Curvature information about the objective function is stored in the form of a limited memory matrix, and plays the dual role of preconditioning the inner conjugate gradient iteration in the HFN method and of providing an initial matrix for L-BFGS iterations. The lengths of the L-BFGS and HFN cycles are adjusted dynamically during the course of the optimization. Numerical experiments indicate that the new algorithms are both effective and not sensitive to the choice of parameters.     

Subspace Barzilai-Borwein Gradient Method for Large-Scale Bound Constrained Optimization

An active set subspace Barzilai-Borwein gradient algorithm for large-scale bound constrained optimization is proposed. The active sets are estimated by an identification technique. The search direction consists of two parts: some of the components are simply defined; the other components are determined by the Barzilai-Borwein gradient method. In this work, a nonmonotone line search strategy that guarantees global convergence is used. Preliminary numerical results show that the proposed method is promising, and competitive with the well-known method SPG on a subset of bound constrained problems from CUTEr collection. This work was supported by the 973 project granted 2004CB719402 and the NSF project of China granted 10471036.     

一簇非线性等式约束优化问题的过滤线搜索修正正割方法

本文提供了一簇新的过滤线搜索修正正割方法求解非线性等式约束优化问题.新算法簇的特点是:用修正正割算法簇中的一个算法获得搜索方向,回代线搜索技术得到步长,过滤准则用来决定是否接受步长,引入二阶校正技术减少不可行性并克服Maratos效应.在合理的假设条件下,分析了算法的总体收敛性.并证明了,通过附加二阶校正步,算法簇克服了Maratos效应,并二步Q-超线性收敛到满足二阶充分最优条件的局部解.数值结果表明了所提供的算法具有有效性.     

解非线性约束方程的拉格朗日全局投影方法

基于最优化方法求解约束非线性方程组的一个突出困难是计算 得到的仅是该优化问题的稳定点或局部极小点,而非方程组的解点.由此引出的问题是如何从一个稳定点出发得到一个相对于方程组解更好的点. 该文采用投影型算法,推广了Nazareth-Qi$^{[8,9]}$ 求解无约束非线性方程组的拉格朗日全局算法(Lagrangian Global-LG)于约束方程上; 理论上证明了从优化问题的稳定点出发,投影LG方法可寻找到一个更好的点. 数值试验证明了LG方法的有效性.     

解非线性约束优化问题的依赖域算法

本文对一般非线性约束优化问题提出了一个信赖域算法,导出了等价的KKT条件.在试探步满足适当条件下,证明了算法的全局收敛性,并进行了数值试验.     

A Trajectory-Based Method for Constrained Nonlinear Optimization Problems

A trajectory-based method for solving constrained nonlinear optimization problems is proposed. The method is an extension of a trajectory-based method for unconstrained optimization. The optimization problem is transformed into a system of second-order differential equations with the aid of the augmented Lagrangian. Several novel contributions are made, including a new penalty parameter updating strategy, an adaptive step size routine for numerical integration and a scaling mechanism. A new criterion is suggested for the adjustment of the penalty parameter. Global convergence properties of the method are established.     

约束优化问题的广义既约型梯度法

本文提出了二类解约束优化问题的广义既约型梯度法,从统一角度研究了投影梯度法和既约梯度法的结构及其全局收敛性.本文结果统一、推广了常见的可行方向法.     

Spectral Gradient Methods for Linearly Constrained Optimization

Linearly constrained optimization problems with simple bounds are considered in the present work. First, a preconditioned spectral gradient method is defined for the case in which no simple bounds are present. This algorithm can be viewed as a quasi-Newton method in which the approximate Hessians satisfy a weak secant equation. The spectral choice of steplength is embedded into the Hessian approximation and the whole process is combined with a nonmonotone line search strategy. The simple bounds are then taken into account by placing them in an exponential penalty term that modifies the objective function. The exponential penalty scheme defines the outer iterations of the process. Each outer iteration involves the application of the previously defined preconditioned spectral gradient method for linear equality constrained problems. Therefore, an equality constrained convex quadratic programming problem needs to be solved at every inner iteration. The associated extended KKT matrix remains constant unless the process is reinitiated. In ordinary inner iterations, only the right-hand side of the KKT system changes. Therefore, suitable sparse factorization techniques can be applied and exploited effectively. Encouraging numerical experiments are presented.This research was supported by FAPESP Grant 2001-04597-4 and Grant 903724-6, FINEP and FAEP-UNICAMP, and the Scientific Computing Center of UCV. The authors thank two anonymous referees whose comments helped us to improve the final version of this paper.     

Hybrid Methods for Nonlinear Least Squares

The conjecture is considered that the hybrid GN-BFGS methodof Al-Baali and Fletcher [1] is superlinearly convergent. Acounterexample is constructed for which the method is only linearlyconvergent whereas the BFGS method alone would be superlinearlyconvergent, thus disproving the conjecture. Two new hybrids methods are suggested, and it is shown thatthese are superlinearly convergent under mild conditions. Awide range of numerical experience is reported and is seen tofavour the new methods. Some interesting theoretical propertiesof a variational Hessian error measure are given.     

关于非线性等式约束优化问题的一点注记

给出了一种求解非线性约束优化问题的算法.利用Lagrange函数,将非线性约束优化问题转化为无约束优化问题,从而得到解决.方法仅仅依靠求解一个线性方程组来求解,因此使得计算量减小,计算速度变快.在一定条件下,给出算法的收敛性证明.数值试验表明方法是有效的.     

约束优化问题的修正共轭梯度投影算法

对闭凸集约束的非线性规划问题构造了一个修正共轭梯度投影下降算法,在去掉迭代点列有界的条件下,分析了算法的全局收敛性.新算法与共轭梯度参数结合,给出了三类结合共轭梯度参数的修正共轭梯度投影算法.数值例子表明算法是有效的.     

一类等式约束非线性优化问题的信赖域新算法

提出一类信赖域新算法用于求解等式约束的非线性优化问题,在构造增广拉格朗日函数的基础上,提出了信赖域子问题的求解公式,研究了拉格朗日乘子和罚因子的修正公式,并使用滤子技巧,放松了接受尝试步的条件,证明了算法的收敛性.最后进行了数值试验.     

Differential Conditions for Constrained Nonlinear Programming via Pareto Optimization

We deal with the differential conditions for local optimality. The conditions that we derive for inequality constrained problems do not require constraint qualifications and are the broadest conditions based on only first-order and second-order derivatives. A similar result is proved for equality constrained problems, although the necessary conditions require the regularity of the equality constraints.