An increasing‐angle property of the conjugate gradient method and the implementation of large‐scale minimization algorithms with line searches

The search direction in unconstrained minimization algorithms for large‐scale problems is usually computed as an iterate of the preconditioned) conjugate gradient method applied to the minimization of a local quadratic model. In line‐search procedures this direction is required to satisfy an angle condition that says that the angle between the negative gradient at the current point and the direction is bounded away from π/2. In this paper, it is shown that the angle between conjugate gradient iterates and the negative gradient strictly increases as far as the conjugate gradient algorithm proceeds. Therefore, the interruption of the conjugate gradient sub‐algorithm when the angle condition does not hold is theoretically justified. Copyright © 2002 John Wiley & Sons, Ltd.     

一类改进的非凸二次规划有效集方法

一类改进的非凸二次规划有效集方法修乃华（河北师范学院数学系）ＡＣＬＡＳＳＯＦＩＭＰＲＯＶＥＤＡＣＴＩＶＥＳＥＴＭＥＴＨＯＤＳＦＯＲＮＯＮＣＯＮＶＥＸＱＵＡＤＲＡＴＩＣＰＲＯＧＲＡＭＭＩＮＧＰＲＯＢＬＥＭ￥ＸｉｕＮａｉ－ｈｕａ（Ｄｅｐｔ．ｏｆＭａｔｈ．...     

An Interior Point Algorithm for Mixed Complementarity Nonlinear Problems

Nonlinear complementarity and mixed complementarity problems arise in mathematical models describing several applications in Engineering, Economics and different branches of physics. Previously, robust and efficient feasible directions interior point algorithm was presented for nonlinear complementarity problems. In this paper, it is extended to mixed nonlinear complementarity problems. At each iteration, the algorithm finds a feasible direction with respect to the region defined by the inequality conditions, which is also monotonic descent direction for the potential function. Then, an approximate line search along this direction is performed in order to define the next iteration. Global and asymptotic convergence for the algorithm is investigated. The proposed algorithm is tested on several benchmark problems. The results are in good agreement with the asymptotic analysis. Finally, the algorithm is applied to the elastic–plastic torsion problem encountered in the field of Solid Mechanics.     

Another Conjugate Gradient Algorithm with Guaranteed Descent and Conjugacy Conditions for Large-scale Unconstrained Optimization

In this paper, we suggest another accelerated conjugate gradient algorithm for which both the descent and the conjugacy conditions are guaranteed. The search direction is selected as a linear combination of the gradient and the previous direction. The coefficients in this linear combination are selected in such a way that both the descent and the conjugacy condition are satisfied at every iteration. The algorithm introduces the modified Wolfe line search, in which the parameter in the second Wolfe condition is modified at every iteration. It is shown that both for uniformly convex functions and for general nonlinear functions, the algorithm with strong Wolfe line search generates directions bounded away from infinity. The algorithm uses an acceleration scheme modifying the step length in such a manner as to improve the reduction of the function values along the iterations. Numerical comparisons with some conjugate gradient algorithms using a set of 75 unconstrained optimization problems with different dimensions show that the computational scheme outperforms the known conjugate gradient algorithms like Hestenes and Stiefel; Polak, Ribière and Polyak; Dai and Yuan or the hybrid Dai and Yuan; CG_DESCENT with Wolfe line search, as well as the quasi-Newton L-BFGS.     

基于动力系统的线性不等式组的解法

本文提出了一种新的求解线性不等式组可行解的方法-基于动力系统的方法．假设线性不等式组的可行域为非空,在可行域的相对内域上建立一个非线性关系表达式,进而得到一个结构简单的动力系统模型．同时,定义了穿越方向。文章最后的数值实验结果表明此算法是有效的．     

An Improved Spectral Conjugate Gradient Algorithm for Nonconvex Unconstrained Optimization Problems

In this paper, an improved spectral conjugate gradient algorithm is developed for solving nonconvex unconstrained optimization problems. Different from the existent methods, the spectral and conjugate parameters are chosen such that the obtained search direction is always sufficiently descent as well as being close to the quasi-Newton direction. With these suitable choices, the additional assumption in the method proposed by Andrei on the boundedness of the spectral parameter is removed. Under some mild conditions, global convergence is established. Numerical experiments are employed to demonstrate the efficiency of the algorithm for solving large-scale benchmark test problems, particularly in comparison with the existent state-of-the-art algorithms available in the literature.     

Some new three-term Hestenes–Stiefel conjugate gradient methods with affine combination

In this paper, a modified Hestenes–Stiefel conjugate gradient method for unconstrained problems is developed, which can achieves the twin goals of generating sufficient descent direction at each iteration as well as being close to the Newton direction. In our methods, the hybridization parameter can also be obtained based on other kinds of conjugacy conditions. Under mild condition, we establish their global convergence for general objective functions. Numerical experimentation with the new method indicates that it efficiently solves the test problems and therefore is promising.     

求解线性不等式组的方法

本提出了一个新的求解线性不等式组可行解的方法－－无约束极值方法。通过在线性不等式组的非空可行域的相对内域上建立一个非线性极值问题，根据对偶关系，得到了一个对偶空间的无约束极值及原始，对偶变量之间的简单线性映射关系，这样将原来线性不等式组问题的求解转化为一个无约束极值问题。中主要讨论了求解无约束极值问题的共轭梯度算法。同时，在寻找不等式组可行解的过程中，定义了穿越方向，这样大大减少计算量。中最后数值实验结果表明此算法是有效的。     

Convergence of a feasible directions algorithm for a distributed optimal control problem of parabolic type with terminal inequality constraints

In this paper, we consider a class of optimal control problems with control and terminal inequality constraints, where the system dynamics is governed by a linear second-order parabolic partial differential equation with first boundary condition. A feasible direction algorithm for solving this class of optimal control problems has already been obtained in the literature. The aim of this paper is to improve the convergence result by using a topology arising in the study of relaxed controls.     

New spectral PRP conjugate gradient method for unconstrained optimization

In this paper, a new spectral PRP conjugate gradient algorithm has been developed for solving unconstrained optimization problems, where the search direction was a kind of combination of the gradient and the obtained direction, and the steplength was obtained by the Wolfe-type inexact line search. It was proved that the search direction at each iteration is a descent direction of objective function. Under mild conditions, we have established the global convergence theorem of the proposed method. Numerical results showed that the algorithm is promising, particularly, compared with the existing several main methods.     

Surrogate Gradient Algorithm for Lagrangian Relaxation

The subgradient method is used frequently to optimize dual functions in Lagrangian relaxation for separable integer programming problems. In the method, all subproblems must be solved optimally to obtain a subgradient direction. In this paper, the surrogate subgradient method is developed, where a proper direction can be obtained without solving optimally all the subproblems. In fact, only an approximate optimization of one subproblem is needed to get a proper surrogate subgradient direction, and the directions are smooth for problems of large size. The convergence of the algorithm is proved. Compared with methods that take effort to find better directions, this method can obtain good directions with much less effort and provides a new approach that is especially powerful for problems of very large size.     

无约束极大极小问题的广义梯度投影算法

本文讨论Rⁿ空间上的无约束极大极小问题. 通过Rⁿ⁺¹空间上的广义梯度投影技术产生Rⁿ上的下降搜索方向,进而结合Armijo非精确线搜索建立了原问题Rⁿ上的一个广义梯度投影型算法.算法在仿射线性无关条件下,具有全局收敛性和强收敛性. 文中对算法进行了初步的数值试验.     

多目标非线性规划的可行方向交互算法

本文提出一个求解多目标非线性规划问题的交互规划算法．在每一轮迭代中,此法仅要求决策者提供目标间权衡比的局部信息．算法中的可行方向是基于求解非线性规划问题的Topkis－Veinott法构千的．我们证明,在一定条件下,此算法收敛于问题的有效解．     

一类非单调线性互补问题的高阶仿射尺度算法

In this paper, a new interior point algorithm-high-order atone scaling for a class of nonmonotonic linear complementary problems is developed. On the basis of idea of primal-dual affine scaling method for linear programming , the search direction of our algorithm is obtained by a linear system of equation at each step . We show that, by appropriately choosing the step size, the algorithm has polynomial time complexity. We also give the numberical results of the algorithm for two test problems.     

An improved interior-type feasible QP-free algorithm for inequality constrained optimization problems

In this paper, an improved interior-type feasible QP-free algorithm for inequality constrained optimization problems is proposed. At each iteration, by solving three systems of linear equations with the same coefficient matrix, a search direction is generated. The algorithm is proved to be globally and superlinearly convergent under some mild conditions. Preliminary numerical results show that the proposed algorithm may be promising. Advantages of the algorithm include: the uniformly nonsingularity of the coefficient matrices without the strictly complementarity condition is obtained. Moreover, the global convergence is achieved even if the number of the stationary points is infinite.     

An interior trust region algorithm for solving linearly constrained nonlinear optimization

In this paper, an interior point algorithm based on trust region techniques is proposed for solving nonlinear optimization problems with linear equality constraints and nonnegative variables. Unlike those existing interior-point trust region methods, this proposed method does not require that a general quadratic subproblem with a trust region bound be solved at each iteration. Instead, a system of linear equations is solved to get a search direction, and then a linesearch of Armijo type is performed in this direction to obtain a new iteration point. From a computational point of view, this approach may in general reduce a computational effort, and thus improve the computational efficiency. Under suitable conditions, it is proven that any accumulation of the sequence generated by the algorithm satisfies the first-order optimality condition.     

An affine scaling algorithm for linear programming problems with inequality constraints

A primal, interior point method is developed for linear programming problems for which the linear objective function is to be maximised over polyhedra that are not necessarily in standard form. This algorithm concurs with the affine scaling method of Dikin when the polyhedron is in standard form, and satisfies the usual conditions imposed for using that method. If the search direction is regarded as a function of the current iterate, then it is shown that this function has a unique, continuous extension to the boundary. In fact, on any given face, this extension is just the value the search direction would have for the problem of maximising the objective function over that face. This extension is exploited to prove convergence. The algorithm presented here can be used to exploit such special constraint structure as bounds, ranges, and free variables without increasing the size of the linear programming problem.This paper is in final form and no version of it will be submitted for publication elsewhere.     

基于有效约束识别技术的一个SSLE算法及其收敛性分析

基于一个有效约束识别技术, 给出了具有不等式约束的非线性最优化问题的一个可行SSLE算法. 为获得搜索方向算法的每步迭代只需解两个或三个具有相同系数矩阵的线性方程组. 在一定的条件下, 算法全局收敛到问题的一个KKT点. 没有严格互补条件, 在比强二阶充分条件弱的条件下算法具有超线性收敛速度.     

求解阵列天线自适应滤波问题的一种调比随机逼近算法

提出了求解阵列天线自适应滤波问题的一种调比随机逼近算法.每一步迭代中,算法选取调比的带噪负梯度方向作为新的迭代方向.相比已有的其他随机逼近算法,这个算法不需要调整稳定性常数,在一定程度上解决了稳定性常数选取难的问题.数值仿真实验表明,算法优于已有的滤波算法,且比经典Robbins-Monro （RM）算法具有更好的稳定性.     

NEW HYBRID CONJUGATE GRADIENT METHOD AS A CONVEX COMBINATION OF LS AND FR METHODS

In this paper, we present a new hybrid conjugate gradient algorithm for unconstrained optimization. This method is a convex combination of Liu-Storey conjugate gradient method and Fletcher-Reeves conjugate gradient method. We also prove that the search direction of any hybrid conjugate gradient method, which is a convex combination of two conjugate gradient methods, satisfies the famous D-L conjugacy condition and in the same time accords with the Newton direction with the suitable condition. Furthermore, this property doesn't depend on any line search. Next, we also prove that, moduling the value of the parameter t,the Newton direction condition is equivalent to Dai-Liao conjugacy condition.The strong Wolfe line search conditions are used.The global convergence of this new method is proved.Numerical comparisons show that the present hybrid conjugate gradient algorithm is the efficient one.