A new derivative-free SCG-type projection method for nonlinear monotone equations with convex constraints

Based on a modified line search scheme, this paper presents a new derivative-free projection method for solving nonlinear monotone equations with convex constraints, which can be regarded as an extension of the scaled conjugate gradient method and the projection method. Under appropriate conditions, the global convergence and linear convergence rate of the proposed method is proven. Preliminary numerical results are also reported to show that this method is promising.     

一个基于近似下降方向的免梯度算法

本构造一个求解非线性无约束优化问题的免梯度算法，该算法基于传统的模矢法，每次不成功迭代后，充分利用已有迭代点的信息，构造近似下降方向，产生新的迭代点。在较弱条件下，算法是总体收敛的。通过数值实验与传统模矢法相比，计算量明显减少。     

初始点任意的解非线性不等式约束优化问题的结合共轭梯度参数的超记忆梯度广义投影算法

本文利用广义投影矩阵，对求解无约束规划的超记忆梯度算法中的参数给出一种新的取值范围以保证得到目标函数的超记忆梯度广义投影下降方向，并与处理任意初始点的方法技巧结合建立求解非线性不等式约束优化问题的一个初始点任意的超记忆梯度广义投影算法，在较弱条件下证明了算法的收敛性．同时给出结合FR，PR，HS共轭梯度参数的超记忆梯度广义投影算法，从而将经典的共轭梯度法推广用于求解约束规划问题．数值例子表明算法是有效的．     

A family of derivative-free conjugate gradient methods for large-scale nonlinear systems of equations

In this paper, we propose a family of derivative-free conjugate gradient methods for large-scale nonlinear systems of equations. They come from two modified conjugate gradient methods [W.Y. Cheng, A two term PRP based descent Method, Numer. Funct. Anal. Optim. 28 (2007) 1217–1230; L. Zhang, W.J. Zhou, D.H. Li, A descent modified Polak–Ribiére–Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal. 26 (2006) 629–640] recently proposed for unconstrained optimization problems. Under appropriate conditions, the global convergence of the proposed method is established. Preliminary numerical results show that the proposed method is promising.     

Spectral DY-Type Projection Method for Nonlinear Monotone Systems of Equations

In this paper, we propose a spectral DY-type projection method for nonlinear monotone systems of equations, which is a reasonable combination of DY conjugate gradient method, the spectral gradient method and the projection technique. Without the differentiability assumption on the system of equations, we establish the global convergence of the proposed method, which does not rely on any merit function. Furthermore, this method is derivative-free and so is very suitable to solve large-scale nonlinear monotone systems. The preliminary numerical results show the feasibility and effectiveness of the proposed method.     

A derivative-free three-term projection algorithm involving spectral quotient for solving nonlinear monotone equations

ABSTRACT

In this paper, we develop a three-term conjugate gradient method involving spectral quotient, which always satisfies the famous Dai-Liao conjugacy condition and quasi-Newton secant equation, independently of any line search. This new three-term conjugate gradient method can be regarded as a variant of the memoryless Broyden-Fletcher-Goldfarb-Shanno quasi-Newton method with regard to spectral quotient. By combining this method with the projection technique proposed by Solodov and Svaiter in 1998, we establish a derivative-free three-term projection algorithm for dealing with large-scale nonlinear monotone system of equations. We prove the global convergence of the algorithm and obtain the R-linear convergence rate under some mild conditions. Numerical results show that our projection algorithm is effective and robust, and is more competitive with the TTDFP algorithm proposed Liu and Li [A three-term derivative-free projection method for nonlinear monotone system of equations. Calcolo. 2016;53:427–450].     

Strong Convergence of an Inexact Proximal Point Algorithm for Equilibrium Problems in Banach Spaces

We develop an inexact proximal point algorithm for solving equilibrium problems in Banach spaces which consists of two principal steps and admits an interesting geometric interpretation. At a certain iterate, first we solve an inexact regularized equilibrium problem with a flexible error criterion to obtain an axillary point. Using this axillary point and the inexact solution of the previous iterate, we construct two appropriate hyperplanes which separate the current iterate from the solution set of the given problem. Then the next iterate is defined as the Bregman projection of the initial point onto the intersection of two halfspaces obtained from the two constructed hyperplanes containing the solution set of the original problem. Assuming standard hypotheses, we present a convergence analysis for our algorithm, establishing that the generated sequence strongly and globally converges to a solution of the problem which is the closest one to the starting point of the algorithm.     

An increasing‐angle property of the conjugate gradient method and the implementation of large‐scale minimization algorithms with line searches

The search direction in unconstrained minimization algorithms for large‐scale problems is usually computed as an iterate of the preconditioned) conjugate gradient method applied to the minimization of a local quadratic model. In line‐search procedures this direction is required to satisfy an angle condition that says that the angle between the negative gradient at the current point and the direction is bounded away from π/2. In this paper, it is shown that the angle between conjugate gradient iterates and the negative gradient strictly increases as far as the conjugate gradient algorithm proceeds. Therefore, the interruption of the conjugate gradient sub‐algorithm when the angle condition does not hold is theoretically justified. Copyright © 2002 John Wiley & Sons, Ltd.     

A derivative-free algorithm for linearly constrained optimization problems

Based on the NEWUOA algorithm, a new derivative-free algorithm is developed, named LCOBYQA. The main aim of the algorithm is to find a minimizer $x^{*} \in\mathbb{R}^{n}$ of a non-linear function, whose derivatives are unavailable, subject to linear inequality constraints. The algorithm is based on the model of the given function constructed from a set of interpolation points. LCOBYQA is iterative, at each iteration it constructs a quadratic approximation (model) of the objective function that satisfies interpolation conditions, and leaves some freedom in the model. The remaining freedom is resolved by minimizing the Frobenius norm of the change to the second derivative matrix of the model. The model is then minimized by a trust-region subproblem using the conjugate gradient method for a new iterate. At times the new iterate is found from a model iteration, designed to improve the geometry of the interpolation points. Numerical results are presented which show that LCOBYQA works well and is very competing against available model-based derivative-free algorithms.     

求解非线性单调方程组的一种无导数投影算法

推广了一种修正的CG_DESCENT共轭梯度方法,并建立了一种有效求解非线性单调方程组问题的无导数投影算法.在适当的线搜索条件下,证明了算法的全局收敛性.由于新算法不需要借助任何导数信息,故它适应于求解大规模非光滑的非线性单调方程组问题.大量的数值试验表明,新算法对给定的测试问题是有效的.     

An Liu-Storey-Type Method for Solving Large-Scale Nonlinear Monotone Equations

An algorithm for solving nonlinear monotone equations is proposed, which combines a modified Liu-Storey conjugate gradient method with hyperplane projection method. Under mild conditions, the global convergence of the proposed method is established with a suitable line search method. The method can be applied to solve large-scale problems for its lower storage requirement. Numerical results indicate that our method is efficient.     

解凸约束非线性单调方程组的无导数谱PRP投影算法

本文在著名PRP共轭梯度算法的基础上研究了一种无导数谱PRP投影算法,并证明了算法在求解带有凸约束条件的非线性单调方程组问题的全局收敛性.由于无导数和储存量小的特性,它更适应于求解大规模非光滑的非线性单调方程组问题.数值试验表明,新算法对给定的测试问题是有效的和稳定的.     

Wolfe步长规则下约束优化问题的共轭梯度投影算法

本文研究了约束优化问题min x∈Ωf(x).利用共轭梯度算法与GLP梯度投影思想相结合的方法,构造了一个新的共轭梯度投影算法,并在Wolfe线搜索下获得了该算法的全局收敛性结果.     

一个求解非线性不等式约束优化问题的带有共轭梯度参数的广义梯度投影算法

本文提出一个求解非线性不等式约束优化问题的带有共轭梯度参数的广义梯度投影算法.算法中的共轭梯度参数是很容易得到的,且算法的初始点可以任意选取.而且,由于算法仅使用前一步搜索方向的信息,因而减少了计算量.在较弱条件下得到了算法的全局收敛性.数值结果表明算法是有效的.     

Conjugate gradient path method without line search technique for derivative-free unconstrained optimization

In this paper, we propose conjugate gradient path method for solving derivative-free unconstrained optimization. The iterative direction is obtained by constructing and solving quadratic interpolation model of the objective function with conjugate gradient methods. The global convergence and local superlinear convergence rate of the proposed algorithm are established under some reasonable conditions. Finally, the numerical results are reported to show the effectiveness of the proposed algorithm.     

一种杂交共轭梯度方法的全局收敛性(英文)

Conjugate gradient method is one of successful methods for solving the unconstrained optimization problems. In this paper, absorbing the advantages of FR and CD methods, a hybrid conjugate gradient method is proposed. Under the general Wolfe linear searches, the proposed method can generate the sufficient descent direction at each iterate,and its global convergence property also can be established. Some preliminary numerical results show that the proposed method is effective and stable for the given test problems.     

求解非线性等式和不等式约束优化问题的三项记忆梯度广义投影算法

利用广义投影矩阵，对求解无约束规划的三项记忆梯度算法中的参数给一条件，确定它们的取值范围，以保证得到目标函数的三项记忆梯度广义投影下降方向，建立了求解非线性等式和不等式约束优化问题的三项记忆梯度广义投影算法，并证明了算法的收敛性．同时给出了结合FR，PR，HS共轭梯度参数的三项记忆梯度广义投影算法，从而将经典的共轭梯度算法推广用于求解约束规划问题．数值例子表明算法是有效的．     

A proximal augmented Lagrangian method for equilibrium problems

Considering a recently proposed proximal point method for equilibrium problems, we construct an augmented Lagrangian method for solving the same problem in reflexive Banach spaces with cone constraints generating a strongly convergent sequence to a certain solution of the problem. This is an inexact hybrid method meaning that at a certain iterate, a solution of an unconstrained equilibrium problem is found, allowing a proper error bound, followed by a Bregman projection of the initial iterate onto the intersection of two appropriate halfspaces. Assuming a set of reasonable hypotheses, we provide a full convergence analysis.     

A computational method for minimization with nonlinear constraints

An iterative technique is developed to solve the problem of minimizing a functionf(y) subject to certain nonlinear constraintsg(y)=0. The variables are separated into the basic variablesx and the independent variablesu. Each iteration consists of a gradient phase and a restoration phase. The gradient phase involves a movement (on a surface that is linear in the basic variables and nonlinear in the independent variables) from a feasible point to a varied point in a direction based on the reduced gradient. The restoration phase involves a movement (in a hyperplane parallel tox-space) from the nonfeasible varied point to a new feasible point.The basic scheme is further modified to implement the method of conjugate gradients. The work required in the restoration phase is considerably reduced when compared with the existing methods.     

带扰动项的FR共轭梯度法

本文提出了两种搜索方向带有扰动项的Fletcher-Reeves (abbr. FR)共轭梯度法.其迭代公式为xk 1=xk αk(sk ωk),其中sk由共轭梯度迭代公式确定,ωk为扰动项,αk采用线搜索确定而不是必须趋于零.我们在很一般的假设条件下证明了两种算法的全局收敛性,而不需要目标函数有下界或水平集有界等有界性条件.