A nonmonotone truncated Newton–Krylov method exploiting negative curvature directions, for large scale unconstrained optimization

We propose a new truncated Newton method for large scale unconstrained optimization, where a Conjugate Gradient (CG)-based technique is adopted to solve Newton’s equation. In the current iteration, the Krylov method computes a pair of search directions: the first approximates the Newton step of the quadratic convex model, while the second is a suitable negative curvature direction. A test based on the quadratic model of the objective function is used to select the most promising between the two search directions. Both the latter selection rule and the CG stopping criterion for approximately solving Newton’s equation, strongly rely on conjugacy conditions. An appropriate linesearch technique is adopted for each search direction: a nonmonotone stabilization is used with the approximate Newton step, while an Armijo type linesearch is used for the negative curvature direction. The proposed algorithm is both globally and superlinearly convergent to stationary points satisfying second order necessary conditions. We carry out a significant numerical experience in order to test our proposal.     

非拟牛顿非凸族的收敛性

１．引言 对于无约束最优化问题拟牛顿法是目前最成熟,应用最广泛的解法之一．近二十多年来,对拟牛顿法收敛性质的研究一直是非线性最优化算法理论研究的热点．带非精确搜索的拟牛顿算法的研究是从１９７６年 Ｐｏｗｅｌｌ［１］开始,他证明了带 Ｗｏｌｆｅ搜索 ＢＦＧＳ算法的全局收敛性和超线性收敛性． １９７８年 Ｂｙｒｄ, Ｎｏｃｅｄａｌ; Ｙａ－Ｘｉａｎｇ Ｙｕａｎ［３］成功地将 Ｐｏｗｅｌｌ的结果推广到限制的 Ｂｒｏｓｄｅｎ凸族． １９８９年, Ｎｏｃｅｄａｌ［４］在目标函数一致凸的条件下,证明了带回追搜索的ＢＦＧ…     

A Linearly Convergent Derivative-Free Descent Method for Strongly Monotone Complementarity Problems

We establish the first rate of convergence result for the class of derivative-free descent methods for solving complementarity problems. The algorithm considered here is based on the implicit Lagrangian reformulation [26, 35] of the nonlinear complementarity problem, and makes use of the descent direction proposed in [42], but employs a different Armijo-type linesearch rule. We show that in the strongly monotone case, the iterates generated by the method converge globally at a linear rate to the solution of the problem.     

Descent method with inexact linesearch for mixed variational inequalities

We propose a descent method with respect to a merit function for the mixed variational inequality involving a general nonlinear mapping and a convex, but not necessarily differentiable function. The method utilizes an inexact linesearch procedure. Its convergence is proved under the additional assumptions of continuity and strong monotonicity of the cost mapping.     

非凸函数极小问题的BFGS算法

本对于非凸函数的无约束优化问题，给出一类修正的BFGS算法。算法的思想是对非凸函数的近似Hesse矩阵进行修正，得到下降方向，并且保证拟牛顿条件成立，当步长采用线性搜索一般模型时，证明了该算法的局部收敛性。     

A block coordinate variable metric linesearch based proximal gradient method

In this paper we propose an alternating block version of a variable metric linesearch proximal gradient method. This algorithm addresses problems where the objective function is the sum of a smooth term, whose variables may be coupled, plus a separable part given by the sum of two or more convex, possibly nonsmooth functions, each depending on a single block of variables. Our approach is characterized by the possibility of performing several proximal gradient steps for updating every block of variables and by the Armijo backtracking linesearch for adaptively computing the steplength parameter. Under the assumption that the objective function satisfies the Kurdyka-?ojasiewicz property at each point of its domain and the gradient of the smooth part is locally Lipschitz continuous, we prove the convergence of the iterates sequence generated by the method. Numerical experience on an image blind deconvolution problem show the improvements obtained by adopting a variable number of inner block iterations combined with a variable metric in the computation of the proximal operator.     

Using approximate secant equations in limited memory methods for multilevel unconstrained optimization

The properties of multilevel optimization problems defined on a hierarchy of discretization grids can be used to define approximate secant equations, which describe the second-order behavior of the objective function. Following earlier work by Gratton and Toint (2009) we introduce a quasi-Newton method (with a linesearch) and a nonlinear conjugate gradient method that both take advantage of this new second-order information. We then present numerical experiments with these methods and formulate recommendations for their practical use.     

Quasi-Newton methods for constrained nonlinear systems: complexity analysis and applications

We address the solution of constrained nonlinear systems by new linesearch quasi-Newton methods. These methods are based on a proper use of the projection map onto the convex constraint set and on a derivative-free and nonmonotone linesearch strategy. The convergence properties of the proposed methods are presented along with a worst-case iteration complexity bound. Several implementations of the proposed scheme are discussed and validated on bound-constrained problems including gas distribution network models. The results reported show that the new methods are very efficient and competitive with an existing affine-scaling procedure.     

无约束最优化线搜索一般模型及BFGS方法的整体收敛性

本文给出了无约束最优化的算法中线性搜索的可接受的步长选择律的一种一般形式，它概括了大多数已有的步长律为其特例，并且研究了它基本性质，最后证明了此线性搜索一般模拟相结合的无约束优化的ＢＦＧＳ算法的整体收敛性。     

Global Convergence of the Broyden‘’s Class of Quasi—Newton Methods with Nonmonotone Linesearch

In this paper,the Broyden class of quasi-Newton methods for unconstrained optimization is inves-tigated.Non-monotone linesearch procedure is introduced,which is combined with the Broyden‘s class.Under the convexity assumption on objective function,the global convergence of the Broyden‘s class is proved.     

Global convergence of an SQP method without boundedness assumptions on any of the iterative sequences

Usual global convergence results for sequential quadratic programming (SQP) algorithms with linesearch rely on some a priori assumptions about the generated sequences, such as boundedness of the primal sequence and/or of the dual sequence and/or of the sequence of values of a penalty function used in the linesearch procedure. Different convergence statements use different combinations of assumptions, but they all assume boundedness of at least one of the sequences mentioned above. In the given context boundedness assumptions are particularly undesirable, because even for non-pathological and well-behaved problems the associated penalty functions (whose descent is used to produce primal iterates) may not be bounded below for any value of the penalty parameter. Consequently, boundedness assumptions on the iterates are not easily justifiable. By introducing a very simple and computationally cheap safeguard in the linesearch procedure, we prove boundedness of the primal sequence in the case when the feasible set is nonempty, convex, and bounded. If, in addition, the Slater condition holds, we obtain a complete global convergence result without any a priori assumptions on the iterative sequences. The safeguard consists of not accepting a further increase of constraints violation at iterates which are infeasible beyond a chosen threshold, which can always be ensured by the proposed modified SQP linesearch criterion. The author is supported in part by CNPq Grants 301508/2005-4, 490200/2005-2, 550317/2005-8, by PRONEX–Optimization, and by FAPERJ Grant E-26/151.942/2004.     

无单调性集值变分不等式的一种投影算法

本文给出了求解无单调性集值变分不等式的一个新的投影算法,该算法所产生的迭代序列在Minty变分不等式解集非空且映射满足一定的连续性条件下收敛到解.对比文献[10]中的算法,本文中的算法使用了不同的线性搜索和半空间,在计算本文所引的两个数值例子时,该算法比文献[10]中的算法所需迭代步更少.     

Derivative-free methods for bound constrained mixed-integer optimization

We consider the problem of minimizing a continuously differentiable function of several variables subject to simple bound constraints where some of the variables are restricted to take integer values. We assume that the first order derivatives of the objective function can be neither calculated nor approximated explicitly. This class of mixed integer nonlinear optimization problems arises frequently in many industrial and scientific applications and this motivates the increasing interest in the study of derivative-free methods for their solution. The continuous variables are handled by a linesearch strategy whereas to tackle the discrete ones we employ a local search-type approach. We propose different algorithms which are characterized by the way the current iterate is updated and by the stationarity conditions satisfied by the limit points of the sequences they produce.     

A new linesearch algorithm for nonlinear least squares problems

A linesearch (steplength) algorithm for unconstrained nonlinear least squares problems is described. To estimate the steplength inside the linesearch algorithm a new method that interpolates the residual vector is used together with a standards method that interpolates the sums of squares. Numerical results are reported that point out the advantage with the new steplength estimation method.     

带扰动项的FR共轭梯度法

本文提出了两种搜索方向带有扰动项的Fletcher-Reeves (abbr. FR)共轭梯度法.其迭代公式为xk 1=xk αk(sk ωk),其中sk由共轭梯度迭代公式确定,ωk为扰动项,αk采用线搜索确定而不是必须趋于零.我们在很一般的假设条件下证明了两种算法的全局收敛性,而不需要目标函数有下界或水平集有界等有界性条件.     

Hybrid projection methods for equilibrium problems with non‐Lipschitz type bifunctions

Based on the extended extragradient‐like method and the linesearch technique, we propose three projection methods for finding a common solution of a finite family of equilibrium problems. The linesearch used in the proposed algorithms has allowed to reduce some conditions imposed on equilibrium bifunctions. The strongly convergent theorems are established without the Lipschitz‐type condition of bifunctions. The paper also helps in the design and analysis of practical algorithms and gives us a generalization of some previously known problems. Copyright © 2017 John Wiley & Sons, Ltd.     

Stopping criteria for linesearch methods without derivatives

In this paper acceptability criteria for the linesearch stepsize are introduced which require only function values. Simple algorithm models based on these criteria are presented. Some modifications of criteria based on the knowledge of the directional derivative are also illustrated. This paper was written while this author was visiting CRAI.     

A New Nonmonotone Linesearch SQP Algorithm for Unconstrained Minimax Problem

In this article, unconstrained minimax problems are discussed, and a sequential quadratic programming (SQP) algorithm with a new nonmonotone linesearch is presented. At each iteration, a search direction of descent is obtained by solving a quadratic programming (QP). To circumvent the Maratos effect, a high-order correction direction is achieved by solving another QP and a new nonmonotone linesearch is performed. Under reasonable conditions, the global convergence and the rate of superlinear convergence are established. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.     

An Armijo-type method for pseudomonotone equilibrium problems and its applications

We present a new algorithm for solving equilibrium problems, where the underlying bifunctions are pseudomonotone and not necessarily Lipschitz-type continuous. The algorithm is based on the auxiliary problem principle and the Armijo-type linesearch techniques. Convergence properties of the algorithms are established, among them the global convergence is proved under few assumptions. Applications to generalized variational inequalities and some numerical results are reported.     

一类非光滑凸优化问题的邻近梯度算法

考虑求解目标函数为光滑损失函数与非光滑正则函数之和的凸优化问题的一种基于线搜索的邻近梯度算法及其收敛性分析,证明了在梯度局部Lipschitz连续条件下该算法是R-线性收敛的,并在非光滑部分为稀疏块LASSO正则函数情况下给出了误差界条件成立的证明,得到了线性收敛率.最后,数值实验结果验证了方法的有效性.