Global Convergence of the Broyden‘’s Class of Quasi—Newton Methods with Nonmonotone Linesearch

In this paper,the Broyden class of quasi-Newton methods for unconstrained optimization is inves-tigated.Non-monotone linesearch procedure is introduced,which is combined with the Broyden‘s class.Under the convexity assumption on objective function,the global convergence of the Broyden‘s class is proved.     

Quadratic and Superlinear Convergence of the Huschens Method for Nonlinear Least Squares Problems

This paper is concerned with quadratic and superlinear convergence of structured quasi-Newton methods for solving nonlinear least squares problems. These methods make use of a special structure of the Hessian matrix of the objective function. Recently, Huschens proposed a new kind of structured quasi-Newton methods and dealt with the convex class of the structured Broyden family, and showed its quadratic and superlinear convergence properties for zero and nonzero residual problems, respectively. In this paper, we extend the results by Huschens to a wider class of the structured Broyden family. We prove local convergence properties of the method in a way different from the proof by Huschens.     

修改Broyden非凸族在一般Wolfe搜索下的收敛性

近来,韦等提出了一类新的拟牛顿方程B_(k 1)S_k=y_k~*=y_k A_kS_k,A_k为一矩阵,并在此基础上给出了两种类型的修改Broyden族(MBC)．作者利用一般Wolfe搜索技术,与修改Broyden族相结合,证明了在适当的条件下修改Broyden非凸族具有全局收敛性和超线性收敛速度．     

成组Broyden修正矩阵的紧凑形式与成组记忆修正算法

1 引言 成组型线性方程组 其中,p是适中的数值,由于其有相当的实际应用背景,人们一直在研究有效的数值方法,特别是近年来,实际问题中归结出来的成组型方程组,其规模越来越大,又具有稀疏结构,因而使用迭代法是一种有效的途径,目前使用比较多的是Krylov子空间方法中的Lanczos方法,CG方法,GMRES方法等等。这种成组型算法的建立,其基本出发点是使算法具有较少的计算量和存储量,具体体现在: 1)成组型算法在应用于问题(1.1)的求解时,也具有有限终止性性质,而其终止步数一般要比单个型算法的步数减少了户倍,由于成组型算法每迭代一步的计算量基本上等同于单个型算法使用户次的计算量,如此,算法的计算量会有明显的改善。 2)当A存储在二级(secondary)内存时,在迭代计算时需要不断地进行存取交换,由于成组型算法的迭代步数减少了户倍,如此,用在这种交换的时间也要减少户倍,相当有效。 3)由于在成组型算法中,出现的多是AX的形式,其中,故成组型算法便于计算并行化。 4)即使用于求解单个方程组,当A的少数几个极端特征值分离甚远时,这种成组型算法也有可能改善其收敛速度,如成组型的CG方法。 目前,这种成组型算法已体现出很大的实用计算价值,然而其进一步的理论分析还有待深入研究。     

A globally convergent BFGS method for nonlinear monotone equations without any merit functions

Since 1965, there has been significant progress in the theoretical study on quasi-Newton methods for solving nonlinear equations, especially in the local convergence analysis. However, the study on global convergence of quasi-Newton methods is relatively fewer, especially for the BFGS method. To ensure global convergence, some merit function such as the squared norm merit function is typically used. In this paper, we propose an algorithm for solving nonlinear monotone equations, which combines the BFGS method and the hyperplane projection method. We also prove that the proposed BFGS method converges globally if the equation is monotone and Lipschitz continuous without differentiability requirement on the equation, which makes it possible to solve some nonsmooth equations. An attractive property of the proposed method is that its global convergence is independent of any merit function.We also report some numerical results to show efficiency of the proposed method.

     

Partitioned quasi-Newton methods for sparse nonlinear equations

In this paper, we present two partitioned quasi-Newton methods for solving partially separable nonlinear equations. When the Jacobian is not available, we propose a partitioned Broyden’s rank one method and show that the full step partitioned Broyden’s rank one method is locally and superlinearly convergent. By using a well-defined derivative-free line search, we globalize the method and establish its global and superlinear convergence. In the case where the Jacobian is available, we propose a partitioned adjoint Broyden method and show its global and superlinear convergence. We also present some preliminary numerical results. The results show that the two partitioned quasi-Newton methods are effective and competitive for solving large-scale partially separable nonlinear equations.     

MIMD机上解非线性方程组的同步化并行拟Newton法

本文给出了适于在MIMD机上解非线性方程组的同步化并行Broyden方法和换列修正拟Newton法的迭代格式,以及它们的局部收敛性定理.数值试验结果也验证了收敛性.     

求解非线性互补问题的逐次逼近拟牛顿法的收敛性分析

提出了求解非线性互补问题的一个逐次逼近拟牛顿算法。在适当的假设下,证明了该算法的全局收敛性和局部超线性收敛性。     

Local Convergence of the Affine-Scaling Interior-Point Algorithm for Nonlinear Programming

This paper addresses the local convergence properties of the affine-scaling interior-point algorithm for nonlinear programming. The analysis of local convergence is developed in terms of parameters that control the interior-point scheme and the size of the residual of the linear system that provides the step direction. The analysis follows the classical theory for quasi-Newton methods and addresses q-linear, q-superlinear, and q-quadratic rates of convergence.     

Disguised and new quasi-Newton methods for nonlinear eigenvalue problems

In this paper, we take a quasi-Newton approach to nonlinear eigenvalue problems (NEPs) of the type M(λ)v =?0, where \(M:\mathbb {C}\rightarrow \mathbb {C}^{n\times n}\) is a holomorphic function. We investigate which types of approximations of the Jacobian matrix lead to competitive algorithms, and provide convergence theory. The convergence analysis is based on theory for quasi-Newton methods and Keldysh’s theorem for NEPs. We derive new algorithms and also show that several well-established methods for NEPs can be interpreted as quasi-Newton methods, and thereby, we provide insight to their convergence behavior. In particular, we establish quasi-Newton interpretations of Neumaier’s residual inverse iteration and Ruhe’s method of successive linear problems.     

On superlinear convergence of quasi-Newton methods for nonsmooth equations

We show that strong differentiability at solutions is not necessary for superlinear convergence of quasi-Newton methods for solving nonsmooth equations. We improve the superlinear convergence result of Ip and Kyparisis for general quasi-Newton methods as well as the Broyden method. For a special example, the Newton method is divergent but the Broyden method is superlinearly convergent.     

Globally Convergent Inexact Quasi-Newton Methods for Solving Nonlinear Systems

Large scale nonlinear systems of equations can be solved by means of inexact quasi-Newton methods. A global convergence theory is introduced that guarantees that, under reasonable assumptions, the algorithmic sequence converges to a solution of the problem. Under additional standard assumptions, superlinear convergence is preserved.     

A BFGS algorithm for solving symmetric nonlinear equations

In this article, we propose a BFGS method for solving symmetric nonlinear equations. The presented method possesses some favourable properties: (a) the generated sequence of iterates is norm descent; (b) the generated sequence of the quasi-Newton matrix is positive definite and (c) this method possesses the global convergence and superlinear convergence. Numerical results show that the presented method is interesting.     

Adjoint-based quasi-Newton methods for partially separable problems

In this paper the concepts of partitioned quasi-Newton methods are applied to adjoint Broyden updates. Consequently a corresponding partitioned adjoint Broyden update is presented and local convergence results are given. Numerical results compare the partitioned adjoint Broyden update methods to the corresponding unpartitioned quasi-Newton method and to Newton's method for nonlinear equations. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

A new smoothing quasi-Newton method for nonlinear complementarity problems

A new smoothing quasi-Newton method for nonlinear complementarity problems is presented. The method is a generalization of Thomas’ method for smooth nonlinear systems and has similar properties as Broyden's method. Local convergence is analyzed for a strictly complementary solution as well as for a degenerate solution. Presented numerical results demonstrate quite similar behavior of Thomas’ and Broyden's methods.     

Broyden's method in Hilbert space

Broyden's method is formulated for the solution of nonlinear operator equations in Hilbert spaces. The algorithm is proven to be well defined and a linear rate of convergence is shown. Under an additional assumption on the initial approximation for the derivative we prove the superlinear rate of convergence.     

Rates of superlinear convergence for classical quasi-Newton methods

Mathematical Programming - We study the local convergence of classical quasi-Newton methods for nonlinear optimization. Although it was well established a long time ago that asymptotically these...     

Quasi-Newton Methods for Nonlinear Least Squares Focusing on Curvatures

A new quasi-Newton method for nonlinear least squares problems is proposed. Two advantages of the method are accomplished by utilizing special geometrical properties in the problem class. First, fast convergence is established for well-conditioned problems by interpolating both the current and the previous step in each iteration. Second, high accuracy is achieved for certain difficult problems, such as ill-conditioned problems and problems with large curvatures in the tangent space. Numerical results for artificial problems and standard test problems are presented and discussed.     

On the limited memory BFGS method for large scale optimization

We study the numerical performance of a limited memory quasi-Newton method for large scale optimization, which we call the L-BFGS method. We compare its performance with that of the method developed by Buckley and LeNir (1985), which combines cycles of BFGS steps and conjugate direction steps. Our numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence. We show that the L-BFGS method can be greatly accelerated by means of a simple scaling. We then compare the L-BFGS method with the partitioned quasi-Newton method of Griewank and Toint (1982a). The results show that, for some problems, the partitioned quasi-Newton method is clearly superior to the L-BFGS method. However we find that for other problems the L-BFGS method is very competitive due to its low iteration cost. We also study the convergence properties of the L-BFGS method, and prove global convergence on uniformly convex problems.This work was supported by the Applied Mathematical Sciences subprogram of the Office of Energy Research, U.S. Department of Energy, under contract DE-FG02-87ER25047, and by National Science Foundation Grant No. DCR-86-02071.     

Operations research and optimization (ORO)

We have recently proposed a structured algorithm for solving constrained nonlinear least-squares problems and established its local two-step Q-superlinear convergence rate. The approach is based on an earlier adaptive structured scheme due to Mahdavi-Amiri and Bartels of the exact penalty method. The structured adaptation makes use of the ideas of Nocedal and Overton for handling quasi-Newton updates of projected Hessians and adapts a structuring scheme due to Engels and Martinez. For robustness, we have employed a specific nonsmooth line search strategy, taking account of the least-squares objective. Numerical results also confirm the practical relevance of our special considerations for the inherent structure of the least squares. Here, we establish global convergence of the proposed algorithm using a weaker condition than the one used by the exact penalty method of Coleman and Conn for general nonlinear programs.