Quadratic and Superlinear Convergence of the Huschens Method for Nonlinear Least Squares Problems

This paper is concerned with quadratic and superlinear convergence of structured quasi-Newton methods for solving nonlinear least squares problems. These methods make use of a special structure of the Hessian matrix of the objective function. Recently, Huschens proposed a new kind of structured quasi-Newton methods and dealt with the convex class of the structured Broyden family, and showed its quadratic and superlinear convergence properties for zero and nonzero residual problems, respectively. In this paper, we extend the results by Huschens to a wider class of the structured Broyden family. We prove local convergence properties of the method in a way different from the proof by Huschens.     

A robust secant method for optimization problems with inequality constraints

This paper presents a secant method, based on R. B. Wilson's formula for the solution of optimization problems with inequality constraints. Global convergence properties are ensured by grafting the secant method onto a phase I - phase II feasible directions method, using a rate of convergence test for crossover control.This research was sponsored by the National Science Foundation, Grant No. ENG-73-08214 and Grant No. (RANN)-ENV-76-04264, and by the Joint Services Electronics Program. Contract No. F44620-76-C-0100.     

Family of optimally conditioned quasi-Newton updates for unconstrained optimization

In this note, a general optimal conditioning problem for updates which satisfy the quasi-Newton equation is solved. The new solution is a family of updates which contains other known optimally conditioned updates but also includes new formulas of increased rank. A new factorization formula for the Broyden family and some preliminary numerical results are also given.     

Variational quasi-Newton methods for unconstrained optimization

In this paper, we propose new members of the Broyden family of quasi-Newton methods. We develop, on the basis of well-known least-change results for the BFGS and DFP updates, a measure for the Broyden family which seeks to take into account the change in both the Hessian approximation and its inverse. The proposal is then to choose the formula which gives the least value of this measure in terms of the two parameters available, and hence to produce an update which is optimal in the sense of the given measure. Several approaches to the problem of minimizing the measure are considered, from which new updates are obtained. In particular, one approach yields a new variational result for the Davidon optimally conditioned method and another yields a reasonable modification to this method. The paper is also concerned with the possibility of estimating, in a certain sense, the size of the eigenvalues of the Hessian approximation on the basis of two available scalars. This allows one to derive further modifications to the above-mentioned methods. Comparisons with the BFGS and Davidson methods are made on a set of standard test problems that show promising results for certain new methods.Part of this work was done during the author's visits at International Centre for Theoretical Physics, Trieste, Italy, at Systems Department, University of Calabria, Cosenza, Italy, and at Ajman University College of Science and Technology, Ajman, United Arab Emirates.The author expresses his gratitude to Professor L. Grandinetti for his encouragement and thanks the anonymous referees for their careful reading of an earlier draft of the paper and valuable comments, which led to a substantial improvement of the original paper.     

A CLASS OF COLLINEAR SCALING ALGORITHMS FOR UNCONSTRAINED OPTIMIZATON

A Class of Collinear Scaling Algorithms for Unconstrained Optimization. An appealing approach to the solution of nonlinear optimization problems based on conic models of the objective function has been in troduced by Davidon (1980). It leads to a broad class of algorithms which can be considered to generalize the existing quasi-Newton methods. One particular member of this class has been deeply discussed by Sorensen (1980), who has proved some interesting theoretical properties. In this paper, we generalize Sorensen's technique to Spedicato three-parameter family of variable-metric updates. Furthermore, we point out that the collinear scaling three- parameter family is essentially equivalent to the Spedicato three-parameter family. In addition, numerical expriments have been carried out to compare some colliner scaling algorithms with a straightforward implementation of the BFGS quasi-Newton method.     

A CHARACTERIZATION OF SELF-SCALING RANK ONE UPDATE

An analysis on the property of Self-Scaling Rank One Update (SSR1) is given. In view of it, we explore whether the SSR1 update without using exact line search possesses the finite termination property when it is applied to minimize a positive definite quadratic objective function.     

Degenerate values for broyden methods

A Broyden method fails when the free parameter takes on a degenerate value; one such value is well known, but this paper shows that many others exist in general. These values have practical significance if the initial Hessian approximation is indefinite. The BFS formula is special in that it avoids these degenerate values. Properties about how the new degenerate values behave and their relationship to the well-known degenerate value are described. A new result is used about a reduced inverse Hessian method, which is equivalent to a Broyden method but is parameter free and provides a simple proof of Dixon's theorem.     

Banach空间中非光滑算子方程的光滑化拟牛顿法

研究Banach空间中非光滑算子方程的光滑化拟牛顿法.构造光滑算子逼近非光滑算子,在光滑逼近算子满足方向可微相容性的条件下,证明了光滑化拟牛顿法具有局部超线性收敛性质.应用说明了算法的有效性.     

A New Family of Trust Region Algorithms for Unconstrained Optimization

Trust region(TR)algorithms are a class of recently developed alogrthms for nonlinear optimization.A new family of TR algorithms for unconstrained optimization,which is the extension of the usual TR method,is pressented in this paper.When the objective function is bounded below and continuously differentiable,and the norm of the Hesse approximations increases at most linearly with the iteration number,we prove the global convergence of the algorithms.Limited numerical results are repoted,which indicate that our new TR algorithm is competitive.     

A Survey of Quasi-Newton Equations and Quasi-Newton Methods for Optimization

Quasi-Newton equations play a central role in quasi-Newton methods for optimization and various quasi-Newton equations are available. This paper gives a survey on these quasi-Newton equations and studies properties of quasi-Newton methods with updates satisfying different quasi-Newton equations. These include single-step quasi-Newton equations that use only gradient information and that use both gradient and function value information in one step, and multi-step quasi-Newton equations that use the gradient information in last m steps. Main properties of quasi-Newton methods with updates satisfying different quasi-Newton equations are studied. These properties include the finite termination property, invariance, heredity of positive definite updates, consistency of search directions, global convergence and local superlinear convergence properties.