Accelerated convergence for the Powell/Hestenes multiplier method

It is known that augmented Lagrangian or multiplier methods for solving constrained optimization problems can be interpreted as techniques for maximizing an augmented dual functionD _c(). For a constantc sufficiently large, by considering maximizing the augmented dual functionD _c() with respect to, it is shown that the Newton iteration for based on maximizingD _c() can be decomposed into taking a Powell/Hestenes iteration followed by a Newton-like correction. Superimposed on the original Powell/Hestenes method, a simple acceleration technique is devised to make use of information from the previous iteration. For problems with only one constraint, the acceleration technique is equivalent to replacing the second (Newton-like) part of the decomposition by a finite difference approximation. Numerical results are presented.     

An implicit function theorem

Suppose thatF:DR ⁿ×R^mRⁿ, withF(x ⁰,y ⁰)=0. The classical implicit function theorem requires thatF is differentiable with respect tox and moreover that ₁ F(x ⁰,y ⁰) is nonsingular. We strengthen this theorem by removing the nonsingularity and differentiability requirements and by replacing them with a one-to-one condition onF as a function ofx.     

Strong uniqueness and second order convergence in nonlinear discrete approximation

Summary Strong uniqueness has proved to be an important condition in demonstrating the second order convergence of the generalised Gauss-Newton method for discrete nonlinear approximation problems [4]. Here we compare strong uniqueness with the multiplier condition which has also been used for this purpose. We describe strong uniqueness in terms of the local geometry of the unit ball and properties of the problem functions at the minimum point. When the norm is polyhedral we are able to give necessary and sufficient conditions for the second order convergence of the generalised Gauss-Newton algorithm.