Local convergence analysis of the Gauss-Newton method under a majorant condition

The Gauss-Newton method for solving nonlinear least squares problems is studied in this paper. Under the hypothesis that the derivative of the function associated with the least square problem satisfies a majorant condition, a local convergence analysis is presented. This analysis allows us to obtain the optimal convergence radius and the biggest range for the uniqueness of stationary point, and to unify two previous and unrelated results.     

Local convergence of Newton’s method under majorant condition

A local convergence analysis of Newton’s method for solving nonlinear equations, under a majorant condition, is presented in this paper. Without assuming convexity of the derivative of the majorant function, which relaxes the Lipschitz condition on the operator under consideration, convergence, the biggest range for uniqueness of the solution, the optimal convergence radius and results on the convergence rate are established. Besides, two special cases of the general theory are presented as applications.     

The convergence analysis of inexact Gauss–Newton methods for nonlinear problems

In this paper, inexact Gauss–Newton methods for nonlinear least squares problems are studied. Under the hypothesis that derivative satisfies some kinds of weak Lipschitz conditions, the local convergence properties of inexact Gauss–Newton and inexact Gauss–Newton like methods for nonlinear problems are established with the modified relative residual control. The obtained results can provide an estimate of convergence ball for inexact Gauss–Newton methods.     

Local convergence analysis of inexact Newton-like methods under majorant condition

We present a local convergence analysis of inexact Newton-like methods for solving nonlinear equations under majorant conditions. This analysis provides an estimate of the convergence radius and a clear relationship between the majorant function, which relaxes the Lipschitz continuity of the derivative, and the nonlinear operator under consideration. It also allow us to obtain some important special cases.     

Convergence of the Gauss-Newton method for convex composite optimization problems under majorant condition on Riemannian manifolds

In this paper, we consider convex composite optimization problems on Riemannian manifolds, and discuss the semi-local convergence of the Gauss-Newton method with quasi-regular initial point and under the majorant condition. As special cases, we also discuss the convergence of the sequence generated by the Gauss-Newton method under Lipschitz-type condition, or under γ-condition.     

A new semi-local convergence theorem for the inexact Newton methods

We present a new semi-local convergence theorem for the inexact Newton methods in the assumption that the derivative satisfies some kind of weak Lipschitz conditions. As special cases of our main result we re-obtain some well-known convergence theorems for Newton methods.     

Local results for the Gauss-Newton method on constrained rank-deficient nonlinear least squares

A nonlinear least squares problem with nonlinear constraints may be ill posed or even rank-deficient in two ways. Considering the problem formulated as subject to the constraints , the Jacobian and/or the Jacobian , , may be ill conditioned at the solution.

We analyze the important special case when and/or do not have full rank at the solution. In order to solve such a problem, we formulate a nonlinear least norm problem. Next we describe a truncated Gauss-Newton method. We show that the local convergence rate is determined by the maximum of three independent Rayleigh quotients related to three different spaces in .

Another way of solving an ill-posed nonlinear least squares problem is to regularize the problem with some parameter that is reduced as the iterates converge to the minimum. Our approach is a Tikhonov based local linear regularization that converges to a minimum norm problem. This approach may be used both for almost and rank-deficient Jacobians.

Finally we present computational tests on constructed problems verifying the local analysis.

     

Inexact Gauss-Newton like methods for injective-overdetermined systems of equations under a majorant condition

In this paper, inexact Gauss-Newton like methods for solving injective-overdetermined systems of equations are studied. We use a majorant condition, defined by a function whose derivative is not necessarily convex, to extend and improve several existing results on the local convergence of the Gauss-Newton methods. In particular, this analysis guarantees the convergence of the methods for two important new cases.     

Local convergence analysis of tensor methods for nonlinear equations

Tensor methods for nonlinear equations base each iteration upon a standard linear model, augmented by a low rank quadratic term that is selected in such a way that the mode is efficient to form, store, and solve. These methods have been shown to be very efficient and robust computationally, especially on problems where the Jacobian matrix at the root has a small rank deficiency. This paper analyzes the local convergence properties of two versions of tensor methods, on problems where the Jacobian matrix at the root has a null space of rank one. Both methods augment the standard linear model by a rank one quadratic term. We show under mild conditions that the sequence of iterates generated by the tensor method based upon an ideal tensor model converges locally and two-step Q-superlinearly to the solution with Q-order 3/2, and that the sequence of iterates generated by the tensor method based upon a practial tensor model converges locally and three-step Q-superlinearly to the solution with Q-order 3/2. In the same situation, it is known that standard methods converge linearly with constant converging to 1/2. Hence, tensor methods have theoretical advantages over standard methods. Our analysis also confirms that tensor methods converge at least quadratically on problems where the Jacobian matrix at the root is nonsingular.This paper is dedicated to Phil Wolfe on the occasion of his 65th birthday.Research supported by AFOSR grant AFOSR-90-0109, ARO grant DAAL 03-91-G-0151, NSF grants CCR-8920519 CCR-9101795.     

On the convergence of inexact block coordinate descent methods for constrained optimization

We consider the problem of minimizing a smooth function over a feasible set defined as the Cartesian product of convex compact sets. We assume that the dimension of each factor set is huge, so we are interested in studying inexact block coordinate descent methods (possibly combined with column generation strategies). We define a general decomposition framework where different line search based methods can be embedded, and we state global convergence results. Specific decomposition methods based on gradient projection and Frank–Wolfe algorithms are derived from the proposed framework. The numerical results of computational experiments performed on network assignment problems are reported.     

A nonsmooth inexact Newton method for the solution of large-scale nonlinear complementarity problems

A new algorithm for the solation of large-scale nonlinear complementarity problems is introduced. The algorithm is based on a nonsmooth equation reformulation of the complementarity problem and on an inexact Levenberg-Marquardt-type algorithm for its solution. Under mild assumptions, and requiring only the approximate solution of a linear system at each iteration, the algorithm is shown to be both globally and superlinearly convergent, even on degenerate problems. Numerical results for problems with up to 10 000 variables are presented. Partially supported by Agenzia Spaziale Italiana, Roma, Italy.     

On the convergence of an inexact Newton-type method

In this paper we give local convergence results of an inexact Newton-type method for monotone equations under a local error bound condition. This condition may hold even for problems with non-isolated solutions, and it therefore is weaker than the standard non-singularity condition.     

A convergence theorem for the Newton-like methods under some kind of weak Lipschitz conditions

We provide sufficient conditions for the convergence of the Newton-like methods in the assumption that the derivative satisfies some kind of weak Lipschitz conditions. Consequently, some important convergence theorems follow from our main result in this paper.     

On the local convergence of a deformed Newton's method under Argyros-type condition

For the iteration which was independently proposed by King [R.F. King, Tangent method for nonlinear equations, Numer. Math. 18 (1972) 298-304] and Werner [W. Werner, Über ein Verfarhren der Ordnung zur Nullstellenbestimmung, Numer. Math. 32 (1979) 333-342] for solving a nonlinear operator equation in Banach space, we established a local convergence theorem under the condition which was introduced recently by Argyros [I.K. Argyros, A unifying local-semilocal convergence analysis and application for two-point Newton-like methods in Banach space, J. Math. Anal. Appl. 298 (2004) 374-397].     

Local convergence of quasi-Newton methods for B-differentiable equations

We study local convergence of quasi-Newton methods for solving systems of nonlinear equations defined by B-differentiable functions. We extend the classical linear and superlinear convergence results for general quasi-Newton methods as well as for Broyden's method. We also show how Broyden's method may be applied to nonlinear complementarity problems and illustrate its computational performance on two small examples.     

Factorized quasi-Newton methods for nonlinear least squares problems

This paper provides a modification to the Gauss—Newton method for nonlinear least squares problems. The new method is based on structured quasi-Newton methods which yield a good approximation to the second derivative matrix of the objective function. In particular, we propose BFGS-like and DFP-like updates in a factorized form which give descent search directions for the objective function. We prove local and q-superlinear convergence of our methods, and give results of computational experiments for the BFGS-like and DFP-like updates.This work was supported in part by the Grant-in-Aid for Encouragement of Young Scientists of the Japanese Ministry of Education: (A)61740133 and (A)62740137.     

Hansen Patrick迭代的局部收敛性

讨论了Hansen-Patrick迭代的局部特征关系式,引入函数T（t),利用逐步归纳技巧,证明了在α为一定条件下Hansen-Patrick迭代过程对方程f(z)=0零点的局部收敛性。     

On the convergence of inexact augmented Lagrangian methods for problems with convex constraints

We consider the augmented Lagrangian method (ALM) for constrained optimization problems in the presence of convex inequality and convex abstract constraints. We focus on the case where the Lagrangian sub-problems are solved up to approximate stationary points, with increasing accuracy. We analyze two different criteria of approximate stationarity for the sub-problems and we prove the global convergence to stationary points of ALM in both cases.     

Local convergence results of Gauss-Newton's like method in weak conditions

In this paper, we study the convergence of Gauss-Newton's like method for nonlinear least squares problems. Under the hypothesis that derivative satisfies some kind of weak Lipschitz condition, we obtain the sharp estimates of the radii of convergence ball of Gauss-Newton's like method and the uniqueness ball of the solution.