Rates of convergence for adaptive Newton methods

We consider Newton-type methods for constrained optimization problems in infinite-dimensional spaces, where at each iteration the first and second derivatives and the feasible set are approximated. The approximations can change at each iteration and conditions are given under which linear and superlinear rates of convergence of the iterates to the optimal point hold. Several applications are discussed.     

Global convergence for Newton methods in mathematical programming

In constrained optimization problems in mathematical programming, one wants to minimize a functionalf(x) over a given setC. If, at an approximate solutionx _n , one replacesf(x) by its Taylor series expansion through quadratic terms atx _n and denotes byx _n+1 the minimizing point for this overC, one has a direct analogue of Newton's method. The local convergence of this has been previously analyzed; here, we give global convergence results for this and the similar algorithm in which the constraint setC is also linearized at each step.This research was supported in part by the Office of Naval Research, Contract No. N00014-67-0126-0015, and was presented by invitation at the Fifth Gatlinburg Symposium on Numerical Algebra, Los Alamos, New Mexico, 1972.     

A convergence analysis for directional two-step Newton methods

We present a Kantorovich-type semilocal convergence analysis of the Newton–Josephy method for solving a certain class of variational inequalities. By using a combination of Lipschitz and center-Lipschitz conditions, and our new idea of recurrent functions, we provide an analysis with the following advantages over the earlier works (Wang 2009, Wang and Shen, Appl Math Mech 25:1291–1297, 2004) (under the same or less computational cost): weaker sufficient convergence conditions, larger convergence domain, finer error bounds on the distances involved, and an at least as precise information on the location of the solution.     

Kantorovich-type semilocal convergence analysis for inexact Newton methods

We provide a new semilocal convergence analysis for generating an inexact Newton method converging to a solution of a nonlinear equation in a Banach space setting. Our analysis is based on our idea of recurrent functions. Our results are compared favorably to earlier ones by others and us (Argyros (2007, 2009) [5] and [6], Argyros and Hilout (2009) [7], Guo (2007) [15], Shen and Li (2008) [18], Li and Shen (2008) [19], Shen and Li (2009) [20]). Numerical examples are provided to show that our results apply, but not earlier ones [15], [18], [19] and [20].     

Concerning the convergence of inexact Newton methods

In this study, we use inexact Newton methods to find solutions of nonlinear operator equations on Banach spaces with a convergence structure. Our technique involves the introduction of a generalized norm as an operator from a linear space into a partially ordered Banach space. In this way the metric properties of the examined problem can be analyzed more precisely. Moreover, this approach allows us to derive from the same theorem, on the one hand, semi-local results of Kantorovich type, and on the other hand, global results based on monotonicity considerations. By imposing very general Lipschitz-like conditions on the operators involved, on the one hand, we cover a wider range of problems, and on the other hand, by choosing our operators appropriately, we can find sharper error bounds on the distances involved than before. Furthermore, we show that special cases of our results reduce to the corresponding ones already in the literature. Finally, our results are used to solve integral equations that cannot be solved with existing methods.     

Affine invariant local convergence theorems for inexact Newton-like methods

Affine invariant sufficient conditions are given for two local convergence theorems involving inexact Newton-like methods. The first uses conditions on the first Fréchet-derivative whereas the second theorem employs hypotheses on the second. Radius of convergence as well as rate of convergence results are derived. Results involving superlinear convergence and known to be true for inexact Newton methods are extended here. Moreover, we show that under hypotheses on the second Fréchet-derivative our radius of convergence is larger than the corresponding one in [10]. This allows a wider choice for the initial guess. A numerical example is also provided to show that our radius of convergence is larger than the one in [10].     

A new semi-local convergence theorem for the inexact Newton methods

We present a new semi-local convergence theorem for the inexact Newton methods in the assumption that the derivative satisfies some kind of weak Lipschitz conditions. As special cases of our main result we re-obtain some well-known convergence theorems for Newton methods.     

Improved convergence theorems for new Hermitian and skew-Hermitian splitting methods

In this note, based on the previous work by Pour and Goughery (New Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems Numer. Algor. 69 (2015) 207–225), we further discuss this new Hermitian and skew-Hermitian splitting (described as NHSS) methods for non-Hermitian positive definite linear systems. Some new convergence conditions of the NHSS method are obtained, which are superior to the results in the above paper.     

Superlinear convergence theorems for Newton-type methods for nonlinear systems of equations

In this paper, someQ-order convergence theorems are given for the problem of solving nonlinear systems of equations when using very general finitely terminating methods for the solution of the associated linear systems. The theorems differ from those of Dembo, Eisenstat, and Steihaug in the different stopping condition and in their applicability to the nonlinear ABS algorithm.Lecture presented at the University of Bergamo, Bergamo, Italy, October 1989.     

Local convergence of tensor methods

Mathematical Programming - In this paper, we study local convergence of high-order Tensor Methods for solving convex optimization problems with composite objective. We justify local superlinear...     

Hessian averaging in stochastic Newton methods achieves superlinear convergence

Mathematical Programming - We consider minimizing a smooth and strongly convex objective function using a stochastic Newton method. At each iteration, the algorithm is given an...     

Local convergence of quasi-Newton methods for B-differentiable equations

We study local convergence of quasi-Newton methods for solving systems of nonlinear equations defined by B-differentiable functions. We extend the classical linear and superlinear convergence results for general quasi-Newton methods as well as for Broyden's method. We also show how Broyden's method may be applied to nonlinear complementarity problems and illustrate its computational performance on two small examples.     

On the convergence of inexact Newton methods for discrete-time algebraic Riccati equations

In this paper, we present a convergence analysis of the inexact Newton method for solving Discrete-time algebraic Riccati equations (DAREs) for large and sparse systems. The inexact Newton method requires, at each iteration, the solution of a symmetric Stein matrix equation. These linear matrix equations are solved approximatively by the alternating directions implicit (ADI) or Smith?s methods. We give some new matrix identities that will allow us to derive new theoretical convergence results for the obtained inexact Newton sequences. We show that under some necessary conditions the approximate solutions satisfy some desired properties such as the d-stability. The theoretical results developed in this paper are an extension to the discrete case of the analysis performed by Feitzinger et al. (2009) [8] for the continuous-time algebraic Riccati equations. In the last section, we give some numerical experiments.     

The convergence analysis of inexact Gauss–Newton methods for nonlinear problems

In this paper, inexact Gauss–Newton methods for nonlinear least squares problems are studied. Under the hypothesis that derivative satisfies some kinds of weak Lipschitz conditions, the local convergence properties of inexact Gauss–Newton and inexact Gauss–Newton like methods for nonlinear problems are established with the modified relative residual control. The obtained results can provide an estimate of convergence ball for inexact Gauss–Newton methods.     

A class of Newton’s methods with third-order convergence

In this work, a class of iterative Newton’s methods, known as power mean Newton’s methods, is proposed. Some known results can be regarded as particular cases. It is shown that the order of convergence of the proposed methods is 3. Numerical results are given to verify the theory and demonstrate the performance.     

Two fundamental convergence theorems for nonlinear conjugate gradient methods and their applications

1. IntroductionWe consider the global convergence of conjugate gradient methods for the unconstrainednonlinear optimization problemadn f(x),where f: Re - RI is continuously dtherelltiable and its gradiellt is denoted by g. Weconsider only the cajse where the methods are implemented without regular restarts. Theiterative formula is given byXk 1 = Xk Akdk, (1'1).and the seaxch direction da is defined bywhere gb is a scalar, ^k is a stenlength, and gb denotes g(xk).The best-known formulas fo…     

On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds

We consider a new algorithm, an interior-reflective Newton approach, for the problem of minimizing a smooth nonlinear function of many variables, subject to upper and/or lower bounds on some of the variables. This approach generatesstrictly feasible iterates by using a new affine scaling transformation and following piecewise linear paths (reflection paths). The interior-reflective approach does not require identification of an activity set. In this paper we establish that the interior-reflective Newton approach is globally and quadratically convergent. Moreover, we develop a specific example of interior-reflective Newton methods which can be used for large-scale and sparse problems.Research partially supported by the Applied Mathematical Sciences Research Program (KC-04-02) of the Office of Energy Research of the U.S. Department of Energy under grant DE-FG02-86ER25013.A000, and in part by NSF, AFOSR, and ONR through grant DMS-8920550, and by the Advanced Computing Research Institute, a unit of the Cornell Theory Center which receives major funding from the National Science Foundation and IBM Corporation, with additional support from New York State and members of its Corporate Research Institute.Corresponding author.     

Local and superlinear convergence for truncated iterated projections methods

Least change secant updates can be obtained as the limit of iterated projections based on other secant updates. We show that these iterated projections can be terminated or truncated after any positive number of iterations and the local and the superlinear rate of convergence are still maintained. The truncated iterated projections method is used to find sparse and symmetric updates that are locally and superlinearly convergent. A part of this paper was presented at the Third International Workshop on Numerical Analysis, IIMAS, University of Mexico, January 1981 and ORSA-TIMS National Meeting, Houston, October 1981.     

Local convergence analysis of tensor methods for nonlinear equations

Tensor methods for nonlinear equations base each iteration upon a standard linear model, augmented by a low rank quadratic term that is selected in such a way that the mode is efficient to form, store, and solve. These methods have been shown to be very efficient and robust computationally, especially on problems where the Jacobian matrix at the root has a small rank deficiency. This paper analyzes the local convergence properties of two versions of tensor methods, on problems where the Jacobian matrix at the root has a null space of rank one. Both methods augment the standard linear model by a rank one quadratic term. We show under mild conditions that the sequence of iterates generated by the tensor method based upon an ideal tensor model converges locally and two-step Q-superlinearly to the solution with Q-order 3/2, and that the sequence of iterates generated by the tensor method based upon a practial tensor model converges locally and three-step Q-superlinearly to the solution with Q-order 3/2. In the same situation, it is known that standard methods converge linearly with constant converging to 1/2. Hence, tensor methods have theoretical advantages over standard methods. Our analysis also confirms that tensor methods converge at least quadratically on problems where the Jacobian matrix at the root is nonsingular.This paper is dedicated to Phil Wolfe on the occasion of his 65th birthday.Research supported by AFOSR grant AFOSR-90-0109, ARO grant DAAL 03-91-G-0151, NSF grants CCR-8920519 CCR-9101795.