Convergence of a generalized Newton and an inexact generalized Newton algorithms for solving nonlinear equations with nondifferentiable terms

In this paper, we consider two versions of the Newton-type method for solving a nonlinear equations with nondifferentiable terms, which uses as iteration matrices, any matrix from B-differential of semismooth terms. Local and global convergence theorems for the generalized Newton and inexact generalized Newton method are proved. Linear convergence of the algorithms is obtained under very mild assumptions. The superlinear convergence holds under some conditions imposed on both terms of equation. Some numerical results indicate that both algorithms works quite well in practice.      

Methods of conjugate directions versus quasi-Newton methods

It is shown that algorithms for minimizing an unconstrained functionF(x), x E ⁿ , which are solely methods of conjugate directions can be expected to exhibit only ann or (n–1) step superlinear rate of convergence to an isolated local minimizer. This is contrasted with quasi-Newton methods which can be expected to exhibit every step superlinear convergence. Similar statements about a quadratic rate of convergence hold when a Lipschitz condition is placed on the second derivatives ofF(x). Research was supported in part by Army Research Office, Contract Number DAHC 19-69-C-0017 and the Office of Naval Research, Contract Number N00014-71-C-0116 (NR 047-99).     

Second-Order Approach to Optimal Semiconductor Design

Second order methods for optimal semiconductor design based on the standard drift diffusion model are developed. Second–order necessary and sufficient conditions are established. Local quadratic convergence for the Newton method is proved. Numerical results for an unsymmetric (n–p) diode are presented. The first author acknowledges financial support from the Collaborative Research Center 609 funded by the German Research Foundation. The second author was supported by the European network HYKE, funded by the EC under Contract HPRN-CT-2002-00282.     

A quasi-Newton method can be obtained from a method of conjugate directions

In a recent paper McCormick and Ritter consider two classes of algorithms, namely methods of conjugate directions and quasi-Newton methods, for the problem of minimizing a function ofn variablesF(x). They show that the former methods possess ann-step superlinear rate of convergence while the latter are every step superlinear and therefore inherently superior. In this paper a simple and computationally inexpensive modification of a method of conjugate directions is presented. It is shown that the modified method is a quasi-Newton method and is thus every step superlinearly convergent. It is also shown that under certain assumptions on the second derivatives ofF the rate of convergence of the modified method isn-step quadratic.This work was supported by the National Research Council of Canada under Research Grant A8189.     

On the convergence of the Newton/log-barrier method

In the Newton/log-barrier method, Newton steps are taken for the log-barrier function for a fixed value of the barrier parameter until a certain convergence criterion is satisfied. The barrier parameter is then decreased and the Newton process is repeated. A naive analysis indicates that Newton’s method does not exhibit superlinear convergence to the minimizer of each instance of the log-barrier function until it reaches a very small neighborhood, namely within O(μ²) of the minimizer, where μ is the barrier parameter. By analyzing the structure of the barrier Hessian and gradient in terms of the subspace of active constraint gradients and the associated null space, we show that this neighborhood is in fact much larger –O(μ^σ) for any σ∈(1,2] – thus explaining why reasonably fast local convergence can be attained in practice. Moreover, we show that the overall convergence rate of the Newton/log-barrier algorithm is superlinear in the number of function/derivative evaluations, provided that the nonlinear program is formulated with a linear objective and that the schedule for decreasing the barrier parameter is related in a certain way to the step length and convergence criteria for each Newton process. Received: October 10, 1997 / Accepted: September 10, 2000?Published online February 22, 2001     

Projection method for unconstrained optimization

A method of conjugate directions, the projection method, for solving unconstrained minimization problems is presented. Under the assumption of uniform strict convexity, the method is shown to converge to the global minimizer of the unconstrained problem and to have an (n – 1)-step superlinear rate of convergence. With a Lipschitz condition on the second derivatives, the rate of convergence is shown to be a modifiedn-step quadratic one.This research was supported in part by the Army Research Office, Contract No. DAHC 19-69-C-0017, and the Office of Naval Research, Contract No. N00014-71-C-0116(NR-047-099).     

A trust region algorithm for nonsmooth optimization

A trust region algorithm is proposed for minimizing the nonsmooth composite functionF(x) = h(f(x)), wheref is smooth andh is convex. The algorithm employs a smoothing function, which is closely related to Fletcher's exact differentiable penalty functions. Global and local convergence results are given, considering convergence to a strongly unique minimizer and to a minimizer satisfying second order sufficiency conditions.     

Local convergence theorems for Newton methods

Affine invariant sufficient conditions are given for two local convergence theorems involving inexact Newton-like methods. The first uses conditions on the first Fréchet-derivative whereas the second theorem employs hypotheses on themth (m ≥ 2 an integer). Radius of convergence as well as rate of convergence results are derived. Results involving superlinear convergence and known to be true for inexact Newton methods are extended here. Moreover, we show that under hypotheses on the mth Fréchet-derivative our radius of convergence can sometimes be larger than the corresponding one in [10]. This allows a wider choice for the initial guess. A numerical example is also provided to show that our radius of convergence is larger than the one in [10].     

Numerical simulation of two-dimensional Bingham fluid flow by semismooth Newton methods

This paper is devoted to the numerical simulation of two-dimensional stationary Bingham fluid flow by semismooth Newton methods. We analyze the modeling variational inequality of the second kind, considering both Dirichlet and stress-free boundary conditions. A family of Tikhonov regularized problems is proposed and the convergence of the regularized solutions to the original one is verified. By using Fenchel’s duality, optimality systems which characterize the original and regularized solutions are obtained. The regularized optimality systems are discretized using a finite element method with (cross-grid P₁)-Q₀ elements for the velocity and pressure, respectively. A semismooth Newton algorithm is proposed in order to solve the discretized optimality systems. Using an additional relaxation, a descent direction is constructed from each semismooth Newton iteration. Local superlinear convergence of the method is also proved. Finally, we perform numerical experiments in order to investigate the behavior and efficiency of the method.     

Affine invariant local convergence theorems for inexact Newton-like methods

Affine invariant sufficient conditions are given for two local convergence theorems involving inexact Newton-like methods. The first uses conditions on the first Fréchet-derivative whereas the second theorem employs hypotheses on the second. Radius of convergence as well as rate of convergence results are derived. Results involving superlinear convergence and known to be true for inexact Newton methods are extended here. Moreover, we show that under hypotheses on the second Fréchet-derivative our radius of convergence is larger than the corresponding one in [10]. This allows a wider choice for the initial guess. A numerical example is also provided to show that our radius of convergence is larger than the one in [10].     

A Regularization Newton Method for Solving Nonlinear Complementarity Problems

In this paper we construct a regularization Newton method for solving the nonlinear complementarity problem (NCP(F )) and analyze its convergence properties under the assumption that F is a P ₀ -function. We prove that every accumulation point of the sequence of iterates is a solution of NCP(F ) and that the sequence of iterates is bounded if the solution set of NCP(F ) is nonempty and bounded. Moreover, if F is a monotone and Lipschitz continuous function, we prove that the sequence of iterates is bounded if and only if the solution set of NCP(F ) is nonempty by setting , where is a parameter. If NCP(F) has a locally unique solution and satisfies a nonsingularity condition, then the convergence rate is superlinear (quadratic) without strict complementarity conditions. At each step, we only solve a linear system of equations. Numerical results are provided and further applications to other problems are discussed. Accepted 25 March 1998     

PICARD ITERATION FOR NONSMOOTH EQUATIONS

1. IntroductionConsider the following nonsmooth equationsF(x) = 0 (l)where F: R" - R" is LipsChitz continuous. A lot of work has been done and is bellg doneto deal with (1). It is basicly a genera1ization of the cIassic Newton method [8,10,11,14],Newton-lthe methods[1,18] and quasiNewton methods [6,7]. As it is discussed in [7], the latter,quasiNewton methods, seem to be lindted when aPplied to nonsmooth caJse in that a boundof the deterioration of uPdating matrir can not be maintained w…     

A globalization procedure for solving nonlinear systems of equations

A new globalization procedure for solving a nonlinear system of equationsF(x)=0 is proposed based on the idea of combining Newton step and the steepest descent step WITHIN each iteration. Starting with an arbitrary initial point, the procedure converges either to a solution of the system or to a local minimizer off(x)=1/2F(x) ^T F(x). Each iteration is chosen to be as close to a Newton step as possible and could be the Newton step itself. Asymptotically the Newton step will be taken in each iteration and thus the convergence is quadratic. Numerical experiments yield positive results. Further generalizations of this procedure are also discussed in this paper.     

The convergence analysis of inexact Gauss–Newton methods for nonlinear problems

In this paper, inexact Gauss–Newton methods for nonlinear least squares problems are studied. Under the hypothesis that derivative satisfies some kinds of weak Lipschitz conditions, the local convergence properties of inexact Gauss–Newton and inexact Gauss–Newton like methods for nonlinear problems are established with the modified relative residual control. The obtained results can provide an estimate of convergence ball for inexact Gauss–Newton methods.     

A preconditioning proximal newton method for nondifferentiable convex optimization

We propose a proximal Newton method for solving nondifferentiable convex optimization. This method combines the generalized Newton method with Rockafellar’s proximal point algorithm. At each step, the proximal point is found approximately and the regularization matrix is preconditioned to overcome inexactness of this approximation. We show that such a preconditioning is possible within some accuracy and the second-order differentiability properties of the Moreau-Yosida regularization are invariant with respect to this preconditioning. Based upon these, superlinear convergence is established under a semismoothness condition. This work is supported by the Australian Research Council.     

Rate of convergence of the generalized newton algorithm using the fixed-point approach

This paper shows that the generalized Newton algorithm [GN(r)], developed by Kalaba and Tishler (Ref. 1), can be described as a fixed-point algorithm. In addition to specifying sufficient conditions for convergence of the GN(r), we show that, forr=1, 2, 3, its rate of convergence increases with the order of the derivatives which are used.The authors are indebted to I. Zang for drawing their attention to an error in an earlier draft of this paper. Suggestions and comments by N. Levin and D. Trietsch are also gratefully acknowledged.     

A proximal point algorithm for the monotone second-order cone complementarity problem

This paper is devoted to the study of the proximal point algorithm for solving monotone second-order cone complementarity problems. The proximal point algorithm is to generate a sequence by solving subproblems that are regularizations of the original problem. After given an appropriate criterion for approximate solutions of subproblems by adopting a merit function, the proximal point algorithm is verified to have global and superlinear convergence properties. For the purpose of solving the subproblems efficiently, we introduce a generalized Newton method and show that only one Newton step is eventually needed to obtain a desired approximate solution that approximately satisfies the appropriate criterion under mild conditions. Numerical comparisons are also made with the derivative-free descent method used by Pan and Chen (Optimization 59:1173–1197, 2010), which confirm the theoretical results and the effectiveness of the algorithm.     

Local analysis of Newton-type methods for variational inequalities and nonlinear programming

This paper presents some new results in the theory of Newton-type methods for variational inequalities, and their application to nonlinear programming. A condition of semistability is shown to ensure the quadratic convergence of Newton's method and the superlinear convergence of some quasi-Newton algorithms, provided the sequence defined by the algorithm exists and converges. A partial extension of these results to nonsmooth functions is given. The second part of the paper considers some particular variational inequalities with unknowns (x, ), generalizing optimality systems. Here only the question of superlinear convergence of {x ^k } is considered. Some necessary or sufficient conditions are given. Applied to some quasi-Newton algorithms they allow us to obtain the superlinear convergence of {x ^k }. Application of the previous results to nonlinear programming allows us to strengthen the known results, the main point being a characterization of the superlinear convergence of {x ^k } assuming a weak second-order condition without strict complementarity.     

A projection-filter method for solving nonlinear complementarity problems

The Josephy-Newton method attacks nonlinear complementarity problems which consists of solving, possibly inexactly, a sequence of linear complementarity problems. Under appropriate regularity assumptions, this method is known to be locally (superlinearly) convergent. Utilizing the filter method, we presented a new globalization strategy for this Newton method applied to nonlinear complementarity problem without any merit function. The strategy is based on the projection-proximal point and filter methodology. Our linesearch procedure uses the regularized Newton direction to force global convergence by means of a projection step which reduces the distance to the solution of the problem. The resulting algorithm is globally convergent to a solution. Under natural assumptions, locally superlinear rate of convergence was established.     

Smale's α-theory for inexact Newton methods under the γ-condition

The present paper is concerned with the convergence problem of inexact Newton methods. Assuming that the nonlinear operator satisfies the γ-condition, a convergence criterion for inexact Newton methods is established which includes Smale's type convergence criterion. The concept of an approximate zero for inexact Newton methods is proposed in this paper and the criterion for judging an initial point being an approximate zero is established. Consequently, Smale's α-theory is generalized to inexact Newton methods. Furthermore, a numerical example is presented to illustrate the applicability of our main results.