Cubic regularization of Newton method and its global performance

In this paper, we provide theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem. For this scheme, we prove general local convergence results. However, the main contribution of the paper is related to global worst-case complexity bounds for different problem classes including some nonconvex cases. It is shown that the search direction can be computed by standard linear algebra technique.     

A global Newton method for the zeros of cylinder functions

The zeros of cylinder functions C _u (x)=cos α, J _u (x) - sin α, Y _u(x) coincide with those of the ratios H _u (x)=C _u (x)/C _u-1 (x) except, perhaps, at x = 0. We show monotonicity properties of H u(x) and f _u (x) = x ^2v-1 H u(x) and their derivatives for x > 0. We then build a Newton-Raphson iterative method based on the monotonic function f u(x) which is shown to be convergent, for any real values of u and α and any starting value x ₀ > 0, to an sth positive root c ,s of C _u (x) = 0, s being such that c _,s and x0 belong to the same interval (c _u-1 ,s', c _{u -1} ,s'+1]. We also show applications of the method. In particular, taking advantage of the fact that the ratio H _u (x) for first kind Bessel functions J u(x) can be evaluated by using a continued fraction, a very simple algorithm is built; it becomes especially efficient for low values of u and s and it allows the evaluation of the real zeros for arbitrary orders u, positive or negative. This revised version was published online in August 2006 with corrections to the Cover Date.     

An extended continuous Newton method

This paper describes a numerical realization of an extended continuous Newton method defined by Diener. It traces a connected set of locally one-dimensional trajectories which contains all critical points of a smooth functionf: ⁿ . The results show that the method is effectively applicable.The authors would like to thank L. C. W. Dixon for pointing out some errors in the original version of this paper and for several suggestions of improvements.     

A new smoothing Newton method for solving constrained nonlinear equations

In this paper, a new smoothing Newton method is proposed for solving constrained nonlinear equations. We first transform the constrained nonlinear equations to a system of semismooth equations by using the so-called absolute value function of the slack variables, and then present a new smoothing Newton method for solving the semismooth equations by constructing a new smoothing approximation function. This new method is globally and quadratically convergent. It needs to solve only one system of unconstrained equations and to perform one line search at each iteration. Numerical results show that the new algorithm works quite well.     

A smoothing Newton method for second-order cone optimization based on a new smoothing function

A new smoothing function is given in this paper by smoothing the symmetric perturbed Fischer-Burmeister function. Based on this new smoothing function, we present a smoothing Newton method for solving the second-order cone optimization (SOCO). The method solves only one linear system of equations and performs only one line search at each iteration. Without requiring strict complementarity assumption at the SOCO solution, the proposed algorithm is shown to be globally and locally quadratically convergent. Numerical results demonstrate that our algorithm is promising and comparable to interior-point methods.     

A modified Newton method for minimization

Some promising ideas for minimizing a nonlinear function, whose first and second derivatives are given, by a modified Newton method, were introduced by Fiacco and McCormick (Ref. 1). Unfortunately, in developing a method around these ideas, Fiacco and McCormick used a potentially unstable, or even impossible, matrix factorization. Using some recently developed techniques for factorizing an indefinite symmetric matrix, we are able to produce a method which is similar to Fiacco and McCormick's original method, but avoids the difficulties of the original method.Both authors gratefully acknowledge the award of a research fellowship from the British Science Research Council.     

A new modified one-step smoothing Newton method for solving the general mixed complementarity problem

In last decades, there has been much effort on the solution and the analysis of the mixed complementarity problem (MCP) by reformulating MCP as an unconstrained minimization involving an MCP function. In this paper, we propose a new modified one-step smoothing Newton method for solving general (not necessarily P₀) mixed complementarity problems based on well-known Chen-Harker-Kanzow-Smale smooth function. Under suitable assumptions, global convergence and locally superlinear convergence of the algorithm are established.     

A generalized Newton method for absolute value equations

A direct generalized Newton method is proposed for solving the NP-hard absolute value equation (AVE) Ax − |x| = b when the singular values of A exceed 1. A simple MATLAB implementation of the method solved 100 randomly generated 1,000-dimensional AVEs to an accuracy of 10⁻⁶ in less than 10 s each. Similarly, AVEs corresponding to 100 randomly generated linear complementarity problems with 1,000 × 1,000 nonsymmetric positive definite matrices were also solved to the same accuracy in less than 29 s each.     

A Newton inexact interior-point method for large scale nonlinear optimization problems

In this paper, we describe a variant of the Newton Interior-Point method in [8] for nonlinear programming problems. In this scheme, the perturbation parameter can be chosen within a range of, values and we can use an iterative method for approximately solving the reduced linear system arising at each step. We have devised the inner termination rule which guarantees the global convergence of this Newton Inexact Interior-Point method. We remark that the required assumptions are weaker than those stated in [8], as shown by some numerical examples. This research was supported by the Italian Ministry for Education, University and Research (MIUR), FIRB Project No. RBAU01JYPN.     

A globally convergent Newton method for convex SC1 minimization problems

This paper presents a globally convergent and locally superlinearly convergent method for solving a convex minimization problem whose objective function has a semismooth but nondifferentiable gradient. Applications to nonlinear minimax problems, stochastic programs with recourse, and their extensions are discussed.The research of the first author is based on work supported by the National Science Foundation under Grants DDM-9104078 and CCR-9213739. This research was carried out while he was visiting the University of New South Wales. The research of the second author is based on work supported by the Australian Research Council.     

A truncated aggregate smoothing Newton method for minimax problems

Aggregate function is a useful smoothing function to the max-function of some smooth functions and has been used to solve minimax problems, linear and nonlinear programming, generalized complementarity problems, etc. The aggregate function is a single smooth but complex function, its gradient and Hessian calculations are time-consuming. In this paper, a truncated aggregate smoothing stabilized Newton method for solving minimax problems is presented. At each iteration, only a small subset of the components in the max-function are aggregated, hence the number of gradient and Hessian calculations is reduced dramatically. The subset is adaptively updated with some truncating criterions, concerning only with computation of function values and not their gradients or Hessians, to guarantee the global convergence and, for the inner iteration, locally quadratic convergence with as few computational cost as possible. Numerical results show the efficiency of the proposed algorithm.     

A smoothing inexact Newton method for nonlinear complementarity problems

In this article, we propose a new smoothing inexact Newton algorithm for solving nonlinear complementarity problems (NCP) base on the smoothed Fischer-Burmeister function. In each iteration, the corresponding linear system is solved only approximately. The global convergence and local superlinear convergence are established without strict complementarity assumption at the NCP solution. Preliminary numerical results indicate that the method is effective for large-scale NCP.     

The relaxed Newton method derivative: Its dynamics and non-linear properties

The dynamic behaviour of the one-dimensional family of maps f(x)=_c2[(a−1)x+_c1]^{−λ/(α−1)}$f\left(x\right)=c_2{[(a-1)x+c_1]}^{-\lambda /(\alpha -1)}$ is examined, for representative values of the control parameters a,_c1$a,c_1$, _c2$c_2$ and λ$\lambda$. The maps under consideration are of special interest, since they are solutions of the relaxed Newton method derivative being equal to a constant a$a$. The maps f(x)$f\left(x\right)$ are also proved to be solutions of a non-linear differential equation with outstanding applications in the field of power electronics. The recurrent form of these maps, after excessive iterations, shows, in an _xn$x_n$ versus λ$\lambda$ plot, an initial exponential decay followed by a bifurcation. The value of λ$\lambda$ at which this bifurcation takes place depends on the values of the parameters a,_c1$a,c_1$ and _c2$c_2$. This corresponds to a switch to an oscillatory behaviour with amplitudes of f(x)$f\left(x\right)$ undergoing a period doubling. For values of a$a$ higher than 1 and at higher values of λ$\lambda$ a reverse bifurcation occurs. The corresponding branches converge and a bleb is formed for values of the parameter _c1$c_1$ between 1 and 1.20. This behaviour is confirmed by calculating the corresponding Lyapunov exponents.     

A one-step smoothing Newton method for second-order cone programming

A new smoothing function for the second-order cone programming is given by smoothing the symmetric perturbed Fischer–Burmeister function. Based on this new function, a one-step smoothing Newton method is presented for solving the second-order cone programming. The proposed algorithm solves only one linear system of equations and performs only one line search at each iteration. This algorithm does not have restrictions regarding its starting point and is Q-quadratically convergent. Numerical results suggest the effectiveness of our algorithm.     

Riemannian Newton method for positive bounded Hessian functions

In this work we study Newton type method for functions on Riemannian manifolds whose Hessian satisfies a double inequality. The main results refer to global convergence and convergence rate estimates.     

A Newton method for a class of quasi-variational inequalities

A variant of the Newton method for nonsmooth equations is applied to solve numerically quasivariational inequalities with monotone operators. For this purpose, we investigate the semismoothness of a certain locally Lipschitz operator coming from the quasi-variational inequality, and analyse the generalized Jacobian of this operator to ensure local convergence of the method. A simplified variant of this approach, applicable to implicit complementarity problems, is also studied. Small test examples have been computed.This work has been supported in parts by a grant from the German Scientific Foundation and by a grant from the Czech Academy of Sciences.     

A reduced Newton method for constrained linear least-squares problems

We propose an iterative method that solves constrained linear least-squares problems by formulating them as nonlinear systems of equations and applying the Newton scheme. The method reduces the size of the linear system to be solved at each iteration by considering only a subset of the unknown variables. Hence the linear system can be solved more efficiently. We prove that the method is locally quadratic convergent. Applications to image deblurring problems show that our method gives better restored images than those obtained by projecting or scaling the solution into the dynamic range.     

A wavelet adaptive Newton method for the solution of nonlinear equations

For the solution of nonlinear equations, we present an adaptive wavelet scheme, which couples an inexact Newton method and the idea of nonlinear wavelet approximation. In particular, we obtain a result of quadratic convergence.     

A projected Newton method in a Cartesian product of balls

We formulate a locally superlinearly convergent projected Newton method for constrained minimization in a Cartesian product of balls. For discrete-time,N-stage, input-constrained optimal control problems with Bolza objective functions, we then show how the required scaled tangential component of the objective function gradient can be approximated efficiently with a differential dynamic programming scheme; the computational cost and the storage requirements for the resulting modified projected Newton algorithm increase linearly with the number of stages. In calculations performed for a specific control problem with 10 stages, the modified projected Newton algorithm is shown to be one to two orders of magnitude more efficient than a standard unscaled projected gradient method.This work was supported by the National Science Foundation, Grant No. DMS-85-03746.     

A nonmonotone smoothing Newton method for circular cone programming

The circular cone programming (CCP) problem is to minimize or maximize a linear function over the intersection of an affine space with the Cartesian product of circular cones. In this paper, we study nondegeneracy and strict complementarity for the CCP, and present a nonmonotone smoothing Newton method for solving the CCP. We reformulate the CCP as a second-order cone programming (SOCP) problem using the algebraic relation between the circular cone and the second-order cone. Then based on a one parametric class of smoothing functions for the SOCP, a smoothing Newton method is developed for the CCP by adopting a new nonmonotone line search scheme. Without restrictions regarding its starting point, our algorithm solves one linear system of equations approximately and performs one line search at each iteration. Under mild assumptions, our algorithm is shown to possess global and local quadratic convergence properties. Some preliminary numerical results illustrate that our nonmonotone smoothing Newton method is promising for solving the CCP.