A matrix-free implementation of Riemannian Newton’s method on the Stiefel manifold

Newton’s method for unconstrained optimization problems on the Euclidean space can be generalized to that on Riemannian manifolds. The truncated singular value problem is one particular problem defined on the product of two Stiefel manifolds, and an algorithm of the Riemannian Newton’s method for this problem has been designed. However, this algorithm is not easy to implement in its original form because the Newton equation is expressed by a system of matrix equations which is difficult to solve directly. In the present paper, we propose an effective implementation of the Newton algorithm. A matrix-free Krylov subspace method is used to solve a symmetric linear system into which the Newton equation is rewritten. The presented approach can be used on other problems as well. Numerical experiments demonstrate that the proposed method is effective for the above optimization problem.     

A Newton’s method for the continuous quadratic knapsack problem

We introduce a new efficient method to solve the continuous quadratic knapsack problem. This is a highly structured quadratic program that appears in different contexts. The method converges after $O(n)$ iterations with overall arithmetic complexity $O(n^2)$ . Numerical experiments show that in practice the method converges in a small number of iterations with overall linear complexity, and is faster than the state-of-the-art algorithms based on median finding, variable fixing, and secant techniques.     

Concerning the radius of convergence of Newton’s method and applications

We present local and semilocal convergence results for Newton’s method in a Banach space setting. In particular, using Lipschitz-type assumptions on the second Fréchet-derivative we find results concerning the radius of convergence of Newton’s method. Such results are useful in the context of predictor-corrector continuation procedures. Finally, we provide numerical examples to show that our results can apply where earlier ones using Lipschitz assumption on the first Fréchet-derivative fail.     

Global convergence of Newton’s method on an interval

The solution of an equation f(x)= given by an increasing function f on an interval I and right-hand side , can be approximated by a sequence calculated according to Newtons method. In this article, global convergence of the method is considered in the strong sense of convergence for any initial value in I and any feasible right-hand side. The class of functions for which the method converges globally is characterized. This class contains all increasing convex and increasing concave functions as well as sums of such functions on the given interval. The characterization is applied to Keplers equation and to calculation of the internal rate of return of an investment project.An earlier version was presented at the Joint National Meeting of TIMS and ORSA, Las Vegas, May 7–9, 1990. Financial support from Økonomisk Forskningsfond, Bodø, Norway, is gratefully acknowledged. The author thanks an anonymous referee for helpful comments and suggestions.     

Local convergence of Newton’s method in the classical calculus of variations

Sufficient conditions for a weak local minimizer in the classical calculus of variations can be expressed without reference to conjugate points. The local quadratic convergence of Newton’s method follows from these sufficient conditions. Newton’s method is applied in the minimization form; that is, the step is generated by minimizing the local quadratic approximation. This allows the extension to a globally convergent line search based algorithm (which will be presented in a future paper).     

A note on inexact gradient and Hessian conditions for cubic regularized Newton’s method

The inexact cubic-regularized Newton’s method (CR) proposed by Cartis, Gould and Toint achieves the same convergence rate as exact CR proposed by Nesterov and Polyak, but the inexact condition is not implementable due to its dependence on a future variable. This note establishes the same convergence rate under a similar but implementable inexact condition, which depends on only current variables. Our proof bounds the function-value decrease over total iterations rather than each iteration in the previous studies.     

Extending the applicability of Newton’s method using nondiscrete induction

We extend the applicability of Newton’s method for approximating a solution of a nonlinear operator equation in a Banach space setting using nondiscrete mathematical induction concept introduced by Potra and Ptak. We obtain new sufficient convergence conditions for Newton’s method using Lipschitz and center-Lipschitz conditions instead of only the Lipschitz condition used in F.A.Potra, V.Ptak, Sharp error bounds for Newton’s process, Numer. Math., 34 (1980), 63–72, and F.A.Potra, V.Ptak, Nondiscrete Induction and Iterative Processes, Research Notes in Mathematics, 103. Pitman Advanced Publishing Program, Boston, 1984. Under the same computational cost as before, we provide: weaker sufficient convergence conditions; tighter error estimates on the distances involved and more precise information on the location of the solution. Numerical examples are also provided in this study.     

On a posteriori approximation of the set of solutions to a system of quadratic equations using Newton’s method

For quadratic systems of algebraic equations we propose an algorithm generating a posteriori estimates of the convex hull of the set of solutions using one step of Newton’s method. Results of some numerical tests are given.     

On the finite element approximation for maxwell’s problem in polynomial domains of the plane

The time-harmonic Maxwell boundary value problem in polygonal domains of R² is considered. The behaviour of the solution in the neighbourhood of nonregular boundary points is given and asymptotic error estimates in L₂- and in curl-div-norm for a finite element approximation of the solution are derived     

On the attraction of Newton’s method to critical lagrange multipliers

The attraction of dual trajectories of Newton’s method for the Lagrange system to critical Lagrange multipliers is analyzed. This stable effect, which has been confirmed by numerical practice, leads to the Newton-Lagrange method losing its superlinear convergence when applied to problems with irregular constraints. At the same time, available theoretical results are of “negative” character; i.e., they show that convergence to a noncritical multiplier is not possible or unlikely. In the case of a purely quadratic problem with a single constraint, a “positive” result is proved for the first time demonstrating that the critical multipliers are attractors for the dual trajectories. Additionally, the influence exerted by the attraction to critical multipliers on the convergence rate of direct and dual trajectories is characterized.     

Influence of the multiplicity of the roots on the basins of attraction of Newton’s method

In this work, we develop and implement two algorithms for plotting and computing the measure of the basins of attraction of rational maps defined on the Riemann sphere. These algorithms are based on the subdivisions of a cubical decomposition of a sphere and they have been made by using different computational environments. As an application, we study the basins of attraction of the fixed points of the rational functions obtained when Newton’s method is applied to a polynomial with two roots of multiplicities m and n. We focus our attention on the analysis of the influence of the multiplicities m and n on the measure of the two basins of attraction. As a consequence of the numerical results given in this work, we conclude that, if m > n, the probability that a point in the Riemann Sphere belongs to the basin of the root with multiplicity m is bigger than the other case. In addition, if n is fixed and m tends to infinity, the probability of reaching the root with multiplicity n tends to zero.     

Accelerating the cubic regularization of Newton’s method on convex problems

In this paper we propose an accelerated version of the cubic regularization of Newton’s method (Nesterov and Polyak, in Math Program 108(1): 177–205, 2006). The original version, used for minimizing a convex function with Lipschitz-continuous Hessian, guarantees a global rate of convergence of order \(O\big({1 \over k^2}\big)\), where k is the iteration counter. Our modified version converges for the same problem class with order \(O\big({1 \over k^3}\big)\), keeping the complexity of each iteration unchanged. We study the complexity of both schemes on different classes of convex problems. In particular, we argue that for the second-order schemes, the class of non-degenerate problems is different from the standard class.     

How to improve the domain of parameters for Newton’s method

We study the influence of a center Lipschitz condition for the first derivative of the operator involved when the solution of a nonlinear equation is approximated by Newton’s method in Banach spaces. As a consequence, we see that the domain of parameters associated to the Newton–Kantorovich theorem is enlarged.     

The numerical solution of Newton’s problem of least resistance

In this paper we consider Newton’s problem of finding a convex body of least resistance. This problem could equivalently be written as a variational problem over concave functions in \({\mathbb {R}}^{2}\) . We propose two different methods for solving it numerically. First, we discretize this problem by writing the concave solution function as a infimum over a finite number of affine functions. The discretized problem could be solved by standard optimization software efficiently. Second, we conjecture that the optimal body has a certain structure. We exploit this structure and obtain a variational problem in \({\mathbb {R}}^{1}\) . Deriving its Euler–Lagrange equation yields a program with two unknowns, which can be solved quickly.     

On finite difference approximation of a matrix-vector product in the Jacobian-free Newton–Krylov method

The Jacobian-free Newton–Krylov (JFNK) method is a special kind of Newton–Krylov algorithm, in which the matrix-vector product is approximated by a finite difference scheme. Consequently, it is not necessary to form and store the Jacobian matrix. This can greatly improve the efficiency and enlarge the application area of the Newton–Krylov method. The finite difference scheme has a strong influence on the accuracy and robustness of the JFNK method. In this paper, several methods for approximating the Jacobian-vector product, including the finite difference scheme and the finite difference step size, are analyzed and compared. Numerical results are given to verify the effectiveness of different finite difference methods.     

Stability in an overdetermined problem for the Green’s function

In the plane, we consider the problem of reconstructing a domain from the normal derivative of its Green’s function (with fixed pole) relative to the Dirichlet problem for the Laplace operator. By means of the theory of conformal mappings, we derive stability estimates of Hölder type.     

On the local convergence of Newton’s method under generalized conditions of Kantorovich

Following an idea similar to that given by Dennis and Schnabel (1996) in [2], we prove a local convergence result for Newton’s method under generalized conditions of Kantorovich type.