On convergence rates for the Iteratively regularized Gauss-Newton method

Received on 3 February 1995. Revised on 20 April 1996. In this paper we prove that the iteratively regularized Gauss-Newtonmethod is a locally convergent method for solving nonlinearill-posed problems, provided the nonlinear operator satisfiesa certain smoothness condition. For perturbed data we proposea priori and a posteriori stopping niles that guarantee convergenceof the iterates, if the noise level goes to zero. Under appropriatecloseness and smoothness conditions on the exact solution weobtain the same convergence rates as for linear ill-posed problems.     

Iteratively regularized methods for irregular nonlinear operator equations with a normally solvable derivative at the solution

A group of iteratively regularized methods of Gauss–Newton type for solving irregular nonlinear equations with smooth operators in a Hilbert space under the condition of normal solvability of the derivative of the operator at the solution is considered. A priori and a posteriori methods for termination of iterations are studied, and estimates of the accuracy of approximations obtained are found. It is shown that, in the case of a priori termination, the accuracy of the approximation is proportional to the error in the input data. Under certain additional conditions, the same estimate is established for a posterior termination from the residual principle. These results generalize known similar estimates for linear equations with a normally solvable operator.     

Newton-type methods for stochastic programming

Stochastic programming is concerned with practical procedures for decision making under uncertainty, by modelling uncertainties and risks associated with decision in a form suitable for optimization. The field is developing rapidly with contributions from many disciplines such as operations research, probability and statistics, and economics. A stochastic linear program with recourse can equivalently be formulated as a convex programming problem. The problem is often large-scale as the objective function involves an expectation, either over a discrete set of scenarios or as a multi-dimensional integral. Moreover, the objective function is possibly nondifferentiable. This paper provides a brief overview of recent developments on smooth approximation techniques and Newton-type methods for solving two-stage stochastic linear programs with recourse, and parallel implementation of these methods. A simple numerical example is used to signal the potential of smoothing approaches.     

Globally convergent Newton-type methods for multiobjective optimization

Computational Optimization and Applications - We propose two Newton-type methods for solving (possibly) nonconvex unconstrained multiobjective optimization problems. The first is directly inspired...     

Weak convergence conditions for Inexact Newton-type methods

We establish a new semilocal convergence results for Inexact Newton-type methods for approximating a locally unique solution of a nonlinear equation in a Banach spaces setting. We show that our sufficient convergence conditions are weaker and the estimates of error bounds are tighter in some cases than in earlier works [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30] and [31]. Special cases and numerical examples are also provided in this study.     

Tail estimates for exponential functionals and applications to SDEs

In this paper, based on techniques of Malliavin calculus, we obtain an explicit bound for tail probabilities of a general class of exponential functionals. We apply the obtained results to derive asymptotic behaviors for the tail of the exponential functional of stochastic differential equations.     

Majorizing functions and two-point Newton-type methods

The semi-local convergence of a Newton-type method used to solve nonlinear equations in a Banach space is studied. We also give, as two important applications, convergence analyses of two classes of two-point Newton-type methods including a method mentioned in [5] and the midpoint method studied in [1], [2] and [12]. Recently, interest has been shown in such methods [3] and [4].     

A general oscillation theorem for self-adjoint differential systems with applications to Sturm-Liouville eigenvalue problems and quadratic functionals

For given 2n×2n matricesS ₁₃,S ₂₄ with rank(S ₁₃,S ₂₄)=2n we consider the eigenvalue problem:u′=A(x)u+B(x)v,v′=C ₁(x;λ)u-A ^T(x)v with

Newton-type methods for constrained optimization with nonregular constraints

The most important classes of Newton-type methods for solving constrained optimization problems are discussed. These are the sequential quadratic programming methods, active set methods, and semismooth Newton methods for Karush-Kuhn-Tucker systems. The emphasis is placed on the behavior of these methods and their special modifications in the case where assumptions concerning constraint qualifications are relaxed or altogether dropped. Applications to optimization problems with complementarity constraints are examined.     

Various Newton-type iterative methods for solving nonlinear equations

The aim of the present paper is to introduce and investigate new ninth and seventh order convergent Newton-type iterative methods for solving nonlinear equations. The ninth order convergent Newton-type iterative method is made derivative free to obtain seventh-order convergent Newton-type iterative method. These new with and without derivative methods have efficiency indices 1.5518 and 1.6266, respectively. The error equations are used to establish the order of convergence of these proposed iterative methods. Finally, various numerical comparisons are implemented by MATLAB to demonstrate the performance of the developed methods.     

Brownian functionals and applications

Hida's theory of generalized Brownian functionals is surveyed with the applications to: (1) stochastic partial differential equations, (2) Feynman integral, (3) an extension of Itô's lemma, and (4) infinite dimensional Fourier transform.This article is based on the lectures delivered at the Department of Mathematics, University of Texas at Austin during July 6–10, 1981. The author is grateful to the department, especially, Professor Klaus R. Bichteler, for the invitation and the hispitality.Research supported by NSF grant MCS-8100728.     

Correction to: Globalized inexact proximal Newton-type methods for nonconvex composite functions

Computational Optimization and Applications -     

On Newton-type methods for multiple roots with cubic convergence

We introduce two families of Newton-type methods for multiple roots with cubic convergence. A further Newton-type method for multiple roots with cubic convergence is presented that is related to quadrature. We also provide numerical tests that show that these new methods are competitive to other known methods for multiple roots.     

Multiple solutions for indefinite functionals and applications to asymptotically linear problems

In this paper, by means of Morse theory of isolated critical points (orbits) we study further the critical points theory of asymptotically quadratic functionals and give some results concerning the existence of multiple critical points (orbits) which generalize a series of previous results due to Amann, Conley, Zehnder and K.C. Chang. As applications, the existence of multiple periodic solutions for asymptotically linear Hamiltonian systems is investigated. And our results generalize some recent ones due to Coti-Zelati, J.Q.Liu, S.Li, etc.This research was supported in part by the National Postdoctoral Science Fund.     

Generalized semi-inner products with applications to regularized learning

We propose a definition of generalized semi-inner products (g.s.i.p.). By relating them to duality mappings from a normed vector space to its dual space, a characterization for all g.s.i.p. satisfying this definition is obtained. We then study the Riesz representation of continuous linear functionals via g.s.i.p. As applications, we establish a representer theorem and characterization equation for the minimizer of a regularized learning from finite or infinite samples in Banach spaces of functions.     

Superlinear convergence theorems for Newton-type methods for nonlinear systems of equations

In this paper, someQ-order convergence theorems are given for the problem of solving nonlinear systems of equations when using very general finitely terminating methods for the solution of the associated linear systems. The theorems differ from those of Dembo, Eisenstat, and Steihaug in the different stopping condition and in their applicability to the nonlinear ABS algorithm.Lecture presented at the University of Bergamo, Bergamo, Italy, October 1989.