The Convergence of a Levenberg–Marquardt Method for Nonlinear Inequalities

In this paper, we consider the least l ₂-norm solution for a possibly inconsistent system of nonlinear inequalities. The objective function of the problem is only first-order continuously differentiable. By introducing a new smoothing function, the problem is approximated by a family of parameterized optimization problems with twice continuously differentiable objective functions. Then a Levenberg–Marquardt algorithm is proposed to solve the parameterized smooth optimization problems. It is proved that the algorithm either terminates finitely at a solution of the original inequality problem or generates an infinite sequence. In the latter case, the infinite sequence converges to a least l ₂-norm solution of the inequality problem. The local quadratic convergence of the algorithm was produced under some conditions.     

A Translation and Rotation Invariant Gauss–Newton Like Scheme for Image Registration

A Gauss–Newton like method is considered to obtain a d–dimensional displacement vector field , which minimizes a suitable distance measure D between two images. The key to find a minimizer is to substitute the Hessian of D with the Sobolev-H²(Ω)^d norm for . Since the kernel of the associated semi-norm consists only of the affine linear functions we can show in this way, that the solution of each Newton step is a linear combination of an affine linear transformation and an affine-free nonlinear deformation. Our approach is based on the solution of a sequence of quadratic subproblems with linear constraints. We show that the resulting Karush–Kuhn–Tucker system, with a 3×3 block structure, can be solved uniquely and the Gauss–Newton like scheme can be separated into two separated iterations. Finally, we report on synthetic as well as on real-life data test runs. AMS subject classification (2000) 65F20, 68U10     

Nonmonotone Self-adaptive Levenberg–Marquardt Approach for Solving Systems of Nonlinear Equations

The well-known Levenberg–Marquardt method is used extensively to solve systems of nonlinear equations. An extension of the Levenberg–Marquardt method based on new nonmonotone technique is described. To decrease the total number of iterations, this method allows the sequence of objective function values to be nonmonotone, especially in the case where the objective function is ill-conditioned. Moreover, the parameter of Levenberg–Marquardt is produced according to the new nonmonotone strategy to use the advantages of the faster convergence of the Gauss–Newton method whenever iterates are near the optimizer, and the robustness of the steepest descent method in the case in which iterates are far away from the optimizer. The global and quadratic convergence of the proposed method is established. The results of numerical experiments are reported.     

A smoothing Levenberg–Marquardt method for the extended linear complementarity problem

We consider the extended linear complementarity problem (XLCP), of which the linear and horizontal linear complementarity problems are two special cases. We reformulate the XLCP to a smooth equation by using some smoothing functions and propose a Levenberg–Marquardt method to solve the system of smooth equation. The global convergence and local superlinear convergence rate are established under certain conditions. Numerical tests show the effectiveness of the proposed algorithm.     

A Levenberg–Marquardt method based on Sobolev gradients

We extend the theory of Sobolev gradients to include variable metric methods, such as Newton’s method and the Levenberg–Marquardt method, as gradient descent iterations associated with stepwise variable inner products. In particular, we obtain existence, uniqueness, and asymptotic convergence results for a gradient flow based on a variable inner product.     

On a regularized Levenberg–Marquardt method for solving nonlinear inverse problems

We consider a regularized Levenberg–Marquardt method for solving nonlinear ill-posed inverse problems. We use the discrepancy principle to terminate the iteration. Under certain conditions, we prove the convergence of the method and obtain the order optimal convergence rates when the exact solution satisfies suitable source-wise representations.     

Levenberg–Marquardt method based on probabilistic Jacobian models for nonlinear equations

In this paper, we propose a Levenberg–Marquardt method based on probabilistic models for nonlinear equations for which the Jacobian cannot be computed accurately or the computation is very expensive. We introduce the definition of the first-order accurate probabilistic Jacobian model, and show how to construct such a model with sample points generated by standard Gaussian distribution. Under certain conditions, we prove that the proposed method converges to a first order stationary point with probability one. Numerical results show the efficiency of the method.

     

A unified local convergence analysis of inexact constrained Levenberg–Marquardt methods

The Levenberg–Marquardt method is a regularized Gauss–Newton method for solving systems of nonlinear equations. If an error bound condition holds it is known that local quadratic convergence to a non-isolated solution can be achieved. This result was extended to constrained Levenberg–Marquardt methods for solving systems of equations subject to convex constraints. This paper presents a local convergence analysis for an inexact version of a constrained Levenberg–Marquardt method. It is shown that the best results known for the unconstrained case also hold for the constrained Levenberg–Marquardt method. Moreover, the influence of the regularization parameter on the level of inexactness and the convergence rate is described. The paper improves and unifies several existing results on the local convergence of Levenberg–Marquardt methods.     

Global Complexity Bound Analysis of the Levenberg–Marquardt Method for Nonsmooth Equations and Its Application to the Nonlinear Complementarity Problem

We investigate a global complexity bound of the Levenberg–Marquardt Method (LMM) for nonsmooth equations. The global complexity bound is an upper bound to the number of iterations required to get an approximate solution that satisfies a certain condition. We give sufficient conditions under which the bound of the LMM for nonsmooth equations is the same as smooth cases. We also show that it can be reduced under some regularity assumption. Furthermore, by applying these results to nonsmooth equations equivalent to the nonlinear complementarity problem (NCP), we get global complexity bounds for the NCP. In particular, we give a reasonable bound when the mapping involved in the NCP is a uniformly P-function.     

Smoothing Levenberg–Marquardt method for general nonlinear complementarity problems under local error bound

By using the F–B function and smoothing technique to convert the nonlinear complementarity problems to smoothing nonlinear systems, and introducing perturbation parameter μ_k into the smoothing Newton equation, we present a new smoothing Levenberg–Marquardt method for general nonlinear complementarity problems. For general mapping F, not necessarily a P₀ function, the algorithm has global convergence. Each accumulation point of the iterative sequence is at least a stationary point of the problem. Under the local error bound condition, which is much weaker than nonsingularity assumption or the strictly complementarity condition, we get the local superlinear convergence. Under some proper condition, quadratic convergence is also obtained.     

On the inexactness level of robust Levenberg–Marquardt methods

Recently, the Levenberg–Marquardt (LM) method has been used for solving systems of nonlinear equations with nonisolated solutions. Under certain conditions it converges Q-quadratically to a solution. The same rate has been obtained for inexact versions of the LM method. In this article the LM method will be called robust, if the magnitude of the regularization parameter occurring in its sub-problems is as large as possible without decreasing the convergence rate. For robust LM methods the article shows that the level of inexactness in the sub-problems can be increased significantly. As an application, the local convergence of a projected robust LM method is analysed.     

On the convergence of a regularizing Levenberg–Marquardt scheme for nonlinear ill-posed problems

In this note, we study the convergence of the Levenberg–Marquardt regularization scheme for nonlinear ill-posed problems. We consider the case that the initial error satisfies a source condition. Our main result shows that if the regularization parameter does not grow too fast (not faster than a geometric sequence), then the scheme converges with optimal convergence rates. Our analysis is based on our recent work on the convergence of the exponential Euler regularization scheme (Hochbruck et al. in Inverse Probl 25(7):075009, 2009).     

Primal-Dual Relationship Between Levenberg–Marquardt and Central Trajectories for Linearly Constrained Convex Optimization

We consider the minimization of a convex function on a bounded polyhedron (polytope) represented by linear equality constraints and non-negative variables. We define the Levenberg–Marquardt and central trajectories starting at the analytic center using the same parameter, and show that they satisfy a primal-dual relationship, being close to each other for large values of the parameter. Based on this, we develop an algorithm that starts computing primal-dual feasible points on the Levenberg–Marquardt trajectory and eventually moves to the central path. Our main theorem is particularly relevant in quadratic programming, where points on the primal-dual Levenberg–Marquardt trajectory can be calculated by means of a system of linear equations. We present some computational tests related to box constrained trust region subproblems.     

WRAO and OWA learning using Levenberg–Marquardt and genetic algorithms

The generalized Weighted Relevance Aggregation Operator (WRAO) is a non-additive aggregation function. The Ordered Weighted Aggregation Operator (OWA) (or its generalized form: Generalized Ordered Weighted Aggregation Operator (GOWA)) is more restricted with the additivity constraint in its weights. In addition, it has an extra weights reordering step making it hard to learn automatically from data. Our intension here is to compare the efficiency (or effectiveness) of learning these two types of aggregation functions from empirical data. We employed two methods to learn WRAO and GOWA: Levenberg–Marquardt (LM) and a Genetic Algorithm (GA) based method. We use UCI (University of California Irvine) benchmark data to compare the aggregation performance of non-additive WRAO and additive GOWA. We found that the non-constrained aggregation function WRAO was learnt well automatically and produced consistent results, while GOWA was learnt less well and quite inconsistently.     

A Symmetric Characteristic Finite Volume Element Scheme for Nonlinear Convection-Diffusion Problems

In this paper, we implement alternating direction strategy and construct a symmetric FVE scheme for nonlinear convection-diffusion problems. Comparing to general FVE methods, our method has two advantages. First, the coefficient matrices of the discrete schemes will be symmetric even for nonlinear problems. Second, since the solution of the algebraic equations at each time step can be inverted into the solution of several one-dimensional problems, the amount of computation work is smaller. We prove the optimal H1-norm error estimates of order O（△t2 ＋ h） and present some numerical examples at the end of the paper.     

An efficient Levenberg–Marquardt method with a new LM parameter for systems of nonlinear equations

A modified Levenberg–Marquardt method for solving singular systems of nonlinear equations was proposed by Fan [J Comput Appl Math. 2003;21;625–636]. Using trust region techniques, the global and quadratic convergence of the method were proved. In this paper, to improve this method, we decide to introduce a new Levenberg–Marquardt parameter while also incorporate a new nonmonotone technique to this method. The global and quadratic convergence of the new method is proved under the local error bound condition. Numerical results show the new algorithm is efficient and promising.