Two-Stage Method of Construction of Regularizing Algorithms for Nonlinear Ill-Posed Problems

For an equation with a nonlinear differentiable operator acting in a Hilbert space, we study a two-stage method of construction of a regularizing algorithm. First, we use the Lavrentiev regularization scheme. Then we apply to the regularized equation either Newton’s method or nonlinear analogs of α-processes: the minimum error method, the minimum residual method, and the steepest descent method. For these processes, we establish the linear convergence rate and the Fejér property of iterations. Two cases are considered: when the operator of the problem is monotone and when the operator is finite-dimensional and its derivative has nonnegative spectrum. For the two-stage method with a monotone operator, we give an error bound, which has optimal order on the class of sourcewise representable solutions. In the second case, the error of the method is estimated by means of the residual. The proposed methods and their modified analogs are implemented numerically for three-dimensional inverse problems of gravimetry and magnetometry. The results of the numerical experiment are discussed.     

Identification of the zeroth-order coefficient and fractional order in a time-fractional reaction-diffusion-wave equation

In this paper, we investigate an inverse problem of recovering the zeroth-order coefficient and fractional order simultaneously in a time-fractional reaction-diffusion-wave equation by using boundary measurement data from both of uniqueness and numerical method. We prove the uniqueness of the considered inverse problem and the Lipschitz continuity of the forward operator. Then the inverse problem is formulated into a variational problem by the Tikhonov-type regularization. Based on the continuity of the forward operator, we prove that the minimizer of the Tikhonov-type functional exists and converges to the exact solution under an a priori choice of regularization parameter. The steepest descent method combined with Nesterov acceleration is adopted to solve the variational problem. Three numerical examples are presented to support the efficiency and rationality of our proposed method.     

Steepest descent preconditioning for nonlinear GMRES optimization

Steepest descent preconditioning is considered for the recently proposed nonlinear generalized minimal residual (N‐GMRES) optimization algorithm for unconstrained nonlinear optimization. Two steepest descent preconditioning variants are proposed. The first employs a line search, whereas the second employs a predefined small step. A simple global convergence proof is provided for the N‐GMRES optimization algorithm with the first steepest descent preconditioner (with line search), under mild standard conditions on the objective function and the line search processes. Steepest descent preconditioning for N‐GMRES optimization is also motivated by relating it to standard non‐preconditioned GMRES for linear systems in the case of a standard quadratic optimization problem with symmetric positive definite operator. Numerical tests on a variety of model problems show that the N‐GMRES optimization algorithm is able to very significantly accelerate convergence of stand‐alone steepest descent optimization. Moreover, performance of steepest‐descent preconditioned N‐GMRES is shown to be competitive with standard nonlinear conjugate gradient and limited‐memory Broyden–Fletcher–Goldfarb–Shanno methods for the model problems considered. These results serve to theoretically and numerically establish steepest‐descent preconditioned N‐GMRES as a general optimization method for unconstrained nonlinear optimization, with performance that appears promising compared with established techniques. In addition, it is argued that the real potential of the N‐GMRES optimization framework lies in the fact that it can make use of problem‐dependent nonlinear preconditioners that are more powerful than steepest descent (or, equivalently, N‐GMRES can be used as a simple wrapper around any other iterative optimization process to seek acceleration of that process), and this potential is illustrated with a further application example. Copyright © 2012 John Wiley & Sons, Ltd.     

Preconditioned iterative methods for a class of nonlinear eigenvalue problems

This paper proposes new iterative methods for the efficient computation of the smallest eigenvalue of symmetric nonlinear matrix eigenvalue problems of large order with a monotone dependence on the spectral parameter. Monotone nonlinear eigenvalue problems for differential equations have important applications in mechanics and physics. The discretization of these eigenvalue problems leads to nonlinear eigenvalue problems with very large sparse ill-conditioned matrices monotonically depending on the spectral parameter. To compute the smallest eigenvalue of large-scale matrix nonlinear eigenvalue problems, we suggest preconditioned iterative methods: preconditioned simple iteration method, preconditioned steepest descent method, and preconditioned conjugate gradient method. These methods use only matrix-vector multiplications, preconditioner-vector multiplications, linear operations with vectors, and inner products of vectors. We investigate the convergence and derive grid-independent error estimates for these methods. Numerical experiments demonstrate the practical effectiveness of the proposed methods for a model problem.     

Inverse thermal imaging in materials with nonlinear conductivity by material and shape derivative method

The material and shape derivative method is used for an inverse problem in thermal imaging. The goal is to identify the boundary of unknown inclusions inside an object by applying a heat source and measuring the induced temperature near the boundary of the sample. The problem is studied in the framework of quasilinear elliptic equations. The explicit form is derived of the equations that are satisfied by material and shape derivatives. The existence of weak material derivative is proved. These general findings are demonstrated on the steepest descent optimization procedure. Simulations involving the level set method for tracing the interface are performed for several materials with nonlinear heat conductivity. Copyright © 2011 John Wiley & Sons, Ltd.     

On a Family of Gradient-Type Projection Methods for Nonlinear Ill-Posed Problems

Abstract

We propose and analyze a family of successive projection methods whose step direction is the same as the Landweber method for solving nonlinear ill-posed problems that satisfy the Tangential Cone Condition (TCC). This family encompasses the Landweber method, the minimal error method, and the steepest descent method; thus, providing an unified framework for the analysis of these methods. Moreover, we define new methods in this family, which are convergent for the constant of the TCC in a range twice as large as the one required for the Landweber and other gradient type methods. The TCC is widely used in the analysis of iterative methods for solving nonlinear ill-posed problems. The key idea in this work is to use the TCC in order to construct special convex sets possessing a separation property, and to successively project onto these sets. Numerical experiments are presented for a nonlinear two-dimensional elliptic parameter identification problem, validating the efficiency of our method.     

Long-time behavior of solutions to the derivative nonlinear Schrödinger equation for soliton-free initial data

The large-time behavior of solutions to the derivative nonlinear Schrödinger equation is established for initial conditions in some weighted Sobolev spaces under the assumption that the initial conditions do not support solitons. Our approach uses the inverse scattering setting and the nonlinear steepest descent method of Deift and Zhou as recast by Dieng and McLaughlin.     

On the steepest descent approximation method for the zeros of generalized accretive operators

In this paper we present certain characteristic conditions for the convergence of the generalized steepest descent approximation process to a zero of a generalized strongly accretive operator, defined on a uniformly smooth Banach space. Our study is based on an important result of Reich [S. Reich, An iterative procedure for constructing zeros of accretive sets in Banach spaces, Nonlinear Anal. 2 (1978) 85–92] and given results extend and improve some of the earlier results which include the steepest descent approximation method.     

Two methods for minimizing convex functions in a class of nonconvex sets

The conditional gradient method and the steepest descent method, which are conventionally used for solving convex programming problems, are extended to the case where the feasible set is the set-theoretic difference between a convex set and the union of several convex sets. Iterative algorithms are proposed, and their convergence is examined.     

Weak convergence theorem for zero points of inverse strongly monotone mapping and fixed points of nonexpansive mapping in Hilbert space

We know that variational inequality problem is very important in the nonlinear analysis. For a variational inequality problem defined over a nonempty fixed point set of a nonexpansive mapping in Hilbert space, the strong convergence theorem has been proposed by I. Yamada. The algorithm in this theorem is named the hybrid steepest descent method. Based on this method, we propose a new weak convergence theorem for zero points of inverse strongly monotone mapping and fixed points of nonexpansive mapping in Hilbert space. Using this result, we obtain some new weak convergence theorems which are useful in nonlinear analysis and optimization problem.     

Steepest‐descent proximal point algorithms for a class of variational inequalities in Banach spaces

In this paper, we present a new approach to the problem of finding a common zero for a system of m‐accretive mappings in a uniformly convex Banach space with a uniformly Gâteaux differentiable norm. We propose an implicit iteration method and two explicit ones, based on compositions of resolvents with the steepest‐descent method. We show that our results contain some iterative methods in literature as special cases. An extension of the Xu's regularization method for the proximal point algorithm from Hilbert spaces onto Banach ones under simple conditions of convergence and a new variant for the method of alternating resolvents are obtained. Numerical experiments are given to affirm efficiency of the methods.     

A numerical method for heat conduction in shape reconstruction

This paper is concerned with the problem of the shape reconstruction of the inverse problem for heat conduction with two different boundary conditions in a multiple connected bounded domain. We derive the representation for domain derivative of the corresponding operator. This allows the investigation of the iterative regularization methods solving such ill-posed and nonlinear problem. The numerical examples show that our theory is useful for practical purpose and the proposed algorithm is feasible.     

求解2—D波动方程系数反问题的迭代方法及收敛性

本文针对地震勘探中提出一类重要的２－Ｄ波动方程反演问题，通过定义一个新的非线性算子将２－Ｄ波动方程的反演问题归结为一个新的非线性算子方程，详细讨论了非线性算子的性质，给出了求解反问题的迭代方法，并证明了这种迭代方法的收敛性。     

A numerical method for solving nonlinear ill-posed problems

A two-step iterative process for the numerical solution of nonlinear problems is suggested. In order to avoid the ill-posed inversion of the Fréchet derivative operator, some regularization parameter is introduced. A convergence theorem is proved. The proposed method is illustrated by a numerical example in which a nonlinear inverse problem of gravimetry is considered. Based on the results of the numerical experiments practical recommendations for the choice of the regularization parameter are given. Some other iterative schemes are considered.     

An iterative method for split inclusion problems without prior knowledge of operator norms

In this paper, we study the approximation of solution (assuming existence) for the split inclusion problem in uniformly convex Banach spaces which are also uniformly smooth. We introduce an iterative algorithm in which the stepsizes are selected without the need for any prior information about the bounded linear operator norm and strong convergence obtained. The novelty of our algorithm is that the bounded linear operator norm is not given a priori and stepsizes are constructed step by step in a natural way. Our results extend and improve many recent and important results obtained in the literature on the split inclusion problem and its variations.     

On degeneracy in linear programming and related problems

Methods related to Wolfe's recursive method for the resolution of degeneracy in linear programming are discussed, and a nonrecursive variant which works with probability one suggested. Numerical results for both nondegenerate problems and problems constructed to have degenerate optima are reported. These are obtained using a careful implementation of the projected gradient algorithm [11]. These are compared with results obtained using a steepest descent approach which can be implemented by means of a closely related projected gradient method, and which is not affected by degeneracy in principle. However, the problem of correctly identifying degenerate active sets occurs with both algorithms. The numerical results favour the more standard projected gradient procedure which resolves the degeneracy explicitly. Extension of both methods to general polyhedral convex function minimization problems is sketched.     

Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints

Regularization of ill-posed linear inverse problems via ? ₁ penalization has been proposed for cases where the solution is known to be (almost) sparse. One way to obtain the minimizer of such an ? ₁ penalized functional is via an iterative soft-thresholding algorithm. We propose an alternative implementation to ? ₁-constraints, using a gradient method, with projection on ? ₁-balls. The corresponding algorithm uses again iterative soft-thresholding, now with a variable thresholding parameter. We also propose accelerated versions of this iterative method, using ingredients of the (linear) steepest descent method. We prove convergence in norm for one of these projected gradient methods, without and with acceleration.     

无约束优化问题的一个下降方法

本文研究了无约束优化问题.利用当前和前面迭代点的信息以及曲线搜索技巧产生新的迭代点,得到了一个新的求解无约束优化问题的下降方法.在较弱条件下证明了算法具有全局收敛性.当目标函数为一致凸函数时,证明了算法具有线性收敛速率.初步的数值试验表明算法是有效的.     

On Weak and Strong Convergence of the Projected Gradient Method for Convex Optimization in Real Hilbert Spaces

This work focuses on convergence analysis of the projected gradient method for solving constrained convex minimization problems in Hilbert spaces. We show that the sequence of points generated by the method employing the Armijo line search converges weakly to a solution of the considered convex optimization problem. Weak convergence is established by assuming convexity and Gateaux differentiability of the objective function, whose Gateaux derivative is supposed to be uniformly continuous on bounded sets. Furthermore, we propose some modifications in the classical projected gradient method in order to obtain strong convergence. The new variant has the following desirable properties: the sequence of generated points is entirely contained in a ball with diameter equal to the distance between the initial point and the solution set, and the whole sequence converges strongly to the solution of the problem that lies closest to the initial iterate. Convergence analysis of both methods is presented without Lipschitz continuity assumption.     

Gradient projection method for stable approximation of quasisolutions to irregular nonlinear operator equations

An iterative process of the gradient projection type is constructed and examined as a tool for approximating quasisolutions to irregular nonlinear operator equations in a Hilbert space. One step of this process combines a gradient descent step in a finite-dimensional affine subspace and the Fejrér operator with respect to the convex closed set to which the quasisolution belongs. It is proved that the approximations generated by the proposed method stabilize in a small neighborhood of the desired quasisolution, and the diameter of this neighborhood is estimated.