Regularization of Discrete Ill-Posed Problems

The paper concerns conditioning aspects of finite-dimensional problems arising when the Tikhonov regularization is applied to discrete ill-posed problems. A relation between the regularization parameter and the sensitivity of the regularized solution is investigated. The main conclusion is that the condition number can be decreased only to the square root of that for the nonregularized problem. The convergence of solutions of regularized discrete problems to the exact generalized solution is analyzed just in the case when the regularization corresponds to the minimal condition number. The convergence theorem is proved under the assumption of the suitable relation between the discretization level and the data error. As an example the method of truncated singular value decomposition with regularization is considered. This revised version was published online in July 2006 with corrections to the Cover Date.     

Relaxed Steepest Descent and Cauchy-Barzilai-Borwein Method

The negative gradient direction to find local minimizers has been associated with the classical steepest descent method which behaves poorly except for very well conditioned problems. We stress out that the poor behavior of the steepest descent methods is due to the optimal Cauchy choice of steplength and not to the choice of the search direction. We discuss over and under relaxation of the optimal steplength. In fact, we study and extend recent nonmonotone choices of steplength that significantly enhance the behavior of the method. For a new particular case (Cauchy-Barzilai-Borwein method), we present a convergence analysis and encouraging numerical results to illustrate the advantages of using nonmonotone overrelaxations of the gradient method.     

L-曲线估计确定正则参数的双网格迭代法

本文考虑对不适定问题离散化得到的大规模不适定线性方程组进行Tiknonov正则化,然后用双网格迭代法求解得到的Tikhonov正则化方程组,并用L-曲线估计法来确定正则参数.试验问题的数值结果表明双网格迭代法求解正则化后的对称正定线性方程组效果很好,且L-曲线估计法确定正则参数计算量很小.     

GMRES, L-Curves, and Discrete Ill-Posed Problems

The GMRES method is a popular iterative method for the solution of large linear systems of equations with a nonsymmetric nonsingular matrix. This paper discusses application of the GMRES method to the solution of large linear systems of equations that arise from the discretization of linear ill-posed problems. These linear systems are severely ill-conditioned and are referred to as discrete ill-posed problems. We are concerned with the situation when the right-hand side vector is contaminated by measurement errors, and we discuss how a meaningful approximate solution of the discrete ill-posed problem can be determined by early termination of the iterations with the GMRES method. We propose a termination criterion based on the condition number of the projected matrices defined by the GMRES method. Under certain conditions on the linear system, the termination index corresponds to the vertex of an L-shaped curve.     

A Regularized Iterative Scheme for Solving Nonlinear Ill-Posed Problems

In this article, we consider a regularized iterative scheme for solving nonlinear ill-posed problems. The convergence analysis and error estimates are derived by choosing the regularization parameter according to both a priori and a posteriori methods. The iterative scheme is stopped using an a posteriori stopping rule, and we prove that the scheme converges to the solution of the well-known Lavrentiev scheme. The salient features of the proposed scheme are: (i) convergence and error estimate analysis require only weaker assumptions compared to standard assumptions followed in literature, and (ii) consideration of an adaptive a posteriori stopping rule and a parameter choice strategy that gives the same convergence rate as that of an a priori method without using the smallness assumption, the source condition. The above features are very useful from theory and application points of view. We also supply the numerical results to illustrate that the method is adaptable. Further, we compare the numerical result of the proposed method with the standard approach to demonstrate that our scheme is stable and achieves good computational output.     

广义Φ-增生算子最速下降迭代序列的收敛定理

证明了任意实Banach空间广义Φ-增生算子的最速下降迭代序列的收敛定理,推广了ZeiqingLiu和周海云等人的近期结果.     

An Iterative Multigrid Regularization Method for Toeplitz Discrete Ill-Posed Problems

Iterative regularization multigrid methods have been successfully applied to signal/image deblurring problems. When zero-Dirichlet boundary conditions are imposed the deblurring matrix has a Toeplitz structure and it is potentially full. A crucial task of a multilevel strategy is to preserve the Toeplitz structure at the coarse levels which can be exploited to obtain fast computations. The smoother has to be an iterative regularization method. The grid transfer operator should preserve the regularization property of the smoother. This paper improves the iterative multigrid method proposed in [11] introducing a wavelet soft-thresholding denoising post-smoother. Such post-smoother avoids the noise amplification that is the cause of the semi-convergence of iterative regularization methods and reduces ringing effects. The resulting iterative multigrid regularization method stabilizes the iterations so that the imprecise (over) estimate of the stopping iteration does not have a deleterious effect on the computed solution. Numerical examples of signal and image deblurring problems confirm the effectiveness of the proposed method.     

应用正则化子建立求解不适定问题的正则化方法的探讨

根据紧算子的奇异系统理论,提出一种新的正则化子进而建立了一类新的求解不适定问题的正则化方法。分别通过正则参数的先验选取和后验确定方法,证明了正则解的收敛性并得到了其最优的渐近收敛阶;验证了应用Newton迭代法计算最佳参数的可行性。最后建立了当算子与右端均有扰动时相应的正则化求解策略。文中所述方法完善了一般优化正则化策略的构造理论。     

A New Gradient Method for Ill-Posed Problems

In this paper, we present a new gradient method for linear and nonlinear ill-posed problems F(x) = y. Combined with the discrepancy principle as stopping rule it is a regularization method that yields convergence to an exact solution if the operator F satisfies a tangential cone condition. If the exact solution satisfies smoothness conditions, then even convergence rates can be proven. Numerical results show that the new method in most cases needs less iteration steps than Landweber iteration, the steepest descent or minimal error method.     

On the Behavior of the Gradient Norm in the Steepest Descent Method

It is well known that the norm of the gradient may be unreliable as a stopping test in unconstrained optimization, and that it often exhibits oscillations in the course of the optimization. In this paper we present results descibing the properties of the gradient norm for the steepest descent method applied to quadratic objective functions. We also make some general observations that apply to nonlinear problems, relating the gradient norm, the objective function value, and the path generated by the iterates.     

Regularization of Ill-Posed Problems by Envelope Guided Conjugate Gradients

Abstract

We propose a new way to iteratively solve large scale ill-posed problems by exploiting the relation between Tikhonov regularization and multiobjective optimization to obtain, iteratively, approximations to the Tikhonov L-curve and its corner. Monitoring the change of the approximate L-curves allows us to adjust the regularization parameter adaptively during a preconditioned conjugate gradient iteration, so that the desired solution can be reconstructed with a low number of iterations. We apply the technique to an idealized image reconstruction problem in positron emission tomography.     

Renormalised Steepest Descent in Hilbert Space Converges to a Two-Point Attractor

The result that for quadratic functions the classical steepest descent algorithm in R ^d converges locally to a two-point attractor was proved by Akaike. In this paper this result is proved for bounded quadratic operators in Hilbert space. The asymptotic rate of convergence is shown to depend on the starting point while, as expected, confirming the Kantorovich bounds. The introduction of a relaxation coefficient in the steepest-descent algorithm completely changes its behaviour, which may become chaotic. Different attractors are presented. We show that relaxation allows a significantly improved rate of convergence.     

Modified Iterative Runge-Kutta-Type Methods for Nonlinear Ill-Posed Problems

This work is devoted to the convergence analysis of a modified Runge-Kutta-type iterative regularization method for solving nonlinear ill-posed problems under a priori and a posteriori stopping rules. The convergence rate results of the proposed method can be obtained under a Hölder-type sourcewise condition if the Fréchet derivative is properly scaled and locally Lipschitz continuous. Numerical results are achieved by using the Levenberg-Marquardt, Lobatto, and Radau methods.     

Steepest descent preconditioning for nonlinear GMRES optimization

Steepest descent preconditioning is considered for the recently proposed nonlinear generalized minimal residual (N‐GMRES) optimization algorithm for unconstrained nonlinear optimization. Two steepest descent preconditioning variants are proposed. The first employs a line search, whereas the second employs a predefined small step. A simple global convergence proof is provided for the N‐GMRES optimization algorithm with the first steepest descent preconditioner (with line search), under mild standard conditions on the objective function and the line search processes. Steepest descent preconditioning for N‐GMRES optimization is also motivated by relating it to standard non‐preconditioned GMRES for linear systems in the case of a standard quadratic optimization problem with symmetric positive definite operator. Numerical tests on a variety of model problems show that the N‐GMRES optimization algorithm is able to very significantly accelerate convergence of stand‐alone steepest descent optimization. Moreover, performance of steepest‐descent preconditioned N‐GMRES is shown to be competitive with standard nonlinear conjugate gradient and limited‐memory Broyden–Fletcher–Goldfarb–Shanno methods for the model problems considered. These results serve to theoretically and numerically establish steepest‐descent preconditioned N‐GMRES as a general optimization method for unconstrained nonlinear optimization, with performance that appears promising compared with established techniques. In addition, it is argued that the real potential of the N‐GMRES optimization framework lies in the fact that it can make use of problem‐dependent nonlinear preconditioners that are more powerful than steepest descent (or, equivalently, N‐GMRES can be used as a simple wrapper around any other iterative optimization process to seek acceleration of that process), and this potential is illustrated with a further application example. Copyright © 2012 John Wiley & Sons, Ltd.     

A Joint Tikhonov Regularization and Augmented Lagrange Approach for Ill-Posed State Constrained Control Problems with Sparse Controls

Abstract

We provide a modified augmented Lagrange method coupled with a Tikhonov regularization for solving ill-posed state constrained elliptic optimal control problems with sparse controls. We consider a linear quadratic optimal control problem without any additional L² regularization terms. The sparsity is guaranteed by an additional L¹ term. Here, the modification of the classical augmented Lagrange method guarantees us uniform boundedness of the multiplier that corresponds to the state constraints. We present a coupling between the regularization parameter introduced by the Tikhonov regularization and the penalty parameter from the augmented Lagrange method, which allows us to prove strong convergence of the controls and their corresponding states. Moreover, convergence results proving the weak convergence of the adjoint state and weak*-convergence of the multiplier are provided. Finally, we demonstrate our method in several numerical examples.     

Steepest Descent Methods with Generalized Distances for Constrained Optimization

We consider the problem s.t. , where C is a closed and covex subset of with nonempty interior, and introduce a family of interior point methods for this problem, which can be seen as approximate versions of generalized proximal point methods. Each step consists of a one-dimensional search along either a curve or a segment in the interior of C. The information about the boundary of C is contained in a generalized distance which defines the segment of the curve, and whose gradient diverges at the boundary of C. The objective of the search is either f or f plus a regularizing term. When , the usual steepest descent method is a particular case of our general scheme, and we manage to extend known convergence results for the steepest descent method to our family: for nonregularized one-dimensional searches,under a level set boundedness assumption on f, the sequence is bounded, the difference between consecutive iterates converges to 0 and every cluster point of the sequence satisfies first-order optimality conditions for the problem, i.e. is a solution if f is convex. For the regularized search and convex f, no boundedness condition on f is needed and full and global convergence of the sequence to a solution of the problem is established.     

A Regularized Hybrid Steepest Descent Method for Variational Inclusions

This article is concerned with a generalization of the hybrid steepest descent method from variational inequalities to the multivalued case. This will be reached by replacing the multivalued operator by its Yosida approximate, which is always Lipschitz continuous. It is worth mentioning that the hybrid steepest descent method is an algorithmic solution to variational inequality problems over the fixed point set of certain nonexpansive mappings and has remarkable applicability to the constrained nonlinear inverse problems like image recovery and MIMO communication systems (see, e.g., [9 I. Yamada , M. Yukawa , and M. Yamagishi ( 2011 ). Minimizing the moreau envelope of nonsmooth convex functions over the fixed point set of certain quasi-nonexpansive mappings . In Fixed Point Algorithms for Inverse Problems in Science and Engineering ( H.H. Bauschke , R. Burachik , P.L. Combettes , V. Elser , D.R. Luke , and H. Wolkowicz , eds.), Springer-Verlag , New York , Chapter 17 , pp. 343 – 388 . [Google Scholar], 10 I. Yamada , Ogura , and N. Shirakawa ( 2002 ). A numerically robust hybrid steepest descent method for the convexly constrained generalized inverse problems . In Inverse Problems, Image Analysis and Medical Imaging. Contemporary Mathematics ( Z. Nashed and O. Scherzer , eds.), American Mathematical Society , Providence , RI , Vol. 313 , pp. 269 – 305 . [Google Scholar]]).     

REGULARIZATION METHODS FOR NONLINEAR ILL-POSED PROBLEMS WITH ACCRETIVE OPERATORS

This article is devoted to the regularization of nonlinear ill-posed problems with accretive operators in Banach spaces. The data involved are assumed to be known approximately. The authors concentrate their discussion on the convergence rates of regular solutions.     

广义最速下降法逼近拟增生算子方程解的一个特征条件

倪仁兴最近的文章研究了广义最速下降法强收敛于拟增生算子方程解的一特征条件．本文对此进行了修正和改进，给出了一个新的特征条件．所得结果同时改进和推广了一些已有的结果．     

拟增生算子方程广义最速下降法的收敛性特征条件

本文给出了广义最速下降法强收敛于定义在一致光滑实Banach空间的真子集上的局部有界拟增生算子的零点的一特征条件.所得的结果推广和统一如徐宗本和蒋耀林等人的相应结果.