On the iteratively regularized Gauss–Newton method in Banach spaces with applications to parameter identification problems

In this paper we propose an extension of the iteratively regularized Gauss–Newton method to the Banach space setting by defining the iterates via convex optimization problems. We consider some a posteriori stopping rules to terminate the iteration and present the detailed convergence analysis. The remarkable point is that in each convex optimization problem we allow non-smooth penalty terms including $L^1$ and total variation like penalty functionals. This enables us to reconstruct special features of solutions such as sparsity and discontinuities in practical applications. Some numerical experiments on parameter identification in partial differential equations are reported to test the performance of our method.     

Iteration-complexity of first-order penalty methods for convex programming

This paper considers a special but broad class of convex programming problems whose feasible region is a simple compact convex set intersected with the inverse image of a closed convex cone under an affine transformation. It studies the computational complexity of quadratic penalty based methods for solving the above class of problems. An iteration of these methods, which is simply an iteration of Nesterov’s optimal method (or one of its variants) for approximately solving a smooth penalization subproblem, consists of one or two projections onto the simple convex set. Iteration-complexity bounds expressed in terms of the latter type of iterations are derived for two quadratic penalty based variants, namely: one which applies the quadratic penalty method directly to the original problem and another one which applies the latter method to a perturbation of the original problem obtained by adding a small quadratic term to its objective function.     

基于埃特金加速的改进Landweber迭代正则化方法

为克服Landweber迭代正则化方法在求解大规模不适定问题时收敛速度慢的不足,将埃特金加速技巧与不动点迭代相结合,构建了能快速收敛的改进Landweber迭代正则化方法.数值实验结果表明:改进的迭代正则化方法在稳定求解不适定问题时,能够快速地收敛至问题的最优解,较Landweber迭代正则化方法大大提高了收敛速度.     

A Runge–Kutta type modified Landweber method for nonlinear ill-posed operator equations

In this paper, we investigate the convergence behavior of a Runge–Kutta type modified Landweber method for nonlinear ill-posed operator equations. In order to improve the stability and convergence of the Landweber iteration, a 2-stage Gauss-type Runge–Kutta method is applied to the continuous analogy of the modified Landweber method, to give a new modified Landweber method, called R–K type modified Landweber method. Under some appropriate conditions, we prove the convergence of the proposed method. We conclude with a numerical example confirming the theoretical results, including comparisons to the modified Landweber iteration.     

求解不适定问题的快速Landweber迭代法

本文从一般迭代法的级数形式出发，将一般迭代法的每一步分解为矩阵计算和求解两步，并对其中的矩阵计算部分进行了修改，在此基础上提出了快速迭代法，最后通过数值实验验证了我们的算法不仅提高了计算速度，同时也大大减少了计算量，是一种效率很高的算法。     

Frozen Landweber Iteration for Nonlinear Ill-Posed Problems

In this paper we propose a modification of the Landweber iteration termed frozen Landweberiteration for nonlinear ill-posed problems.A convergence analysis for this iteration is presented.The numericalperformance of this frozen Landweber iteration for a nonlinear Hammerstein integral equation is compared withthat of the Landweber iteration.We obtain a shorter running time of the frozen Landweber iteration based onthe same convergence accuracy.     

Convergence results of Landweber iterations for linear systems

The Landweber scheme is a method for algebraic image reconstructions. The convergence behavior of the Landweber scheme is of both theoretical and practical importance. Using the diagonalization of matrix, we derive a neat iterative representation formula for the Landweber schemes and consequently establish the convergence conditions of Landweber iteration. This work refines our previous convergence results on the Landweber scheme.     

An implicit Landweber method for nonlinear ill-posed operator equations

In this paper, we are interested in the solution of nonlinear inverse problems of the form F(x)=y. We propose an implicit Landweber method, which is similar to the third-order midpoint Newton method in form, and consider the convergence behavior of the implicit Landweber method. Using the discrepancy principle as a stopping criterion, we obtain a regularization method for ill-posed problems. We conclude with numerical examples confirming the theoretical results, including comparisons with the classical Landweber iteration and presented modified Landweber methods.     

An efficient augmented Lagrangian method with applications to total variation minimization

Based on the classic augmented Lagrangian multiplier method, we propose, analyze and test an algorithm for solving a class of equality-constrained non-smooth optimization problems (chiefly but not necessarily convex programs) with a particular structure. The algorithm effectively combines an alternating direction technique with a nonmonotone line search to minimize the augmented Lagrangian function at each iteration. We establish convergence for this algorithm, and apply it to solving problems in image reconstruction with total variation regularization. We present numerical results showing that the resulting solver, called TVAL3, is competitive with, and often outperforms, other state-of-the-art solvers in the field.     

Scheduling for a processor sharing system with linear slowdown

We consider the problem of scheduling arrivals to a congestion system with a finite number of users having identical deterministic demand sizes. The congestion is of the processor sharing type in the sense that all users in the system at any given time are served simultaneously. However, in contrast to classical processor sharing congestion models, the processing slowdown is proportional to the number of users in the system at any time. That is, the rate of service experienced by all users is linearly decreasing with the number of users. For each user there is an ideal departure time (due date). A centralized scheduling goal is then to select arrival times so as to minimize the total penalty due to deviations from ideal times weighted with sojourn times. Each deviation penalty is assumed quadratic, or more generally convex. But due to the dynamics of the system, the scheduling objective function is non-convex. Specifically, the system objective function is a non-smooth piecewise convex function. Nevertheless, we are able to leverage the structure of the problem to derive an algorithm that finds the global optimum in a (large but) finite number of steps, each involving the solution of a constrained convex program. Further, we put forward several heuristics. The first is the traversal of neighbouring constrained convex programming problems, that is guaranteed to reach a local minimum of the centralized problem. This is a form of a “local search”, where we use the problem structure in a novel manner. The second is a one-coordinate “global search”, used in coordinate pivot iteration. We then merge these two heuristics into a unified “local–global” heuristic, and numerically illustrate the effectiveness of this heuristic.     

A convergence analysis of an inexact Newton-Landweber iteration method for nonlinear problem

In this paper, we study the convergence and the convergence rates of an inexact Newton–Landweber iteration method for solving nonlinear inverse problems in Banach spaces. Opposed to the traditional methods, we analyze an inexact Newton–Landweber iteration depending on the Hölder continuity of the inverse mapping when the data are not contaminated by noise. With the namely Hölder-type stability and the Lipschitz continuity of DF, we prove convergence and monotonicity of the residuals defined by the sequence induced by the iteration. Finally, we discuss the convergence rates.     

A total variation based approach to correcting surface coil magnetic resonance images

Magnetic resonance images which are corrupted by noise and by smooth modulations are corrected using a variational formulation incorporating a total variation like penalty for the image and a high order penalty for the modulation. The optimality system is derived and numerically discretized. The cost functional used is non-convex, but it possesses a bilinear structure which allows the ambiguity among solutions to be resolved technically by regularization and practically by normalizing the maximum value of the modulation. Since the cost is convex in each single argument, convex analysis is used to formulate the optimality condition for the image in terms of a primal-dual system. To solve the optimality system, a nonlinear Gauss-Seidel outer iteration is used in which the cost is minimized with respect to one variable after the other using an inner generalized Newton iteration. Favorable computational results are shown for artificial phantoms as well as for realistic magnetic resonance images. Reported computational times demonstrate the feasibility of the approach in practice.     

Exterior point algorithms for nearest points and convex quadratic programs

We consider the problem of finding the nearest point (by Euclidean distance) in a simplicial cone to a given point, and develop an exterior penalty algorithm for it. Each iteration in the algorithm consists of a single Newton step following a reduction in the value of the penalty parameter. Proofs of convergence of the algorithm are given. Various other versions of exterior penalty algorithms for nearest point problems in nonsimplicial polyhedral cones and for convex quadratic programs, all based on a single descent step following a reduction in the value of the penalty parameter per iteration, are discussed. The performance of these algorithms in large scale computational experiments is very encouraging. It shows that the number of iterations grows very slowly, if at all, with the dimension of the problem.Partially supported by NSF Grant No. ECS-8521183, and by the two universities.     

Asymptotic L-based comparison of Tikhonov regularization and Landweber iteration

The distance between the Tikhonov and Landweber regularized solutions of a linear inverse problem is partly controlled by the L^∞-norm of the difference in their corresponding singular value filters. For large Landweber iteration number, the regularization parameter of the closest Tikhonov filter to a given Landweber filter is determined. This asymptotically computed parameter compares well with the numerically computed value even for moderate sized iteration number. A consequence of the analysis is to determine the range of singular values to which the difference in regularized solutions is most sensitive.     

Coupling the Proximal Point Algorithm with Approximation Methods

We study the convergence of a diagonal process for minimizing a closed proper convex function f, in which a proximal point iteration is applied to a sequence of functions approximating f. We prove that, when the approximation is sufficiently fast, and also when it is sufficiently slow, the sequence generated by the method converges toward a minimizer of f. Comparison to previous work is provided through examples in penalty methods for linear programming and Tikhonov regularization.     

Nonstationary iterated Tikhonov regularization in Banach spaces with uniformly convex penalty terms

We consider the nonstationary iterated Tikhonov regularization in Banach spaces which defines the iterates via minimization problems with uniformly convex penalty term. The penalty term is allowed to be non-smooth to include \(L^1\) and total variation (TV) like penalty functionals, which are significant in reconstructing special features of solutions such as sparsity and discontinuities in practical applications. We present the detailed convergence analysis and obtain the regularization property when the method is terminated by the discrepancy principle. In particular we establish the strong convergence and the convergence in Bregman distance which sharply contrast with the known results that only provide weak convergence for a subsequence of the iterative solutions. Some numerical experiments on linear integral equations of first kind and parameter identification in differential equations are reported.     

A nonlinear programming algorithm based on non-coercive penalty functions

 We consider first the differentiable nonlinear programming problem and study the asymptotic behavior of methods based on a family of penalty functions that approximate asymptotically the usual exact penalty function. We associate two parameters to these functions: one is used to control the slope and the other controls the deviation from the exact penalty. We propose a method that does not change the slope for feasible iterates and show that for problems satisfying the Mangasarian-Fromovitz constraint qualification all iterates will remain feasible after a finite number of iterations. The same results are obtained for non-smooth convex problems under a Slater qualification condition. Received: September 2000 / Accepted: June 2002 Published online: March 21, 2003 Research partially supported by CAPES, Brazil Research partially supported by CNPq, Brazil, and CONICIT, Venezuela. Mathematics Subject Classification (1991): 20E28, 20G40, 20C20     

Hilbert尺度下求解非线性不适定问题的多水平迭代法

本文吸取了多水平方法的思想，采用多水平方法提供了离散化参数和迭代初值的合理的选择方法，提出了Hilbert尺度下求解非线性不适定问题的多水平Landweber迭代算法，并给出了算法的收敛性分析，证明了算法在整体上提高了Hilbert尺度下的Landweber迭代法的迭代效率。     

Iterative hard thresholding methods for l_0 regularized convex cone programming

In this paper we consider \(l_0\) regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving \(l_0\) regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer. Also, we establish the iteration complexity of the IHT method for finding an \({{\epsilon }}\) -local-optimal solution. We then propose a method for solving \(l_0\) regularized convex cone programming by applying the IHT method to its quadratic penalty relaxation and establish its iteration complexity for finding an \({{\epsilon }}\) -approximate local minimizer. Finally, we propose a variant of this method in which the associated penalty parameter is dynamically updated, and show that every accumulation point is a local izer of the problem.     

An iterative multi level algorithm for solving nonlinear ill–posed problems

Summary. The convergence analysis of Landweber's iteration for the solution of nonlinear ill–posed problem has been developed recently by Hanke, Neubauer and Scherzer. In concrete applications, sufficient conditions for convergence of the Landweber iterates developed there (although quite natural) turned out to be complicated to verify analytically. However, in numerical realizations, when discretizations are considered, sufficient conditions for local convergence can usually easily be stated. This paper is motivated by these observations: Initially a discretization is fixed and a discrete Landweber iteration is implemented until an appropriate stopping criterion becomes active. The output is used as an initial guess for a finer discretization. An advantage of this method is that the convergence analysis can be considered in a family of finite dimensional spaces. The numerical performance of this multi level algorithm is compared with Landweber's iteration. Received October 21, 1996 / Revised version received July 28, 1997