Fractional Tikhonov regularization for linear discrete ill-posed problems

Tikhonov regularization is one of the most popular methods for solving linear systems of equations or linear least-squares problems with a severely ill-conditioned matrix A. This method replaces the given problem by a penalized least-squares problem. The present paper discusses measuring the residual error (discrepancy) in Tikhonov regularization with a seminorm that uses a fractional power of the Moore-Penrose pseudoinverse of AA ^T as weighting matrix. Properties of this regularization method are discussed. Numerical examples illustrate that the proposed scheme for a suitable fractional power may give approximate solutions of higher quality than standard Tikhonov regularization.     

The Lagrange method for the regularization of discrete ill-posed problems

In many science and engineering applications, the discretization of linear ill-posed problems gives rise to large ill-conditioned linear systems with the right-hand side degraded by noise. The solution of such linear systems requires the solution of minimization problems with one quadratic constraint, depending on an estimate of the variance of the noise. This strategy is known as regularization. In this work, we propose a modification of the Lagrange method for the solution of the noise constrained regularization problem. We present the numerical results of test problems, image restoration and medical imaging denoising. Our results indicate that the proposed Lagrange method is effective and efficient in computing good regularized solutions of ill-conditioned linear systems and in computing the corresponding Lagrange multipliers. Moreover, our numerical experiments show that the Lagrange method is computationally convenient. Therefore, the Lagrange method is a promising approach for dealing with ill-posed problems. This work was supported by the Italian FIRB Project “Parallel algorithms and Nonlinear Numerical Optimization” RBAU01JYPN.     

GKB-FP: an algorithm for large-scale discrete ill-posed problems

We describe an algorithm for large-scale discrete ill-posed problems, called GKB-FP, which combines the Golub-Kahan bidiagonalization algorithm with Tikhonov regularization in the generated Krylov subspace, with the regularization parameter for the projected problem being chosen by the fixed-point method by Bazán (Inverse Probl. 24(3), 2008). The fixed-point method selects as regularization parameter a fixed-point of the function ‖r _λ ‖₂/‖f _λ ‖₂, where f _λ is the regularized solution and r _λ is the corresponding residual. GKB-FP determines the sought fixed-point by computing a finite sequence of fixed-points of functions ||r_l^(k)||₂/||f_l^(k)||₂\|r_{\lambda}^{(k)}\|_{2}/\|f_{\lambda}^{(k)}\|_{2}, where f_l^(k)f_{\lambda}^{(k)} approximates f _λ in a k-dimensional Krylov subspace and r_l^(k)r_{\lambda}^{(k)} is the corresponding residual. Based on this and provided the sought fixed-point is reached, we prove that the regularized solutions f_l^(k)f_{\lambda}^{(k)} remain unchanged and therefore completely insensitive to the number of iterations. This and the performance of the method when applied to well-known test problems are illustrated numerically.     

Carleman estimates for the regularization of ill-posed Cauchy problems

This work is a survey of results for ill-posed Cauchy problems for PDEs of the author with co-authors starting from 1991. A universal method of the regularization of these problems is presented here. Even though the idea of this method was previously discussed for specific problems, a universal approach of this paper was not discussed, at least in detail. This approach consists in constructing of such Tikhonov functionals which are generated by unbounded linear operators of those PDEs. The approach is quite general one, since it is applicable to all PDE operators for which Carleman estimates are valid. Three main types of operators of the second order are among them: elliptic, parabolic and hyperbolic ones. The key idea is that convergence rates of minimizers are established using Carleman estimates. Generalizations to nonlinear inverse problems, such as problems of reconstructions of obstacles and coefficient inverse problems are also feasible.     

Error estimates for large-scale ill-posed problems

The computation of an approximate solution of linear discrete ill-posed problems with contaminated data is delicate due to the possibility of severe error propagation. Tikhonov regularization seeks to reduce the sensitivity of the computed solution to errors in the data by replacing the given ill-posed problem by a nearby problem, whose solution is less sensitive to perturbation. This regularization method requires that a suitable value of the regularization parameter be chosen. Recently, Brezinski et al. (Numer Algorithms 49, 2008) described new approaches to estimate the error in approximate solutions of linear systems of equations and applied these estimates to determine a suitable value of the regularization parameter in Tikhonov regularization when the approximate solution is computed with the aid of the singular value decomposition. This paper discusses applications of these and related error estimates to the solution of large-scale ill-posed problems when approximate solutions are computed by Tikhonov regularization based on partial Lanczos bidiagonalization of the matrix. The connection between partial Lanczos bidiagonalization and Gauss quadrature is utilized to determine inexpensive bounds for a family of error estimates. In memory of Gene H. Golub. This work was supported by MIUR under the PRIN grant no. 2006017542-003 and by the University of Cagliari.     

The discrete picard condition for discrete ill-posed problems

We investigate the approximation properties of regularized solutions to discrete ill-posed least squares problems. A necessary condition for obtaining good regularized solutions is that the Fourier coefficients of the right-hand side, when expressed in terms of the generalized SVD associated with the regularization problem, on the average decay to zero faster than the generalized singular values. This is the discrete Picard condition. We illustrate the importance of this condition theoretically as well as experimentally.This work was carried out during a visit to Dept. of Mathematics, UCLA, and was supported by the Danish Natural Science Foundation, by the National Science Foundation under contract NSF-DMS87-14612, and by the Army Research Office under contract No. DAAL03-88-K-0085.     

Combining approximate solutions for linear discrete ill-posed problems

Linear discrete ill-posed problems of small to medium size are commonly solved by first computing the singular value decomposition of the matrix and then determining an approximate solution by one of several available numerical methods, such as the truncated singular value decomposition or Tikhonov regularization. The determination of an approximate solution is relatively inexpensive once the singular value decomposition is available. This paper proposes to compute several approximate solutions by standard methods and then extract a new candidate solution from the linear subspace spanned by the available approximate solutions. We also describe how the method may be used for large-scale problems.     

Decomposition methods for large linear discrete ill-posed problems

The solution of large linear discrete ill-posed problems by iterative methods continues to receive considerable attention. This paper presents decomposition methods that split the solution space into a Krylov subspace that is determined by the iterative method and an auxiliary subspace that can be chosen to help represent pertinent features of the solution. Decomposition is well suited for use with the GMRES, RRGMRES, and LSQR iterative schemes.     

Iterative exponential filtering for large discrete ill-posed problems

Summary. We describe a new iterative method for the solution of large, very ill-conditioned linear systems of equations that arise when discretizing linear ill-posed problems. The right-hand side vector represents the given data and is assumed to be contaminated by measurement errors. Our method applies a filter function of the form with the purpose of reducing the influence of the errors in the right-hand side vector on the computed approximate solution of the linear system. Here is a regularization parameter. The iterative method is derived by expanding in terms of Chebyshev polynomials. The method requires only little computer memory and is well suited for the solution of large-scale problems. We also show how a value of and an associated approximate solution that satisfies the Morozov discrepancy principle can be computed efficiently. An application to image restoration illustrates the performance of the method. Received January 25, 1997 / Revised version received February 9, 1998 / Published online July 28, 1999     

An a posteriori parameter choice for simplified regularization of ill-posed problems

An a posteriori parameter choice strategy is proposed for the simplified regularization of ill-posed problems where no information about the smoothness of the unknown solution is required. If the smoothness of the solution is known then, as a particular case, the optimal rate is achieved. Our result also includes a recent result of Guacanme (1990).     

Minimization of functionals on the solution of a large-scale discrete ill-posed problem

In this work we study the minimization of a linear functional defined on a set of approximate solutions of a discrete ill-posed problem. The primary application of interest is the computation of confidence intervals for components of the solution of such a problem. We exploit the technique introduced by Eldén in 1990, utilizing a parametric programming reformulation involving the solution of a sequence of quadratically constrained least squares problems. Our iterative method, which uses the connection between Lanczos bidiagonalization and Gauss-type quadrature rules to bound certain matrix functionals, is well-suited for large-scale problems, and offers a significant reduction in matrix-vector product evaluations relative to available methods.     

On the second-order asymptotical regularization of linear ill-posed inverse problems

ABSTRACT

In this paper, we establish an initial theory regarding the second-order asymptotical regularization (SOAR) method for the stable approximate solution of ill-posed linear operator equations in Hilbert spaces, which are models for linear inverse problems with applications in the natural sciences, imaging and engineering. We show the regularizing properties of the new method, as well as the corresponding convergence rates. We prove that, under the appropriate source conditions and by using Morozov's conventional discrepancy principle, SOAR exhibits the same power-type convergence rate as the classical version of asymptotical regularization (Showalter's method). Moreover, we propose a new total energy discrepancy principle for choosing the terminating time of the dynamical solution from SOAR, which corresponds to the unique root of a monotonically non-increasing function and allows us to also show an order optimal convergence rate for SOAR. A damped symplectic iterative regularizing algorithm is developed for the realization of SOAR. Several numerical examples are given to show the accuracy and the acceleration effect of the proposed method. A comparison with other state-of-the-art methods are provided as well.     

一类非线性不适定问题的Tikhonov正则化

本文在B anach空间中,研究一类非线性不适定问题的正则化,所研究的算子是多值的,且是近拟的,假定原始问题是可解的,利用带双参数的T ikhonov正则化方法构造出逼近步骤.     

On spline functions and statistical regularization of ill-posed problems

Interpolating natural splines are used for the algebraization and smoothing regularization of linear Fredholm integral equations of the first kind. A simplified version of statistical regularization is presented and, in turn, applied to data graduation by smoothing natural splines.     

Vector extrapolation enhanced TSVD for linear discrete ill-posed problems

The truncated singular value decomposition (TSVD) is a popular solution method for small to moderately sized linear ill-posed problems. The truncation index can be thought of as a regularization parameter; its value affects the quality of the computed approximate solution. The choice of a suitable value of the truncation index generally is important, but can be difficult without auxiliary information about the problem being solved. This paper describes how vector extrapolation methods can be combined with TSVD, and illustrates that the determination of the proper value of the truncation index is less critical for the combined extrapolation-TSVD method than for TSVD alone. The numerical performance of the combined method suggests a new way to determine the truncation index. In memory of Gene H. Golub.     

A wavelet multilevel method for ill-posed problems stabilized by Tikhonov regularization

Summary. An additive Schwarz iteration is described for the fast resolution of linear ill-posed problems which are stabilized by Tikhonov regularization. The algorithm and its analysis are presented in a general framework which applies to integral equations of the first kind discretized either by spline functions or Daubechies wavelets. Numerical experiments are reported on to illustrate the theoretical results and to compare both discretization schemes. Received March 6, 1995 / Revised version received December 27, 1995     

An L-ribbon for large underdetermined linear discrete ill-posed problems

The L-curve is a popular aid for determining a suitable value of the regularization parameter when solving ill-conditioned linear systems of equations with a right-hand side vector, which is contaminated by errors of unknown size. However, for large problems, the computation of the L-curve can be quite expensive, because the determination of a point on the L-curve requires that both the norm of the regularized approximate solution and the norm of the corresponding residual vector be available. Recently, an approximation of the L-curve, referred to as the L-ribbon, was introduced to address this difficulty. The present paper discusses how to organize the computation of the L-ribbon when the matrix of the linear system of equations has many more columns than rows. Numerical examples include an application to computerized tomography.     

Noise-reducing cascadic multilevel methods for linear discrete ill-posed problems

Cascadic multilevel methods for the solution of linear discrete ill-posed problems with noise-reducing restriction and prolongation operators recently have been developed for the restoration of blur- and noise-contaminated images. This is a particular ill-posed problem. The multilevel methods were found to determine accurate restorations with fairly little computational work. This paper describes noise-reducing multilevel methods for the solution of general linear discrete ill-posed problems.     

Subspace-restricted singular value decompositions for linear discrete ill-posed problems

The truncated singular value decomposition is a popular solution method for linear discrete ill-posed problems. These problems are numerically underdetermined. Therefore, it can be beneficial to incorporate information about the desired solution into the solution process. This paper describes a modification of the singular value decomposition that permits a specified linear subspace to be contained in the solution subspace for all truncations. Modifications that allow the range to contain a specified subspace, or that allow both the solution subspace and the range to contain specified subspaces also are described.