Limitations of the L-curve method in ill-posed problems

This paper considers the Tikhonov regularization method with the regularization parameter chosen by the so-called L-curve criterion. An infinite dimensional example is constructed for which the selected regularization parameter vanishes too rapidly as the noise to signal ratio in the data goes to zero. As a consequence the computed reconstructions do not converge to the true solution. Numerical examples are given to show that similar phenomena can be observed under more general assumptions in discrete ill-posed problems provided the exact solution of the problem is smooth.This work was partially supported by NATO grant CRG 930044.     

An interior-point method for large constrained discrete ill-posed problems

Ill-posed problems are numerically underdetermined. It is therefore often beneficial to impose known properties of the desired solution, such as nonnegativity, during the solution process. This paper proposes the use of an interior-point method in conjunction with truncated iteration for the solution of large-scale linear discrete ill-posed problems with box constraints. An estimate of the error in the data is assumed to be available. Numerical examples demonstrate the competitiveness of this approach.     

Combining approximate solutions for linear discrete ill-posed problems

Linear discrete ill-posed problems of small to medium size are commonly solved by first computing the singular value decomposition of the matrix and then determining an approximate solution by one of several available numerical methods, such as the truncated singular value decomposition or Tikhonov regularization. The determination of an approximate solution is relatively inexpensive once the singular value decomposition is available. This paper proposes to compute several approximate solutions by standard methods and then extract a new candidate solution from the linear subspace spanned by the available approximate solutions. We also describe how the method may be used for large-scale problems.     

An efficient discretization scheme for solving ill-posed problems

In this paper, we consider a finite-dimensional approximation scheme combined with Tikhonov regularization for solving ill-posed problems. Error estimates are obtained by an a priori parameter choice strategy and the results show that the amount of discrete information required for solving the problem is far less than the traditional finite-dimensional approach.     

An iterative method for linear discrete ill-posed problems with box constraints

Many questions in science and engineering give rise to linear discrete ill-posed problems. Often it is desirable that the computed approximate solution satisfies certain constraints, e.g., that some or all elements of the computed solution be nonnegative. This paper describes an iterative method of active set-type for the solution of large-scale problems of this kind. The method employs conjugate gradient iteration with a stopping criterion based on the discrepancy principle and allows updates of the active set by more than one index at a time.     

The Tikhonov regularization method in elastoplasticity

The numerical simulation of the mechanical behavior of industrial materials is widely used for viability verification, improvement and optimization of designs. Elastoplastic models have been used to forecast the mechanical behavior of different materials. The numerical solution of most elastoplastic models comes across problems of ill-condition matrices. A complete representation of the nonlinear behavior of such structures involves the nonlinear equilibrium path of the body and handling of singular (limit) points and/or bifurcation points. Several techniques to solve numerical problems associated to these points have been disposed in the specialized literature. Two examples are the load-controlled Newton–Raphson method and displacement controlled techniques. However, most of these methods fail due to convergence problems (ill-conditioning) in the neighborhood of limit points, specially when the structure presents snap-through or snap-back equilibrium paths. This study presents the main ideas and formalities of the Tikhonov regularization method and shows how this method can be used in the analysis of dynamic elastoplasticity problems. The study presents a rigorous mathematical demonstration of existence and uniqueness of the solution of well-posed dynamic elastoplasticity problems. The numerical solution of dynamic elastoplasticity problems using Tikhonov regularization is presented in this paper. The Galerkin method is used in this formulation. Effectiveness of Tikhonov’s approach in the regularization of the solution of elastoplasticity problems is demonstrated by means of some simple numerical examples.     

The residual method for regularizing ill-posed problems

Although the residual method, or constrained regularization, is frequently used in applications, a detailed study of its properties is still missing. This sharply contrasts the progress of the theory of Tikhonov regularization, where a series of new results for regularization in Banach spaces has been published in the recent years. The present paper intends to bridge the gap between the existing theories as far as possible. We develop a stability and convergence theory for the residual method in general topological spaces. In addition, we prove convergence rates in terms of (generalized) Bregman distances, which can also be applied to non-convex regularization functionals.We provide three examples that show the applicability of our theory. The first example is the regularized solution of linear operator equations on L^p-spaces, where we show that the results of Tikhonov regularization generalize unchanged to the residual method. As a second example, we consider the problem of density estimation from a finite number of sampling points, using the Wasserstein distance as a fidelity term and an entropy measure as regularization term. It is shown that the densities obtained in this way depend continuously on the location of the sampled points and that the underlying density can be recovered as the number of sampling points tends to infinity. Finally, we apply our theory to compressed sensing. Here, we show the well-posedness of the method and derive convergence rates both for convex and non-convex regularization under rather weak conditions.     

Condition numbers and perturbation analysis for the Tikhonov regularization of discrete ill‐posed problems

One of the most successful methods for solving the least‐squares problem min_x∥Ax?b∥₂ with a highly ill‐conditioned or rank deficient coefficient matrix A is the method of Tikhonov regularization. In this paper, we derive the normwise, mixed and componentwise condition numbers and componentwise perturbation bounds for the Tikhonov regularization. Our results are sharper than the known results. Some numerical examples are given to illustrate our results. Copyright © 2010 John Wiley & Sons, Ltd.     

Error estimates for large-scale ill-posed problems

The computation of an approximate solution of linear discrete ill-posed problems with contaminated data is delicate due to the possibility of severe error propagation. Tikhonov regularization seeks to reduce the sensitivity of the computed solution to errors in the data by replacing the given ill-posed problem by a nearby problem, whose solution is less sensitive to perturbation. This regularization method requires that a suitable value of the regularization parameter be chosen. Recently, Brezinski et al. (Numer Algorithms 49, 2008) described new approaches to estimate the error in approximate solutions of linear systems of equations and applied these estimates to determine a suitable value of the regularization parameter in Tikhonov regularization when the approximate solution is computed with the aid of the singular value decomposition. This paper discusses applications of these and related error estimates to the solution of large-scale ill-posed problems when approximate solutions are computed by Tikhonov regularization based on partial Lanczos bidiagonalization of the matrix. The connection between partial Lanczos bidiagonalization and Gauss quadrature is utilized to determine inexpensive bounds for a family of error estimates. In memory of Gene H. Golub. This work was supported by MIUR under the PRIN grant no. 2006017542-003 and by the University of Cagliari.     

A numerical technique for solving IHCPs using Tikhonov regularization method

This study is intended to provide a numerical algorithm for solving a one-dimensional inverse heat conduction problem. The given heat conduction equation, the boundary conditions, and the initial condition are presented in a dimensionless form. The numerical approach is developed based on the use of the solution to the auxiliary problem as a basis function. To regularize the resultant ill-conditioned linear system of equations, we apply the Tikhonov regularization method to obtain the stable numerical approximation to the solution.     

Tikhonov regularization for a general nonlinear constrained optimization problem

The goal of this study is to analyze the Tikhonov regularization method as applied to a general nonlinear optimization problem that has been previously reduced to an unconstrained optimization problem. The stability properties of the method are examined, and its convergence is proved. The text was submitted by the author in English.     

Tikhonov regularization method for a backward problem for the inhomogeneous time-fractional diffusion equation

Fractional (nonlocal) diffusion equations replace the integer-order derivatives in space and time by their fractional-order analogs and they are used to model anomalous diffusion, especially in physics. In this paper, we study a backward problem for an inhomogeneous time-fractional diffusion equation with variable coefficients in a general bounded domain. Such a backward problem is of practically great importance because we often do not know the initial density of substance, but we can observe the density at a positive moment. The backward problem is ill-posed and we propose a regularizing scheme by using Tikhonov regularization method. We also prove the convergence rate for the regularized solution by using an a priori regularization parameter choice rule. Numerical examples illustrate applicability and high accuracy of the proposed method.     

Cascadic multilevel methods for ill-posed problems

Multilevel methods are popular for the solution of well-posed problems, such as certain boundary value problems for partial differential equations and Fredholm integral equations of the second kind. However, little is known about the behavior of multilevel methods when applied to the solution of linear ill-posed problems, such as Fredholm integral equations of the first kind, with a right-hand side that is contaminated by error. This paper shows that cascadic multilevel methods with a conjugate gradient-type method as basic iterative scheme are regularization methods. The iterations are terminated by a stopping rule based on the discrepancy principle.     

An efficient method to set up a Lanczos based preconditioner for discrete ill-posed problems

Rezghi and Hosseini [M. Rezghi, S.M. Hosseini, Lanczos based preconditioner for discrete ill-posed problems, Computing 88 (2010) 79–96] presented a Lanczos based preconditioner for discrete ill-posed problems. Their preconditioner is constructed by using few steps (e.g., k) of the Lanczos bidiagonalization and corresponding computed singular values and right Lanczos vectors. In this article, we propose an efficient method to set up such preconditioner. Some numerical examples are given to show the effectiveness of the method.     

A numerical method for solving nonlinear ill-posed problems

A two-step iterative process for the numerical solution of nonlinear problems is suggested. In order to avoid the ill-posed inversion of the Fréchet derivative operator, some regularization parameter is introduced. A convergence theorem is proved. The proposed method is illustrated by a numerical example in which a nonlinear inverse problem of gravimetry is considered. Based on the results of the numerical experiments practical recommendations for the choice of the regularization parameter are given. Some other iterative schemes are considered.     

The Tikhonov regularization extended to equilibrium problems involving pseudomonotone bifunctions

We extend the Tikhonov regularization method widely used in optimization and monotone variational inequality studies to equilibrium problems. It is shown that the convergence results obtained from the monotone variational inequality remain valid for the monotone equilibrium problem. For pseudomonotone equilibrium problems, the Tikhonov regularized subproblems have a unique solution only in the limit, but any Tikhonov trajectory tends to the solution of the original problem, which is the unique solution of the strongly monotone equilibrium problem defined on the basis of the regularization bifunction.     

Tikhonov regularization of nonlinear III-posed problems in hilbert scales

In this paper we consider nonlinear ill-posed problems F(x) = y ₀, where x and y ₀ are elements of Hilbert spaces X and Y, respectively. We solve these problems by Tikhonov regularization in a Hilbert scale. This means that the regularizing norm is stronger than the norm in X. Smoothness conditions are given that guarantee convergence rates with respect to the data noise in the original norm in X. We also propose a variant of Tikhonov regularization that yields these rates without needing the knowledge of the smoothness conditions. In this variant F is allowed to be known only approximately and X can be approximated by a finite-dimensional subspace. Finally, we illustrate the required conditions for a simple parameter estimation problem for regularization in Sobolev spaces.     

Discrepancy principles for Tikhonov regularization of ill-posed problems leading to optimal convergence rates

We propose a class ofa posteriori parameter choice strategies for Tikhonov regularization (including variants of Morozov's and Arcangeli's methods) that lead to optimal convergence rates toward the minimal-norm, least-squares solution of an ill-posed linear operator equation in the presence of noisy data.     

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

LSQR, a Lanczos bidiagonalization based Krylov subspace iterative method, and its mathematically equivalent conjugate gradient for least squares problems (CGLS) applied to normal equations system, are commonly used for large-scale discrete ill-posed problems. It is well known that LSQR and CGLS have regularizing effects, where the number of iterations plays the role of the regularization parameter. However, it has long been unknown whether the regularizing effects are good enough to find best possible regularized solutions. Here a best possible regularized solution means that it is at least as accurate as the best regularized solution obtained by the truncated singular value decomposition (TSVD) method. We establish bounds for the distance between the k-dimensional Krylov subspace and the k-dimensional dominant right singular space. They show that the Krylov subspace captures the dominant right singular space better for severely and moderately ill-posed problems than for mildly ill-posed problems. Our general conclusions are that LSQR has better regularizing effects for the first two kinds of problems than for the third kind, and a hybrid LSQR with additional regularization is generally needed for mildly ill-posed problems. Exploiting the established bounds, we derive an estimate for the accuracy of the rank k approximation generated by Lanczos bidiagonalization. Numerical experiments illustrate that the regularizing effects of LSQR are good enough to compute best possible regularized solutions for severely and moderately ill-posed problems, stronger than our theory predicts, but they are not for mildly ill-posed problems and additional regularization is needed.     

An L-ribbon for large underdetermined linear discrete ill-posed problems

The L-curve is a popular aid for determining a suitable value of the regularization parameter when solving ill-conditioned linear systems of equations with a right-hand side vector, which is contaminated by errors of unknown size. However, for large problems, the computation of the L-curve can be quite expensive, because the determination of a point on the L-curve requires that both the norm of the regularized approximate solution and the norm of the corresponding residual vector be available. Recently, an approximation of the L-curve, referred to as the L-ribbon, was introduced to address this difficulty. The present paper discusses how to organize the computation of the L-ribbon when the matrix of the linear system of equations has many more columns than rows. Numerical examples include an application to computerized tomography.