An interior-point method for large constrained discrete ill-posed problems

Ill-posed problems are numerically underdetermined. It is therefore often beneficial to impose known properties of the desired solution, such as nonnegativity, during the solution process. This paper proposes the use of an interior-point method in conjunction with truncated iteration for the solution of large-scale linear discrete ill-posed problems with box constraints. An estimate of the error in the data is assumed to be available. Numerical examples demonstrate the competitiveness of this approach.     

An efficient method to set up a Lanczos based preconditioner for discrete ill-posed problems

Rezghi and Hosseini [M. Rezghi, S.M. Hosseini, Lanczos based preconditioner for discrete ill-posed problems, Computing 88 (2010) 79–96] presented a Lanczos based preconditioner for discrete ill-posed problems. Their preconditioner is constructed by using few steps (e.g., k) of the Lanczos bidiagonalization and corresponding computed singular values and right Lanczos vectors. In this article, we propose an efficient method to set up such preconditioner. Some numerical examples are given to show the effectiveness of the method.     

A reduced Newton method for constrained linear least-squares problems

We propose an iterative method that solves constrained linear least-squares problems by formulating them as nonlinear systems of equations and applying the Newton scheme. The method reduces the size of the linear system to be solved at each iteration by considering only a subset of the unknown variables. Hence the linear system can be solved more efficiently. We prove that the method is locally quadratic convergent. Applications to image deblurring problems show that our method gives better restored images than those obtained by projecting or scaling the solution into the dynamic range.     

A truncated projected SVD method for linear discrete ill-posed problems

Truncated singular value decomposition is a popular solution method for linear discrete ill-posed problems. However, since the singular value decomposition of the matrix is independent of the right-hand side, there are linear discrete ill-posed problems for which this method fails to yield an accurate approximate solution. This paper describes a new approach to incorporating knowledge about properties of the desired solution into the solution process through an initial projection of the linear discrete ill-posed problem. The projected problem is solved by truncated singular value decomposition. Computed examples illustrate that suitably chosen projections can enhance the accuracy of the computed solution.     

The Lagrange method for the regularization of discrete ill-posed problems

In many science and engineering applications, the discretization of linear ill-posed problems gives rise to large ill-conditioned linear systems with the right-hand side degraded by noise. The solution of such linear systems requires the solution of minimization problems with one quadratic constraint, depending on an estimate of the variance of the noise. This strategy is known as regularization. In this work, we propose a modification of the Lagrange method for the solution of the noise constrained regularization problem. We present the numerical results of test problems, image restoration and medical imaging denoising. Our results indicate that the proposed Lagrange method is effective and efficient in computing good regularized solutions of ill-conditioned linear systems and in computing the corresponding Lagrange multipliers. Moreover, our numerical experiments show that the Lagrange method is computationally convenient. Therefore, the Lagrange method is a promising approach for dealing with ill-posed problems. This work was supported by the Italian FIRB Project “Parallel algorithms and Nonlinear Numerical Optimization” RBAU01JYPN.     

The residual method for regularizing ill-posed problems

Although the residual method, or constrained regularization, is frequently used in applications, a detailed study of its properties is still missing. This sharply contrasts the progress of the theory of Tikhonov regularization, where a series of new results for regularization in Banach spaces has been published in the recent years. The present paper intends to bridge the gap between the existing theories as far as possible. We develop a stability and convergence theory for the residual method in general topological spaces. In addition, we prove convergence rates in terms of (generalized) Bregman distances, which can also be applied to non-convex regularization functionals.We provide three examples that show the applicability of our theory. The first example is the regularized solution of linear operator equations on L^p-spaces, where we show that the results of Tikhonov regularization generalize unchanged to the residual method. As a second example, we consider the problem of density estimation from a finite number of sampling points, using the Wasserstein distance as a fidelity term and an entropy measure as regularization term. It is shown that the densities obtained in this way depend continuously on the location of the sampled points and that the underlying density can be recovered as the number of sampling points tends to infinity. Finally, we apply our theory to compressed sensing. Here, we show the well-posedness of the method and derive convergence rates both for convex and non-convex regularization under rather weak conditions.     

Preconditioned iterative methods for linear discrete ill-posed problems from a Bayesian inversion perspective

In this paper we revisit the solution of ill-posed problems by preconditioned iterative methods from a Bayesian statistical inversion perspective. After a brief review of the most popular Krylov subspace iterative methods for the solution of linear discrete ill-posed problems and some basic statistics results, we analyze the statistical meaning of left and right preconditioners, as well as projected-restarted strategies. Computed examples illustrating the interplay between statistics and preconditioning are also presented.     

An iterative method for ill-posed problems with inequalities

Ill-posed problems for integral and operator equations with nonnegativity and band inequality constraints arise in a wide range of applications. The effect and propagation of data perturbations in mathematical programming problems are highly dramatized in the area of ill-posed problems. In this note an iterative method for solving an ill-posed integral inequality and its moment discretization is described.     

An extrapolated TSVD method for linear discrete ill-posed problems with Kronecker structure

This paper describes a new numerical method for the solution of large linear discrete ill-posed problems, whose matrix is a Kronecker product. Problems of this kind arise, for instance, from the discretization of Fredholm integral equations of the first kind in two space-dimensions with a separable kernel. The available data (right-hand side) of many linear discrete ill-posed problems that arise in applications is contaminated by measurement errors. Straightforward solution of these problems generally is not meaningful because of severe error propagation. We discuss how to combine the truncated singular value decomposition (TSVD) with reduced rank vector extrapolation to determine computed approximate solutions that are fairly insensitive to the error in the data. Exploitation of the structure of the problem keeps the computational effort quite small.     

Algorithms for range restricted iterative methods for linear discrete ill-posed problems

Range restricted iterative methods based on the Arnoldi process are attractive for the solution of large nonsymmetric linear discrete ill-posed problems with error-contaminated data (right-hand side). Several derivations of this type of iterative methods are compared in Neuman et al. (Linear Algebra Appl. in press). We describe MATLAB codes for the best of these implementations. MATLAB codes for range restricted iterative methods for symmetric linear discrete ill-posed problems are also presented.     

The structure of iterative methods for symmetric linear discrete ill-posed problems

The iterative solution of large linear discrete ill-posed problems with an error contaminated data vector requires the use of specially designed methods in order to avoid severe error propagation. Range restricted minimal residual methods have been found to be well suited for the solution of many such problems. This paper discusses the structure of matrices that arise in a range restricted minimal residual method for the solution of large linear discrete ill-posed problems with a symmetric matrix. The exploitation of the structure results in a method that is competitive with respect to computer storage, number of iterations, and accuracy.     

Implementations of range restricted iterative methods for linear discrete ill-posed problems

This paper is concerned with iterative solution methods for large linear systems of equations with a matrix of ill-determined rank and an error-contaminated right-hand side. The numerical solution is delicate, because the matrix is very ill-conditioned and may be singular. It is natural to require that the computed iterates live in the range of the matrix when the latter is symmetric, because then the iterates are orthogonal to the null space. Computational experience indicates that it can be beneficial to require that the iterates live in the range of the matrix also when the latter is nonsymmetric. We discuss the design and implementation of iterative methods that determine iterates with this property. New implementations that are particularly well suited for use with the discrepancy principle are described.     

Regularization of ill-posed problems with unbounded operators

Variational regularization and the method of quasisolutions are justified for unbounded closed operators.     

Decomposition methods for large linear discrete ill-posed problems

The solution of large linear discrete ill-posed problems by iterative methods continues to receive considerable attention. This paper presents decomposition methods that split the solution space into a Krylov subspace that is determined by the iterative method and an auxiliary subspace that can be chosen to help represent pertinent features of the solution. Decomposition is well suited for use with the GMRES, RRGMRES, and LSQR iterative schemes.     

An L-ribbon for large underdetermined linear discrete ill-posed problems

The L-curve is a popular aid for determining a suitable value of the regularization parameter when solving ill-conditioned linear systems of equations with a right-hand side vector, which is contaminated by errors of unknown size. However, for large problems, the computation of the L-curve can be quite expensive, because the determination of a point on the L-curve requires that both the norm of the regularized approximate solution and the norm of the corresponding residual vector be available. Recently, an approximation of the L-curve, referred to as the L-ribbon, was introduced to address this difficulty. The present paper discusses how to organize the computation of the L-ribbon when the matrix of the linear system of equations has many more columns than rows. Numerical examples include an application to computerized tomography.     

The conjugate gradient method for linear ill-posed problems with operator perturbations

We consider an ill-posed problem Ta = f_* in Hilbert spaces and suppose that the linear bounded operator T is approximately available, with a known estimate for the operator perturbation at the solution. As a numerical scheme the CGNR-method is considered, that is, the classical method of conjugate gradients by Hestenes and Stiefel applied to the associated normal equations. Two a posteriori stopping rules are introduced, and convergence results are provided for the corresponding approximations, respectively. As a specific application, a parameter estimation problem is considered.     

An iterative method for a system of linear complementarity problems with perturbations and interval data

In this paper, we introduce a total step method for solving a system of linear complementarity problems with perturbations and interval data. It is applied to two interval matrices [A] and [B] and two interval vectors [b] and [c]. We prove that the sequence generated by the total step method converges to ([x^∗],[y^∗]) which includes the solution set for the system of linear complementarity problems defined by any fixed A∈[A],B∈[B],b∈[b] and c∈[c]. We also consider a modification of the method and show that, if we start with two interval vectors containing the limits, then the iterates contain the limits. We close our paper with two examples which illustrate our theoretical results.     

Cascadic multilevel methods for ill-posed problems

Multilevel methods are popular for the solution of well-posed problems, such as certain boundary value problems for partial differential equations and Fredholm integral equations of the second kind. However, little is known about the behavior of multilevel methods when applied to the solution of linear ill-posed problems, such as Fredholm integral equations of the first kind, with a right-hand side that is contaminated by error. This paper shows that cascadic multilevel methods with a conjugate gradient-type method as basic iterative scheme are regularization methods. The iterations are terminated by a stopping rule based on the discrepancy principle.     

Combining approximate solutions for linear discrete ill-posed problems

Linear discrete ill-posed problems of small to medium size are commonly solved by first computing the singular value decomposition of the matrix and then determining an approximate solution by one of several available numerical methods, such as the truncated singular value decomposition or Tikhonov regularization. The determination of an approximate solution is relatively inexpensive once the singular value decomposition is available. This paper proposes to compute several approximate solutions by standard methods and then extract a new candidate solution from the linear subspace spanned by the available approximate solutions. We also describe how the method may be used for large-scale problems.     

On finite termination of an iterative method for linear complementarity problems

Based on a well-known reformulation of the linear complementarity problem (LCP) as a nondifferentiable system of nonlinear equations, a Newton-type method will be described for the solution of LCPs. Under certain assumptions, it will be shown that this method has a finite termination property, i.e., if an iterate is sufficiently close to a solution of LCP, the method finds this solution in one step. This result will be applied to a recently proposed algorithm by Harker and Pang in order to prove that their algorithm also has the finite termination property.