An extrapolated TSVD method for linear discrete ill-posed problems with Kronecker structure

This paper describes a new numerical method for the solution of large linear discrete ill-posed problems, whose matrix is a Kronecker product. Problems of this kind arise, for instance, from the discretization of Fredholm integral equations of the first kind in two space-dimensions with a separable kernel. The available data (right-hand side) of many linear discrete ill-posed problems that arise in applications is contaminated by measurement errors. Straightforward solution of these problems generally is not meaningful because of severe error propagation. We discuss how to combine the truncated singular value decomposition (TSVD) with reduced rank vector extrapolation to determine computed approximate solutions that are fairly insensitive to the error in the data. Exploitation of the structure of the problem keeps the computational effort quite small.     

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

LSQR, a Lanczos bidiagonalization based Krylov subspace iterative method, and its mathematically equivalent conjugate gradient for least squares problems (CGLS) applied to normal equations system, are commonly used for large-scale discrete ill-posed problems. It is well known that LSQR and CGLS have regularizing effects, where the number of iterations plays the role of the regularization parameter. However, it has long been unknown whether the regularizing effects are good enough to find best possible regularized solutions. Here a best possible regularized solution means that it is at least as accurate as the best regularized solution obtained by the truncated singular value decomposition (TSVD) method. We establish bounds for the distance between the k-dimensional Krylov subspace and the k-dimensional dominant right singular space. They show that the Krylov subspace captures the dominant right singular space better for severely and moderately ill-posed problems than for mildly ill-posed problems. Our general conclusions are that LSQR has better regularizing effects for the first two kinds of problems than for the third kind, and a hybrid LSQR with additional regularization is generally needed for mildly ill-posed problems. Exploiting the established bounds, we derive an estimate for the accuracy of the rank k approximation generated by Lanczos bidiagonalization. Numerical experiments illustrate that the regularizing effects of LSQR are good enough to compute best possible regularized solutions for severely and moderately ill-posed problems, stronger than our theory predicts, but they are not for mildly ill-posed problems and additional regularization is needed.     

ON THE ACCURACY OF THE LEAST SQUARES AND THE TOTAL LEAST SQUARES METHODS

Consider solving an overdetermined system of linear algebraic equations by both the least squares method (LS) and the total least squares method (TLS). Extensive published computational evidence shows that when the original system is consistent. one often obtains more accurate solutions by using the TLS method rather than the LS method. These numerical observations contrast with existing analytic perturbation theories for the LS and TLS methods which show that the upper bounds for the LS solution are always smaller than the corresponding upper bounds for the TLS solutions. In this paper we derive a new upper bound for the TLS solution and indicate when the TLS method can be more accurate than the LS method.Many applied problems in signal processing lead to overdetermined systems of linear equations where the matrix and right hand side are determined by the experimental observations (usually in the form of a lime series). It often happens that as the number of columns of the matrix becomes larger, the ra     

On a class of ill-posed non-linear integral equations arising in the interpretation of indirect measurements

In this paper we consider a class of specific Urysohn integral equations for which the solutions are only determined with the exception of rearrangements of function values and associated arguments. As an alternative to Tikhonov's regularization method approximating minimum-norm solutions for this ill-posed class of inverse problems, a constrained least-squares approach is presented. This approach is aimed at finding decreasing rearrangements serving as appropriate solution representatives. It is shown that the inverses of these decresing solutions solve a Fredholm linear integral equation of the first kind.     

GMRES, L-Curves, and Discrete Ill-Posed Problems

The GMRES method is a popular iterative method for the solution of large linear systems of equations with a nonsymmetric nonsingular matrix. This paper discusses application of the GMRES method to the solution of large linear systems of equations that arise from the discretization of linear ill-posed problems. These linear systems are severely ill-conditioned and are referred to as discrete ill-posed problems. We are concerned with the situation when the right-hand side vector is contaminated by measurement errors, and we discuss how a meaningful approximate solution of the discrete ill-posed problem can be determined by early termination of the iterations with the GMRES method. We propose a termination criterion based on the condition number of the projected matrices defined by the GMRES method. Under certain conditions on the linear system, the termination index corresponds to the vertex of an L-shaped curve.     

Chebyshev and L1 solutions of overdetermined systems of linear equations with bounded variables

Two algorithms are here presented. The first one is for obtaining a Chebyshev solution of an overdetermined system of linear equations subject to bounds on the elements of the solution vector. The second algorithm is for obtaining an L₁ solution of an overdetermined system of linear equations subject to the same constraints. Efficient solutions are obtained using linear programming techniques. Numerical results and comments are given.     

A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations

An iterative method based on Lanczos bidiagonalization is developed for computing regularized solutions of large and sparse linear systems, which arise from discretizations of ill-posed problems in partial differential or integral equations. Determination of the regularization parameter and termination criteria are discussed. Comments are given on the computational implementation of the algorithm.Dedicated to Peter Naur on the occasion of his 60th birthday     

An algorithm for finding normal solutions of consistent systems of linear equations

We propose a numerical algorithm for finding the normal solution of consistent systems of linear algebraic equations of incomplete rank of rows. Results of a comparison of a numerical realization of the proposed algorithm with some known subroutines are given.     

Tensor Approximation Approach to Calculation of Singular Values and Vectors for SVD Problem

A new method of calculation of singular values and left and right singular vectors of arbitrary nonsquare matrices is proposed. The method allows one to avoid solutions of high rank systems of linear equations of singular value decomposition problems, which makes it not sensitive to ill-conditioness of the decomposed matrix. On the base of the Eckart–Young theorem, it was shown that each second order r-rank tensor can be represented as a sum of the first rank r-order “coordinate” tensors. A new system of equations for “coordinate” tensor generator vectors was obtained. An iterative method of solution of the system was elaborated. Results of the method were compared with classical methods of solutions of singular value decomposition problems.     

A rational Arnoldi approach for ill-conditioned linear systems

For the solution of full-rank ill-posed linear systems a new approach based on the Arnoldi algorithm is presented. Working with regularized systems, the method theoretically reconstructs the true solution by means of the computation of a suitable function of matrix. In this sense, the method can be referred to as an iterative refinement process. Numerical experiments arising from integral equations and interpolation theory are presented. Finally, the method is extended to work in connection with the standard Tikhonov regularization with the right-hand side contaminated by noise.     

Error estimates for large-scale ill-posed problems

The computation of an approximate solution of linear discrete ill-posed problems with contaminated data is delicate due to the possibility of severe error propagation. Tikhonov regularization seeks to reduce the sensitivity of the computed solution to errors in the data by replacing the given ill-posed problem by a nearby problem, whose solution is less sensitive to perturbation. This regularization method requires that a suitable value of the regularization parameter be chosen. Recently, Brezinski et al. (Numer Algorithms 49, 2008) described new approaches to estimate the error in approximate solutions of linear systems of equations and applied these estimates to determine a suitable value of the regularization parameter in Tikhonov regularization when the approximate solution is computed with the aid of the singular value decomposition. This paper discusses applications of these and related error estimates to the solution of large-scale ill-posed problems when approximate solutions are computed by Tikhonov regularization based on partial Lanczos bidiagonalization of the matrix. The connection between partial Lanczos bidiagonalization and Gauss quadrature is utilized to determine inexpensive bounds for a family of error estimates. In memory of Gene H. Golub. This work was supported by MIUR under the PRIN grant no. 2006017542-003 and by the University of Cagliari.     

An algorithm for linear least squares problems with equality and nonnegativity constraints

We present a new algorithm for solving a linear least squares problem with linear constraints. These are equality constraint equations and nonnegativity constraints on selected variables. This problem, while appearing to be quite special, is the core problem arising in the solution of the general linearly constrained linear least squares problem. The reduction process of the general problem to the core problem can be done in many ways. We discuss three such techniques.The method employed for solving the core problem is based on combining the equality constraints with differentially weighted least squares equations to form an augmented least squares system. This weighted least squares system, which is equivalent to a penalty function method, is solved with nonnegativity constraints on selected variables.Three types of examples are presented that illustrate applications of the algorithm. The first is rank deficient, constrained least squares curve fitting. The second is concerned with solving linear systems of algebraic equations with Hilbert matrices and bounds on the variables. The third illustrates a constrained curve fitting problem with inconsistent inequality constraints.     

On the almost rank deficient case of the least squares problem

This paper studies properties of the solutions to overdetermined systems of linear equations whose matrices are almost rank deficient. Let such a system be approximated by the system of rankr which is closest in the euclidean matrix norm. The residual of the approximate solution depends on the scaling of the independent variable. Sharp bounds are given for the sensitivity of the residual to the scaling of the independent variable. It turns out that these bounds depend critically on a few factors which can be computed in connection with the singular value decomposition. Further the influence from the scaling on the pseudo-inverse solution of a rank deficient system is estimated.This work was sponsored by the Swedish Institute of Applied Mathematics.     

On nonlinear time-dependent multivelocity transport equations

The aim of this paper is to show the existence and uniqueness of a solution for a class of nonlinear multivelocity neutron transport equations and to present a method for the determination of the solution. A recursion formula for the construction of a solution by the method of successive approximations and an error estimate for approximate solutions are given. Both global and local solutions are discussed. The conditions imposed on the nonlinear equation are directly applicable to the linear multivelocity transport problem, and a recursion formula for this system is also given. In the process of approximations, only successive integrations are involved. This fact indicates a promising possibility for the calculation of numerical results by using a computer. A brief discussion of this possibility and the applicability of the results on nonlinear problems are included.     

A preconditioned low‐rank CG method for parameter‐dependent Lyapunov matrix equations

This paper is concerned with the numerical solution of symmetric large‐scale Lyapunov equations with low‐rank right‐hand sides and coefficient matrices depending on a parameter. Specifically, we consider the situation when the parameter dependence is sufficiently smooth, and the aim is to compute solutions for many different parameter samples. On the basis of existing results for Lyapunov equations and parameter‐dependent linear systems, we prove that the tensor containing all solution samples typically allows for an excellent low multilinear rank approximation. Stacking all sampled equations into one huge linear system, this fact can be exploited by combining the preconditioned CG method with low‐rank truncation. Our approach is flexible enough to allow for a variety of preconditioners based, for example, on the sign function iteration or the alternating direction implicit method. Copyright © 2013 John Wiley & Sons, Ltd.     

解不适定问题的一种多尺度方法

1 前言 数学物理反问题是应用数学领域中成长和发展最快的领域之一.反问题大多是不适定的.对于不适定问题的解法已有不少的学者进行探索和研究,Tikhonov正则化方法是一种理论上最完备而在实践上行之有效的方法(参见[5,6,7,8,13]).     

Parametric continuation of the solitary traveling pulse solution in the reaction-diffusion system using the Newton-Krylov method

The matrix-free Newton-Krylov method that uses the GMRES algorithm (an iterative algorithm for solving systems of linear algebraic equations) is used for the parametric continuation of the solitary traveling pulse solution in a three-component reaction-diffusion system. Using the results of integration on a short time interval, we replace the original system of nonlinear algebraic equations by another system that has more convenient (from the viewpoint of the spectral properties of the GMRES algorithm) Jacobi matrix. The proposed parametric continuation proved to be efficient for large-scale problems, and it made it possible to thoroughly examine the dependence of localized solutions on a parameter of the model.     

A Modified Tikhonov Regularization for Linear Operator Equations

We construct with the aid of regularizing filters a new class of improved regularization methods, called modified Tikhonov regularization (MTR), for solving ill-posed linear operator equations. Regularizing properties and asymptotic order of the regularized solutions are analyzed in the presence of noisy data and perturbation error in the operator. With some accurate estimates in the solution errors, optimal convergence order of the regularized solutions is obtained by a priori choice of the regularization parameter. Furthermore, numerical results are given for several ill-posed integral equations, which not only roughly coincide with the theoretical results but also show that MTR can be more accurate than ordinary Tikhonov regularization (OTR).     

Locally extra-optimal regularizing algorithms and a posteriori estimates of the accuracy for ill-posed problems with discontinuous solutions

Local a posteriori estimates of the accuracy of approximate solutions to ill-posed inverse problems with discontinuous solutions from the classes of functions of several variables with bounded variations of the Hardy or Giusti type are studied. Unlike global estimates (in the norm), local estimates of accuracy are carried out using certain linear estimation functionals (e.g., using the mean value of the solution on a given fragment of its support). The concept of a locally extra-optimal regularizing algorithm for solving ill-posed inverse problems, which has an optimal in order local a posteriori estimate, was introduced. A method for calculating local a posteriori estimates of accuracy with the use of some distinguished classes of linear functionals for the problems with discontinuous solutions is proposed. For linear inverse problems, the method is bases on solving specialized convex optimization problems. Examples of locally extra-optimal regularizing algorithms and results of numerical experiments on a posteriori estimation of the accuracy of solutions for different linear estimation functionals are presented.     

Cascadic multilevel methods for ill-posed problems

Multilevel methods are popular for the solution of well-posed problems, such as certain boundary value problems for partial differential equations and Fredholm integral equations of the second kind. However, little is known about the behavior of multilevel methods when applied to the solution of linear ill-posed problems, such as Fredholm integral equations of the first kind, with a right-hand side that is contaminated by error. This paper shows that cascadic multilevel methods with a conjugate gradient-type method as basic iterative scheme are regularization methods. The iterations are terminated by a stopping rule based on the discrepancy principle.