Preconditionings and splittings for rectangular systems

Summary Recently Eiermann, Marek, and Niethammer have shown how to applysemiiterative methods to a fixed point systemx=Tx+c which isinconsistent or in whichthe powers of the fixed point operator T have no limit, to obtain iterative methods which converge to some approximate solution to the fixed point system. In view of their results we consider here stipulations on apreconditioning QAx=Qb of the systemAx=b and, separately, on asplitting A=M–N which lead to fixed point systems such that, with the aid of a semiiterative method, the iterative scheme converges to a weighted Moore-Penrose solution to the systemAx=b. We show in several ways that to obtain a meaningful limit point from a semiiterative method requires less restrictions on the splittings or the reconditionings than those which have been required in the classical Picard iterative method (see, e.g., the works of Berman and Plemmons, Berman and Neumann, and Tanabe).We pay special attention to the case when the weighted Moore-Penrose solution which is sought is the minimal norm least squares solution toAx=b.Research supported by the Deutsche ForschungsgemeinschaftPartially supported by AFOSR research grant 88-0047     

Finite alogorithms for robust linear regression

In this paper Hubert's M-estimator for robust linear regression is analyzed. Newton type methods for solution of the problem are defined and analyzed, and finite convergence is proved. Numerical experiments with a large number of test problems demonstrate efficiency and indicate that this kind of approach may be useful also in solving thel ₁ problem.     

On regularization methods based on dynamic programming techniques

In this article, we investigate the connection between regularization theory for inverse problems and dynamic programming theory. This is done by developing two new regularization methods, based on dynamic programming techniques. The aim of these methods is to obtain stable approximations to the solution of linear inverse ill-posed problems. We follow two different approaches and derive a continuous and a discrete regularization method. Regularization properties for both methods are proved as well as rates of convergence. A numerical benchmark problem concerning integral operators with convolution kernels is used to illustrate the theoretical results.     

Convergence rates of iterated Tikhonov regularized solutions of nonlinear III — posed problems

Summary In this paper we investigate iterated Tikhonov regularization for the solution of nonlinear ill-posed problems. In the case of linear ill-posed problems it is well-known that (under appropriate assumptions) then-th iterated regularized solutions can converge likeO(²² /(2n+1)), where denotes the noise level of the data perturbation. We give conditions that guarantee this convergence rate also for nonlinear ill-posed problems, and motivate these conditions by the mapping degree. The results are derived by a comparison of the iterated regularized solutions of the nonlinear problem with the iterated regularized solutions of its linearization. Numerical examples are presented.Supported by the Austrian Fonds zur Förderung der wissenschaftlichen Forschung,project P-7869 PHY, and by the Christian Doppler Society     

A Modified Tikhonov Regularization for Linear Operator Equations

We construct with the aid of regularizing filters a new class of improved regularization methods, called modified Tikhonov regularization (MTR), for solving ill-posed linear operator equations. Regularizing properties and asymptotic order of the regularized solutions are analyzed in the presence of noisy data and perturbation error in the operator. With some accurate estimates in the solution errors, optimal convergence order of the regularized solutions is obtained by a priori choice of the regularization parameter. Furthermore, numerical results are given for several ill-posed integral equations, which not only roughly coincide with the theoretical results but also show that MTR can be more accurate than ordinary Tikhonov regularization (OTR).     

Convergent nonnegative matrices and iterative methods for consistent linear systems

Summary In this paper we study linear stationary iterative methods with nonnegative iteration matrices for solving singular and consistent systems of linear equationsAx=b. The iteration matrices for the schemes are obtained via regular and weak regular splittings of the coefficients matrixA. In certain cases when only some necessary, but not sufficient, conditions for the convergence of the iterations schemes exist, we consider a transformation on the iteration matrices and obtain new iterative schemes which ensure convergence to a solution toAx=b. This transformation is parameter-dependent, and in the case where all the eigenvalues of the iteration matrix are real, we show how to choose this parameter so that the asymptotic convergence rate of the new schemes is optimal. Finally, some applications to the problem of computing the stationary distribution vector for a finite homogeneous ergodic Markov chain are discussed.Research sponsored in part by US Army Research Office     

A stationary iterative pseudoinverse algorithm

Iterative methods applied to the normal equationsA ^T Ax=A ^T b are sometimes used for solving large sparse linear least squares problems. However, when the matrix is rank-deficient many methods, although convergent, fail to produce the unique solution of minimal Euclidean norm. Examples of such methods are the Jacobi and SOR methods as well as the preconditioned conjugate gradient algorithm. We analyze here an iterative scheme that overcomes this difficulty for the case of stationary iterative methods. The scheme combines two stationary iterative methods. The first method produces any least squares solution whereas the second produces the minimum norm solution to a consistent system. This work was supported by the Swedish Research Council for Engineering Sciences, TFR.     

Two-level preconditioners for regularized inverse problems I: Theory

Summary. We compare additive and multiplicative Schwarz preconditioners for the iterative solution of regularized linear inverse problems, extending and complementing earlier results of Hackbusch, King, and Rieder. Our main findings are that the classical convergence estimates are not useful in this context: rather, we observe that for regularized ill-posed problems with relevant parameter values the additive Schwarz preconditioner significantly increases the condition number. On the other hand, the multiplicative version greatly improves conditioning, much beyond the existing theoretical worst-case bounds. We present a theoretical analysis to support these results, and include a brief numerical example. More numerical examples with real applications will be given elsewhere. Received May 28, 1998 / Published online: July 7, 1999     

The composite Milstein methods for the numerical solution of Ito stochastic differential equations

In this paper, we present the composite Milstein methods for the strong solution of Ito stochastic differential equations. These methods are a combination of semi-implicit and implicit Milstein methods. We give a criterion for choosing either the implicit or the semi-implicit scheme at each step of our numerical solution. The stability and convergence properties are investigated and discussed for the linear test equation. The convergence properties for the nonlinear case are shown numerically to be the same as the linear case. The stability properties of the composite Milstein methods are found to be more superior compared to those of the Milstein, the Euler and even better than the composite Euler method. This superiority in stability makes the methods a better candidate for the solution of stiff SDEs.     

On the balancing principle for some problems of Numerical Analysis

We discuss a choice of weight in penalization methods. The motivation for the use of penalization in computational mathematics is to improve the conditioning of the numerical solution. One example of such improvement is a regularization, where a penalization substitutes an ill-posed problem for a well-posed one. In modern numerical methods for PDEs a penalization is used, for example, to enforce a continuity of an approximate solution on non-matching grids. A choice of penalty weight should provide a balance between error components related with convergence and stability, which are usually unknown. In this paper we propose and analyze a simple adaptive strategy for the choice of penalty weight which does not rely on a priori estimates of above mentioned components. It is shown that under natural assumptions the accuracy provided by our adaptive strategy is worse only by a constant factor than one could achieve in the case of known stability and convergence rates. Finally, we successfully apply our strategy for self-regularization of Volterra-type severely ill-posed problems, such as the sideways heat equation, and for the choice of a weight in interior penalty discontinuous approximation on non-matching grids. Numerical experiments on a series of model problems support theoretical results.     

Subspace-restricted singular value decompositions for linear discrete ill-posed problems

The truncated singular value decomposition is a popular solution method for linear discrete ill-posed problems. These problems are numerically underdetermined. Therefore, it can be beneficial to incorporate information about the desired solution into the solution process. This paper describes a modification of the singular value decomposition that permits a specified linear subspace to be contained in the solution subspace for all truncations. Modifications that allow the range to contain a specified subspace, or that allow both the solution subspace and the range to contain specified subspaces also are described.     

The effect of non-optimal bases on the convergence of Krylov subspace methods

Summary There are many examples where non-orthogonality of a basis for Krylov subspace methods arises naturally. These methods usually require less storage or computational effort per iteration than methods using an orthonormal basis (optimal methods), but the convergence may be delayed. Truncated Krylov subspace methods and other examples of non-optimal methods have been shown to converge in many situations, often with small delay, but not in others. We explore the question of what is the effect of having a non-optimal basis. We prove certain identities for the relative residual gap, i.e., the relative difference between the residuals of the optimal and non-optimal methods. These identities and related bounds provide insight into when the delay is small and convergence is achieved. Further understanding is gained by using a general theory of superlinear convergence recently developed. Our analysis confirms the observed fact that in exact arithmetic the orthogonality of the basis is not important, only the need to maintain linear independence is. Numerical examples illustrate our theoretical results.This revised version was published online in June 2005 due to a typesetting mistake in the footnote on page 7.     

Two class of synchronous matrix multisplitting schemes for solving linear complementarity problems

Many problems in the areas of scientific computing and engineering applications can lead to the solution of the linear complementarity problem LCP (M,q). It is well known that the matrix multisplitting methods have been found very useful for solving LCP (M,q). In this article, by applying the generalized accelerated overrelaxation (GAOR) and the symmetric successive overrelaxation (SSOR) techniques, we introduce two class of synchronous matrix multisplitting methods to solve LCP (M,q). Convergence results for these two methods are presented when M is an H-matrix (and also an M-matrix). Also the monotone convergence of the new methods is established. Finally, the numerical results show that the introduced methods are effective for solving the large and sparse linear complementary problems.     

A class of Uzawa-SOR methods for saddle point problems

In this paper, we consider a class of Uzawa-SOR methods for saddle point problems, and prove the convergence of the proposed methods. We solve a lower triangular system per iteration in the proposed methods, instead of solving a linear equation Az=b. Actually, the new methods can be considered as an inexact iteration method with the Uzawa as the outer iteration and the SOR as the inner iteration. Although the proposed methods cannot achieve the same convergence rate as the GSOR methods proposed by Bai et al. [Z.-Z. Bai, B.N. Parlett, Z.-Q. Wang, On generalized successive overrelaxation methods for augmented linear systems, Numer. Math. 102 (2005) 1-38], but our proposed methods have less workloads per iteration step. Experimental results show that our proposed methods are feasible and effective.     

Quadratic spline collocation methods for elliptic partial differential equations

We consider Quadratic Spline Collocation (QSC) methods for linear second order elliptic Partial Differential Equations (PDEs). The standard formulation of these methods leads to non-optimal approximations. In order to derive optimal QSC approximations, high order perturbations of the PDE problem are generated. These perturbations can be applied either to the PDE problem operators or to the right sides, thus leading to two different formulations of optimal QSC methods. The convergence properties of the QSC methods are studied. OptimalO(h ^3–j ) global error estimates for thejth partial derivative are obtained for a certain class of problems. Moreover,O(h ^4–j ) error bounds for thejth partial derivative are obtained at certain sets of points. Results from numerical experiments verify the theoretical behaviour of the QSC methods. Performance results also show that the QSC methods are very effective from the computational point of view. They have been implemented efficiently on parallel machines.This research was supported in part by David Ross Foundation (U.S.A) and NSERC (Natural Sciences and Engineering Research Council of Canada).     

Collocation methods for solving linear differential-algebraic boundary value problems

Summary. We consider boundary value problems for linear differential-algebraic equations with variable coefficients with no restriction on the index. A well-known regularisation procedure yields an equivalent index one problem with d differential and a=n-d algebraic equations. Collocation methods based on the regularised BVP approximate the solution x by a continuous piecewise polynomial of degree k and deliver, in particular, consistent approximations at mesh points by using the Radau schemes. Under weak assumptions, the collocation problems are uniquely and stably solvable and, if the unique solution x is sufficiently smooth, convergence of order min {k+1,2k-1} and superconvergence at mesh points of order 2k-1 is shown. Finally, some numerical experiments illustrating these results are presented. Received October 1, 1999 / Revised version received April 25, 2000 / Published online December 19, 2000     

Linear best approximation using a class of polyhedral norms

A class of polyhedral norms is introduced, which contains thel ₁ andl norms as special cases. Of primary interest is the solution of linear best approximation problems using these norms. Best approximations are characterized, and an algorithm is developed. This is a methods of descent type which may be interpreted as a generalization of existing well-known methods for solving thel ₁ andl problems. Numerical results are given to illustrate the performance of two variants of the algorithm on some problems.Communicated by C. Brezinski     

Discrepancy principles for Tikhonov regularization of ill-posed problems leading to optimal convergence rates

We propose a class ofa posteriori parameter choice strategies for Tikhonov regularization (including variants of Morozov's and Arcangeli's methods) that lead to optimal convergence rates toward the minimal-norm, least-squares solution of an ill-posed linear operator equation in the presence of noisy data.     

Regularisierung schlecht gestellter Probleme durch Projektionsverfahren

Summary The numerical solution of ill-posed problems by projection methods is considered. Regularization is carried out simply by choosing an optimal discretization parameter. It is shown by asymptotic estimates and by numerical examples that this kind of regularization is as efficient as the method of Tikhonov and Phillips.

     

Iteration methods for convexly constrained ill-posed problems in hilbert space

Minimization problems in Hilbert space with quadratic objective function and closed convex constraint set C are considered. In case the minimum is not unique we are looking for the solution of minimal norm. If a problem is ill-posed, i.e. if the solution does not depend continuously on the data, and if the data are subject to errors then it has to be solved by means of regularization methods. The regularizing properties of some gradient projection methods—i.e. convergence for exact data, order of convergence under additional assumptions on the solution and stability for perturbed data—are the main issues of this paper.