THE SINE TRANSFORM OPERATOR IN THE BANACH SPACE OF SYMMETRIC MATRICES AND ITS APPLICATION IN IMAGE RESTORATION

In this paper, we study an operator s which maps every n-by-n symmetric matrix A, to a matrix s(A_n) that minimizes || B_n-A_n || F over the set of all matrices B_n, that can be diagonalized by the sine transform. The matrix s(A_n), called the optimal sine transform preconditioner, is defined for any n-by-n symmetric matrices A_n. The cost of constructing s(A_n) is the same as that of optimal circulant preconditioner c(A_n) which is defined in [8], The s(A_n) has been proved in [6] to be a good preconditioner in solving symmetric Toeplitz systems with the preconditioned conjugate gradient (PCG) method. In this paper, we discuss the algebraic and geometric properties of the operator s, and compute its operator norms in Banach spaces of symmetric matrices. Some numerical tests and an application in image restoration are also given.     

Right preconditioned MINRES for singular systems†

We consider solving large sparse symmetric singular linear systems. We first introduce an algorithm for right preconditioned minimum residual (MINRES) and prove that its iterates converge to the preconditioner weighted least squares solution without breakdown for an arbitrary right‐hand‐side vector and an arbitrary initial vector even if the linear system is singular and inconsistent. For the special case when the system is consistent, we prove that the iterates converge to a min‐norm solution with respect to the preconditioner if the initial vector is in the range space of the right preconditioned coefficient matrix. Furthermore, we propose a right preconditioned MINRES using symmetric successive over‐relaxation (SSOR) with Eisenstat's trick. Some numerical experiments on semidefinite systems in electromagnetic analysis and so forth indicate that the method is efficient and robust. Finally, we show that the residual norm can be further reduced by restarting the iterations.     

Solution of projection problems over polytopes

Summary In this paper, we present variants of a convergent projection and contraction algorithm [25] for solving projection problems over polytope. By using the special struture of the projection problems, an iterative algorithm with constant step-size is given, which is globally linearly convergent. These algorithms are simple to implement and each step of the method requires only a few matrix-vector multiplications. Especially, for minimums norm problems over transportation or general network polytopes onlyO(n) additions andO(n) multiplications are needed at each iteration. Numerical results for randomly generated test problems over network polytopes, up to 10000 variables, indicate that the presented algorithms are simple and efficient even for large problems.     

FAST DENSE MATRIX METHOD FOR THE SOLUTION OF INTEGRAL EQUATIONS OF THE SECOND KIND

We present a fast algorithm based on polynomial interpolation to approximate matrices arising from the discretization of second-kind integral equations where the kernel function is either smooth, non-oscillatory and possessing only a finite number of singularities or a product of such function with a highly oscillatory coefficient function. Contrast to wavelet-like approximations, ourapproximation matrix is not sparse. However, the approximation can be construced in O(n) operations and requires O(n) storage, where n is the number of quadrature points used in the discretization. Moreover, the matrix-vector multiplication cost is of order O(nlogn). Thus our scheme is well suitable for conjugate gradient type methods. Our numerical results indicate that the algorithm is very accurate and stable for high degree polynomial interpolation.     

Balancing sparse Hamiltonian eigenproblems

Balancing a matrix by a simple and accurate similarity transformation can improve the speed and accuracy of numerical methods for computing eigenvalues. We describe balancing strategies for a large and sparse Hamiltonian matrix H.It is first shown how to permute H to irreducible form while retaining its structure. This form can be used to decompose the Hamiltonian eigenproblem into smaller-sized problems. Next, we discuss the computation of a symplectic scaling matrix D so that the norm of D⁻¹HD is reduced. The considered scaling algorithm is solely based on matrix-vector products and thus particularly suitable if the elements of H are not explicitly given. The merits of balancing for eigenvalue computations are illustrated by several practically relevant examples.     

A Proposal for Toeplitz Matrix Calculations

In contrast to the usual (and successful) direct methods for Toeplitz systems Ax = b, we propose an algorithm based on the conjugate gradient method. The preconditioner is a circulant, so that all matrices have constant diagonals and all matrix-vector multiplications use the Fast Fourier Transform. We also suggest a technique for the eigenvalue problem, where current methods are less satisfactory. If the first indications are supported by further experiment, this new approach may have useful applications—including nearly Toeplitz systems, and parallel computations.     

Schwarz preconditioned CG algorithm for the mortar finite element

We propose a simple and effective hybrid (multiplicative) Schwarz precondtioner for solving systems of algebraic equations resulting from the mortar finite element discretization of second order elliptic problems on nonmatching meshes. The preconditioner is embedded in a variant of the classical preconditioned conjugate gradient (PCG) for an effective implementation reducing the cost of computing the matrix-vector multiplication in each iteration of the PCG. In fact, it serves as a framework for effective implementation of a class of hybrid Schwarz preconditioners. The preconditioners of this class are based on solving a sequence of non-overlapping local subproblems exactly, and the coarse problems either exactly or inexactly (approximately). The classical PCG algorithm is reformulated in order to make reuse of the results of matrix-vector multiplications that are already available from the preconditioning step resulting in an algorithm which is cost effective. An analysis of the proposed preconditioner, with numerical results, showing scalability with respect to the number of subdomains, and a convergence which is independent of the jumps of the coefficients are given.     

A FAST COMPACT DIFFERENCE METHOD FOR TWO-DIMENSIONAL NONLINEAR SPACE-FRACTIONAL COMPLEX GINZBURG-LANDAU EQUATIONS

This paper focuses on a fast and high-order finite difference method for two-dimensional space-fractional complex Ginzburg-Landau equations.We firstly establish a three-level finite difference scheme for the time variable followed by the linearized technique of the nonlinear term.Then the fourth-order compact finite difference method is employed to discretize the spatial variables.Hence the accuracy of the discretization is O(τ2 + h41 +h42) in L2-norm,where τ is the temporal step-size,both h1 and h2 denote spatial mesh sizes in x-and y-directions,respectively.The rigorous theoretical analysis,including the uniqueness,the almost unconditional stability,and the convergence,is studied via the energy argument.Practically,the discretized system holds the block Toeplitz structure.Therefore,the coefficient Toeplitz-like matrix only requires O(M1M2) memory storage,and the matrix-vector multiplication can be carried out in O(M1M2(logM1 + log M2))computational complexity by the fast Fourier transformation,where M1 and M2 denote the numbers of the spatial grids in two different directions.In order to solve the resulting Toeplitz-like system quickly,an efficient preconditioner with the Krylov subspace method is proposed to speed up the iteration rate.Numerical results are given to demonstrate the well performance of the proposed method.     

Superoptimal Preconditioners for Functions of Matrices

For any given matrix $A∈\mathbb{C}^{n×n}$, a preconditioner $t_U(A)$ called the superoptimal preconditioner was proposed in 1992 by Tyrtyshnikov. It has been shown that $t_U(A)$ is an efficient preconditioner for solving various structured systems, for instance, Toeplitz-like systems. In this paper, we construct the superoptimal preconditioners for different functions of matrices. Let $f$ be a function of matrices from$\mathbb{C}^{n×n}$ to$\mathbb{C}^{n×n}$. For any $A∈\mathbb{C}^{n×n}$, one may construct two superoptimal preconditioners for $f(A)$: $t_U(f(A))$ and $f(t_U(A))$. We establish basic properties of $t_U(f(A))$) and $f(t_U(A))$ for different functions of matrices. Some numerical tests demonstrate that the proposed preconditioners are very efficient for solving the system $f(A)x=b$.     

Diagonal forms of the translation operators in the fast multipole algorithm for scattering problems

The integral equations of acoustic and electromagnetic scattering generate large dense systems of linear equations. These systems are efficiently solved with iterative methods where the matrix-vector multiplication is computed using a special fast method, such as the fast Fourier transform or the fast multipole method (FMM). In this paper, the so called diagonal forms of the translation operators for the fast multipole method are derived starting from integral representations of certain special functions. Error analysis of the FMM is given, considering both the truncation error of potential expansions and the errors from the use of numerical integration in the diagonal translation theorem. The implications of the error bounds on the FMM algorithm are discussed.This work has been financially supported by the Jenny and Antti Wihuri Foundation and by the Cultural Foundation of Finland.     

Approximate inverse preconditioner by computing approximate solution of Sylvester equation

This paper presents a new method for obtaining a matrix M which is an approximate inverse preconditioner for a given matrix A, where the eigenvalues of A all either have negative real parts or all have positive real parts. This method is based on the approximate solution of the special Sylvester equation AX + XA = 2I. We use a Krylov subspace method for obtaining an approximate solution of this Sylvester matrix equation which is based on the Arnoldi algorithm and on an integral formula. The computation of the preconditioner can be carried out in parallel and its implementation requires only the solution of very simple and small Sylvester equations. The sparsity of the preconditioner is preserved by using a proper dropping strategy. Some numerical experiments on test matrices from Harwell–Boing collection for comparing the numerical performance of the new method with an available well-known algorithm are presented.     

广义块Toeplitz特征值问题的基于sine变换的预处理子

在求块Toeplitz矩阵束（Amn，Bmn）特征值的Lanczos过程中，通过对移位块Toepltz矩阵Amn-ρBmn进行基于sine变换的块预处理，从而改进了位移块Toeplitz矩阵的谱分布，加速了Lanczos过程的收敛速度．该块预处理方法能通过快速算法有效快速执行．本文证明了预处理后Lanczos过程收敛迅速，并通过实验证明该算法求解大规模矩阵问题尤其有效．     

Fast Fourier Transform Solvers and Preconditioners for Quadratic Spline Collocation

Quadratic Spline Collocation (QSC) methods of optimal order of convergence have been recently developed for the solution of elliptic Partial Differential Equations (PDEs). In this paper, linear solvers based on Fast Fourier Transforms (FFT)are developed for the solution of the QSC equations. The complexity of the FFT solvers is O(N ² log N), where N is the gridsize in one dimension. These direct solvers can handle PDEs with coefficients in one variable or constant, and Dirichlet, Neumann, alternating Dirichlet-Neumann or periodic boundary conditions, along at least one direction of a rectangular domain. General variable coefficient PDEs are handled by preconditioned iterative solvers. The preconditioner is the QSC matrix arising from a constant coefficient PDE. The convergence analysis of the preconditioner is presented. It is shown that, under certain conditions, the convergence rate is independent of the gridsize. The preconditioner is solved by FFT techniques, and integrated with one-step or acceleration methods, giving rise to asymptotically almost optimal linear solvers, with complexity O(N ² log N). Numerical experiments verify the effectiveness of the solvers and preconditioners, even on problems more general than the analysis assumes. The development and analysis of FFT solvers and preconditioners is extended to QSC equations corresponding to systems of elliptic PDEs.     

Fast Construction of Optimal Circulant Preconditioners for Matrices from the Fast Dense Matrix Method

In this paper, we consider solving non-convolution type integral equations by the preconditioned conjugate gradient method. The fast dense matrix method is a fast multiplication scheme that provides a dense discretization matrix A approximating a given integral equation. The dense matrix A can be constructed in O(n) operations and requires only O(n) storage where n is the size of the matrix. Moreover, the matrix-vector multiplication A xcan be done in O(n log n) operations. Thus if the conjugate gradient method is used to solve the discretized system, the cost per iteration is O(n log n) operations. However, for some integral equations, such as the Fredholm integral equations of the first kind, the system will be ill-conditioned and therefore the convergence rate of the method will be slow. In these cases, preconditioning is required to speed up the convergence rate of the method. A good choice of preconditioner is the optimal circulant preconditioner which is the minimizer of C – A _F in Frobenius norm over all circulant matrices C. It can be obtained by taking arithmetic averages of all the entries of A and therefore the cost of constructing the preconditioner is of O(n ²) operations for general dense matrices. In this paper, we develop an O(n log n) method of constructing the preconditioner for dense matrices A obtained from the fast dense matrix method. Application of these ideas to boundary integral equations from potential theory will be given. These equations are ill-conditioned whereas their optimal circulant preconditioned equations will be well-conditioned. The accuracy of the approximation A, the fast construction of the preconditioner and the fast convergence of the preconditioned systems will be illustrated by numerical examples.     

广义鞍点问题的松弛维数分解预条件子

本文将Benzi等提出的松弛维数分解(Relaxed dimensionalfactorization, RDF)预条件子进一步推广到广义鞍点问题上,并称为GRDF(Generalized RDF)预条件子.该预条件子可看做是用维数分裂迭代法求解广义鞍点问题而导出的改进维数分裂(Modified dimensional split, MDS)预条件子的松弛形式, 它相比MDS预条件子更接近于系数矩阵, 因而结合Krylov子空间方法(如GMRES)有更快的收敛速度.文中分析了GRDF预处理矩阵特征值的一些性质,并用数值算例验证了新预条件子的有效性.     

Parallel preconditioners and multigrid solvers for stochastic polynomial chaos discretizations of the diffusion equation at the large scale

This paper presents parallel preconditioners and multigrid solvers for solving linear systems of equations arising from stochastic polynomial chaos formulations of the diffusion equation with random coefficients. These preconditioners and solvers are extensions of the preconditioner developed in an earlier paper for strongly coupled systems of elliptic partial differential equations that are norm equivalent to systems that can be factored into an algebraic coupling component and a diagonal differential component. The first preconditioner, which is applied to the norm equivalent system, is obtained by sparsifying the inverse of the algebraic coupling component. This sparsification leads to an efficient method for solving these systems at the large scale, even for problems with large statistical variations in the random coefficients. An extension of this preconditioner leads to stand‐alone multigrid methods that can be applied directly to the actual system rather than to the norm equivalent system. These multigrid methods exploit the algebraic/differential factorization of the norm equivalent systems to produce variable‐decoupled systems on the coarse levels. Moreover, the structure of these methods allows easy software implementation through re‐use of robust high‐performance software such as the Hypre library package. Two‐grid matrix bounds will be established, and numerical results will be given. Copyright © 2015 John Wiley & Sons, Ltd.     

对称r-循环矩阵的快速算法和并行算法

借助于快速付立叶变换(FFT),给出了一种判断对称r-循环线性系统是否有解的快速算法,并且在有解的情况下求出其解,该算法的计算复杂度为O(nlogn),且具有很好的并行性,若使用n台处理机并行处理该算法则只需要O(logn)步.当r=0时,对称r-循环矩阵变成一个上三角型Hankel矩阵,我们也给出了此类矩阵求逆的一种算法.最后将该算法推广到线性同余系统,其运算量仅为O(nlogn).     

Finite iterative method for solving coupled Sylvester-transpose matrix equations

This paper is concerned with solutions to the so-called coupled Sylvester-transpose matrix equations, which include the generalized Sylvester matrix equation and Lyapunov matrix equation as special cases. By extending the idea of conjugate gradient method, an iterative algorithm is constructed to solve this kind of coupled matrix equations. When the considered matrix equations are consistent, for any initial matrix group, a solution group can be obtained within finite iteration steps in the absence of roundoff errors. The least Frobenius norm solution group of the coupled Sylvester-transpose matrix equations can be derived when a suitable initial matrix group is chosen. By applying the proposed algorithm, the optimal approximation solution group to a given matrix group can be obtained by finding the least Frobenius norm solution group of new general coupled matrix equations. Finally, a numerical example is given to illustrate that the algorithm is effective.     

GENUINE-OPTIMAL CIRCULANT PRECONDITIONERS FOR WIENER-HOPF EQUATIONS

1. IntroductionWienerHopf equations are integral equations defined on the haif line:where rr > 0, a(.) C L1(ro and g(.) E L2(at). Here R = (--oo,oo) and ty [0,oo). Inou-r discussions, we assume that a(.) is colljugate symmetric, i.e. a(--t) = a(t). WienerHop f equations arise in a variety of practical aPplicatiolls in mathematics and ellgineering, forinstance, in the linear prediction problems fOr stationary stochastic processes [8, pp.145--146],diffuSion problems and scattering problems […     

Matrix-free interior point method

In this paper we present a redesign of a linear algebra kernel of an interior point method to avoid the explicit use of problem matrices. The only access to the original problem data needed are the matrix-vector multiplications with the Hessian and Jacobian matrices. Such a redesign requires the use of suitably preconditioned iterative methods and imposes restrictions on the way the preconditioner is computed. A two-step approach is used to design a preconditioner. First, the Newton equation system is regularized to guarantee better numerical properties and then it is preconditioned. The preconditioner is implicit, that is, its computation requires only matrix-vector multiplications with the original problem data. The method is therefore well-suited to problems in which matrices are not explicitly available and/or are too large to be stored in computer memory. Numerical properties of the approach are studied including the analysis of the conditioning of the regularized system and that of the preconditioned regularized system. The method has been implemented and preliminary computational results for small problems limited to 1 million of variables and 10 million of nonzero elements demonstrate the feasibility of the approach.