A general iterative scheme with applications to convex optimization and related fields

A general iterative scheme including relaxation and a corresponding problem class are presented. Some global convergence results are given. The acceleration of convergence is discussed, The scheme comprises a lot of known iterative methods such as subgradient methods and methods of successive orthogonal projections with relaxation. Applications to convex optimization, convex feasibility problems, systems of convex inequalities, variational inequalities, operator equations and systems of linear equations are given.     

用非精确的平行割线法求解奇异问题

奇异方程经常出现在很多实际非线性问题中,如反应扩散系统等.因此,研究奇异非线性方程的求解具有十分重要的意义.平行割线法是一种经典的求解非线性方程的迭代方法,它收敛阶较高,计算量较少.但在解决实际问题时,一方面,抽象出的数学模型与实际问题总是存在着一定的偏差,另外,在数据的计算中难免存在着一定的计算误差,所以研究用非精确的平行割线法求解非线性奇异问题具有很重要的现实意义,使得求解奇异问题具有更高的实用性和可行性.采用在平行割线法的迭代公式中加入摄动项的方法,构造出新的加速迭代格式,证明了新的迭代格式的收敛性,给出了收敛速率,得到了误差估计.     

Asymptotisches Verhalten und Konvergenzbeschleunigung von Iterationsfolgen

Summary The effectivity of iterative numerical methods depends on the rate of convergence. In this note general procedures to accelerate the convergence of finite-dimensional stationary one-step-methods (fixed point methods) by extrapolation methods are studied. In this connection the investigation of the asymptotic behaviour of the sequences is fundamental. Differentiability and contractivity qualities supposed in the following an asymptotic expansion for such iterative sequences is proved. Neglecting the remainder the expansion fulfils a linear difference equation with constant coefficients. Wynn's -algorithm work off this expansion term by term, and the attainable acceleration can be exactly estimated. Skelboe's convergence statement is refuted. First test results demonstrate the advantage of acceleration methods.

     

An efficient higher order family of root finders

A one parameter family of iterative methods for the simultaneous approximation of simple complex zeros of a polynomial, based on a cubically convergent Hansen–Patrick's family, is studied. We show that the convergence of the basic family of the fourth order can be increased to five and six using Newton's and Halley's corrections, respectively. Since these corrections use the already calculated values, the computational efficiency of the accelerated methods is significantly increased. Further acceleration is achieved by applying the Gauss–Seidel approach (single-step mode). One of the most important problems in solving nonlinear equations, the construction of initial conditions which provide both the guaranteed and fast convergence, is considered for the proposed accelerated family. These conditions are computationally verifiable; they depend only on the polynomial coefficients, its degree and initial approximations, which is of practical importance. Some modifications of the considered family, providing the computation of multiple zeros of polynomials and simple zeros of a wide class of analytic functions, are also studied. Numerical examples demonstrate the convergence properties of the presented family of root-finding methods.     

Convergence of Compressible Euler-Maxwell Equations to Compressible Euler-Poisson Equations

In this paper,the convergence of time-dependent Euler-Maxwell equations to compressible Euler-Poisson equations in a torus via the non-relativistic limit is studied. The local existence of smooth solutions to both systems is proved by using energy esti- mates for first order symmetrizable hyperbolic systems.For well prepared initial data the convergence of solutions is rigorously justified by an analysis of asymptotic expansions up to any order.The authors perform also an initial layer analysis for general initial data and prove the convergence of asymptotic expansions up to first order.     

Efficient methods for the inclusion of polynomial zeros

Using a suitable zero-relation and the inclusion isotonicity property, new interval iterative methods for the simultaneous inclusion of simple complex zeros of a polynomial are derived. These methods produce disks in the complex plane that contain the polynomial zeros in each iteration, providing in this manner an information about upper error bounds of approximations. Starting from the basic method of the fourth order, two accelerated methods with Newton’s and Halley’s corrections, having the order of convergence five and six respectively, are constructed. This increase of the convergence rate is obtained without any additional operations, which means that the methods with corrections are very efficient. The convergence analysis of the basic method and the methods with corrections is performed under computationally verifiable initial conditions, which is of practical importance. Two numerical examples are presented to demonstrate the convergence behavior of the proposed interval methods.     

Methods of aggregation

We study a class of methods for accelerating the convergence of iterative methods for solving linear systems. The methods proceed by replacing the given linear system with a derived one of smaller size, the aggregated system. The solution of the latter is used to accelerate the original iterative process. The construction of the aggregated system as well as the passage of information between it and the original system depends on one or more approximations of the solution of the latter. A number of variants are introduced, estimates of the acceleration are obtained, and numerical experiments are performed. The theory and computations show the methods to be effective.     

Iterative and non-iterative methods for non-linear Volterra integro-differential equations

Iterative and non-iterative methods for the solution of nonlinear Volterra integro-differential equations are presented and their local convergence is proved. The iterative methods provide a sequence solution and make use of fixed-point theory, whereas the non-iterative ones result in series solutions and also make use of fixed-point principles. By means of integration by parts and use of certain integral identities, it is shown that the initial conditions that appear in the iterative methods presented here can be eliminated and the resulting iterative technique is identical to the variational iteration method which is derived here without making any use at all of Lagrange multipliers and constrained variations. It is also shown that the formulation presented here can be applied to initial-value problems in ordinary differential, Volterra’s integral and integro-differential, pantograph, and nonlinear and linear algebraic equations. A technique for improving/accelerating the convergence of the iterative methods presented here is also presented and results in a Lipschitz constant that may be varied as the iteration progresses. It is shown that this acceleration technique is related to preconditioning methods for the solution of linear algebraic equations. It is also argued that the non-iterative methods presented in this paper may not competitive with iterative ones because of possible cancellation errors, if implemented numerically. An analytical continuation procedure based on dividing the interval of integration into disjoint subintervals is also presented and its limitations are discussed.     

Convergence acceleration of preconditioned conjugate gradient solver based on error vector sampling for a sequence of linear systems

In this article, we focus on solving a sequence of linear systems that have identical (or similar) coefficient matrices. For this type of problem, we investigate subspace correction (SC) and deflation methods, which use an auxiliary matrix (subspace) to accelerate the convergence of the iterative method. In practical simulations, these acceleration methods typically work well when the range of the auxiliary matrix contains eigenspaces corresponding to small eigenvalues of the coefficient matrix. We develop a new algebraic auxiliary matrix construction method based on error vector sampling in which eigenvectors with small eigenvalues are efficiently identified in the solution process. We use the generated auxiliary matrix for convergence acceleration in the following solution step. Numerical tests confirm that both SC and deflation methods with the auxiliary matrix can accelerate the solution process of the iterative solver. Furthermore, we examine the applicability of our technique to the estimation of the condition number of the coefficient matrix. We also present the algorithm of the preconditioned conjugate gradient method with condition number estimation.     

An asymptotically hierarchy-consistent, iterative sequence transformation for convergence acceleration of Fourier series

We derive the I transformation, an iterative sequence transformation that is useful for the convergence acceleration of certain Fourier series. The derivation is based on the concept of hierarchical consistency in the asymptotic regime. We show that this sequence transformation is a special case of the J transformation. Thus, many properties of the I transformation can be deduced from the known properties of the J transformation (like the kernel, determinantal representations, and theorems on convergence behavior and stability). Besides explicit formulas for the kernel, some basic convergence theorems for the I transformation are given here. Further, numerical results are presented that show that suitable variants of the I transformation are powerful nonlinear convergence accelerators for Fourier series with coefficients of monotonic behavior. This revised version was published online in August 2006 with corrections to the Cover Date.     

A Parameterized Class of Complex Nonsymmetric Algebraic Riccati Equations

In this paper, by introducing a definition of parameterized comparison matrix of a given complex square matrix, the solvability of a parameterized class of complex nonsymmetric algebraic Riccati equations (NAREs) is discussed. The existence and uniqueness of the extremal solutions of the NAREs is proved. Some classical numerical methods can be applied to compute the extremal solutions of the NAREs, mainly including the Schur method, the basic fixed-point iterative methods, Newton's method and the doubling algorithms. Furthermore, the linear convergence of the basic fixed-point iterative methods and the quadratic convergence of Newton's method and the doubling algorithms are also shown. Moreover, some concrete parameter selection strategies in complex number field for the doubling algorithms are also given. Numerical experiments demonstrate that our numerical methods are effective.     

Polynomial Acceleration Methods for Solving Singular Systems of Linear Equations

In this paper we study the polynomial acceleration methods for solving singular linear systems. We establish iterative schemes, show their convergence and find iteration error bounds.     

鞍点问题的修正对称超松弛迭代算法

为了提高求解鞍点问题的迭代算法的速度,通过设置合适的加速变量,对修正超松弛迭代算法（简记作MSOR-like算法）和广义对称超松弛迭代算法（简记作GSSOR-like算法）进行了修正,给出了修正对称超松弛迭代算法,即MSSOR-like （modified symmetric successiveover-relaxation）算法,并研究了该算法收敛的充分必要条件.最后,通过数值例子表明,选择合适的参数后,新算法的迭代速度和迭代次数均优于MSOR-like （modified successive overrelaxation）和GSSOR-like （generalized symmetric successive over-relaxation）算法,因此,它是一种较好的解决鞍点问题的算法.     

矩阵分裂序列与线性二级迭代法

本文讨论线性非定常二级迭代法的收敛性．对于一般的基于矩阵分裂序列的迭代法,针对分裂序列本身找到了一种新的且相对较弱的收敛性条件,并因此得到了由非定常二级迭代法推广而来的广义二级迭代法的收敛结果．从而,用一种新的方法证明了非定常二级迭代法的收敛性．     

On efficient two-parameter methods for solving nonlinear equations

Derivative free methods for solving nonlinear equations of Steffensen’s type are presented. Using two self-correcting parameters, calculated by Newton’s interpolatory polynomials of second and third degree, the order of convergence is increased from 2 to 3.56. This method is used as a corrector for a family of biparametric two-step derivative free methods with and without memory with the accelerated convergence rate up to order 7. Significant acceleration of convergence is attained without any additional function calculations, which provides very high computational efficiency of the proposed methods. Another advantage is a convenient fact that the proposed methods do not use derivatives. Numerical examples are given to demonstrate excellent convergence behavior of the proposed methods and good coincidence with theoretical results.     

On the complexity of extending the convergence ball of Wang’s method for finding a zero of a derivative

Ball convergence results are very important, since they demonstrate the complexity in choosing initial points for iterative methods. One of the most important problems in the study of iterative methods is to determine the convergence ball. This ball is small in general restricting the choice of initial points. We address this problem in the case of Wang’s method utilized to determine a zero of a derivative. Finding such a zero has many applications in computational fields, especially in function optimization. In particular, we find the convergence ball of Wang’s method using hypotheses up to the second derivative in contrast to earlier studies using hypotheses up to the fourth derivative. This way, we also extend the applicability of Wang’s method. Numerical experiments used to test the convergence criteria complete this study.     

Efficient derivative-free variants of Hansen-Patrick’s family with memory for solving nonlinear equations

In this paper, we present a new tri-parametric derivative-free family of Hansen-Patrick type methods for solving nonlinear equations numerically. The proposed family requires only three functional evaluations to achieve optimal fourth order of convergence. In addition, acceleration of convergence speed is attained by suitable variation of free parameters in each iterative step. The self-accelerating parameters are estimated from the current and previous iteration. These self-accelerating parameters are calculated using Newton’s interpolation polynomials of third and fourth degrees. Consequently, the R-order of convergence is increased from 4 to 7, without any additional functional evaluation. Furthermore, the most striking feature of this contribution is that the proposed schemes can also determine the complex zeros without having to start from a complex initial guess as would be necessary with other methods. Numerical experiments and the comparison of the existing robust methods are included to confirm the theoretical results and high computational efficiency.     

The self-validated method for polynomial zeros of high efficiency

The improved iterative method of Newton’s type for the simultaneous inclusion of all simple complex zeros of a polynomial is proposed. The presented convergence analysis, which uses the concept of the R-order of convergence of mutually dependent sequences, shows that the convergence rate of the basic third order method is increased from 3 to 6 using Ostrowski’s corrections. The new inclusion method with Ostrowski’s corrections is more efficient compared to all existing methods belonging to the same class. To demonstrate the convergence properties of the proposed method, two numerical examples are given.     

A Non-Krylov Subspace Method for Solving Large and Sparse Linear System of Equations

Most current prevalent iterative methods can be classified into the socalled extended Krylov subspace methods, a class of iterative methods which do not fall into this category are also proposed in this paper. Comparing with traditional Krylov subspace methods which always depend on the matrix-vector multiplication with a fixed matrix, the newly introduced methods (the so-called (progressively) accumulated projection methods, or AP (PAP) for short) use a projection matrix which varies in every iteration to form a subspace from which an approximate solution is sought. More importantly, an accelerative approach (called APAP) is introduced to improve the convergence of PAP method. Numerical experiments demonstrate some surprisingly improved convergence behaviors. Comparisons between benchmark extended Krylov subspace methods (Block Jacobi and GMRES) are made and one can also see remarkable advantage of APAP in some examples. APAP is also used to solve systems with extremely ill-conditioned coefficient matrix (the Hilbert matrix) and numerical experiments shows that it can bring very satisfactory results even when the size of system is up to a few thousands.     

On the convergence of basic iterative methods for convection–diffusion equations

In this paper we analyze convergence of basic iterative Jacobi and Gauss–Seidel type methods for solving linear systems which result from finite element or finite volume discretization of convection–diffusion equations on unstructured meshes. In general the resulting stiffness matrices are neither M‐matrices nor satisfy a diagonal dominance criterion. We introduce two newmatrix classes and analyse the convergence of the Jacobi and Gauss–Seidel methods for matrices from these classes. A new convergence result for the Jacobi method is proved and negative results for the Gauss–Seidel method are obtained. For a few well‐known discretization methods it is shown that the resulting stiffness matrices fall into the new matrix classes. Copyright © 1999 John Wiley & Sons, Ltd.