Nonlinear Schwarz iterations with reduced rank extrapolation

Extrapolation methods can be a very effective technique used for accelerating the convergence of vector sequences. In this paper, these methods are used to accelerate the convergence of Schwarz iterative methods for nonlinear problems. A new implementation of the reduced-rank-extrapolation (RRE) method is introduced. Some convergence analysis is presented, and it is shown numerically that certain extrapolation methods can indeed be very effective in accelerating the convergence of Schwarz methods.     

Convergence of intermediate rows of minimal polynomial and reduced rank extrapolation tables

Let {x _m} _m ^=0/ be a vector sequence obtained from a linear fixed point iterative technique in a general inner product space. In two previous papers [6,9] the convergence properties of the minimal polynomial and reduced rank extrapolation methods, as they are applied to the vector sequence above, were analyzed. In particular, asymptotically optimal convergence results pertaining to some of the rows of the tables associated with these two methods were obtained. In the present work we continue this analysis and provide analogous results for the remaining (intermediate) rows of these tables. In particular, when {x _m} _m ^=0/ is a convergent sequence, the main result of this paper says, roughly speaking, that all of the rows converge, and it also gives the rate of convergence for each row. The results are demonstrated numerically through an example.     

Toward a unified theory for third R-order iterative methods for operators with unbounded second derivative

In this paper, we provide a semilocal convergence analysis for a family of Newton-like methods, which contains the best-known third-order iterative methods for solving a nonlinear equation F(x)=0 in Banach spaces. It is assumed that the operator F is twice Fréchet differentiable and F^″ satisfies a Lipschitz type condition but it is unbounded. By using majorant sequences, we provide sufficient convergence conditions to obtain cubic semilocal convergence. Results on existence and uniqueness of solutions, and error estimates are also given. Finally, a numerical example is provided.     

Squared polynomial extrapolation methods with cycling: an application to the positron emission tomography problem

Roland and Varadhan (Appl. Numer. Math., 55:215–226, 2005) presented a new idea called “squaring” to improve the convergence of Lemaréchal’s scheme for solving nonlinear fixed-point problems. Varadhan and Roland (Squared extrapolation methods: A new class of simple and efficient numerical schemes for accelerating the convergence of the EM algorithm, Department of Biostatistics Working Paper. Johns Hopkins University, , 2004) noted that Lemaréchal’s scheme can be viewed as a member of the class of polynomial extrapolation methods with cycling that uses two fixed-point iterations per cycle. Here we combine these two ideas, cycled extrapolation and squaring, and construct a new class of methods, called squared polynomial methods (SQUAREM), for accelerating the convergence of fixed-point iterations. Our main goal is to evaluate whether the squaring device is effective in improving the rate of convergence of cycled extrapolation methods that use more than two fixed-point iterations per cycle. We study the behavior of the new schemes on an image reconstruction problem for positron emission tomography (PET) using simulated data. Our numerical experiments show the effectiveness of first- and higher-order squared polynomial extrapolation methods in accelerating image reconstruction, and also their relative superiority compared to the classical, “unsquared” vector polynomial methods.     

General Study of Iterative Processes of <Emphasis Type="Italic">R</Emphasis>-Order at Least Three under Weak Convergence Conditions

We consider a family of Newton-type iterative processes solving nonlinear equations in Banach spaces, that generalizes the usually iterative methods of R-order at least three. The convergence of this family in Banach spaces is usually studied when the second derivative of the operator involved is Lipschitz continuous and bounded. In this paper, we relax the first condition, assuming that ‖F″(x)−F″(y)‖≤ω(‖x−y‖), where ω is a nondecreasing continuous real function. We prove that the different R-orders of convergence that we can obtain depend on the quasihomogeneity of the function ω. We end the paper by applying the study to some nonlinear integral equations. This work was supported by the Ministry of Science and Technology (BFM 2002-00222), the University of La Rioja (API-04/13) and the Government of La Rioja (ACPI 2003/2004).     

Variable-step multilevel preconditioning methods,I: Self-adjoint and positive definite elliptic problems

When solving large size systems of equations by preconditioned iterative solution methods, one normally uses a fixed preconditioner which may be defined by some eigenvalue information, such as in a Chebyshev iteration method. In many problems, however, it may be more effective to use variable preconditioners, in particular when the eigenvalue information is not available. In the present paper, a recursive way of constructing variable-step of, in general, nonlinear multilevel preconditioners for selfadjoint and coercive second-order elliptic problems, discretized by the finite element method is proposed. The preconditioner is constructed recursively from the coarsest to finer and finer levels. Each preconditioning step requires only block-diagonal solvers at all levels except at every k₀, k₀ ≥ 1 level where we perform a sufficient number ν, ν ≥ 1 of GCG-type variable-step iterations that involve the use again of a variable-step preconditioning for that level. It turns out that for any sufficiently large value of k₀ and, asymptotically, for ν sufficiently large, but not too large, the method has both an optimal rate of convergence and an optimal order of computational complexity, both for two and three space dimensional problem domains. The method requires no parameter estimates and the convergence results do not depend on the regularity of the elliptic problem.     

Asymptotisches Verhalten und Konvergenzbeschleunigung von Iterationsfolgen

Summary The effectivity of iterative numerical methods depends on the rate of convergence. In this note general procedures to accelerate the convergence of finite-dimensional stationary one-step-methods (fixed point methods) by extrapolation methods are studied. In this connection the investigation of the asymptotic behaviour of the sequences is fundamental. Differentiability and contractivity qualities supposed in the following an asymptotic expansion for such iterative sequences is proved. Neglecting the remainder the expansion fulfils a linear difference equation with constant coefficients. Wynn's -algorithm work off this expansion term by term, and the attainable acceleration can be exactly estimated. Skelboe's convergence statement is refuted. First test results demonstrate the advantage of acceleration methods.

     

Projektive Newton-Verfahren und Anwendungen auf nichtlineare Randwertaufgaben

Summary In this paper we consider the following Newton-like methods for the solution of nonlinear equations. In each step of the Newton method the linear equations are solved approximatively by a projection method. We call this a Projective Newton method. For a fixed projection method the approximations often are the same as those of the Newton method applied to a nonlinear projection method. But the efficiency can be increased by adapting the accuracy of the projection method to the convergence of the approximations. We investigate the convergence and the order of convergence for these methods. The results are applied to some Projective Newton methods for nonlinear two point boundary value problems. Some numerical results indicate the efficiency of these methods.

     

Convergence of a nonlinear finite difference scheme for the Kuramoto–Tsuzuki equation

A nonlinear finite difference scheme is studied for solving the Kuramoto–Tsuzuki equation. Because the maximum estimate of the numerical solution can not be obtained directly, it is difficult to prove the stability and convergence of the scheme. In this paper, we introduce the Brouwer-type fixed point theorem and induction argument to prove the unique existence and convergence of the nonlinear scheme. An iterative algorithm is proposed for solving the nonlinear scheme, and its convergence is proved. Based on the iterative algorithm, some linearized schemes are presented. Numerical examples are carried out to verify the correction of the theory analysis. The extrapolation technique is applied to improve the accuracy of the schemes, and some interesting results are obtained.     

A simple iterative method for water distribution network analysis

In this paper, a new method categorized as a modification to the Q–h equations is proposed for the analysis of pipe networks. The method is inspired by the Kani method which is used for analysis of structural frames. The new method can be considered as an iterative procedure which does not require a large system of nonlinear equations to be solved simultaneously. The main advantages of the proposed method are relative simplicity in formulation and programming, and rapid and smooth convergence of initially guessed values. To show the robustness and convergence of the present method, several pipe networks are analyzed, and the results are compared with those of the conventional methods.     

Optimum first and second order extrapolations of Successive Overrelaxation type methods for certain classes of matrices

This paper deals with the iterative solution of the linear systemx=Bx+c when its Jacobi matrixB is weakly 2-cyclic consistently ordered and has a complex eigenvalue spectrum which lies on a straight-line segment. The optimization problem of the following three methods is considered and solved: i) The extrapolation of the optimum Successive Overrelaxation (SOR) ii) The second order extrapolation of a good SOR and iii) The second order extrapolation of the Gauss-Seidel method. In addition a variant of the second order methods considered, suitable for the solution of the system even ifB isnot necessarily weakly 2-cyclic consistently ordered, is proposed. Finally a reference to a theoretical comparison of the various optimum methods in the paper is made and their asymptotic convergence factors for selected eigenvalue spectra are illustrated in a Table in support of the theory developed.     

A two-grid finite element method for the Allen-Cahn equation with the logarithmic potential

In this paper, we present a two-grid finite element method for the Allen-Cahn equation with the logarithmic potential. This method consists of two steps. In the first step, based on a fully implicit finite element method, the Allen-Cahn equation is solved on a coarse grid with mesh size H. In the second step, a linearized system whose nonlinear term is replaced by the value of the first step is solved on a fine grid with mesh size h. We give the energy stabilities of the traditional finite element method and the two-grid finite element method. The optimal convergence order of the two-grid finite element method in H¹ norm is achieved when the mesh sizes satisfy h = O(H²). Numerical examples are given to demonstrate the validity of the proposed scheme. The results show that the two-grid method can save the CPU time while keeping the same convergence rate.     

Extrapolation methods for fixed‐point multilinear PageRank computations

Nonnegative tensors arise very naturally in many applications that involve large and complex data flows. Due to the relatively small requirement in terms of memory storage and number of operations per step, the (shifted) higher order power method is one of the most commonly used technique for the computation of positive Z‐eigenvectors of this type of tensors. However, unlike the matrix case, the method may fail to converge even for irreducible tensors. Moreover, when it converges, its convergence rate can be very slow. These two drawbacks often make the computation of the eigenvectors demanding or unfeasible for large problems. In this work, we consider a particular class of nonnegative tensors associated with the multilinear PageRank modification of higher order Markov chains. Based on the simplified topological ε‐algorithm in its restarted form, we introduce an extrapolation‐based acceleration of power method type algorithms, namely, the shifted fixed‐point method and the inner‐outer method. The accelerated methods show remarkably better performance, with faster convergence rates and reduced overall computational time. Extensive numerical experiments on synthetic and real‐world datasets demonstrate the advantages of the introduced extrapolation techniques.     

Numerical method for solving an inverse problem for a population model

The inverse problem of determining the growth rate coefficient of biological objects from additional information on their time-dependent density is considered. Two nonlinear integral equations are derived for the unknown coefficient, which is determined on part of its domain from one equation and on the remaining part from the other equation. The nonlinear integral equations are solved by iterative methods. The convergence conditions for the iterative methods are formulated, and results of numerical experiments are presented.     

On the convergence of the sequences of Gerschgorin-like disks

Carstensen’s results from 1991, connected with Gerschgorin’s disks, are used to establish a theorem concerning the localization of polynomial zeros and to derive an a posteriori error bound method. The presented quasi-interval method possesses useful property of inclusion methods to produce disks containing all simple zeros of a polynomial. The centers of these disks behave as approximations generated by a cubic derivative free method where the use of quantities already calculated in the previous iterative step decreases the computational cost. We state initial convergence conditions that guarantee the convergence of error bound method and prove that the method has the order of convergence three. Initial conditions are computationally verifiable since they depend only on the polynomial coefficients, its degree and initial approximations. Some computational aspects and the possibility of implementation on parallel computers are considered, including two numerical examples.In honor of Professor Richard S. Varga.     

Waveform relaxation methods for implicit differential equations

We apply a Runge-Kutta-based waveform relaxation method to initial-value problems for implicit differential equations. In the implementation of such methods, a sequence of nonlinear systems has to be solved iteratively in each step of the integration process. The size of these systems increases linearly with the number of stages of the underlying Runge-Kutta method, resulting in high linear algebra costs in the iterative process for high-order Runge-Kutta methods. In our earlier investigations of iterative solvers for implicit initial-value problems, we designed an iteration method in which the linear algebra costs are almost independent of the number of stages when implemented on a parallel computer system. In this paper, we use this parallel iteration process in the Runge-Kutta waveform relaxation method. In particular, we analyse the convergence of the method. The theoretical results are illustrated by a few numerical examples.     

Interval Iterative Methods Under Partial Ordering（Ⅰ）

Many types of nonlinear systems, which can be solved by ordered iterative methods, are discussed in unified form in the present paper. Under different initial conditions, some generalized ordered iterative methods are given, and the existence and uniqueness of the solution and the convergence of the methods are proved.     

Equivalent iterative methods for p-cyclic matrices

An overview is given of the simplifications which arise when p-cyclic systems are solved by iterative methods. Besides the classic iterative methods, we treat the Chebyshev semi-iterative method and the OR and MR variants of the class of Krylov subspace methods. Particular emphasis is given to equivalent iterations applied to the cyclically reduced system.     

The stable ATA-orthogonal s-step Orthomin(k) algorithm with the CADNA library

The major drawback of the s-step iterative methods for nonsymmetric linear systems of equations is that, in the floating-point arithmetic, a quick loss of orthogonality of s-dimensional direction subspaces can occur, and consequently slow convergence and instability in the algorithm may be observed as s gets larger than 5. In [18], Swanson and Chronopoulos have demonstrated that the value of s in the s-step Orthomin(k) algorithm can be increased beyond s=5 by orthogonalizing the s direction vectors in each iteration, and have shown that the A^TA-orthogonal s-step Orthomin(k) is stable for large values of s (up to s=16). The subject of this paper is to show how by using the CADNA library, it is possible to determine a good value of s for A^TA-orthogonal s-step Orthomin(k), and during the run of its code to detect the numerical instabilities and to stop the process correctly, and to restart the A^TA-orthogonal s-step Orthomin(k) in order to improve the computed solution. Numerical examples are used to show the good numerical properties. This revised version was published online in June 2006 with corrections to the Cover Date.