Parallel iterative linear solvers for multistep Runge-Kutta methods

This paper deals with solving stiff systems of differential equations by implicit Multistep Runge-Kutta (MRK) methods. For this type of methods, nonlinear systems of dimension sd arise, where s is the number of Runge-Kutta stages and d the dimension of the problem. Applying a Newton process leads to linear systems of the same dimension, which can be very expensive to solve in practice. With a parallel iterative linear system solver, especially designed for MRK methods, we approximate these linear systems by s systems of dimension d, which can be solved in parallel on a computer with s processors. In terms of Jacobian evaluations and LU-decompositions, the k-steps-stage MRK applied with this technique is on s processors equally expensive as the widely used k-step Backward Differentiation Formula on 1 processor, whereas the stability properties are better than that of BDF. A simple implementation of both methods shows that, for the same number of Newton iterations, the accuracy delivered by the new method is higher than that of BDF.     

A choice of forcing terms in inexact Newton method

Inexact Newton method is one of the effective tools for solving systems of nonlinear equations. In each iteration step of the method, a forcing term, which is used to control the accuracy when solving the Newton equations, is required. The choice of the forcing terms is of great importance due to their strong influence on the behavior of the inexact Newton method, including its convergence, efficiency, and even robustness. To improve the efficiency and robustness of the inexact Newton method, a new strategy to determine the forcing terms is given in this paper. With the new forcing terms, the inexact Newton method is locally Q-superlinearly convergent. Numerical results are presented to support the effectiveness of the new forcing terms.     

New quasi-Newton method for solving systems of nonlinear equations

We propose a new Broyden method for solving systems of nonlinear equations, which uses the first derivatives, but is more efficient than the Newton method (measured by the computational time) for larger dense systems. The new method updates QR or LU decompositions of nonsymmetric approximations of the Jacobian matrix, so it requires O(n ²) arithmetic operations per iteration in contrast with the Newton method, which requires O(n ³) operations per iteration. Computational experiments confirm the high efficiency of the new method.     

Comparison of preconditioned Krylov subspace iteration methods for PDE-constrained optimization problems

We construct a novel multi-step iterative method for solving systems of nonlinear equations by introducing a parameter θ to generalize the multi-step Newton method while keeping its order of convergence and computational cost. By an appropriate selection of θ, the new method can both have faster convergence and have larger radius of convergence. The new iterative method only requires one Jacobian inversion per iteration, and therefore, can be efficiently implemented using Krylov subspace methods. The new method can be used to solve nonlinear systems of partial differential equations, such as complex generalized Zakharov systems of partial differential equations, by transforming them into systems of nonlinear equations by discretizing approaches in both spatial and temporal independent variables such as, for instance, the Chebyshev pseudo-spectral discretizing method. Quite extensive tests show that the new method can have significantly faster convergence and significantly larger radius of convergence than the multi-step Newton method.     

Solvability of Newton equations in smoothing-type algorithms for the SOCCP

In this paper, we first investigate the invertibility of a class of matrices. Based on the obtained results, we then discuss the solvability of Newton equations appearing in the smoothing-type algorithm for solving the second-order cone complementarity problem (SOCCP). A condition ensuring the solvability of such a system of Newton equations is given. In addition, our results also show that the assumption that the Jacobian matrix of the function involved in the SOCCP is a P₀-matrix is not enough for ensuring the solvability of such a system of Newton equations, which is different from the one of smoothing-type algorithms for solving many traditional optimization problems in ℜⁿ.     

Approximate Newton Methods for Nonsmooth Equations

We develop general approximate Newton methods for solving Lipschitz continuous equations by replacing the iteration matrix with a consistently approximated Jacobian, thereby reducing the computation in the generalized Newton method. Locally superlinear convergence results are presented under moderate assumptions. To construct a consistently approximated Jacobian, we introduce two main methods: the classic difference approximation method and the -generalized Jacobian method. The former can be applied to problems with specific structures, while the latter is expected to work well for general problems. Numerical tests show that the two methods are efficient. Finally, a norm-reducing technique for the global convergence of the generalized Newton method is briefly discussed.     

An interior penalty method for a finite-dimensional linear complementarity problem in financial engineering

In this work we study an interior penalty method for a finite-dimensional large-scale linear complementarity problem (LCP) arising often from the discretization of stochastic optimal problems in financial engineering. In this approach, we approximate the LCP by a nonlinear algebraic equation containing a penalty term linked to the logarithmic barrier function for constrained optimization problems. We show that the penalty equation has a solution and establish a convergence theory for the approximate solutions. A smooth Newton method is proposed for solving the penalty equation and properties of the Jacobian matrix in the Newton method have been investigated. Numerical experimental results using three non-trivial test examples are presented to demonstrate the rates of convergence, efficiency and usefulness of the method for solving practical problems.     

A new class of smoothing functions and a smoothing Newton method for complementarity problems

In this paper, we introduce a new class of smoothing functions, which include some popular smoothing complementarity functions. We show that the new smoothing functions possess a system of favorite properties. The existence and continuity of a smooth path for solving the nonlinear complementarity problem (NCP) with a P ₀ function are discussed. The Jacobian consistency of this class of smoothing functions is analyzed. Based on the new smoothing functions, we investigate a smoothing Newton algorithm for the NCP and discuss its global and local superlinear convergence. Some preliminary numerical results are reported.     

Half-explicit Runge-Kutta methods for semi-explicit differential-algebraic equations of index 1

Summary For the numerical solution of non-stiff semi-explicit differentialalgebraic equations (DAEs) of index 1 half-explicit Runge-Kutta methods (HERK) are considered that combine an explicit Runge-Kutta method for the differential part with a simplified Newton method for the (approximate) solution of the algebraic part of the DAE. Two principles for the choice of the initial guesses and the number of Newton steps at each stage are given that allow to construct HERK of the same order as the underlying explicit Runge-Kutta method. Numerical tests illustrate the efficiency of these methods.     

PICARD ITERATION FOR NONSMOOTH EQUATIONS

1. IntroductionConsider the following nonsmooth equationsF(x) = 0 (l)where F: R" - R" is LipsChitz continuous. A lot of work has been done and is bellg doneto deal with (1). It is basicly a genera1ization of the cIassic Newton method [8,10,11,14],Newton-lthe methods[1,18] and quasiNewton methods [6,7]. As it is discussed in [7], the latter,quasiNewton methods, seem to be lindted when aPplied to nonsmooth caJse in that a boundof the deterioration of uPdating matrir can not be maintained w…     

Generalized Newton’s method based on graphical derivatives

This paper concerns developing a numerical method of the Newton type to solve systems of nonlinear equations described by nonsmooth continuous functions. We propose and justify a new generalized Newton algorithm based on graphical derivatives, which have never been used to derive a Newton-type method for solving nonsmooth equations. Based on advanced techniques of variational analysis and generalized differentiation, we establish the well-posedness of the algorithm, its local superlinear convergence, and its global convergence of the Kantorovich type. Our convergence results hold with no semismoothness and Lipschitzian assumptions, which is illustrated by examples. The algorithm and main results obtained in the paper are compared with well-recognized semismooth and B-differentiable versions of Newton’s method for nonsmooth Lipschitzian equations.     

Solving the generalized regularized long wave equation on the basis of a reproducing kernel space

On the basis of a reproducing kernel space, an iterative algorithm for solving the generalized regularized long wave equation is presented. The analytical solution in the reproducing kernel space is shown in a series form and the approximate solution u_n is constructed by truncating the series to n terms. The convergence of u_n to the analytical solution is also proved. Results obtained by the proposed method imply that it can be considered as a simple and accurate method for solving such evolution equations.     

Splitting methods for second‐order initial value problems

We consider implicit integration methods for the solution of stiff initial value problems for second-order differential equations of the special form y' = f(y). In implicit methods, we are faced with the problem of solving systems of implicit relations. This paper focuses on the construction and analysis of iterative solution methods which are effective in cases where the Jacobian of the right‐hand side of the differential equation can be split into a sum of matrices with a simple structure. These iterative methods consist of the modified Newton method and an iterative linear solver to deal with the linear Newton systems. The linear solver is based on the approximate factorization of the system matrix associated with the linear Newton systems. A number of convergence results are derived for the linear solver in the case where the Jacobian matrix can be split into commuting matrices. Such problems often arise in the spatial discretization of time‐dependent partial differential equations. Furthermore, the stability matrix and the order of accuracy of the integration process are derived in the case of a finite number of iterations. This revised version was published online in August 2006 with corrections to the Cover Date.     

The numerical solution of the pre-elimination models of cable configurations

The pre-elimination model of the segmented representation of cable shape consists of a system of n nonlinear algebraic equations with the orientations of the n discrete segments as unknown variables. In this paper the banded structure of the Jacobian of the system is utilised to derive an efficient Gauss-Newton solution of the nonlinear equations. It is shown that the finite difference approximation to the Jacobian can be obtained with only three vector function evaluations instead of n. It follows that the computational effort for each iterative adjustment to the approximate solution is linear in n, which means that such pre-elimination cable modelling with the associated numerical solution method is an economical way of solving cable problems.     

A newton-penalty method for a simplified liquid crystal model

In this paper we are concerned with the computation of a liquid crystal model defined by a simplified Oseen-Frank energy functional and a (sphere) nonlinear constraint. A particular case of this model defines the well known harmonic maps. We design a new iterative method for solving such a minimization problem with the nonlinear constraint. The main ideas are to linearize the nonlinear constraint by Newton’s method and to define a suitable penalty functional associated with the original minimization problem. It is shown that the solution sequence of the new minimization problems with the linear constraints converges to the desired solutions provided that the penalty parameters are chosen by a suitable rule. Numerical results confirm the efficiency of the new method.     

Preconditioning in implicit initial-value problem methods on parallel computers

Implicit step-by-step methods for numerically solving the initial-value problem {y=f(y),y(0)=y ₀} usually lead to implicit relations of which the Jacobian can be approximated by a matrix of the special formK=I–hM J, whereM is a matrix characterizing the step-by-step method andJ is the Jacobian off. Similar implicit relations are encountered in discretizing initial-value problems for other types of functional equations such as VIEs, VIDEs and DDEs. Application of (modified) Newton iteration for solving these implicit relations requires the LU-decomposition ofK. Ifs andd are the dimensions ofM andJ, respectively, then this LU-decomposition is anO(s ³ d ³) process, which is extremely costly for large values ofsd. We shall discuss parallel iteration methods for solving the implicit relations that exploit the special form of Jacobian matrixK. Their main characteristic is that each processor is required to compute LU-decompositions of matrices of dimensiond, so that this part of the computational work is reduced by a factors ³. On the other hand, the number of iterations in these parallel iteration methods is usually much larger than in Newton iteration. In this contribution, we will try to reduce the number of iterations by improving the convergence of such parallel iteration methods by means of preconditioning.This paper is presented as an outcome of the LMS Durham Symposium convened by Professor C.T.H. Baker on 4–14 July 1992, with support from the SERC under grant reference number GR/H03964.     

Improved Newton’s method without direct function evaluations

For solving systems of nonlinear equations, we have recently developed a Newton’s method to manage issues with inaccurate function values or problems with high computational cost. In this work we introduce a modification of the above method, reducing the total computational cost and improving, in general, its overall performance. Moreover, the proposed version retains the quadratic convergence, the good behavior over singular and ill-conditioned cases of Jacobian matrix, and its capability to be ideal for imprecise function problems. Numerical results demonstrate the efficiency of the new proposed method.     

Convergence rate of Newton's method for L 2 spectral estimation

In the paper, we prove the Hölder continuous property of the Jacobian of the function generated from the dual of the power spectrum estimation problem. It follows that the convergence of the Newton method for the problem is at least of order where m is the order of the trigonometric bases. This result theoretically confirms the numerical observation by Potter (1990) and Cole and Goodrich (1993).     

ADI preconditioned Krylov methods for large Lyapunov matrix equations

In the present paper, we propose preconditioned Krylov methods for solving large Lyapunov matrix equations AX+XA^T+BB^T=0. Such problems appear in control theory, model reduction, circuit simulation and others. Using the Alternating Direction Implicit (ADI) iteration method, we transform the original Lyapunov equation to an equivalent symmetric Stein equation depending on some ADI parameters. We then define the Smith and the low rank ADI preconditioners. To solve the obtained Stein matrix equation, we apply the global Arnoldi method and get low rank approximate solutions. We give some theoretical results and report numerical tests to show the effectiveness of the proposed approaches.     

A cascadic multigrid algorithm for semilinear elliptic problems

Summary. We propose a cascadic multigrid algorithm for a semilinear elliptic problem. The nonlinear equations arising from linear finite element discretizations are solved by Newton's method. Given an approximate solution on the coarsest grid on each finer grid we perform exactly one Newton step taking the approximate solution from the previous grid as initial guess. The Newton systems are solved iteratively by an appropriate smoothing method. We prove that the algorithm yields an approximate solution within the discretization error on the finest grid provided that the start approximation is sufficiently accurate and that the initial grid size is sufficiently small. Moreover, we show that the method has multigrid complexity. Received February 12, 1998 / Revised version received July 22, 1999 / Published online June 8, 2000