A new family of global methods for linear systems with multiple right-hand sides

The global bi-conjugate gradient (Gl-BCG) method is an attractive matrix Krylov subspace method for solving nonsymmetric linear systems with multiple right-hand sides, but it often show irregular convergence behavior in many applications. In this paper, we present a new family of global $A$-biorthogonal methods by using short two-term recurrences and formal orthogonal polynomials, which contain the global bi-conjugate residual (Gl-BCR) algorithm and its improved version. Finally, numerical experiments illustrate that the proposed methods are highly competitive and often superior to originals.     

Variants of the groupwise update strategy for short-recurrence Krylov subspace methods

Krylov subspace methods often use short-recurrences for updating approximations and the corresponding residuals. In the bi-conjugate gradient (Bi-CG) type methods, rounding errors arising from the matrix–vector multiplications used in the recursion formulas influence the convergence speed and the maximum attainable accuracy of the approximate solutions. The strategy of a groupwise update has been proposed for improving the convergence of the Bi-CG type methods in finite-precision arithmetic. In the present paper, we analyze the influence of rounding errors on the convergence properties when using alternative recursion formulas, such as those used in the bi-conjugate residual (Bi-CR) method, which are different from those used in the Bi-CG type methods. We also propose variants of a groupwise update strategy for improving the convergence speed and the accuracy of the approximate solutions. Numerical experiments demonstrate the effectiveness of the proposed method.     

An improved parallel hybrid bi-conjugate gradient method suitable for distributed parallel computing

An improved parallel hybrid bi-conjugate gradient method (IBiCGSTAB(2) method, in brief) for solving large sparse linear systems with nonsymmetric coefficient matrices is proposed for distributed parallel environments. The method reduces five global synchronization points to two by reconstructing the BiCGSTAB(2) method in [G.L.G. Sleijpen, H.A. van der Vorst, Hybrid bi-conjugate gradient methods for CFD problems, in: M. Hafez, K. Oshima (Eds.), Computational Fluid Dynamics Review 1995, John Wiley & Sons Ltd, Chichester, 1995, pp. 457–476] and the communication time required for the inner product can be efficiently overlapped with useful computation. The cost is only slightly increased computation time, which can be ignored, compared with the reduction of communication time. Performance and isoefficiency analysis shows that the IBiCGSTAB(2) method has better parallelism and scalability than the BiCGSTAB(2) method. Numerical experiments show that the scalability can be improved by a factor greater than 2.5 and the improvement in parallel communication performance approaches 60%.     

Analysis of the finite precision bi-conjugate gradient algorithm for nonsymmetric linear systems

In this paper we analyze the bi-conjugate gradient algorithm in finite precision arithmetic, and suggest reasons for its often observed robustness. By using a tridiagonal structure, which is preserved by the finite precision bi-conjugate gradient iteration, we are able to bound its residual norm by a minimum polynomial of a perturbed matrix (i.e. the residual norm of the exact GMRES applied to a perturbed matrix) multiplied by an amplification factor. This shows that occurrence of near-breakdowns or loss of biorthogonality does not necessarily deter convergence of the residuals provided that the amplification factor remains bounded. Numerical examples are given to gain insights into these bounds.

     

An improved GPBi-CG algorithm suitable for distributed parallel computing

An improved generalized product-type bi-conjugate gradient (GPBi-CG) method (IGPBi-CG method, in brief) for solving large sparse linear systems with unsymmetrical coefficient matrices is proposed for distributed parallel environments. The method reduces three global synchronization points to two by reconstructing GPBi-CG method and the communication time required for the inner product can be efficiently overlapped with useful computation. The cost is only slightly increased computation time, which can be ignored compared with the reduction of communication time. Performance and isoefficiency analysis show that the IGPBi-CG method has better parallelism and scalability than the GPBi-CG method. Numerical experiments show that the scalability can be improved by a factor greater than 1.5 and the improvement in parallel communication performance approaches 33.3˙%.     

Novel numerical techniques for the finite moment log stable computational model for European call option

Option pricing models are often used to describe the dynamic characteristics of prices in financial markets. Unlike the classical Black–Scholes (BS) model, the finite moment log stable (FMLS) model can explain large movements of prices during small time steps. In the FMLS, the second-order spatial derivative of the BS model is replaced by a fractional operator of order α which generates an α-stable Lévy process. In this paper, we consider the finite difference method to approximate the FMLS model. We present two numerical schemes for this approximation: the implicit numerical scheme and the Crank–Nicolson scheme. We carry out convergence and stability analyses for the proposed schemes. Since the fractional operator routinely generates dense matrices which often require high computational cost and storage memory, we explore three methods for solving the approximation schemes: the Gaussian elimination method, the bi-conjugate gradient stabilized method (Bi-CGSTAB) and the fast Bi-CGSTAB (FBi-CGSTAB) in order to compare the cost of calculations. Finally, two numerical examples with exact solutions are presented where we also use extrapolation techniques to achieve higher-order convergence. The results suggest that the proposed schemes are unconditionally stable and convergent, and the FMLS model is useful for pricing options.     

Suitable iterative methods for solving the linear system arising in the three fields domain decomposition method

There are two approaches for applying substructuring preconditioner for the linear system corresponding to the discrete Steklov–Poincaré operator arising in the three fields domain decomposition method for elliptic problems. One of them is to apply the preconditioner in a common way, i.e. using an iterative method such as preconditioned conjugate gradient method [S. Bertoluzza, Substructuring preconditioners for the three fields domain decomposition method, I.A.N.-C.N.R, 2000] and the other one is to apply iterative methods like for instance bi-conjugate gradient method, conjugate gradient square and etc. which are efficient for nonsymmetric systems (the preconditioned system will be nonsymmetric). In this paper, second approach will be followed and extensive numerical tests will be presented which imply that the considered iterative methods are efficient.     

Block GPBi-CG method for solving nonsymmetric linear systems with multiple right-hand sides and its convergence analysis

Numerical Algorithms - In this paper, the block generalized product-type bi-conjugate gradient (GPBi-CG) method for solving large, sparse nonsymmetric linear systems of equations with multiple...     

Investigating a hybrid simulated annealing and local search algorithm for constrained optimization

Constrained Optimization Problems (COP) often take place in many practical applications such as kinematics, chemical process optimization, power systems and so on. These problems are challenging in terms of identifying feasible solutions when constraints are non-linear and non-convex. Therefore, finding the location of the global optimum in the non-convex COP is more difficult as compared to non-convex bound-constrained global optimization problems. This paper proposes a Hybrid Simulated Annealing method (HSA), for solving the general COP. HSA has features that address both feasibility and optimality issues and here, it is supported by a local search procedure, Feasible Sequential Quadratic Programming (FSQP). We develop two versions of HSA. The first version (HSAP) incorporates penalty methods for constraint handling and the second one (HSAD) eliminates the need for imposing penalties in the objective function by tracing feasible and infeasible solution sequences independently. Numerical experiments show that the second version is more reliable in the worst case performance.     

Optimization and homotopy methods for the Gibbs free energy of simple magmatic mixtures

In this paper we consider a mathematical model for magmatic mixtures based on the Gibbs free energy. Different reformulations of the problem are presented and some theoretical results about the existence and number of solutions are derived. Finally, two homotopy methods and a global optimization one are introduced and computationally tested. One of the homotopy methods returns a single solution of the problem, while the other is able to return multiple solutions (often all of them). The global optimization method is a branch-and-reduce one with a theoretical guarantee of detecting all the solutions, although some numerical difficulties, resulting in a loss of a few of them, may have to be faced.     

A one-step 7-stage Hermite-Birkhoff-Taylor ODE solver of order 11

A one-step 7-stage Hermite-Birkhoff-Taylor method of order 11, denoted by HBT(11)7, is constructed for solving nonstiff first-order initial value problems y^′=f(t,y), y(t₀)=y₀. The method adds the derivatives y^′ to y⁽⁶⁾, used in Taylor methods, to a 7-stage Runge-Kutta method of order 6. Forcing an expansion of the numerical solution to agree with a Taylor expansion of the true solution to order 11 leads to Taylor- and Runge-Kutta-type order conditions. These conditions are reorganized into Vandermonde-type linear systems whose solutions are the coefficients of the method. The new method has a larger scaled interval of absolute stability than the Dormand-Prince DP87 and a larger unscaled interval of absolute stability than the Taylor method, T11, of order 11. HBT(11)7 is superior to DP87 and T11 in solving several problems often used to test higher-order ODE solvers on the basis of the number of steps, CPU time, and maximum global error. Numerical results show the benefit of adding high-order derivatives to Runge-Kutta methods.     

A new class of efficient one-step contractivity preserving high-order time discretization methods of order 5 to 14

A family of one-step, explicit, contractivity preserving, multi-stage, multi-derivative, Hermite–Birkhoff–Taylor methods of order p =?5,6,…,14, that we denote by CPHBT_RK4(d,s,p), with nonnegative coefficients are constructed by casting s-stage Runge–Kutta methods of order 4 with Taylor methods of order d. The constructed CPHBT_RK4 methods are implemented using efficient variable step control and are compared to other well-known methods on a variety of initial value problems. A selected method: CP 6-stages 9-derivative HBT method of order 12, denoted by CPHBTRK412, has larger region of absolute stability than Dormand–Prince DP(8,7)13M and Taylor method T(12) of order 12. It is superior to DP(8,7)13M and T(12) methods on the basis the number of steps, CPU time, and maximum global error on several problems often used to test higher-order ODE solvers. Also, we show that the contractivity preserving property of CPHBTRK412is very efficient in suppressing the effect of the propagation of discretization errors and the new method compares positively with explicit 17 stages Runge-Kutta-Nyström pair of order 12 by Sharp et al. on a long-term integration of a standard N-body problem. The selected CPHBTRK412is listed in the Appendix.     

Error analysis of explicit TSERKN methods for highly oscillatory systems

In this paper, we are concerned with the error analysis for the two-step extended Runge-Kutta-Nyström-type (TSERKN) methods [Comput. Phys. Comm. 182 (2011) 2486–2507] for multi-frequency and multidimensional oscillatory systems y″(t) + My(t) = f(t, y(t)), where high-frequency oscillations in the solutions are generated by the linear part My(t). TSERKN methods extend the two-step hybrid methods [IMA J. Numer. Anal. 23 (2003) 197–220] by reforming both the internal stages and the updates so that they are adapted to the oscillatory properties of the exact solutions. However, the global error analysis for the TSERKN methods has not been investigated. In this paper we construct a new three-stage explicit TSERKN method of order four and present the global error bound for the new method, which is proved to be independent of ∥M∥ under suitable assumptions. This property of our new method is very important for solving highly oscillatory systems (1), where ∥M∥ may be arbitrarily large. We also analyze the stability and phase properties for the new method. Numerical experiments are included and the numerical results show that the new method is very competitive and promising compared with the well-known high quality methods proposed in the scientific literature.     

One-step 9-stage Hermite–Birkhoff–Taylor ODE solver of order 10

A one-step 9-stage Hermite–Birkhoff–Taylor method of order 10, denoted by HBT(10)9, is constructed for solving nonstiff systems of first-order differential equations of the form y′=f(x,y), y(x ₀)=y ₀. The method uses y′ and higher derivatives y ⁽²⁾ to y ⁽⁴⁾ as in Taylor methods and is combined with a 9-stage Runge–Kutta method. Forcing a Taylor expansion of the numerical solution to agree with an expansion of the true solution leads to Taylor- and Runge–Kutta-type order conditions which are reorganized into Vandermonde-type linear systems whose solutions are the coefficients of the method. The new method has a larger scaled interval of absolute stability than Dormand–Prince DP(8,7)13M. The stepsize is controlled by means of y ⁽²⁾ and y ⁽⁴⁾. HBT(10)9 is superior to DP(8,7)13M and Taylor method of order 10 in solving several problems often used to test high-order ODE solvers on the basis of the number of steps, CPU time, and maximum global error. These numerical results show the benefits of adding high-order derivatives to Runge–Kutta methods.     

Optimal algorithms for global optimization in case of unknown Lipschitz constant

We consider the global optimization problem for d-variate Lipschitz functions which, in a certain sense, do not increase too slowly in a neighborhood of the global minimizer(s). On these functions, we apply optimization algorithms which use only function values. We propose two adaptive deterministic methods. The first one applies in a situation when the Lipschitz constant L is known. The second one applies if L is unknown. We show that for an optimal method, adaptiveness is necessary and that randomization (Monte Carlo) yields no further advantage. Both algorithms presented have the optimal rate of convergence.     

几乎各向同性的高维空间分数阶扩散方程的分块快速正则Hermite分裂预处理方法

一类空间分数阶扩散方程经过有限差分离散后所得到的离散线性方程组的系数矩阵是两个对角矩阵与Toeplitz型矩阵的乘积之和.在本文中,对于几乎各向同性的二维或三维空间分数阶扩散方程的离散线性方程组,采用预处理Krylov子空间迭代方法,我们利用其系数矩阵的特殊结构和具体性质构造了一类分块快速正则Hermite分裂预处理子.通过理论分析,我们证明了所对应的预处理矩阵的特征值大部分都聚集于1的附近.数值实验也表明,这类分块快速正则Hermite分裂预处理子可以明显地加快广义极小残量(GMRES)方法和稳定化的双共轭梯度(BiCGSTAB)方法等Krylov子空间迭代方法的收敛速度.     

Why Gaussian quadrature in the complex plane?

This paper synthesizes formally orthogonal polynomials, Gaussian quadrature in the complex plane and the bi-conjugate gradient method together with an application. Classical Gaussian quadrature approximates an integral over (a region of) the real line. We present an extension of Gaussian quadrature over an arc in the complex plane, which we call complex Gaussian quadrature. Since there has not been any particular interest in the numerical evaluation of integrals over the long history of complex function theory, complex Gaussian quadrature is in need of motivation. Gaussian quadrature in the complex plane yields approximations of certain sums connected with the bi-conjugate gradient method. The scattering amplitude c ^T A ^–1 b is an example where A is a discretization of a differential–integral operator corresponding to the scattering problem and b and c are given vectors. The usual method to estimate this is to use c ^T x ^(k). A result of Warnick is that this is identically equal to the complex Gaussian quadrature estimate of 1/. Complex Gaussian quadrature thereby replaces this particular inner product in the estimate of the scattering amplitude.     

A coordinate gradient descent method for linearly constrained smooth optimization and support vector machines training

Support vector machines (SVMs) training may be posed as a large quadratic program (QP) with bound constraints and a single linear equality constraint. We propose a (block) coordinate gradient descent method for solving this problem and, more generally, linearly constrained smooth optimization. Our method is closely related to decomposition methods currently popular for SVM training. We establish global convergence and, under a local error bound assumption (which is satisfied by the SVM QP), linear rate of convergence for our method when the coordinate block is chosen by a Gauss-Southwell-type rule to ensure sufficient descent. We show that, for the SVM QP with n variables, this rule can be implemented in O(n) operations using Rockafellar’s notion of conformal realization. Thus, for SVM training, our method requires only O(n) operations per iteration and, in contrast to existing decomposition methods, achieves linear convergence without additional assumptions. We report our numerical experience with the method on some large SVM QP arising from two-class data classification. Our experience suggests that the method can be efficient for SVM training with nonlinear kernel.     

A solution of the affine quadratic inverse eigenvalue problem

The quadratic inverse eigenvalue problem (QIEP) is to find the three matrices M,C, and K, given a set of numbers, closed under complex conjugations, such that these numbers become the eigenvalues of the quadratic pencil P(λ)=λ²M+λC+K. The affine inverse quadratic eigenvalue problem (AQIEP) is the QIEP with an additional constraint that the coefficient matrices belong to an affine family, that is, these matrices are linear combinations of substructured matrices. An affine family of matrices very often arise in vibration engineering modeling and analysis. Research on QIEP and AQIEP are still at developing stage. In this paper, we propose three methods and the associated mathematical theories for solving AQIEP: A Newton method, an alternating projections method, and a hybrid method combining the two. Validity of these methods are illustrated with results on numerical experiments on a spring-mass problem and comparisons are made with these three methods amongst themselves and with another Newton method developed by Elhay and Ram (2002) [12]. The results of our experiments show that the hybrid method takes much smaller number of iterations and converges faster than any of these methods.     

Transpose-free formulations of Lanczos-type methods for nonsymmetric linear systems

We present a transpose-free version of the nonsymmetric scaled Lanczos procedure. It generates the same tridiagonal matrix as the classical algorithm, using two matrix–vector products per iteration without accessing A^T. We apply this algorithm to obtain a transpose-free version of the Quasi-minimal residual method of Freund and Nachtigal [15] (without look-ahead), which requires three matrix–vector products per iteration. We also present a related transpose-free version of the bi-conjugate gradients algorithm. This revised version was published online in June 2006 with corrections to the Cover Date.