Multirate infinitesimal step methods for atmospheric flow simulation

The numerical solution of the Euler equations requires the treatment of processes in different temporal scales. Sound waves propagate fast compared to advective processes. Based on a spatial discretisation on staggered grids, a multirate time integration procedure is presented here generalising split-explicit Runge-Kutta methods. The advective terms are integrated by a Runge-Kutta method with a macro stepsize restricted by the CFL number. Sound wave terms are treated by small time steps respecting the CFL restriction dictated by the speed of sound.Split-explicit Runge-Kutta methods are generalised by the inclusion of fixed tendencies of previous stages. The stability barrier for the acoustics equation is relaxed by a factor of two.Asymptotic order conditions for the low Mach case are given. The relation to commutator-free exponential integrators is discussed. Stability is analysed for the linear acoustic equation. Numerical tests are executed for the linear acoustics and the nonlinear Euler equations.     

Arbitrary-order functionally fitted energy-diminishing methods for gradient systems

It is well known that for gradient systems in Euclidean space or on a Riemannian manifold, the energy decreases monotonically along solutions. In this letter we derive and analyse functionally fitted energy-diminishing methods to preserve this key property of gradient systems. It is proved that the novel methods are energy-diminishing and can achieve damping for very stiff gradient systems. We also show that the methods can be of arbitrarily high order and discuss their implementations. A numerical test is reported to illustrate the efficiency of the new methods in comparison with three existing numerical methods in the literature.     

Computational comparison of two methods for constrained global optimization

For constrained concave global minimization problems, two very different solution techniques have been investigated. The first such method is a stochastic mulitstart approach which typically finds, with high probability, all local minima for the problem. The second method is deterministic and guarantees a global minimum solution to within any user specified tolerance. It is the purpose of this paper to make a careful comparison of these two methods on a range of test problems using separable concave objectives over compact polyhedral sets, and to investigate in this way the advantages and disadvantages of each method. A direct computational comparison, on the same set of over 140 problems, is presented.     

On Warner's algorithm for the solution of boundary-value problems for ordinary differential equations

The paper discusses the solution of boundary-value problems for ordinary differential equations by Warner's algorithm. This shooting algorithm requires that only the original system of differential equations is solved once in each iteration, while the initial conditions for a new iteration are evaluated from a matrix equation. Numerical analysis performed shows that the algorithm converges even for very bad starting values of the unknown initial conditions and that the number of iterations is small and weakly dependent on the starting point. Based on this algorithm, a general subroutine can be realized for the solution of a large class of boundary-value problems.     

一种新型压缩预处理CGS算法

CGS算法是求解大型非对称线性方程组的常用算法，然而该算法无极小残差性质，因此它常因出现较大的中间剩余向量而出现典型的不规则收敛行为．本根据IRA方法提出了一种压缩预处理CGS方法，数值实验表明这种算法在一定程度上减小了迭代算法在收敛过程中的剩余问题，从而使得算法具有更好的稳定性，该法构造简单，减少了收敛次数，加快了收敛速度．     

Fourth-Order Splitting Methods for Time-Dependant Differential Equations

This study was suggested by previous work on the simulation of evolution equations with scale-dependent processes,e.g.,wave-propagation or heat-transfer,that are modeled by wave equations or heat equations.Here,we study both parabolic and hyperbolic equations.We focus on ADI (alternating direction implicit) methods and LOD (locally one-dimensional) methods,which are standard splitting methods of lower order,e.g.second-order.Our aim is to develop higher-order ADI methods,which are performed by Richardson extrapolation,Crank-Nicolson methods and higher-order LOD methods,based on locally higher-order methods.We discuss the new theoretical results of the stability and consistency of the ADI methods.The main idea is to apply a higher- order time discretization and combine it with the ADI methods.We also discuss the dis- cretization and splitting methods for first-order and second-order evolution equations. The stability analysis is given for the ADI method for first-order time derivatives and for the LOD (locally one-dimensional) methods for second-order time derivatives.The higher-order methods are unconditionally stable.Some numerical experiments verify our results.     

A primal-dual trust region algorithm for nonlinear optimization

This paper concerns general (nonconvex) nonlinear optimization when first and second derivatives of the objective and constraint functions are available. The proposed method is based on finding an approximate solution of a sequence of unconstrained subproblems parameterized by a scalar parameter. The objective function of each unconstrained subproblem is an augmented penalty-barrier function that involves both primal and dual variables. Each subproblem is solved using a second-derivative Newton-type method that employs a combined trust region and line search strategy to ensure global convergence. It is shown that the trust-region step can be computed by factorizing a sequence of systems with diagonally-modified primal-dual structure, where the inertia of these systems can be determined without recourse to a special factorization method. This has the benefit that off-the-shelf linear system software can be used at all times, allowing the straightforward extension to large-scale problems. Numerical results are given for problems in the COPS test collection.Mathematics Subject Classification (2000): 49M37, 65F05, 65K05, 90C30This paper is dedicated to Roger Fletcher on the occasion of his 65th birthday     

Two-Phase Model Algorithm with Global Convergence for Nonlinear Programming

The family of feasible methods for minimization with nonlinear constraints includes the nonlinear projected gradient method, the generalized reduced gradient method (GRG), and many variants of the sequential gradient restoration algorithm (SGRA). Generally speaking, a particular iteration of any of these methods proceeds in two phases. In the restoration phase, feasibility is restored by means of the resolution of an auxiliary nonlinear problem, generally a nonlinear system of equations. In the minimization phase, optimality is improved by means of the consideration of the objective function, or its Lagrangian, on the tangent subspace to the constraints. In this paper, minimal assumptions are stated on the restoration phase and the minimization phase that ensure that the resulting algorithm is globally convergent. The key point is the possibility of comparing two successive nonfeasible iterates by means of a suitable merit function that combines feasibility and optimality. The merit function allows one to work with a high degree of infeasibility at the first iterations of the algorithm. Global convergence is proved and a particular implementation of the model algorithm is described.     

A convergence proof for an affine-scaling algorithm for convex quadratic programming without nondegeneracy assumptions

This paper presents a theoretical result on convergence of a primal affine-scaling method for convex quadratic programs. It is shown that, as long as the stepsize is less than a threshold value which depends on the input data only, Ye and Tse's interior ellipsoid algorithm for convex quadratic programming is globally convergent without nondegeneracy assumptions. In addition, its local convergence rate is at least linear and the dual iterates have an ergodically convergent property.Research supported in part by the NSF under grant DDM-8721709.     

Sequential conjugate-gradient-restoration algorithm for optimal control problems. Part 1. Theory

This paper considers the problem of minimizing a functionalI which depends on the statex(t), the controlu(t), and the parameter π. Here,I is a scalar,x ann-vector,u anm-vector, and π ap-vector. At the initial point, the state is prescribed. At the final point, the state and the parameter are required to satisfyq scalar relations. Along the interval of integration, the state, the control, and the parameter are required to satisfyn scalar differential equations. First, the case of a quadratic functional subject to linear constraints is considered, and a conjugate-gradient algorithm is derived. Nominal functionsx(t),u(t), π satisfying all the differential equations and boundary conditions are assumed. Variations Δx(t), δu(t), Δπ are determined so that the value of the functional is decreased. These variations are obtained by minimizing the first-order change of the functional subject to the differential equations, the boundary conditions, and a quadratic constraint on the variations of the control and the parameter. Next, the more general case of a nonquadratic functional subject to nonlinear constraints is considered. The algorithm derived for the linear-quadratic case is employed with one modification: a restoration phase is inserted between any two successive conjugate-gradient phases. In the restoration phase, variations Δx(t), Δu(t), Δπ are determined by requiring the least-square change of the control and the parameter subject to the linearized differential equations and the linearized boundary conditions. Thus, a sequential conjugate-gradient-restoration algorithm is constructed in such a way that the differential equations and the boundary conditions are satisfied at the end of each complete conjugate-gradient-restoration cycle. Several numerical examples illustrating the theory of this paper are given in Part 2 (see Ref. 1). These examples demonstrate the feasibility as well as the rapidity of convergence of the technique developed in this paper. This research was supported by the Office of Scientific Research, Office of Aerospace Research, United States Air Force, Grant No. AF-AFOSR-72-2185. The authors are indebted to Professor A. Miele for stimulating discussions. Formerly, Graduate Studient in Aero-Astronautics, Department of Mechanical and Aerospace Engineering and Materials Science, Rice University, Houston, Texas.     

On quasi-Newton and pseudo-Newton algorithms

This paper surveys some of the existing approaches to quasi-Newton methods and introduces a new way for constructing inverse Hessian approximations for such algorithms. This new approach is based on restricting Newton's method to subspaces over which the inverse Hessian is assumed to be known, while expanding this subspace using gradient information. It is shown that this approach can lead to some well-known formulas for updating the inverse Hessian approximation. Deriving such updates through this approach provides new understanding of these formulas and their relation to the pseudo-Newton-Raphson algorithm.     

Numerical Verification Methods for Solutions of the Free Boundary Problem

We propose two methods to enclose the solution of an ordinary free boundary problem. The problem is reformulated as a nonlinear boundary value problem on a fixed interval including an unknown parameter. By appropriately setting a functional space that depends on the finite element approximation, the solution is represented as a fixed point of a compact map. Then, by using the finite element projection with constructive error estimates, a Newton-type verification procedure is derived. In addition, numerical examples confirming the effectiveness of current methods are given.     

Primal—dual methods for linear programming

Many interior-point methods for linear programming are based on the properties of the logarithmic barrier function. After a preliminary discussion of the convergence of the (primal) projected Newton barrier method, three types of barrier method are analyzed. These methods may be categorized as primal, dual and primal—dual, and may be derived from the application of Newton's method to different variants of the same system of nonlinear equations. A fourth variant of the same equations leads to a new primal—dual method.In each of the methods discussed, convergence is demonstrated without the need for a nondegeneracy assumption or a transformation that makes the provision of a feasible point trivial. In particular, convergence is established for a primal—dual algorithm that allows a different step in the primal and dual variables and does not require primal and dual feasibility.Finally, a new method for treating free variables is proposed.Presented at the Second Asilomar Workshop on Progress in Mathematical Programming, February 1990, Asilomar, CA, United StatesThe material contained in this paper is based upon research supported by the National Science Foundation Grant DDM-9204208 and the Office of Naval Research Grant N00014-90-J-1242.     

Improvements of Some Projection Methods for Monotone Nonlinear Variational Inequalities

In this paper, we study the relationship of some projection-type methods for monotone nonlinear variational inequalities and investigate some improvements. If we refer to the Goldstein–Levitin–Polyak projection method as the explicit method, then the proximal point method is the corresponding implicit method. Consequently, the Korpelevich extragradient method can be viewed as a prediction-correction method, which uses the explicit method in the prediction step and the implicit method in the correction step. Based on the analysis in this paper, we propose a modified prediction-correction method by using better prediction and correction stepsizes. Preliminary numerical experiments indicate that the improvements are significant.     

On Producing Multiple Solutions Using Repeated Trials

The number of trials that is required by an algorithm to produce a given fraction of the problem solutions with a specified level of confidence is analyzed. The analysis indicates that the number of trials required to find a large fraction of the solutions rapidly decreases as the number of solutions obtained on each trial by an algorithm increases. In applications where multiple solutions are sought, this decrease in the number of trials could potentially offset the additional computational cost of algorithms that produce multiple solutions on a single trial. The analysis framework presented is used to compare the efficiency of a homotopy algorithm to that of a Newton method by measuring both the number of trials and the number of calculations required to obtain a specified fraction of the solutions.     

A Survey of Algebraic Multilevel Iteration (AMLI) Methods

For the solution by preconditioned conjugate gradient methods of symmetric positive definite equations as arising in boundary value problems we consider preconditioning methods of AMLI type. Particular attention is devoted to providing methods of optimal order of computational complexity which in addition promise to be robust, i.e. with a convergence rate which is bounded above independently of size of discretization parameter h, jumps in problem coefficients, and shape of finite elements or, equivalently, anisotropy of problem coefficients. In addition, the computational cost per iteration step must have optimal order.New results on upper bounds of one of the important parameters in the methods, the Cauchy—Bunyakowski—Schwarz constant are given and an algebraic method how to improve its value is presented.This revised version was published online in October 2005 with corrections to the Cover Date.     

Global optimization by multilevel search

A new approach to the global optimization of functions with extremely rugged graphs is introduced. This multilevel search method is both an algorithm and a meta-algorithm, a logic for regulating optimization done by other algorithms. First, it is examined in the one-dimensional case theoretically and through simple examples. Then, to deal with higher dimensions, multilevel search is combined with the Monte Carlo method; this hybrid algorithm is tested on standard problems and is found to perform extremely well for a derivative-free method.     

On the Computation of Optimal Static Output Feedback Controllers for Discrete-Time Systems

In this article, we consider the static output feedback problem for discrete-time systems when complete set of state variables is not available. It has been reported that quasi-Newton methods have substandard performance on this problem. A structured quasi-Newton method with trust region globalization is analyzed and studied for solving this problem. Moreover, the classical Anderson-Moore method is enhanced by using the trust region mechanism to ensure global convergence instead of the line search technique. The algorithms are tested numerically on test problems of engineering applications.     

Convergence of multistep discretizations of DAEs

Standard ODE methods such as linear multistep methods encounter difficulties when applied to differential-algebraic equations (DAEs) of index greater than 1. In particular, previous results for index 2 DAEs have practically ruled out the use of all explicit methods and of implicit multistep methods other than backward difference formulas (BDFs) because of stability considerations. In this paper we embed known results for semi-explicit index 1 and 2 DAEs in a more comprehensive theory based on compound multistep and one-leg discretizations. This explains and characterizes the necessary requirements that a method must fulfill in order to be applicable to semi-explicit DAEs. Thus we conclude that the most useful discretizations are those that avoid discretization of the constraint. A freer use of e.g. explicit methods for the non-stiff differential part of the DAE is then possible.Dedicated to Germund Dahlquist on the occasion of his 70th birthdayThis author thanks the Centro de Estadística y Software Matemático de la Universidad Simón Bolivar (CESMa) for permitting her free use of its research facilities.Partial support by the Swedish Research Council for Engineering Sciences TFR under contract no. 222/91-405.     

Linearized ridge-path method for function minimization

A modification based on a linearization of a ridge-path optimization method is presented. The linearized ridge-path method is a nongradient, conjugate direction method which converges quadratically in half the number of search directions required for Powell's method of conjugate directions. The ridge-path method and its modification are compared with some basic algorithms, namely, univariate method, steepest descent method, Powell's conjugate direction method, conjugate gradient method, and variable-metric method. The assessment indicates that the ridge-path method, with modifications, could present a promising technique for optimization.This work was in partial fulfillment of the requirements for the MS degree of the first author at Cairo University, Cairo, Egypt. The authors would like to acknowledge the helpful and constructive suggestions of the reviewer.