A quasi-Newton trust-region method

The classical trust-region method for unconstrained minimization can be augmented with a line search that finds a point that satisfies the Wolfe conditions. One can use this new method to define an algorithm that simultaneously satisfies the quasi-Newton condition at each iteration and maintains a positive-definite approximation to the Hessian of the objective function. This new algorithm has strong global convergence properties and is robust and efficient in practice.     

线性方程组迭代方法残差光滑技术的一个推广

1　引　　言我们考虑求解线性方程组Ax=b,A∈Rn×n,b,x∈Rn.(1)的迭代方法.迭代序列{xk}的性态常常由与之对应的残差范数序列{‖rk‖}的特性来决定.人们自然希望{‖rk‖}光滑地(单调地)收敛到0.在所有Krylov子空间方法中,GMRES[7]方法因为可使{‖rk‖}最优地趋于0,故是一个较为成功的方法.但是,GMRES方法的工作量和存贮量却随着迭代步数的增加而迅速增加.而BCG[4]和CGS[10]等方法具有运算量小,收敛快等突出优点.但它们的残差范数性态却很不规则,{‖rk‖}振荡不定.这给判断收敛性及何时停机带来很大的不便.残差光滑技术是一个行之有…     

New least-square algorithms

New algorithms are presented for approximating the minimum of the sum of squares ofM real and differentiable functions over anN-dimensional space. These algorithms update estimates for the location of a minimum after each one of the functions and its first derivatives are evaluated, in contrast with other least-square algorithms which evaluate allM functions and their derivatives at one point before using any of this information to make an update. These new algorithms give estimates which fluctuate about a minimum rather than converging to it. For many least-square problems, they give an adequate approximation for the solution more quickly than do other algorithms.It is a pleasure to thank J. Chesick of Haverford College for suggesting and implementing the application of this algorithm to x-ray crystallography.     

A direct method for the general solution of a system of linear equations

A computationally stable method for the general solution of a system of linear equations is given. The system isA ^Tx–B=0, where then-vectorx is unknown and then×q matrixA and theq-vectorB are known. It is assumed that the matrixA ^T and the augmented matrix [A ^T,B] are of the same rankm, wheremn, so that the system is consistent and solvable. Whenm<n, the method yields the minimum modulus solutionx _m and a symmetricn ×n matrixH _m of rankn–m, so thatx=x _m+H _my satisfies the system for ally, ann-vector. Whenm=n, the matrixH _m reduces to zero andx _m becomes the unique solution of the system.The method is also suitable for the solution of a determined system ofn linear equations. When then×n coefficient matrix is ill-conditioned, the method can produce a good solution, while the commonly used elimination method fails.This research was supported by the National Science Foundation, Grant No. GP-41158.     

Sequential conjugate-gradient-restoration algorithm for optimal control problems. Part 1. Theory

This paper considers the problem of minimizing a functionalI which depends on the statex(t), the controlu(t), and the parameter π. Here,I is a scalar,x ann-vector,u anm-vector, and π ap-vector. At the initial point, the state is prescribed. At the final point, the state and the parameter are required to satisfyq scalar relations. Along the interval of integration, the state, the control, and the parameter are required to satisfyn scalar differential equations. First, the case of a quadratic functional subject to linear constraints is considered, and a conjugate-gradient algorithm is derived. Nominal functionsx(t),u(t), π satisfying all the differential equations and boundary conditions are assumed. Variations Δx(t), δu(t), Δπ are determined so that the value of the functional is decreased. These variations are obtained by minimizing the first-order change of the functional subject to the differential equations, the boundary conditions, and a quadratic constraint on the variations of the control and the parameter. Next, the more general case of a nonquadratic functional subject to nonlinear constraints is considered. The algorithm derived for the linear-quadratic case is employed with one modification: a restoration phase is inserted between any two successive conjugate-gradient phases. In the restoration phase, variations Δx(t), Δu(t), Δπ are determined by requiring the least-square change of the control and the parameter subject to the linearized differential equations and the linearized boundary conditions. Thus, a sequential conjugate-gradient-restoration algorithm is constructed in such a way that the differential equations and the boundary conditions are satisfied at the end of each complete conjugate-gradient-restoration cycle. Several numerical examples illustrating the theory of this paper are given in Part 2 (see Ref. 1). These examples demonstrate the feasibility as well as the rapidity of convergence of the technique developed in this paper. This research was supported by the Office of Scientific Research, Office of Aerospace Research, United States Air Force, Grant No. AF-AFOSR-72-2185. The authors are indebted to Professor A. Miele for stimulating discussions. Formerly, Graduate Studient in Aero-Astronautics, Department of Mechanical and Aerospace Engineering and Materials Science, Rice University, Houston, Texas.     

Error estimates in in covolume methods for elliptic and parabolic problems: A unified approach

In this paper we consider covolume or finite volume element methods for variable coefficient elliptic and parabolic problems on convex smooth domains in the plane. We introduce a general approach for connecting these methods with finite element method analysis. This unified approach is used to prove known convergence results in the norms and new results in the max-norm. For the elliptic problems we demonstrate that the error between the exact solution and the approximate solution in the maximum norm is in the linear element case. Furthermore, the maximum norm error in the gradient is shown to be of first order. Similar results hold for the parabolic problems.

     

Arbitrary-order functionally fitted energy-diminishing methods for gradient systems

It is well known that for gradient systems in Euclidean space or on a Riemannian manifold, the energy decreases monotonically along solutions. In this letter we derive and analyse functionally fitted energy-diminishing methods to preserve this key property of gradient systems. It is proved that the novel methods are energy-diminishing and can achieve damping for very stiff gradient systems. We also show that the methods can be of arbitrarily high order and discuss their implementations. A numerical test is reported to illustrate the efficiency of the new methods in comparison with three existing numerical methods in the literature.     

Matrix conditioning and nonlinear optimization

In a series of recent papers, Oren, Oren and Luenberger, Oren and Spedicato, and Spedicato have developed the self-scaling variable metric algorithms. These algorithms alter Broyden's single parameter family of approximations to the inverse Hessian to a double parameter family. Conditions are given on the new parameter to minimize a bound on the condition number of the approximated inverse Hessian while insuring improved step-wise convergence.Davidon has devised an update which also minimizes the bound on the condition number while remaining in the Broyden single parameter family.This paper derives initial scalings for the approximate inverse Hessian which makes members of the Broyden class self-scaling. The Davidon, BFGS, and Oren—Spedicato updates are tested for computational efficiency and stability on numerous test functions, with the results indicating strong superiority computationally for the Davidon and BFGS update over the self-scaling update, except on a special class of functions, the homogeneous functions.     

Improvements of Some Projection Methods for Monotone Nonlinear Variational Inequalities

In this paper, we study the relationship of some projection-type methods for monotone nonlinear variational inequalities and investigate some improvements. If we refer to the Goldstein–Levitin–Polyak projection method as the explicit method, then the proximal point method is the corresponding implicit method. Consequently, the Korpelevich extragradient method can be viewed as a prediction-correction method, which uses the explicit method in the prediction step and the implicit method in the correction step. Based on the analysis in this paper, we propose a modified prediction-correction method by using better prediction and correction stepsizes. Preliminary numerical experiments indicate that the improvements are significant.     

Enhancing energy conserving methods

Recent observations [5] indicate that energy-momentum methods might be better suited for the numerical integration of highly oscillatory Hamiltonian systems than implicit symplectic methods. However, the popular energy-momentum method, suggested in [3], achieves conservation of energy by a global scaling of the force field. This leads to an undesirable coupling of all degrees of freedom that is not present in the original problem formulation. We suggest enhancing this energy-momentum method by splitting the force field and using separate adjustment factors for each force. In case that the potential energy function can be split into a strong and a weak part, we also show how to combine an energy conserving discretization of the strong forces with a symplectic discretization of the weak contributions. We demonstrate the numerical properties of our method by simulating particles that interact through Lennard-Jones potentials and by integrating the Sine-Gordon equation.This work was partly supported by NIH Grant P41RR05969, DOE/NSF Grant DE-FG02-91-ER25099/DMS-9304268, and NSF GCAG/HPCC ASC-9318159.     

Convergence of multistep discretizations of DAEs

Standard ODE methods such as linear multistep methods encounter difficulties when applied to differential-algebraic equations (DAEs) of index greater than 1. In particular, previous results for index 2 DAEs have practically ruled out the use of all explicit methods and of implicit multistep methods other than backward difference formulas (BDFs) because of stability considerations. In this paper we embed known results for semi-explicit index 1 and 2 DAEs in a more comprehensive theory based on compound multistep and one-leg discretizations. This explains and characterizes the necessary requirements that a method must fulfill in order to be applicable to semi-explicit DAEs. Thus we conclude that the most useful discretizations are those that avoid discretization of the constraint. A freer use of e.g. explicit methods for the non-stiff differential part of the DAE is then possible.Dedicated to Germund Dahlquist on the occasion of his 70th birthdayThis author thanks the Centro de Estadística y Software Matemático de la Universidad Simón Bolivar (CESMa) for permitting her free use of its research facilities.Partial support by the Swedish Research Council for Engineering Sciences TFR under contract no. 222/91-405.     

A Survey of Algebraic Multilevel Iteration (AMLI) Methods

For the solution by preconditioned conjugate gradient methods of symmetric positive definite equations as arising in boundary value problems we consider preconditioning methods of AMLI type. Particular attention is devoted to providing methods of optimal order of computational complexity which in addition promise to be robust, i.e. with a convergence rate which is bounded above independently of size of discretization parameter h, jumps in problem coefficients, and shape of finite elements or, equivalently, anisotropy of problem coefficients. In addition, the computational cost per iteration step must have optimal order.New results on upper bounds of one of the important parameters in the methods, the Cauchy—Bunyakowski—Schwarz constant are given and an algebraic method how to improve its value is presented.This revised version was published online in October 2005 with corrections to the Cover Date.     

Multirate infinitesimal step methods for atmospheric flow simulation

The numerical solution of the Euler equations requires the treatment of processes in different temporal scales. Sound waves propagate fast compared to advective processes. Based on a spatial discretisation on staggered grids, a multirate time integration procedure is presented here generalising split-explicit Runge-Kutta methods. The advective terms are integrated by a Runge-Kutta method with a macro stepsize restricted by the CFL number. Sound wave terms are treated by small time steps respecting the CFL restriction dictated by the speed of sound.Split-explicit Runge-Kutta methods are generalised by the inclusion of fixed tendencies of previous stages. The stability barrier for the acoustics equation is relaxed by a factor of two.Asymptotic order conditions for the low Mach case are given. The relation to commutator-free exponential integrators is discussed. Stability is analysed for the linear acoustic equation. Numerical tests are executed for the linear acoustics and the nonlinear Euler equations.     

Comparison of Two Kinds of Prediction-Correction Methods for Monotone Variational Inequalities

In this paper, we study the relationship between the forward-backward splitting method and the extra-gradient method for monotone variational inequalities. Both of the methods can be viewed as prediction-correction methods. The only difference is that they use different search directions in the correction-step. Our analysis explains theoretically why the extra-gradient methods usually outperform the forward-backward splitting methods. We suggest some modifications for the two methods and numerical results are given to verify the superiority of the modified methods.     

The SMAA-PROMETHEE method

PROMETHEE methods are widely used in Multiple Criteria Decision Aiding (MCDA) to deal with real world decision making problems. In this paper, we propose to apply the Stochastic Multicriteria Acceptability Analysis (SMAA) to the family of PROMETHEE methods in order to explore the whole set of parameters compatible with some preference information provided by the Decision Maker (DM). The application of the presented methodology is described in a didactic example.     

Decomposition methods in stochastic programming

Stochastic programming problems have very large dimension and characteristic structures which are tractable by decomposition. We review basic ideas of cutting plane methods, augmented Lagrangian and splitting methods, and stochastic decomposition methods for convex polyhedral multi-stage stochastic programming problems.     

解一类结构变分不等式问题的非精确并行交替方向法

带线性约束的具有两分块结构的单调变分不等式问题, 出现在许多现代应用中, 如交通和经济问题等. 基于该问题良好的可分结构, 分裂型算法被广泛研究用于其求解. 提出新的带回代的非精确并行交替方向法解该类问题, 在每一步迭代中,首先以并行模式通过投影得到预测点, 然后对其校正得到下一步的迭代点. 在压缩型算法的理论框架下, 在适当条件下证明了所提算法的全局收敛性. 数值结果表明了算法的有效性. 此外, 该算法可推广到求解具有多分块结构的问题.     

Statistical gap Tauberian theorems in metric spaces

By using the concept of statistical convergence we present statistical Tauberian theorems of gap type for the Cesàro, Euler-Borel family and the Hausdorff families applicable in arbitrary metric spaces. In contrast to the classical gap Tauberian theorems, we show that such theorems exist in the statistical sense for the convolution methods which include the Taylor and the Borel matrix methods. We further provide statistical analogs of the gap Tauberian theorems for the Hausdorff methods and provide an explanation as to how the Tauberian rates over the gaps may differ from those of the classical Tauberian theorems.     

一种新型压缩预处理CGS算法

CGS算法是求解大型非对称线性方程组的常用算法，然而该算法无极小残差性质，因此它常因出现较大的中间剩余向量而出现典型的不规则收敛行为．本根据IRA方法提出了一种压缩预处理CGS方法，数值实验表明这种算法在一定程度上减小了迭代算法在收敛过程中的剩余问题，从而使得算法具有更好的稳定性，该法构造简单，减少了收敛次数，加快了收敛速度．