A strong variational algorithm for delay systems

In this paper, we present a convergent extension of the first-order strong-variational algorithm by Mayne and Polak (Ref. 1) for solving optimal control problems with control constraints to delay systems. Although the algorithm is similar to the one presented in Ref. 1, the proof of convergence is different, since the differential dynamic techniques used by Mayne and Polak are not applicable.This work forms part of the author's PhD Dissertation and was conducted at the Imperial College of Science and Technology under a studentship awarded by the UK Science and Engineering Research Council. This assistance is gratefully acknowledged. The author also wishes to thank Dr. R. B. Vinter for his encouragement and help.     

Equality and inequality constrained optimization algorithms with convergent stepsizes

In this paper, we consider two algorithms for nonlinear equality and inequality constrained optimization. Both algorithms utilize stepsize strategies based on differentiable penalty functions and quadratic programming subproblems. The essential difference between the algorithms is in the stepsize strategies used. The objective function in the quadratic subproblem includes a linear term that is dependent on the penalty functions. The quadratic objective function utilizes an approximate Hessian of the Lagrangian augmented by the penalty functions. In this approximation, it is possible to ignore the second-derivative terms arising from the constraints in the penalty functions.The penalty parameter is determined using a strategy, slightly different for each algorithm, that ensures boundedness as well as a descent property. In particular, the boundedness follows as the strategy is always satisfied for finite values of the parameter.These properties are utilized to establish global convergence and the condition under which unit stepsizes are achieved. There is also a compatibility between the quadratic objective function and the stepsize strategy to ensure the consistency of the properties for unit steps and subsequent convergence rates.This research was funded by SERC and ESRC research contracts. The author is grateful to Professors Laurence Dixon and David Mayne for their comments. The numerical results in the paper were obtained using a program written by Mr. Robin Becker.     

Stochastic penalty function methods for nonsmooth constrained minimization

This paper presents a stochastic algorithm with proper stopping rules for nonsmooth inequality-constrained minimization problems. The algorithm is based on an augmented Lagrangian dual problem transformed from a primal one, and it consists of two loops: an outer loop, which is the iteration for the approximate Lagrange multipliers, and an inner loop, which is a nonsmooth unconstrained minimization subroutine. Under mild assumptions, the algorithm is proved to be almost surely convergent.This work was partially supported by the Science Foundation of Ningbo University. The author is grateful to Professor D. Q. Mayne for his help with this work and to two referees for their helpful comments.     

On the convergence of two branch-and-bound algorithms for nonconvex programming problems

This note presents a new convergence property for each of two branch-and-bound algorithms for nonconvex programming problems (Falk-Soland algorithms and Horst algorithms). For each algorithm, it has been shown previously that, under certain conditions, whenever the algorithm generates an infinite sequence of points, at least one accumulation point of this sequence is a global minimum. We show here that, for each algorithm, in fact, under these conditions, every accumulation point of such a sequence is a global minimum.The author would like to thank Professor R. M. Soland for his helpful comments concerning this paper.     

Numerical computation of the value function of optimally controlled stochastic switching processes by multi-grid techniques

By the dynamic programming principle the value function of an optimally controlled stochasticswitching process can be shown to satisfy a boundary value problem for a fully nonlinear second-order elliptic differential equation of Hamilton-Jacobi-Bellman (HJB-) type. For the numerical solution of that HJB-equation we present a multi-grid algorithm whose main features arethe use of nonlinear Gauss-Seidel iteration in the smoothing process and an adaptive local choice of prolongations and restrictions in the coarse-to-fine and fine-to-coarse transfers. Local convergence is proved by combining nonlinear multi-grid convergence theory and elementarysubdifferential calculus. The efficiency of the algorithm is demonstrated for optimal advertising in stochastic dynamic sales response models of Vidale-Wolfe type.     

A Dual Algorithm for Minimizing a Quadratic Function with Two Quadratic Constraints

In this paper, we present a dual algorithm for minimizing a convex quadratic function with two quadratic constraints. Such a minimization problem is a subproblem that appears in some trust region algorithms for general nonlinear programming. Some theoretical properties of the dual problem are given. Global convergence of the algorithm is proved and a local superlinear convergence result is presented. Numerical examples are also provided.     

A parallel variable-metric dynamic programming algorithm

This paper presents a potentially parallel iterative algorithm for the solution of the unconstrainedN-stage decision problem of dynamic programming. The basis of the algorithm is the use of variable-metric minimization techniques to develop a quadratic approximation to the cost function at each stage. The algorithm is applied to various problems, and comparisons with other algorithms are made.This research forms part of the author's PhD program, and is supported by the Department of Scientific and Industrial Research of the New Zealand Government. The author is indebted to Dr. B. A. Murtagh, PhD supervisor, for his encouragement and support during the preparation of this paper.     

Contractivity of Bellman operator in risk averse dynamic programming with infinite horizon

The paper deals with a risk averse dynamic programming problem with infinite horizon. First, the required assumptions are formulated to have the problem well defined. Then the Bellman equation is derived, which may be also seen as a standalone reinforcement learning problem. The fact that the Bellman operator is contraction is proved, guaranteeing convergence of various solution algorithms used for dynamic programming as well as reinforcement learning problems, which we demonstrate on the value iteration and the policy iteration algorithms.     

A cutting-plane algorithm with linear and geometric rates of convergence

This paper presents a cutting-plane algorithm for nonlinear programming which, under suitable conditions, exhibits a linear or geometric global rate of convergence. Other known rates of convergence for cutting-plane algorithms are no better than arithmetic for problems not satisfying a Haar condition. The feature responsible for this improved rate of convergence is the addition at each iteration of a new cut for each constraint, rather than adding only one new cut corresponding to the most violated constraint as is typically the case. Certain cuts can be dropped at each iteration, and there is a uniform upper bound on the number of old cuts retained. Geometric convergence is maintained if the subproblems at each iteration are approximated, rather than solved exactly, so the algorithm is implementable. The algorithm is flexible with respect to the point used to generate new cuts.The author is grateful to W. Oettli for bringing to his attention the linearly convergent cutting-plane algorithm of Ref. 15 and to the referee for a comment that stimulated an extension of the convergence rate results from an earlier version where _k depended on certain parameters of the problem.     

一类求解凸规划的鞍点法

根据凸规划的Kuhn-Tucker定理,有a)假如(x~*,y~*)是L(x,y)在D上的鞍点,那么 (1)x~*是(CP)问题的最优解,     

Newton-KKT interior-point methods for indefinite quadratic programming

Two interior-point algorithms are proposed and analyzed, for the (local) solution of (possibly) indefinite quadratic programming problems. They are of the Newton-KKT variety in that (much like in the case of primal-dual algorithms for linear programming) search directions for the “primal” variables and the Karush-Kuhn-Tucker (KKT) multiplier estimates are components of the Newton (or quasi-Newton) direction for the solution of the equalities in the first-order KKT conditions of optimality or a perturbed version of these conditions. Our algorithms are adapted from previously proposed algorithms for convex quadratic programming and general nonlinear programming. First, inspired by recent work by P. Tseng based on a “primal” affine-scaling algorithm (à la Dikin) [J. of Global Optimization, 30 (2004), no. 2, 285–300], we consider a simple Newton-KKT affine-scaling algorithm. Then, a “barrier” version of the same algorithm is considered, which reduces to the affine-scaling version when the barrier parameter is set to zero at every iteration, rather than to the prescribed value. Global and local quadratic convergence are proved under nondegeneracy assumptions for both algorithms. Numerical results on randomly generated problems suggest that the proposed algorithms may be of great practical interest. The work of the first author was supported in part by the School of Computational Science of Florida State University through a postdoctoral fellowship. Part of this work was done while this author was a Research Fellow with the Belgian National Fund for Scientific Research (Aspirant du F.N.R.S.) at the University of Liège. The work of the second author was supported in part by the National Science Foundation under Grants DMI9813057 and DMI-0422931 and by the US Department of Energy under Grant DEFG0204ER25655. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation or those of the US Department of Energy.     

Convergence property of the Iri-Imai algorithm for some smooth convex programming problems

In this paper, the Iri-Imai algorithm for solving linear and convex quadratic programming is extended to solve some other smooth convex programming problems. The globally linear convergence rate of this extended algorithm is proved, under the condition that the objective and constraint functions satisfy a certain type of convexity, called the harmonic convexity in this paper. A characterization of this convexity condition is given. The same convexity condition was used by Mehrotra and Sun to prove the convergence of a path-following algorithm.The Iri-Imai algorithm is a natural generalization of the original Newton algorithm to constrained convex programming. Other known convergent interior-point algorithms for smooth convex programming are mainly based on the path-following approach.     

Forward differential dynamic programming

The dynamic programming formulation of the forward principle of optimality in the solution of optimal control problems results in a partial differential equation with initial boundary condition whose solution is independent of terminal cost and terminal constraints. Based on this property, two computational algorithms are described. The first-order algorithm with minimum computer storage requirements uses only integration of a system of differential equations with specified initial conditions and numerical minimization in finite-dimensional space. The second-order algorithm is based on the differential dynamic programming approach. Either of the two algorithms may be used for problems with nondifferentiable terminal cost or terminal constraints, and the solution of problems with complicated terminal conditions (e.g., with free terminal time) is greatly simplified.     

Successive element correction algorithms for sparse unconstrained optimization

This paper presents a successive element correction algorithm and a secant modification of this algorithm. The new algorithms are designed to use the gradient evaluations as efficiently as possible in forming the approximate Hessian. The estimates of theq-convergence andr-convergence rates show that the new algorithms may have good local convergence properties. Some restricted numerical results and comparisons with some previously established algorithms suggest that the new algorithms may be efficient in practice.The author would like to thank T. F. Coleman for his many important and helpful suggestions and corrections on the preliminary draft of this paper. The author is also grateful to R. A. Tapia, the editors, and the referees for helpful suggestions and corrections.     

连续型动态规划的新算法研究

提出了求解一维连续型动态规划问题的自创算法----离散近似迭代法,并结合双收敛方法求解多维连续型动态规划问题. 该算法的基本思路为：在给定其它状态向
量序列的基础上,每次对一个状态变量序列进行离散近似迭代,并找出该状态变量的最优序列,直到所有状态向量序列都检查完.当模型为非凸非凹动态规划时,
证明了该算法的收敛性.当模型为凸动态规划时,证明了该算法的线性收敛性. 最后,以一个具体算例验证了该模型和算法的有效性.     

A Continuous Method for Convex Programming Problems

In this paper, we present a continuous method for convex programming (CP) problems. Our approach converts first the convex problem into a monotone variational inequality (VI) problem. Then, a continuous method, which includes both a merit function and an ordinary differential equation (ODE), is introduced for the resulting variational inequality problem. The convergence of the ODE solution is proved for any starting point. There is no Lipschitz condition required in our proof. We show also that this limit point is an optimal solution for the original convex problem. Promising numerical results are presented.This research was supported in part by Grants FRG/01-02/I-39 and FRG/01-02/II-06 of Hong Kong Baptist University and Grant HKBU2059/02P from the Research Grant Council of Hong Kong.The author thanks Professor Bingsheng He for many helpful suggestions and discussions. The author is also grateful for the comments and suggestions of two anonymous referees. In particular, the author is indebted to one referee who drew his attention to References 15, 17, 18.     

A continuous-time linear complementarity system for dynamic user equilibria in single bottleneck traffic flows

This paper formally introduces a linear complementarity system (LCS) formulation for a continuous-time, multi-user class, dynamic user equilibrium (DUE) model for the determination of trip timing decisions in a simplified single bottleneck model. Existence of a Lipschitz solution trajectory to the model is established by a constructive time-stepping method whose convergence is rigorously analyzed. The solvability of the time-discretized subproblems by Lemke’s algorithm is also proved. Combining linear complementarity with ordinary differential equations and being a new entry to the mathematical programming field, the LCS provides a computational tractable framework for the rigorous treatment of the DUE problem in continuous time; this paper makes a positive contribution in this promising research venue pertaining to the application of differential variational theory to dynamic traffic problems.     

风险敏感度激励学习的广义平均算法

提出了一种新的算法．这个算法通过潜在地牺牲控制策略的最优性来获取其鲁棒性．这是因为,如果在理论模型与实际的物理系统之间存在不匹配,或者实际系统是非静态的,或者控制动作的可使用性随时间的变化而变化时,那么鲁棒性就可能成为一个十分重要的问题．主要工作是给出了一组逼近算法和它们的收敛结果．利用广义平均算子来替代最优算子max(或min),对激励学习中的一类最重要的算法——动态规划算法——进行了研究,并讨论了它们的收敛性,目的就是为了提高激励学习算法的鲁棒性．同时使用了更具一般性的风险敏感度性能评价体系,发现基于动态规划的学习算法中的一般结论在这种体系之下并不完全成立．     

Analysis and Computational Methods of Dirichlet Boundary Optimal Control Problems for 2D Boussinesq Equations

We study Dirichlet boundary optimal control problems for 2D Boussinesq equations. The existence of the solution of the optimization problem is proved and an optimality system of partial differential equations is derived from which optimal controls and states may be determined. Then, we present some computational methods to get the solution of the optimality system. The iterative algorithms are given explicitly. We also prove the convergence of the gradient algorithm.     

A new technique for inconsistent QP problems in the SQP method

Successful treatment of inconsistent QP problems is of major importance in the SQP method, since such occur quite often even for well behaved nonlinear programming problems. This paper presents a new technique for regularizing inconsistent QP problems, which compromises in its properties between the simple technique of Pantoja and Mayne [36] and the highly successful, but expensive one of Tone [47]. Global convergence of a corresponding algorithm is shown under reasonable weak conditions. Numerical results are reported which show that this technique, combined with a special method for the case of regular subproblems, is quite competitive to highly appreciated established ones.