On saddle-point optimality in differential games

A family of two-person, zero-sum differential games in which the admissible strategies are Borel measurable is defined, and two types of saddle-point conditions are introduced as optimality criteria. In one, saddle-point candidates are compared at each point of the state space with all playable pairs at that point; and, in the other, they are compared only with strategy pairs playable on the entire state space. As a theorem, these two types of optimality are shown to be equivalent for the defined family of games. Also, it is shown that a certain closure property is sufficient for this equivalence. A game having admissible strategies everywhere constant, in which the two types of saddle-point candidates are not equivalent, is discussed.This paper is based on research supported by ONR.     

A survey of recent results on continuous-time Markov decision processes

This paper is a survey of recent results on continuous-time Markov decision processes (MDPs) withunbounded transition rates, and reward rates that may beunbounded from above and from below. These results pertain to discounted and average reward optimality criteria, which are the most commonly used criteria, and also to more selective concepts, such as bias optimality and sensitive discount criteria. For concreteness, we consider only MDPs with a countable state space, but we indicate how the results can be extended to more general MDPs or to Markov games. Research partially supported by grants NSFC, DRFP and NCET. Research partially supported by CONACyT (Mexico) Grant 45693-F.     

First-and second-order optimality conditions for mathematical programs with vanishing constraints

We consider a special class of optimization problems that we call Mathematical Programs with Vanishing Constraints, MPVC for short, which serves as a unified framework for several applications in structural and topology optimization. Since an MPVC most often violates stronger standard constraint qualification, first-order necessary optimality conditions, weaker than the standard KKT-conditions, were recently investigated in depth. This paper enlarges the set of optimality criteria by stating first-order sufficient and second-order necessary and sufficient optimality conditions for MPVCs. Dedicated to Jiří V. Outrata on the occasion of his 60th birthday. This research was partially supported by the DFG (Deutsche Forschungsgemeinschaft) under grant KA1296/15-1.     

Necessary Optimality Conditions for a Dead Oil Isotherm Optimal Control Problem

We study a system of nonlinear partial differential equations resulting from the traditional modelling of oil engineering within the framework of the mechanics of a continuous medium. Recent results on the problem provide existence, uniqueness and regularity of the optimal solution. Here we obtain the first necessary optimality conditions. Work supported by the Portuguese Foundation for Science and Technology (FCT) through the Centre for Research in Optimization and Control (CEOC) of the University of Aveiro, cofinanced by the European Community fund FEDER/POCTI. The first author was also supported by the postdoctoral fellowship SFRH/BPD/20934/2004.     

Alternative proof and interpretations for a recent state-dependent importance sampling scheme

Recently, a state-dependent change of measure for simulating overflows in the two-node tandem queue was proposed by Dupuis et al. (Ann. Appl. Probab. 17(4):1306–1346, 2007), together with a proof of its asymptotic optimality. In the present paper, we present an alternative, shorter and simpler proof. As a side result, we obtain interpretations for several of the quantities involved in the change of measure in terms of likelihood ratios. Part of this research has been funded by the Dutch BSIK/BRICKS project; part of this research was done while the first author was visiting INRIA/IRISA, Rennes, France.     

On optimal control problems with general boundary conditions

It is shown that the necessary optimality conditions for optimal control problems with terminal constraints and with given initial state allow also to obtain in a straightforward way the necessary optimality conditions for problems involving parameters and general (mixed) boundary conditions. In a similar manner, the corresponding numerical algorithms can be adapted to handle this class of optimal control problems.This research was supported in part by the Commission on International Relations, National Academy of Sciences, under Exchange Visitor Program No. P-1-4174.The author is indebted to the anonymous reviewer bringing to his attention Ref. 9 and making him aware of the possible use of generalized inverse notation when formulating the optimality conditions.     

Recent results on conditions for the existence of average optimal stationary policies

This paper concerns countable state space Markov decision processes endowed with a (long-run expected)average reward criterion. For these models we summarize and, in some cases,extend some recent results on sufficient conditions to establish the existence of optimal stationary policies. The topics considered are the following: (i) the new assumptions introduced by Sennott in [20–23], (ii)necessary and sufficient conditions for the existence of a bounded solution to the optimality equation, and (iii) equivalence of average optimality criteria. Some problems are posed.This research was partially supported by the Third World Academy of Sciences (TWAS) under Grant No. TWAS RG MP 898-152.     

Weak and Strong Optimality Conditions for Constrained Control Problems with Discontinuous Control

In recent years, sufficient optimality criteria and solution stability in optimal control have been investigated widely and used in the analysis of discrete numerical methods. These results were concerned mainly with weak local optima, whereas strong optimality has been considered often as a purely theoretical aspect. In this paper, we show via an example problem how weak the weak local optimality can be and derive new strong optimality conditions. The criteria are suitable for practical verification and can be applied to the case of discontinuous controls with changes in the set of active constraints.     

A note on nonzero-sum differential games with bargaining solution

The extension of Nash's bargaining solution to differential games is discussed. It is shown that a closed-loop solution verifies very stringent necessary conditions and that an open-loop solution can present serious weakness from a normative point of view.This research has been supported by the Canada Council (S73-0935) and the Ministère de l'Education du Québec (DGES).     

Statistical verification of optimality conditions for stochastic programs with recourse

Statistically motivated algorithms for the solution of stochastic programming problems typically suffer from their inability to recognize optimality of a given solution algorithmically. Thus, the quality of solutions provided by such methods is difficult to ascertain. In this paper, we develop methods for verification of optimality conditions within the framework of Stochastic Decomposition (SD) algorithms for two stage linear programs with recourse. Consistent with the stochastic nature of an SD algorithm, we provide termination criteria that are based on statistical verification of traditional (deterministic) optimality conditions. We propose the use of bootstrap methods to confirm the satisfaction of generalized Kuhn-Tucker conditions and conditions based on Lagrange duality. These methods are illustrated in the context of a power generation planning model, and the results are encouraging.This work was supported in part by Grant No. AFOSR-88-0076 from the Air Force Office of Scientific Research and Grant No. DDM-89-10046 from the National Science Foundation.     

Linear pursuit-evasion games with bounded state spaces

The problem of linear pursuit-evasion games with bounded state spaces is considered. Some sufficient conditions for optimality are established, and an example is given.This research was carried out while the author was a Visiting Associate Research Engineer at the University of California at Berkeley. The research was supported by the Office of Naval Research, Grant No. N00014-69-A-0200-1012. The author would like to express his gratitude to Professor G. Leitmann for discussions and for making possible his visit at Berkeley.     

Global optimality of extremals: An example

The question of the existence and the location of Darboux points (beyond which global optimality is lost) is crucial for minimal sufficient conditions for global optimality and for computation of optimal trajectories. Here, we investigate numerically the Darboux points and their relationship with conjugate points for a problem of minimum fuel, constant velocity, horizontal aircraft turns to capture a line. This simple second-order optimal control problem shows that ignoring the possible existence of Darboux points may play havoc with the computation of optimal trajectories.The authors are indebted to G. Moyer for his constructive comments. This research was supported, for the first author, by a National Research Council Associateship at NASA Ames Research Center.on leave from the Technion, Israel Institute of Technology, Haifa, Israel.