Markov games with incomplete information

We consider zero-sum Markov games with incomplete information. Here, the second player is never informed about the current state of the underlying Markov chain. The existence of a value and of optimal strategies for both players is shown. In particular, we present finite algorithms for computing optimal strategies for the informed and uninformed player. The algorithms are based on linear programming results.     

Randomized stopping games and Markov market games

We study nonzero-sum stopping games with randomized stopping strategies. The existence of Nash equilibrium and ɛ-equilibrium strategies are discussed under various assumptions on players random payoffs and utility functions dependent on the observed discrete time Markov process. Then we will present a model of a market game in which randomized stopping times are involved. The model is a mixture of a stochastic game and stopping game. Research supported by grant PBZ-KBN-016/P03/99.     

Formulation and analysis of combat problems as zero-sum bicriterion differential games

In this paper, we present a formulation and analysis of a combat game between two players as a zero-sum bicriterion differential game. Each player's twin objectives of terminating the game on his own target set, while simultaneously avoiding his opponent's target set, are quantified in this approach. The solution in open-loop pure strategies is sought from among the Pareto-optimal security strategies of the players. A specific preference ordering on the outcomes is used to classify initial events in the assured win, draw, and mutual kill regions for the players. The method is compared with the event-constrained differential game approach, recently proposed by others. Finally, a simple example of the turret game is solved to illustrate the use of this method.     

Robust Markov perfect equilibria

In this paper we study a Markov decision model with quasi-hyperbolic discounting and transition probability function depending on an unknown parameter. Assuming that the set of parameters is finite, the sets of states and actions are Borel and the transition probabilities satisfy some additivity conditions and are atomless, we prove the existence of a non-randomised robust Markov perfect equilibrium.     

On linear stochastic differential games with average cost criterions

In this paper, we consider scalar linear stochastic differential games with average cost criterions. We solve the dynamic programming equations for these games and give the synthesis of saddle-point and Nash equilibrium solutions.The authors wish to thank A. Ichikawa for providing the initial impetus and helpful advice.     

Existence of optimal strategies in Markov games with incomplete information

The existence of a value and optimal strategies is proved for the class of two-person repeated games where the state follows a Markov chain independently of players’ actions and at the beginning of each stage only Player 1 is informed about the state. The results apply to the case of standard signaling where players’ stage actions are observable, as well as to the model with general signals provided that Player 1 has a nonrevealing repeated game strategy. The proofs reduce the analysis of these repeated games to that of classical repeated games with incomplete information on one side. This research was supported in part by Israeli Science Foundation grants 382/98, 263/03, and 1123/06, and by the Zvi Hermann Shapira Research Fund.     

On a problem of stochastic differential games

The process of bargaining between management and union during a strike is modelled by a nonlinear stochastic differential game. It is assumed that the two sides bargain in the mood of a cooperative game. A pair of Pareto-optimal strategies is obtained.     

On the uniqueness of Nash strategies for a class of analytic differential games

The uniqueness of Nash equilibria is shown for the case where the data of the problem are analytic functions and the admissible strategy spaces are restricted to analytic functions of the current state and time.This work was supported in part by the Joint Services Electronics Program (US Army, US Navy, and US Air Force) under Contract No. DAAB-07-72-C-0259, in part by the National Science Foundation under Grant No. ENG-74-20091, and in part by the Department of Energy, Electric Energy Systems Division under Contract No. US ERDA EX-76-C-01-2088.     

Algorithms for uniform optimal strategies in two-player zero-sum stochastic games with perfect information

We deal with zero-sum two-player stochastic games with perfect information. We propose two algorithms to find the uniform optimal strategies and one method to compute the optimality range of discount factors. We prove the convergence in finite time for one algorithm. The uniform optimal strategies are also optimal for the long run average criterion and, in transient games, for the undiscounted criterion as well.     

Bicriterion differential games with qualitative outcomes

Combat games are studied as bicriterion differential games with qualitative outcomes determined by threshold values on the criterion functions. Survival and capture strategies of the players are defined using the notion of security levels. Closest approach survival strategies (CASS) and minimum risk capture strategies (MRCS) are important strategies for the players identified as solutions to four optimization problems involving security levels. These are used, in combination with the preference orderings of the qualitative outcomes by the players, to delineate the win regions and the secured draw and mutual kill regions for the players. It is shown that the secured draw regions and the secured mutual kill regions for the two players are not necessarily the same. Simple illustrative examples are given.This paper is based partially on research supported by the Council of Scientific and Industrial Research, India, through a Research Associateship Grant to the second author.     

A note on nonzero-sum differential games with bargaining solution

The extension of Nash's bargaining solution to differential games is discussed. It is shown that a closed-loop solution verifies very stringent necessary conditions and that an open-loop solution can present serious weakness from a normative point of view.This research has been supported by the Canada Council (S73-0935) and the Ministère de l'Education du Québec (DGES).     

Combat games

We propose a mathematical formulation of a combat game between two opponents with offensive capabilities and offensive objectives. Resolution of the combat involves solving two differential games with state constraints. Depending on the game dynamics and parameters, the combat can terminate in one of four ways: (i) the first player wins, (ii) the second player wins, (iii) a draw (neither wins), or (iv) joint capture. In the first two cases, the optimal strategies of the two players are determined from suitable zero-sum games, whereas in the latter two the relevant games are nonzero-sum. Further, to avoid certain technical difficulties, the concept of a -combat game is introduced.Dedicated to G. LeitmannThe first author wishes to acknowledge the friendship and guidance of George Leitmann, beginning in the author's student days at Berkeley and continuing to the present time. All the authors thank George Leitmann for many recent fruitful discussions on differential games.on sabbatical leave from Technion, Israel Institute of Technology, Haifa, Israel.     

Stackelberg strategies in linear-quadratic stochastic differential games

This paper obtains the Stackelberg solution to a class of two-player stochastic differential games described by linear state dynamics and quadratic objective functionals. The information structure of the problem is such that the players make independent noisy measurements of the initial state and are permitted to utilize only this information in constructing their controls. Furthermore, by the very nature of the Stackelberg solution concept, one of the players is assumed to know, in advance, the strategy of the other player (the leader). For this class of problems, we first establish existence and uniqueness of the Stackelberg solution and then relate the derivation of the leader's Stackelberg solution to the optimal solution of a nonstandard stochastic control problem. This stochastic control problem is solved in a more general context, and its solution is utilized in constructing the Stackelberg strategy of the leader. For the special case Gaussian statistics, it is shown that this optimal strategy is affine in observation of the leader. The paper also discusses numerical aspects of the Stackelberg solution under general statistics and develops algorithms which converge to the unique Stackelberg solution.This work was performed while the second author was on sabbatical leave at the Department of Applied Mathematics, Twente University of Technology, Enschede, Holland.     

Statistical linear differential games: A treatment of incomplete information

A zero-sum, two-player linear differential game of fixed duration is considered in the case when the information is incomplete but a statistical structure gives both players the possibility tospy the value of an unknown parameter in the payoff. Considerations of topological vector spaces and functional analysis allow one to demonstrate, via a classical Sion's theorem, sufficient conditions for the existence of a value.The author is indebted to Professor J. Fichefet for his helpful remarks and indications.     

Stochastic two-target pursuit-evasion differential games in the plane

A stochastic version of a two-target homicidal chauffeur, pursuit-evasion differential game (using polar coordinates) is considered. This is used to model a dogfight between a very agile playerQ and a less maneuverable playerP. First, the case where both players have complete observation of the state of the game is considered. A numerical study is conducted, by solving numerically a nonlinear partial differential equation on a torus in ², to investigate the role of the parameters of speed, maneuverability, and performance of the weapon systems, in the encounter. Second, the model is extended to include the case where playerP is jamming playerQ's measurements of , where denotes the bearing ofQ fromP. A numerical study is conducted, by solving numerically a nonlinear partial differential equation on a generalized torus in ³, to investigate the role of the jamming parameter on the outcome of the combat.     

Sufficient conditions for Nash bargaining in differential games

Sufficient conditions for Nash bargaining in differential games are given. These conditions are compared with the sufficient conditions given by Liu (Ref. 1).     

State transformations and the derivation of Nash closed-loop equilibria for non-zero-sum differential games

In this paper the usefulness of state transformations in differential games is demonstrated. It is shown that different (admissible) state transformations give rise to different closed-loop Nash equilibrium candidates, which may all be found by solving systems of ordinary differential equations. A linear-quadratic duopoly differential game is solved to illustrate the results.     

N-person differential games governed by infinite-dimensional systems

This paper discussesN-person differential games governed by infinite-dimensional systems. The minimax principle, which is a necessary condition for the existence of open-loop equilibrium strategies, is proved. For linear-quadraticN-person differential games, global necessary and sufficient conditions for the existence of open-loop and closed-loop equilibrium strategies are derived.This work was supported by the Science Fund of the Chinese Academy of Sciences and the Research Foundation of Purdue University.The problems discussed in this paper were proposed by Professor G. Chen, during the author's visit to Pensylvania State University, and were completed at Purdue University. The author would like to thank Professors L. D. Berkovitz and G. Chen for their hospitality.