Differential games with mixed leadership: The open-loop solution

This paper introduces the notion of mixed leadership in nonzero-sum differential games, where there is no fixed hierarchy in decision making with respect to the players. Whether a particular player is leader or follower depends on the instrument variable s/he is controlling, and it is possible for a player to be both leader and follower, depending on the control variable. The paper studies two-player open-loop differential games in this framework, and obtains a complete set of equations (differential and algebraic) which yield the controls in the mixed-leadership Stackelberg solution. The underlying differential equations are coupled and have mixed-boundary conditions. The paper also discusses the special case of linear-quadratic differential games, in which case solutions to the coupled differential equations can be expressed in terms of solutions to coupled Riccati differential equations which are independent of the state trajectory.     

A BSDE approach to Nash equilibrium payoffs for stochastic differential games with nonlinear cost functionals

In this paper, we study Nash equilibrium payoffs for two-player nonzero-sum stochastic differential games via the theory of backward stochastic differential equations. We obtain an existence theorem and a characterization theorem of Nash equilibrium payoffs for two-player nonzero-sum stochastic differential games with nonlinear cost functionals defined with the help of doubly controlled backward stochastic differential equations. Our results extend former ones by Buckdahn et al. (2004) [3] and are based on a backward stochastic differential equation approach.     

On the saddle-point solution of a class of stochastic differential games

This paper deals with the saddle-point solution of a class of stochastic differential games described by linear state dynamics and quadratic objective functionals. The information structure of the problem is such that both players have access to a common noisy linear measurement of the state and they are permitted to utilize only this information in constructing their controls. The saddle-point solution of such differential game problems has been discussed earlier in Ref. 1, but the conclusions arrived there are incorrect, as is explicitly shown in this paper. We extensively discuss the role of information structure on the saddle-point solution of such stochastic games (specifically within the context of an illustrative discrete-time example) and then obtain the saddle-point solution of the problem originally formulated by employing an indirect approach.This work was done while the author was on sabbatical leave at Twente University of Technology, Department of Applied Mathematics, Enschede, Holland, from Applied Mathematics Division, Marmara Scientific and Industrial Research Institute, Gebze, Kocaeli, Turkey.     

Stackelberg strategies in linear-quadratic stochastic differential games

This paper obtains the Stackelberg solution to a class of two-player stochastic differential games described by linear state dynamics and quadratic objective functionals. The information structure of the problem is such that the players make independent noisy measurements of the initial state and are permitted to utilize only this information in constructing their controls. Furthermore, by the very nature of the Stackelberg solution concept, one of the players is assumed to know, in advance, the strategy of the other player (the leader). For this class of problems, we first establish existence and uniqueness of the Stackelberg solution and then relate the derivation of the leader's Stackelberg solution to the optimal solution of a nonstandard stochastic control problem. This stochastic control problem is solved in a more general context, and its solution is utilized in constructing the Stackelberg strategy of the leader. For the special case Gaussian statistics, it is shown that this optimal strategy is affine in observation of the leader. The paper also discusses numerical aspects of the Stackelberg solution under general statistics and develops algorithms which converge to the unique Stackelberg solution.This work was performed while the second author was on sabbatical leave at the Department of Applied Mathematics, Twente University of Technology, Enschede, Holland.     

Subgame Consistent Solutions of a Cooperative Stochastic Differential Game with Nontransferable Payoffs

Subgame consistency is a fundamental element in the solution of cooperative stochastic differential games. In particular, it ensures that the extension of the solution policy to a later starting time and any possible state brought about by the prior optimal behavior of the players would remain optimal. Recently, mechanisms for the derivation of subgame consistent solutions in stochastic cooperative differential games with transferable payoffs have been found. In this paper, subgame consistent solutions are derived for a class of cooperative stochastic differential games with nontransferable payoffs. The previously intractable subgame consistent solution for games with nontransferable payoffs is rendered tractable.This research was supported by the Research Grant Council of Hong Kong, Grant HKBU2056/99H and by Hong Kong Baptist University, Grant FRG/02-03/II16.Communicated by G. Leitmann     

On linear stochastic differential games with average cost criterions

In this paper, we consider scalar linear stochastic differential games with average cost criterions. We solve the dynamic programming equations for these games and give the synthesis of saddle-point and Nash equilibrium solutions.The authors wish to thank A. Ichikawa for providing the initial impetus and helpful advice.     

An exponential differential game which admits a simple Nash solution

This paper deals with a class ofN-person nonzero-sum differential games where the control variables enter into the state equations as well as the payoff functionals in an exponential way. Due to the structure of the game, Nash-optimal controls are easily determined. The equilibrium in open-loop controls is also a closed-loop equilibrium. An example of optimal exploitation of an exhaustible resource is presented.The helpful comments of Professor Y. C. Ho and Dipl. Ing. E. Dockner are gratefully acknowledged.     

Total Reward Stochastic Games and Sensitive Average Reward Strategies

In this paper, total reward stochastic games are surveyed. Total reward games are motivated as a refinement of average reward games. The total reward is defined as the limiting average of the partial sums of the stream of payoffs. It is shown that total reward games with finite state space are strategically equivalent to a class of average reward games with an infinite countable state space. The role of stationary strategies in total reward games is investigated in detail. Further, it is outlined that, for total reward games with average reward value 0 and where additionally both players possess average reward optimal stationary strategies, it holds that the total reward value exists.     

Representation Formulas for Solutions of the HJI Equations with Discontinuous Coefficients and Existence of Value in Differential Games

In this paper, we study the Hamilton-Jacobi-Isaacs equation of zerosum differential games with discontinuous running cost. For such class of equations, the uniqueness of the solutions is not guaranteed in general. We prove principles of optimality for viscosity solutions where one of the players can play either causal strategies or only a subset of continuous strategies. This allows us to obtain nonstandard representation formulas for the minimal and maximal viscosity solutions and prove that a weak form of the existence of value is always satisfied. We state also an explicit uniqueness result for the HJI equations for piecewise continuous coefficients, in which case the usual statement on the existence of value holds.     

On stochastic games with additive reward and transition structure

In this paper, we introduce a new class of two-person stochastic games with nice properties. For games in this class, the payoffs as well as the transitions in each state consist of a part which depends only on the action of the first player and a part dependent only on the action of the second player.For the zero-sum games in this class, we prove that the orderfield property holds in the infinite-horizon case and that there exist optimal pure stationary strategies for the discounted as well as the undiscounted payoff criterion. For both criteria also, finite algorithms are given to solve the game. An example shows that, for nonzero sum games in this class, there are not necessarily pure stationary equilibria. But, if such a game possesses a stationary equilibrium point, then there also exists a stationary equilibrium point which uses in each state at most two pure actions for each player.     

Monitoring cooperative equilibria in a stochastic differential game

This paper deals with a class of equilibria which are based on the use of memory strategies in the context of continuous-time stochastic differential games. In order to get interpretable results, we will focus the study on a stochastic differential game model of the exploitation of one species of fish by two competing fisheries. We explore the possibility of defining a so-called cooperative equilibrium, which will implement a fishing agreement. In order to obtain that equilibrium, one defines a monitoring variable and an associated retaliation scheme. Depending on the value of the monitoring variable, which provides some evidence of a deviation from the agreement, the probability increases that the mode of a game will change from a cooperative to a punitive one. Both the monitoring variable and the parameters of this jump process are design elements of the cooperative equilibrium. A cooperative equilibrium designed in this way is a solution concept for a switching diffusion game. We solve that game using the sufficient conditions for a feedback equilibrium which are given by a set of coupled HJB equations. A numerical analysis, approximating the solution of the HJB equations through an associated Markov game, enables us to show that there exist cooperative equilibria which dominate the classical feedback Nash equilibrium of the original diffusion game model.This research was supported by FNRS-Switzerland, NSERC-Canada, FCAR-Quebec.     

State transformations and the derivation of Nash closed-loop equilibria for non-zero-sum differential games

In this paper the usefulness of state transformations in differential games is demonstrated. It is shown that different (admissible) state transformations give rise to different closed-loop Nash equilibrium candidates, which may all be found by solving systems of ordinary differential equations. A linear-quadratic duopoly differential game is solved to illustrate the results.     

Differential games with noise-corrupted measurements

In this paper, readily computable strategies for zero-sum, linear-quadratic differential games with noise-corrupted measurements are developed. Of particular significance is the fact that the governing differential equations no longer require the solution of an often difficult nonlinear, two-point boundary-value problem, but again satisfy the separation principle of linear-quadratic optimal control. The implications of the payoff relationships are considered.In a subsequent paper, we will apply the theory developed in this paper to a detailed example of a pursuit-evasion game. We discuss a missile and an airplane system where the missile supported by its launch platform has perfect state measurements and the airplane has noise-corrupted measurements.     

On the existence of good stationary strategies for nonleavable stochastic games

This paper discusses the problem regarding the existence of optimal or nearly optimal stationary strategies for a player engaged in a nonleavable stochastic game. It is known that, for these games, player I need not have an -optimal stationary strategy even when the state space of the game is finite. On the contrary, we show that uniformly -optimal stationary strategies are available to player II for nonleavable stochastic games with finite state space. Our methods will also yield sufficient conditions for the existence of optimal and -optimal stationary strategies for player II for games with countably infinite state space. With the purpose of introducing and explaining the main results of the paper, special consideration is given to a particular class of nonleavable games whose utility is equal to the indicator of a subset of the state space of the game.     

Saddle point for differential games with strongly convex-concave integrand

On a fixed time interval we consider zero-sum nonlinear differential games for which the integrand in the criterion functional is a sufficiently strongly convex-concave function of chosen controls. It is shown that in our setting there exists a saddle point in the class of programmed strategies, and a minimax principle similar to Pontryagin's maximum principle is a necessary and sufficient condition for optimality. An example in which the class of games under study is compared with two known classes of differential games is given. Translated fromMatematicheskie Zametki, Vol. 62, No. 5, pp. 725–743, November, 1997. Translated by N. K. Kulman     

Conditional Viability for Impulse Differential Games

We introduce in this paper the concept of “impulse evolutionary game”. Examples of evolutionary games are usual differential games, differentiable games with history (path-dependent differential games), mutational differential games, etc. Impulse evolutionary systems and games cover in particular “hybrid systems” as well as “qualitative systems”. The conditional viability kernel of a constrained set (with a target) is the set of initial states such that for all strategies (regarded as continuous feedbacks) played by the second player, there exists a strategy of the first player such that the associated run starting from this initial state satisfies the constraints until it hits the target. This paper characterizes the concept of conditional viability kernel for “qualitative games” and of conditional valuation function for “qualitative games” maximinimizing an intertemporal criterion. The theorems obtained so far about viability/capturability issues for evolutionary systems, conditional viability for differential games and about impulse and hybrid systems are used to provide characterizations of conditional viability under impulse evolutionary games.     

Subgame Consistent Cooperative Solutions in Stochastic Differential Games

Subgame consistency is a fundamental element in the solution of cooperative stochastic differential games. In particular, it ensures that: (i) the extension of the solution policy to a later starting time and to any possible state brought about by the prior optimal behavior of the players would remain optimal; (ii) all players do not have incentive to deviate from the initial plan. In this paper, we develop a mechanism for the derivation of the payoff distribution procedures of subgame consistent solutions in stochastic differential games with transferable payoffs. The payoff distribution procedure of the subgame consistent solution can be identified analytically under different optimality principles. Demonstration of the use of the technique for specific optimality principles is shown with an explicitly solvable game. For the first time, analytically tractable solutions of cooperative stochastic differential games with subgame consistency are derived.     

Pure subgame-perfect equilibria in free transition games

We consider a class of stochastic games, where each state is identified with a player. At any moment during play, one of the players is called active. The active player can terminate the game, or he can announce any player, who then becomes the active player. There is a non-negative payoff for each player upon termination of the game, which depends only on the player who decided to terminate. We give a combinatorial proof of the existence of subgame-perfect equilibria in pure strategies for the games in our class.     

Stochastic evolution equations in Hilbert spaces

In this paper we consider Ito's stochastic differential equation in Hilbert spaces. A strong solution is generated by the difference approximation. A regularity result is obtained for solutions to a class of parabolic stochastic partial differential equations. Hyperbolic stochastic evolution equations are also discussed.     

On Coincidence of Feedback Nash Equilibria and Stackelberg Equilibria in Economic Applications of Differential Games

The scope of the applicability of the feedback Stackelberg equilibrium concept in differential games is investigated. First, conditions for obtaining the coincidence between the stationary feedback Nash equilibrium and the stationary feedback Stackelberg equilibrium are given in terms of the instantaneous payoff functions of the players and the state equations of the game. Second, a class of differential games representing the underlying structure of a good number of economic applications of differential games is defined; for this class of differential games, it is shown that the stationary feedback Stackelberg equilibrium coincides with the stationary feedback Nash equilibrium. The conclusion is that the feedback Stackelberg solution is generally not useful to investigate leadership in the framework of a differential game, at least for a good number of economic applications This paper was presented at the 8th Viennese Workshop on Optimal Control, Dynamic Games, and Nonlinear Dynamics: Theory and Applications in Economics and OR/MS, Vienna, Austria, May 14–16, 2003, at the Seminar of the Instituto Complutense de Analisis Economico, Madrid, Spain, June 20, 2003, and at the Sevilla Workshop on Dynamic Economics and the Environment, Sevilla, Spain, July 2–3, 2003. The author is grateful to the participants in these sessions, in particular F.J. Andre and J. Ruiz, for their comments. Five referees provided particularly helpful suggestions. Financial support from the Ministerio de Ciencia y Tecnologia under Grant BEC2000-1432 is gratefully acknowledged.