Series Nash solution of two-person,nonzero-sum,linear-quadratic differential games

It is well-known that the Nash equilibrium solution of a two-person, nonzero-sum, linear differential game with a quadratic cost function can be expressed in terms of the solution of coupled generalized Riccati-type matrix differential equations. For high-order games, the numerical determination of the solution of the nonlinear coupled equations may be difficult or even impossible when the application dictates the use of small-memory computers. In this paper, a series solution is suggested by means of a parameter imbedding method. Instead of solving a high-order matrix-Riccati equation, a lower-order matrix-Riccati equation corresponding to a zero-sum game is solved. In addition, lower-order linear equations have to be solved. These solutions to lower-order equations are the coefficients of the series solution for the nonzero-sum game. Cost functions corresponding to truncated solutions are compared with those for exact Nash equilibrium solutions.This research was supported in part by the National Science Foundation under Grant No. GK-3893, in part by the Air Force under Grant No. AFOSR-68-1579B, and in part by the Joint Services Electronics Program under Contract No. DAAB-07-67-C-0199 with the Coordinated Science Laboratory, University of Illinois, Urbana, Illinois.     

Additional aspects of the Stackelberg strategy in nonzero-sum games

The Stackelberg strategy in nonzero-sum games is a reasonable solution concept for games where, either due to lack of information on the part of one player about the performance function of the other, or due to different speeds in computing the strategies, or due to differences in size or strength, one player dominates the entire game by imposing a solution which is favorable to himself. This paper discusses some properties of this solution concept when the players use controls that are functions of the state variables of the game in addition to time. The difficulties in determining such controls are also pointed out. A simple two-stage finite state discrete game is used to illustrate these properties.This work was supported in part by the U.S. Air Force under Grant No. AFOSR-68-1579D, in part by NSF under Grant No. GK-36276, and in part by the Joint Services Electronics Program under Contract No. DAAB-07-72-C-0259 with the Coordinated Science Laboratory, University of Illinois, Urbana, Illinois.     

Differential games with mixed leadership: The open-loop solution

This paper introduces the notion of mixed leadership in nonzero-sum differential games, where there is no fixed hierarchy in decision making with respect to the players. Whether a particular player is leader or follower depends on the instrument variable s/he is controlling, and it is possible for a player to be both leader and follower, depending on the control variable. The paper studies two-player open-loop differential games in this framework, and obtains a complete set of equations (differential and algebraic) which yield the controls in the mixed-leadership Stackelberg solution. The underlying differential equations are coupled and have mixed-boundary conditions. The paper also discusses the special case of linear-quadratic differential games, in which case solutions to the coupled differential equations can be expressed in terms of solutions to coupled Riccati differential equations which are independent of the state trajectory.     

A class of singularly perturbed,nonlinear, fixed-endpoint control problems

Singular perturbation techniques are applied to a class of nonlinear, fixed-endpoint control problems to decompose the full-order problem into three lower-order problems, namely, the reduced problem and the left and right boundary-layer problems. The boundary-layer problems are linear-quadratic and, contrary to previous singular perturbation works, the reduced problem has a simple formulation. The solutions of these lower-order problems are combined to yield an approximate solution to the full nonlinear problem. Based on the properties of the lower-order problems, the full problem is shown to possess an asymptotic series solution.This work was supported in part by the National Science Foundation under Grant No. ENG-47-20091 and in part by the US Air Force under Grant No. AFOSR-73-2570.The author acknowledges the helpful suggestions of Professor P. V. Kokotovic, University of Illinois, Urbana, Illinois.     

On the relative leadership property of Stackelberg strategies

The relative leadership property of Stackelberg strategies has been investigated via a scalar nonzero-sum, two-person differential game problem. It is shown that, depending on the parameters of the game, there exist three different types of solutions for his class of games.This work was partially supported by the National Science Foundation, Grant No. GK-31511, and the U.S. Office of Naval Research under the Joint Services Electronics Program, Contract No. N00014-67-A-0298-0006. The author would like to thank Professor Y. C. Ho for several valuable discussions.     

Nonzero-sum differential games

The theory of differential games is extended to the situation where there areN players and where the game is nonzero-sum, i.e., the players wish to minimize different performance criteria. Dropping the usual zero-sum condition adds several interesting new features. It is no longer obvious what should be demanded of asolution, and three types of solutions are discussed:Nash equilibrium, minimax, andnoninferior set of strategies. For one special case, the linear-quadratic game, all three of these solutions can be obtained by solving sets of ordinary matrix differential equations. To illustrate the differences between zero-sum and nonzero-sum games, the results are applied to a nonzero-sum version of a simple pursuit-evasion problem first considered by Ho, Bryson, and Baron (Ref. 1).Negotiated solutions are found to exist which give better results forboth players than the usualsaddle-point solution. To illustrate that the theory may find interesting applications in economic analysis, a problem is outlined involving the dividend policies of firms operating in an imperfectly competitive market.This research was supported by Joint Services Electronics Contracts Nos. N00014-67-A-0298-0006, 0005, 0008 and by NASA Grant No. NGR 22-007-068.     

On a class of nonzero-sum,linear-quadratic differential games

A class of two-player, nonzero-sum, linear-quadratic differential games is investigated for Nash equilibrium solutions when both players use closed-loop control and when one or both of the players are required to use open-loop control. For three formulations of the game, necessary and sufficient conditions are obtained for a particular strategy set to be a Nash equilibrium strategy set. For a fourth formulation of the game, where both players use open-loop control, necessary and sufficient conditions for the existence of a Nash equilibrium strategy set are developed. Several examples are presented in order to illustrate the differences between this class of differential games and its zero-sum analog.This research was supported by the National Science Foundation under Grant No. GK-3341.     

A note on dynamic Stackelberg games with leaders in turn

Recently, a model of dynamic Stackelberg games with leaders in turn has been proposed, and dynamic Stackelberg games with leaders in turn have been exploited under a feedback information structure. This paper characterizes dynamic Stackelberg games with leaders in turn under other information structures, both closed-loop and open-loop information structures. Explicit solutions are given for linear-quadratic systems under an open-loop information structure for dynamic Stackelberg games with leaders in turn.     

Sufficient conditions for Stackelberg and Nash strategies with memory

Sufficiency conditions for Stackelberg strategies for a class of deterministic differential games are derived when the players have recall of the previous trajectory. Sufficient conditions for Nash strategies when the players have recall of the trajectory are also derived. The state equation is linear, and the cost functional is quadratic. The admissible strategies are restricted to be affine in the information available.This work was supported in part by the Joint Services Electronics Program under Contract No. N00014-79-C-0424, in part by the National Science Foundation under Grant No. ECS-79-19396, and in part by Department of Energy under Contract No. EX-76-C-01-2088.     

Performance versus informativeness in linear-quadratic Gaussian noncooperative games

In this paper, we study the impact of informativeness on the performance of linear quadratic Gaussian Nash and Stackelberg games. We first show that, in two-person static Nash games, if one of the players acquires more information, then this extra information is beneficial to him, provided that it is orthogonal to both players' information. A special case is that when one of the players is informationally stronger than the other, then any new information is beneficial to him. We then show that a similar result holds for dynamic Nash games. In the dynamic games, the players use strategies that are linear functions of the current estimates of the state, generated by two Kalman filters. The same properties are proved to hold in static and feedback Stackelberg games as well.This work was partially supported by the US Air Force Office of Scientific Research under Grant No. AFOSR-82-0174.     

Pricing problems with a continuum of customers as stochastic Stackelberg games

The pricing problem where a company sells a certain kind of product to a continuum of customers is considered. It is formulated as a stochastic Stackelberg game with nonnested information structure. The inducible region concept, recently developed for deterministic Stackelberg games, is extended to treat the stochastic pricing problem. Necessary and sufficient conditions for a pricing scheme to be optimal are derived, and the pricing problem is solved by first delineating its inducible region, and then solving a constrained optimal control problem.The research work reported here as supported in part by the National Science Foundation under Grant ECS-81-05984, Grant ECS-82-10673, and by the Air Force Office of Scientific Research under AFOSR Grant 80-0098.     

Asymptotic Analysis of Linear Feedback Nash Equilibria in Nonzero-Sum Linear-Quadratic Differential Games

In this paper, we discuss nonzero-sum linear-quadratic differential games. For this kind of games, the Nash equilibria for different kinds of information structures were first studied by Starr and Ho. Most of the literature on the topic of nonzero-sum linear-quadratic differential games is concerned with games of fixed, finite duration; i.e., games are studied over a finite time horizon t _f. In this paper, we study the behavior of feedback Nash equilibria for t _f.In the case of memoryless perfect-state information, we study the so-called feedback Nash equilibrium. Contrary to the open-loop case, we note that the coupled Riccati equations for the feedback Nash equilibrium are inherently nonlinear. Therefore, we limit the dynamic analysis to the scalar case. For the special case that all parameters are scalar, a detailed dynamical analysis is given for the quadratic system of coupled Riccati equations. We show that the asymptotic behavior of the solutions of the Riccati equations depends strongly on the specified terminal values. Finally, we show that, although the feedback Nash equilibrium over any fixed finite horizon is generically unique, there can exist several different feedback Nash equilibria in stationary strategies for the infinite-horizon problem, even when we restrict our attention to Nash equilibria that are stable in the dynamical sense.     

Maximum principle for differential games of forward–backward stochastic systems with applications

This paper is concerned with a maximum principle for both zero-sum and nonzero-sum games. The most distinguishing feature, compared with the existing literature, is that the game systems are described by forward–backward stochastic differential equations. This kind of games is motivated by linear-quadratic differential game problems with generalized expectation. We give a necessary condition and a sufficient condition in the form of maximum principle for the foregoing games. Finally, an example of a nonzero-sum game is worked out to illustrate that the theories may find interesting applications in practice. In terms of the maximum principle, the explicit form of an equilibrium point is obtained.     

Informational uniqueness of closed-loop Nash equilibria for a class of nonstandard dynamic games

This paper discusses an extension of the currently available theory of noncooperative dynamic games to game models whose state equations are of order higher than one. In a discrete-time framework, it first elucidates the reasons why the theory developed for first-order systems is not applicable to higher-order systems, and then presents a general procedure to obtain an informationally unique Nash equilibrium solution in the presence of random disturbances. A numerical example solved in the paper illustrates the general approach.Dedicated to G. LeitmannResearch that led to this paper was supported in part by the Office of Naval Research under Contract No N00014-82-K-0469 and in part by the U.S. Air Force under Grant No. AFOSR-84-0054.     

Stackelberg strategies in linear-quadratic stochastic differential games

This paper obtains the Stackelberg solution to a class of two-player stochastic differential games described by linear state dynamics and quadratic objective functionals. The information structure of the problem is such that the players make independent noisy measurements of the initial state and are permitted to utilize only this information in constructing their controls. Furthermore, by the very nature of the Stackelberg solution concept, one of the players is assumed to know, in advance, the strategy of the other player (the leader). For this class of problems, we first establish existence and uniqueness of the Stackelberg solution and then relate the derivation of the leader's Stackelberg solution to the optimal solution of a nonstandard stochastic control problem. This stochastic control problem is solved in a more general context, and its solution is utilized in constructing the Stackelberg strategy of the leader. For the special case Gaussian statistics, it is shown that this optimal strategy is affine in observation of the leader. The paper also discusses numerical aspects of the Stackelberg solution under general statistics and develops algorithms which converge to the unique Stackelberg solution.This work was performed while the second author was on sabbatical leave at the Department of Applied Mathematics, Twente University of Technology, Enschede, Holland.     

A stability theorem for symmetrically rational counterplanning

In the absence of contrary information, it would seem prudent for a competitor to attribute to his opponents the same level of rationality that he himself employs. In the context of a general, linear-quadratic, nonzero-sum, two-person game, it is shown that a counterplanning procedure consistent with this principle of symmetrical rationality always converges to the unique Nash equilibrium for the game.The work of R. Kalaba and L. Tesfatsion was partially supported by the National Science Foundation under Grant No. ENG-77-28432 and by the National Institutes of Health under Grant No. GM-23732-03.     

The synthesis of optimal controls for linear,time-optimal problems with retarded controls

Optimization problems involving linear systems with retardations in the controls are studied in a systematic way. Some physical motivation for the problems is discussed. The topics covered are: controllability, existence and uniqueness of the optimal control, sufficient conditions, techniques of synthesis, and dynamic programming. A number of solved examples are presented.This research was supported in part by the National Aeronautics and Space Administration under Grant No. NGL-40-002-015, in part by the Air Force Office of Scientific Research under Grant No. AF-AFOSR-693-67B, and in part by the National Science Foundation under Grant No. GP-15132.     

Sequential Stackelberg equilibria in two-person games

The concept of sequential Stackelberg equilibrium is introduced in the general framework of dynamic, two-person games defined in the Denardo contracting operator formalism. A relationship between this solution concept and the sequential Nash equilibrium for an associated extended game is established. This correspondence result, which can be related to previous results obtained by Baar and Haurie (1984), is then used for studying the existence of such solutions in a class of sequential games. For the zero-sum case, the sequential Stackelberg equilibrium corresponds to a sequential maxmin equilibrium. An algorithm is proposed for the computation of this particular case of equilibrium.This research was supported by SSHRC Grant No. 410-83-1012, NSERC Grant No. A4952, and FCAR Grants Nos. 86-CE-130 and EQ-0428.The authors thank T. R. Bielecki and J. A. Filar, who pointed out some mistakes and helped improving the paper.At the time of this research, this author was with GERMA, Ecole Mohammedia d'Ingénieurs, Rabat, Morocco.     

A maximum principle for Markov regime-switching forward–backward stochastic differential games and applications

In this paper, we present an optimal control problem for stochastic differential games under Markov regime-switching forward–backward stochastic differential equations with jumps. First, we prove a sufficient maximum principle for nonzero-sum stochastic differential games problems and obtain equilibrium point for such games. Second, we prove an equivalent maximum principle for nonzero-sum stochastic differential games. The zero-sum stochastic differential games equivalent maximum principle is then obtained as a corollary. We apply the obtained results to study a problem of robust utility maximization under a relative entropy penalty and to find optimal investment of an insurance firm under model uncertainty.     

Dynamic feedback Stackelberg games with non-unique solutions

Stackelberg games, which was originally introduced by Stackelberg, are widely applied in such fields as economics, management, politics and behavioral sciences. Stackelberg games can be modelled as a bi-level optimization problem. There exists an extensive literature about static bi-level optimization problems. However, the studies on dynamic bi-level optimization problems are fairly scarce in spite of the importance in explaining and predicting some phenomena rationally. In this paper, we consider discrete time dynamic Stackelberg games with feedback information. In general, the lower-level strategies are non-unique in practice. For a unique solution, dynamic programming algorithms have been presented with multiple players. We revisit dynamic programming for feedback information dynamic Stackelberg games with non-unique lower-level solution. First, we define some kind of solutions related to the decisions styles. Then, we analyze them, respectively. Moreover, dynamic programming algorithm is successful in solving solve feedback information dynamic Stackelberg games with non-unique lower-level solutions.