Stackelberg strategies in linear-quadratic stochastic differential games

This paper obtains the Stackelberg solution to a class of two-player stochastic differential games described by linear state dynamics and quadratic objective functionals. The information structure of the problem is such that the players make independent noisy measurements of the initial state and are permitted to utilize only this information in constructing their controls. Furthermore, by the very nature of the Stackelberg solution concept, one of the players is assumed to know, in advance, the strategy of the other player (the leader). For this class of problems, we first establish existence and uniqueness of the Stackelberg solution and then relate the derivation of the leader's Stackelberg solution to the optimal solution of a nonstandard stochastic control problem. This stochastic control problem is solved in a more general context, and its solution is utilized in constructing the Stackelberg strategy of the leader. For the special case Gaussian statistics, it is shown that this optimal strategy is affine in observation of the leader. The paper also discusses numerical aspects of the Stackelberg solution under general statistics and develops algorithms which converge to the unique Stackelberg solution.This work was performed while the second author was on sabbatical leave at the Department of Applied Mathematics, Twente University of Technology, Enschede, Holland.     

Dynamic feedback Stackelberg games with non-unique solutions

Stackelberg games, which was originally introduced by Stackelberg, are widely applied in such fields as economics, management, politics and behavioral sciences. Stackelberg games can be modelled as a bi-level optimization problem. There exists an extensive literature about static bi-level optimization problems. However, the studies on dynamic bi-level optimization problems are fairly scarce in spite of the importance in explaining and predicting some phenomena rationally. In this paper, we consider discrete time dynamic Stackelberg games with feedback information. In general, the lower-level strategies are non-unique in practice. For a unique solution, dynamic programming algorithms have been presented with multiple players. We revisit dynamic programming for feedback information dynamic Stackelberg games with non-unique lower-level solution. First, we define some kind of solutions related to the decisions styles. Then, we analyze them, respectively. Moreover, dynamic programming algorithm is successful in solving solve feedback information dynamic Stackelberg games with non-unique lower-level solutions.     

A theoretical approximation scheme for Stackelberg problems

We consider the Stackelberg problem corresponding to a two-player game in which one of the two players has the leadership in playing the game. We present a general approach for approximating the considered hierarchical programming problem by a sequence of two-level optimization problems. From a practical point of view, we also give some results for asymptotically Stackelberg approximating sequences and for problems with perturbed constraints.This paper is based upon results first presented at Journées Fermat: Mathematics for Optimization, Toulouse, France, May 1985.     

On stochastic games with additive reward and transition structure

In this paper, we introduce a new class of two-person stochastic games with nice properties. For games in this class, the payoffs as well as the transitions in each state consist of a part which depends only on the action of the first player and a part dependent only on the action of the second player.For the zero-sum games in this class, we prove that the orderfield property holds in the infinite-horizon case and that there exist optimal pure stationary strategies for the discounted as well as the undiscounted payoff criterion. For both criteria also, finite algorithms are given to solve the game. An example shows that, for nonzero sum games in this class, there are not necessarily pure stationary equilibria. But, if such a game possesses a stationary equilibrium point, then there also exists a stationary equilibrium point which uses in each state at most two pure actions for each player.     

A note on dynamic Stackelberg games with leaders in turn

Recently, a model of dynamic Stackelberg games with leaders in turn has been proposed, and dynamic Stackelberg games with leaders in turn have been exploited under a feedback information structure. This paper characterizes dynamic Stackelberg games with leaders in turn under other information structures, both closed-loop and open-loop information structures. Explicit solutions are given for linear-quadratic systems under an open-loop information structure for dynamic Stackelberg games with leaders in turn.     

Note Equilibrium payoffs in stochastic games of incomplete information: the general symmetric case

We prove that the existence of equilibrium payoffs for stochastic games of incomplete symmetric information follows from the same result for stochastic games with complete information. Received January 1999/Revised October 2001     

Zero-sum stochastic games with unbounded costs: Discounted and average cost cases

Zero-sum stochastic games with countable state space and with finitely many moves available to each player in a given state are treated. As a function of the current state and the moves chosen, player I incurs a nonnegative cost and player II receives this as a reward. For both the discounted and average cost cases, assumptions are given for the game to have a finite value and for the existence of an optimal randomized stationary strategy pair. In the average cost case, the assumptions generalize those given in Sennott (1993) for the case of a Markov decision chain. Theorems of Hoffman and Karp (1966) and Nowak (1992) are obtained as corollaries. Sufficient conditions are given for the assumptions to hold. A flow control example illustrates the results.     

Nonzero-sum stochastic games with unbounded costs: Discounted and average cost cases

We treat non-cooperative stochastic games with countable state space and with finitely many players each having finitely many moves available in a given state. As a function of the current state and move vector, each player incurs a nonnegative cost. Assumptions are given for the expected discounted cost game to have a Nash equilibrium randomized stationary strategy. These conditions hold for bounded costs, thereby generalizing Parthasarathy (1973) and Federgruen (1978). Assumptions are given for the long-run average expected cost game to have a Nash equilibrium randomized stationary strategy, under which each player has constant average cost. A flow control example illustrates the results. This paper complements the treatment of the zero-sum case in Sennott (1993a).     

Uniqueness of Nash equilibrium for linear-convex stochastic differential games

The uniqueness of Nash equilibria is shown for a class of stochastic differential games where the dynamic constraints are linear in the control variables. The result is applied to an oligopoly.This paper benefitted from comments by two anonymous referees and by L. Blume and C. Simon.     

Existence of equilibrium stationary strategies in discounted noncooperative stochastic games with uncountable state space

This paper considers discounted noncooperative stochastic games with uncountable state space and compact metric action spaces. We assume that the transition law is absolutely continuous with respect to some probability measure defined on the state space. We prove, under certain additional continuity and integrability conditions, that such games have -equilibrium stationary strategies for each >0. To prove this fact, we provide a method for approximating the original game by a sequence of finite or countable state games. The main result of this paper answers partially a question raised by Parthasarathy in Ref. 1.     

Characterizing properties of the value function of stochastic games

In the present note, the axiomatic characterization of the value function of two-person, zero-sum games in normal form by Vilkas and Tijs is extended to the value function of discounted, two-person, zero-sum stochastic games. The characterizing axioms can be indicated by the following terms: objectivity, monotony, and sufficiency for both players; or sufficiency for one of the players and symmetry. Also, a characterization without using the monotony axiom is given.     

Zero-sum risk-sensitive stochastic games on a countable state space

Infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled Markov chains with countably many states are analyzed. Upper and lower values for these games are established. The existence of value and saddle-point equilibria in the class of Markov strategies is proved for the discounted-cost game. The existence of value and saddle-point equilibria in the class of stationary strategies is proved under the uniform ergodicity condition for the ergodic-cost game. The value of the ergodic-cost game happens to be the product of the inverse of the risk-sensitivity factor and the logarithm of the common Perron–Frobenius eigenvalue of the associated controlled nonlinear kernels.     

On the saddle-point solution of a class of stochastic differential games

This paper deals with the saddle-point solution of a class of stochastic differential games described by linear state dynamics and quadratic objective functionals. The information structure of the problem is such that both players have access to a common noisy linear measurement of the state and they are permitted to utilize only this information in constructing their controls. The saddle-point solution of such differential game problems has been discussed earlier in Ref. 1, but the conclusions arrived there are incorrect, as is explicitly shown in this paper. We extensively discuss the role of information structure on the saddle-point solution of such stochastic games (specifically within the context of an illustrative discrete-time example) and then obtain the saddle-point solution of the problem originally formulated by employing an indirect approach.This work was done while the author was on sabbatical leave at Twente University of Technology, Department of Applied Mathematics, Enschede, Holland, from Applied Mathematics Division, Marmara Scientific and Industrial Research Institute, Gebze, Kocaeli, Turkey.     

Risk-sensitive stochastic differential games with reflecting diffusions

We study risk-sensitive differential games for controlled reflecting diffusion processes in a bounded domain. We consider both nonzero-sum and zero-sum cases. We treat two cost evaluation criteria; namely, discounted cost and ergodic cost. Under certain assumptions we establish the existence of Nash/saddle-point equilibria for relevant cases.     

Nonrandomized strategy equilibria in noncooperative stochastic games with additive transition and reward structure

In this paper, we study a discounted noncooperative stochastic game with an abstract measurable state space, compact metric action spaces of players, and additive transition and reward structure in the sense of Himmelberget al. (Ref. 1) and Parthasarathy (Ref. 2). We also assume that the transition law of the game is absolutely continuous with respect to some probability distributionp of the initial state and together with the reward functions of players satisfies certain continuity conditions. We prove that such a game has an equilibrium stationary point, which extends a result of Parthasarathy from Ref. 2, where the action spaces of players are assumed to be finite sets. Moreover, we show that our game has a nonrandomized (- )-equilibrium stationary point for each >0, provided that the probability distributionp is nonatomic. The latter result is a new existence theorem.     

Equilibria in repeated games of incomplete information: The general symmetric case

Every two person repeated game of symmetric incomplete information, in which the signals sent at each stage to both players are identical and generated by a state and moves dependent probability distribution on a given finite alphabet, has an equilibrium payoff. Received March 1996/Revised version January 1997/Final version May 1997     

A comparison of different quantity discount pricing policies in a two-echelon channel with stochastic and asymmetric demand information

In this paper, we study quantity discount pricing policies in a channel of one manufacturer and one retailer. The paper assumes that the channel faces a stochastic price-sensitive demand but the retailer can privately observe the realization of an uncertain demand parameter. The problem is analyzed as a Stackelberg game in which the manufacturer declares quantity discount pricing schemes to the retailer and then the retailer follows by selecting the retail price and associated quantity. Proposed in the paper are four quantity-discount pricing policies: “regular quantity discount”; “fixed percentage discount”; “incremental volume discount” and “fixed marginal-profit-rate discount”. Optimal solutions are derived, and numerical examples are presented to illustrate the efficiency of each discount policy.     

On discounted stochastic games with incomplete information on payoffs and a security application

This paper presents a robust optimization model for n$n$-person finite state/action stochastic games with incomplete information on payoffs. For polytopic uncertainty sets, we propose an explicit mathematical programming formulation for an equilibrium calculation. It turns out that a global optimal of this mathematical program yields an equilibrium point and epsilon-equilibria can be calculated based on this result. We briefly describe an incomplete information version of a security application that can benefit from robust game theory.     

On nash equilibrium solutions in nonzero-sum stochastic games with complete information

Two-person nonzero-sum stochastic games with complete information are considered. It is shown that it is sufficient to search the equilibrium solutions in a class of deterministic strategy pairs — the so-calledintimidation strategy pairs. Furthermore, properties of the set of all equilibrium losses of such strategy pairs are proved.