Algorithms for uniform optimal strategies in two-player zero-sum stochastic games with perfect information

We deal with zero-sum two-player stochastic games with perfect information. We propose two algorithms to find the uniform optimal strategies and one method to compute the optimality range of discount factors. We prove the convergence in finite time for one algorithm. The uniform optimal strategies are also optimal for the long run average criterion and, in transient games, for the undiscounted criterion as well.     

Recall in extensive form games

This paper considers characterizations of perfect recall in extensive form games. It is shown that perfect recall can be expressed in terms of choices without any reference to infomation sets. When information sets are taken into account, it is decomposable into an ordering of information sets and that players do not forget what they knew nor what they did. Thus, if information sets are partially ordered, then perfect recall is implied by the player's inability to refine her information from the memory. Received: August 1997/final version: September 1998     

On Pareto equilibria in vector-valued extensive form games

In this paper we investigate the existence of Pareto equilibria in vector-valued extensive form games. In particular we show that every vector-valued extensive form game with perfect information has at least one subgame perfect Pareto equilibrium in pure strategies. If one tries to prove this and develop a vector-valued backward induction procedure in analogy to the real-valued one, one sees that different effects may occur which thus have to be taken into account: First, suppose the deciding player at a nonterminal node makes a choice such that the equilibrium payoff vector of the subgame he would enter is undominated under the equilibrium payoff vectors of the other subgames he might enter. Then this choice need not to lead to a Pareto equilibrium. Second, suppose at a nonterminal node a chance move may arise. The combination of the Pareto equilibria of the subgames to give a strategy combination of the entire game need not be a Pareto equilibrium of the entire game.     

Pure subgame-perfect equilibria in free transition games

We consider a class of stochastic games, where each state is identified with a player. At any moment during play, one of the players is called active. The active player can terminate the game, or he can announce any player, who then becomes the active player. There is a non-negative payoff for each player upon termination of the game, which depends only on the player who decided to terminate. We give a combinatorial proof of the existence of subgame-perfect equilibria in pure strategies for the games in our class.     

Equilibria in repeated games of incomplete information: The general symmetric case

Every two person repeated game of symmetric incomplete information, in which the signals sent at each stage to both players are identical and generated by a state and moves dependent probability distribution on a given finite alphabet, has an equilibrium payoff. Received March 1996/Revised version January 1997/Final version May 1997     

Markov games with incomplete information

We consider zero-sum Markov games with incomplete information. Here, the second player is never informed about the current state of the underlying Markov chain. The existence of a value and of optimal strategies for both players is shown. In particular, we present finite algorithms for computing optimal strategies for the informed and uninformed player. The algorithms are based on linear programming results.     

Existence of optimal strategies in Markov games with incomplete information

The existence of a value and optimal strategies is proved for the class of two-person repeated games where the state follows a Markov chain independently of players’ actions and at the beginning of each stage only Player 1 is informed about the state. The results apply to the case of standard signaling where players’ stage actions are observable, as well as to the model with general signals provided that Player 1 has a nonrevealing repeated game strategy. The proofs reduce the analysis of these repeated games to that of classical repeated games with incomplete information on one side. This research was supported in part by Israeli Science Foundation grants 382/98, 263/03, and 1123/06, and by the Zvi Hermann Shapira Research Fund.     

Statistical linear differential games: A treatment of incomplete information

A zero-sum, two-player linear differential game of fixed duration is considered in the case when the information is incomplete but a statistical structure gives both players the possibility tospy the value of an unknown parameter in the payoff. Considerations of topological vector spaces and functional analysis allow one to demonstrate, via a classical Sion's theorem, sufficient conditions for the existence of a value.The author is indebted to Professor J. Fichefet for his helpful remarks and indications.     

Stochastic games without perfect monitoring

A two-person zero-sum stochastic game with finitely many states and actions is considered. The classical assumption of perfect monitoring is relaxed. Instead of being informed of the previous action of his opponent, each player receives a random signal, the law of which depending on both previous actions and on the previous state. We prove the existence of the max-min and dually of the min-max, thus extending both the result of Mertens-Neyman about the existence of the value in case of perfect monitoring and a theorem obtained by the author on a subclass of stochastic games: the absorbing games. It is a pleasure to thank J. Filar and V. Gaitsgory for their friendly hospitality at the School of Mathematics, University of South-Australia, where this work was initiated. There, the author was supported by a grant from the Australian Research Council no. A69703141. I would like also to thank S. Sorin for his keen interest and finally an anonymous referee for his careful reading of the proof.     

Learning algorithms for repeated bimatrix Nash games with incomplete information

The purpose of this paper is to study a particular recursive scheme for updating the actions of two players involved in a Nash game, who do not know the parameters of the game, so that the resulting costs and strategies converge to (or approach a neighborhood of) those that could be calculated in the known parameter case. We study this problem in the context of a matrix Nash game, where the elements of the matrices are unknown to both players. The essence of the contribution of this paper is twofold. On the one hand, it shows that learning algorithms which are known to work for zero-sum games or team problems can also perform well for Nash games. On the other hand, it shows that, if two players act without even knowing that they are involved in a game, but merely thinking that they try to maximize their output using the learning algorithm proposed, they end up being in Nash equilibrium.This research was supported in part by NSF Grant No. ECS-87-14777.     

On differential games with Markov perfect triggering

This paper is an attempt to throw some light on the issue of whether defining history-dependent state variables in a differential game does allow for a dynamic play of precommitment equilibria. We suggest the application of trigger strategies in a state-feedback context. Based on a punishment mode in Markov perfect strategies and being able to detect even past deviations followed by histories returning to equilibrium, these subgame-perfect strategies lead to the enforcement of certain outcomes by means of dynamic rules of strategic interaction. The last part of our exposition is devoted to specific game structures under which a trigger equilibrium can be used as well as a punishment mode.     

Dimension of complete simple games with minimum

Weighted majority games have the property that players are totally ordered by the desirability relation (introduced by Isbell in [J.R. Isbell, A class of majority games, Quarterly Journal of Mathematics, 7 (1956) 183–187]) because weights induce it. Games for which this relation is total are called complete simple games. Taylor and Zwicker proved in [A.D. Taylor, W.S. Zwicker, Weighted voting, multicameral representation, and power, Games and Economic Behavior 5 (1993) 170–181] that every simple game (or monotonic finite hypergraph) can be represented by an intersection of weighted majority games and consider the dimension of a game as the needed minimum number of them to get it. They provide the existence of non-complete simple games of every dimension and left open the problem for complete simple games.     

Subgame-consistent cooperative solutions in randomly furcating stochastic differential games

The paradigm of randomly-furcating stochastic differential games incorporates additional stochastic elements via randomly branching payoffs in stochastic differential games. This paper considers dynamically stable cooperative solutions in randomly furcating stochastic differential games. Analytically tractable payoff distribution procedures contingent upon specific random realizations of the state and payoff structure are derived. This new approach widens the application of cooperative differential game theory to problems where the evolution of the state and future environments are not known with certainty. Important cases abound in regional economic cooperation, corporate joint ventures and environmental control. An illustration in cooperative resource extraction is presented.     

Generic uniqueness of the bias vector of finite zero-sum stochastic games with perfect information

Mean-payoff zero-sum stochastic games can be studied by means of a nonlinear spectral problem. When the state space is finite, the latter consists in finding an eigenpair $(u,\lambda )$ solution of $T(u)=\lambda e+u$, where $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is the Shapley (or dynamic programming) operator, λ is a scalar, e is the unit vector, and $u\in {\mathbb{R}}^n$. The scalar λ yields the mean payoff per time unit, and the vector u, called bias, allows one to determine optimal stationary strategies in the mean-payoff game. The existence of the eigenpair $(u,\lambda )$ is generally related to ergodicity conditions. A basic issue is to understand for which classes of games the bias vector is unique (up to an additive constant). In this paper, we consider perfect-information zero-sum stochastic games with finite state and action spaces, thinking of the transition payments as variable parameters, transition probabilities being fixed. We show that the bias vector, thought of as a function of the transition payments, is generically unique (up to an additive constant). The proof uses techniques of nonlinear Perron–Frobenius theory. As an application of our results, we obtain an explicit perturbation scheme allowing one to solve degenerate instances of stochastic games by policy iteration.     

On some geometry and equivalence classes of normal form games

Equivalence classes of normal form games are defined using the discontinuities of correspondences of standard equilibrium concepts like correlated, Nash, and robust equilibrium, or risk dominance and rationalizability. Resulting equivalence classes are fully characterized and compared across different equilibrium concepts for 2 ×  2 games; larger games are also studied. It is argued that the procedure leads to broad and game-theoretically meaningful distinctions of games as well as to alternative ways of representing, comparing and testing equilibrium concepts.     

Perfect information two-person zero-sum markov games with imprecise transition probabilities

Based on an extension of the controlled Markov set-chain model by Kurano et al. (in J Appl Prob 35:293–302, 1998) into competitive two-player game setting, we provide a model of perfect information two-person zero-sum Markov games with imprecise transition probabilities. We define an equilibrium value for the games formulated with the model in terms of a partial order and then establish the existence of an equilibrium policy pair that achieves the equilibrium value. We further analyze finite-approximation error bounds obtained from a value iteration-type algorithm and discuss some applications of the model.     

Note Equilibrium payoffs in stochastic games of incomplete information: the general symmetric case

We prove that the existence of equilibrium payoffs for stochastic games of incomplete symmetric information follows from the same result for stochastic games with complete information. Received January 1999/Revised October 2001     

On nash equilibrium solutions in nonzero-sum stochastic games with complete information

Two-person nonzero-sum stochastic games with complete information are considered. It is shown that it is sufficient to search the equilibrium solutions in a class of deterministic strategy pairs — the so-calledintimidation strategy pairs. Furthermore, properties of the set of all equilibrium losses of such strategy pairs are proved.     

Dynamic consistency in incomplete information games under ambiguity

We develop a general framework of incomplete information games under ambiguity which extends the traditional framework of Bayesian games to the context of Ellsberg-type ambiguity. We then propose new solution concepts called ex ante and interim Γ-maximin equilibrium for solving such games. We show that, unlike the standard notion of Bayesian Nash equilibrium, these concepts may lead to rather different recommendations for the same game under ambiguity. This phenomenon is often referred to as dynamic inconsistency. Moreover, we characterize the sufficient condition under which dynamic consistency is assured in this generalized framework.     

Some thoughts on saddle-point conditions and information structures in zero-sum differential games

For a very simple two-stage, linear-quadratic, zero-sum difference game with dynamic information structure, we show that (i) there exist nonlinear saddle-point strategies which require the same existence conditions as the well-known linear, closed-loop, no-memory solution and (ii) there exist both linear and nonlinear saddle-point strategies which require more stringent conditions than the unique open-loop solution. We then discuss the implication of this result with respect to the existence of saddle points in zero-sum differential games for different information patterns.