On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points

A new class of nonzero-sum Borel state space discounted stochastic games having stationary Nash equilibria is presented. Some applications to economic theory are also included. Received: January 2002/Revised: July 2002     

On a Noncooperative Stochastic Game Played by Internally Cooperating Generations

We study a model of intergenerational stochastic game with general state space in which each generation consists of n players. The main objective is to prove the existence of a perfect stationary equilibrium in an infinite-horizon intergenerational game in which cooperation is assumed inside every generation. A suitable change in the terminology used in this paper provides a new equilibrium theorem for stochastic games with so-called “hyperbolic players”. A discussion of perfect equilibria in games of noncooperative generations is also given. Some applications to economic theory are included.     

Perfect information stochastic games and related classes

Forn-person perfect information stochastic games and forn-person stochastic games with Additive Rewards and Additive Transitions (ARAT) we show the existence of pure limiting average equilibria. Using a similar approach we also derive the existence of limiting average ε-equilibria for two-person switching control stochastic games. The orderfield property holds for each of the classes mentioned, and algorithms to compute equilibria are pointed out.     

Monotone and bounded interval equilibria in a coordination game with information aggregation

We analyze how private learning in a class of games with common stochastic payoffs affects the form of equilibria, and how properties such as player welfare and the extent of strategic miscoordination relate across monotone and non-monotone equilibria. Researchers typically focus on monotone equilibria. We provide conditions under which non-monotone equilibria also exist, where players attempt to coordinate to obtain the stochastic payoff whenever signals are in a bounded interval. In bounded interval equilibria (BIE), an endogenous fear of miscoordination discourages players from coordinating to obtain the stochastic payoff when their signals suggest coordination is most beneficial. In contrast to monotone equilibria, expected payoffs from successful coordination in BIE are lower than the ex-ante expected payoff from ignoring signals and always trying to coordinate to obtain the stochastic payoff. We show that BIE only exist when, absent private information, the game would be a coordination game.     

Pareto efficiency,simple game stability,and social structure in finitely repeated games

Simple game (sensu Brown and Vincent, 1987) evolutionary theory, when coupled with social structure measured as non‐random encounter of strategy “clones”, often permits equilibrium refinement leading to Pareto superior outcomes (e.g., Axelrod, 1981; Myerson et al., 1991), a foundational goal of economic game theory (Myerson, 1991: 370–375). This conclusion, derived from analyses of one‐shot and infinitely repeated games, fails for finitely repeated games. While mutant cluster invasion enhances Pareto efficiency of equilibria in the former, it can depress Pareto efficiency in the latter. Cooperative equilibria of finitely repeated games (under economic analysis) can be susceptible to cluster‐invasion by even more Pareto efficient strategies which are not themselves evolutionarily stable. Evolutionary (simple) game theory's ability to eliminate Pareto inferior Nash equilibrium strategies induces vulnerabilities foreign to economic analysis. Simple game analysis of finitely repeated games suggests that social structure, modeled as perennial invasion by mutant‐clusters, can induce cyclic invasion, saturation, and loss of cooperation.     

On Nikaido-Isoda type theorems for discounted stochastic games

We present a class of countable state space stochastic games with discontinuous payoff functions satisfying some assumptions similar to the ones of Nikaido and Isoda for one-stage games. We prove that these games possess stationary equilibria. We show that after adding some concavity assumptions these equilibria are nonrandomized. Further, we present an example of input (or production) dynamic game satisfying the assumptions of our model. We give a closed-form solution for this game.     

Constructions of Nash Equilibria in Stochastic Games of Resource Extraction with Additive Transition Structure

A class of N-person stochastic games of resource extraction with discounted payoffs in discrete time is considered. It is assumed that transition probabilities have special additive structure. It is shown that the Nash equilibria and corresponding payoffs in finite horizon games converge as horizon goes to infinity. This implies existence of stationary Nash equilibria in the infinite horizon case. In addition the algorithm for finding Nash equilibria in infinite horizon games is discussed     

An application of optimization theory to the study of equilibria for games: a survey

This contribution is a survey about potential games and their applications. In a potential game the information that is sufficient to determine Nash equilibria can be summarized in a single function on the strategy space: the potential function. We show that the potential function enable the application of optimization theory to the study of equilibria. Potential games and their generalizations are presented. Two special classes of games, namely team games and separable games, turn out to be potential games. Several properties satisfied by potential games are discussed and examples from concrete situations as congestion games, global emission games and facility location games are illustrated.     

Non-cooperative stochastic dominance games

A non-cooperative stochastic dominance game is a non-cooperative game in which the only knowledge about the players' preferences and risk attitudes is presumed to be their preference orders on the set ofn-tuples of pure strategies. Stochastic dominance equilibria are defined in terms of mixed strategies for the players that are efficient in the stochastic dominance sense against the strategies of the other players. It is shown that the set of SD equilibria equals all Nash equilibria that can be obtained from combinations of utility functions that are consistent with the players' known preference orders. The latter part of the paper looks at antagonistic stochastic dominance games in which some combination of consistent utility functions is zero-sum over then-tuples of pure strategies.     

Cooperative games with stochastic payoffs

This paper introduces a new class of cooperative games arising from cooperative decision making problems in a stochastic environment. Various examples of decision making problems that fall within this new class of games are provided. For a class of games with stochastic payoffs where the preferences are of a specific type, a balancedness concept is introduced. A variant of Farkas' lemma is used to prove that the core of a game within this class is non-empty if and only if the game is balanced. Further, other types of preferences are discussed. In particular, the effects the preferences have on the core of these games are considered.     

Two Different Approaches to Nonzero-Sum Stochastic Differential Games

We make the link between two approaches to Nash equilibria for nonzero-sum stochastic differential games: the first one using backward stochastic differential equations and the second one using strategies with delay. We prove that, when both exist, the two notions of Nash equilibria coincide.     

Stochastic differential games: Occupation measure based approach

A new approach based on occupation measures is introduced for studying stochastic differential games. For two-person zero-sum games, the existence of values and optimal strategies for both players is established for various payoff criteria. ForN-person games, the existence of equilibria in Markov strategies is established for various cases.     

On equilibria in repeated games with absorbing states

We prove the existence of ε-(Nash) equilibria in two-person non-zerosum limiting average repeated games with absorbing states. These are stochastic games in which all states but one are absorbing. A state is absorbing if the probability of ever leaving that state is zero for all available pairs of actions.     

A general characterization of the mean field limit for stochastic differential games

The mean field limit of large-population symmetric stochastic differential games is derived in a general setting, with and without common noise, on a finite time horizon. Minimal assumptions are imposed on equilibrium strategies, which may be asymmetric and based on full information. It is shown that approximate Nash equilibria in the n-player games admit certain weak limits as n tends to infinity, and every limit is a weak solution of the mean field game (MFG). Conversely, every weak MFG solution can be obtained as the limit of a sequence of approximate Nash equilibria in the n-player games. Thus, the MFG precisely characterizes the possible limiting equilibrium behavior of the n-player games. Even in the setting without common noise, the empirical state distributions may admit stochastic limits which cannot be described by the usual notion of MFG solution.     

Uniqueness of Nash equilibrium for linear-convex stochastic differential games

The uniqueness of Nash equilibria is shown for a class of stochastic differential games where the dynamic constraints are linear in the control variables. The result is applied to an oligopoly.This paper benefitted from comments by two anonymous referees and by L. Blume and C. Simon.     

Continuous dependence estimates for viscosity solutions of integro-PDEs

We present a general framework for deriving continuous dependence estimates for, possibly polynomially growing, viscosity solutions of fully nonlinear degenerate parabolic integro-PDEs. We use this framework to provide explicit estimates for the continuous dependence on the coefficients and the “Lévy measure” in the Bellman/Isaacs integro-PDEs arising in stochastic control/differential games. Moreover, these explicit estimates are used to prove regularity results and rates of convergence for some singular perturbation problems. Finally, we illustrate our results on some integro-PDEs arising when attempting to price European/American options in an incomplete stock market driven by a geometric Lévy process. Many of the results obtained herein are new even in the convex case where stochastic control theory provides an alternative to our pure PDE methods.     

Generic stability of Nash equilibria for noncooperative differential games

The stability of Nash equilibria against the perturbation of the right-hand side functions of state equations for noncooperative differential games is investigated. By employing the set-valued analysis theory, we show that the differential games whose equilibria are all stable form a dense residual set, and every differential game can be approximated arbitrarily by a sequence of stable differential games, that is, in the sense of Baire’s category most of the differential games are stable.     

A link between sequential semi-anonymous nonatomic games and their large finite counterparts

We show that obtainable equilibria of a multi-period nonatomic game can be used by players in its large finite counterparts to achieve near-equilibrium payoffs. Such equilibria in the form of random state-to-action rules are parsimonious in form and easy to execute, as they are both oblivious of past history and blind to other players’ present states. Our transient results can be extended to a stationary case, where the finite multi-period games are special discounted stochastic games. In both nonatomic and finite games, players’ states influence their payoffs along with actions they take; also, the random evolution of one particular player’s state is driven by all players’ states as well as actions. The finite games can model diverse situations such as dynamic price competition. But they are notoriously difficult to analyze. Our results thus suggest ways to tackle these problems approximately.     

Structure of extreme correlated equilibria: a zero-sum example and its implications

We exhibit the rich structure of the set of correlated equilibria by analyzing the simplest of polynomial games: the mixed extension of matching pennies. We show that while the correlated equilibrium set is convex and compact, the structure of its extreme points can be quite complicated. In finite games the ratio of extreme correlated to extreme Nash equilibria can be greater than exponential in the size of the strategy spaces. In polynomial games there can exist extreme correlated equilibria which are not finitely supported; we construct a large family of examples using techniques from ergodic theory. We show that in general the set of correlated equilibrium distributions of a polynomial game cannot be described by conditions on finitely many moments (means, covariances, etc.), in marked contrast to the set of Nash equilibria which is always expressible in terms of finitely many moments.     

Zero-sum risk-sensitive stochastic games on a countable state space

Infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled Markov chains with countably many states are analyzed. Upper and lower values for these games are established. The existence of value and saddle-point equilibria in the class of Markov strategies is proved for the discounted-cost game. The existence of value and saddle-point equilibria in the class of stationary strategies is proved under the uniform ergodicity condition for the ergodic-cost game. The value of the ergodic-cost game happens to be the product of the inverse of the risk-sensitivity factor and the logarithm of the common Perron–Frobenius eigenvalue of the associated controlled nonlinear kernels.