Stationary,completely mixed and symmetric optimal and equilibrium strategies in stochastic games

In this paper, we address various types of two-person stochastic games—both zero-sum and nonzero-sum, discounted and undiscounted. In particular, we address different aspects of stochastic games, namely: (1) When is a two-person stochastic game completely mixed? (2) Can we identify classes of undiscounted zero-sum stochastic games that have stationary optimal strategies? (3) When does a two-person stochastic game possess symmetric optimal/equilibrium strategies? Firstly, we provide some necessary and some sufficient conditions under which certain classes of discounted and undiscounted stochastic games are completely mixed. In particular, we show that, if a discounted zero-sum switching control stochastic game with symmetric payoff matrices has a completely mixed stationary optimal strategy, then the stochastic game is completely mixed if and only if the matrix games restricted to states are all completely mixed. Secondly, we identify certain classes of undiscounted zero-sum stochastic games that have stationary optima under specific conditions for individual payoff matrices and transition probabilities. Thirdly, we provide sufficient conditions for discounted as well as certain classes of undiscounted stochastic games to have symmetric optimal/equilibrium strategies—namely, transitions are symmetric and the payoff matrices of one player are the transpose of those of the other. We also provide a sufficient condition for the stochastic game to have a symmetric pure strategy equilibrium. We also provide examples to show the sharpness of our results.     

On the probability of existence of pure equilibria in matrix games

We examine the probability that a randomly chosen matrix game admits pure equilibria and its behavior as the number of actions of the players or the number of players increases. We show that, for zero-sum games, the probability of having pure equilibria goes to zero as the number of actions goes to infinity, but it goes to a nonzero constant for a two-player game. For many-player games, if the number of players goes to infinity, the probability of existence of pure equilibria goes to zero even if the number of actions does not go to infinity.This research was supported in part by NSF Grant CCR-92-22734.     

On a class of nonzero-sum,linear-quadratic differential games

A class of two-player, nonzero-sum, linear-quadratic differential games is investigated for Nash equilibrium solutions when both players use closed-loop control and when one or both of the players are required to use open-loop control. For three formulations of the game, necessary and sufficient conditions are obtained for a particular strategy set to be a Nash equilibrium strategy set. For a fourth formulation of the game, where both players use open-loop control, necessary and sufficient conditions for the existence of a Nash equilibrium strategy set are developed. Several examples are presented in order to illustrate the differences between this class of differential games and its zero-sum analog.This research was supported by the National Science Foundation under Grant No. GK-3341.     

Perfect information two-person zero-sum markov games with imprecise transition probabilities

Based on an extension of the controlled Markov set-chain model by Kurano et al. (in J Appl Prob 35:293–302, 1998) into competitive two-player game setting, we provide a model of perfect information two-person zero-sum Markov games with imprecise transition probabilities. We define an equilibrium value for the games formulated with the model in terms of a partial order and then establish the existence of an equilibrium policy pair that achieves the equilibrium value. We further analyze finite-approximation error bounds obtained from a value iteration-type algorithm and discuss some applications of the model.     

Values for strategic games in which players cooperate

In this paper we propose a new method to associate a coalitional game with each strategic game. The method is based on the lower value of finite two-player zero-sum games. We axiomatically characterize this new method, as well as the method that was described in Von Neumann and Morgenstern (1944). As an intermediate step, we provide axiomatic characterizations of the value and the lower value of matrix games and finite two-player zero-sum games, respectively.The authors acknowledge the financial support of Ministerio de Ciencia y Tecnologia, FEDER andXunta de Galicia through projects BEC2002-04102-C02-02 and PGIDIT03PXIC20701PN.We wish to thank Professor William Thomson as well as an anonymous referee for useful comments.     

Noncooperative stochastic games under a n-stage local contraction assumption

We consider a class of noncooperative stochastic games with general state and action spaces and with a state dependent discount factor. The expected time duration between any two stages of the game is not bounded away from zero, so that the usual N-stage contraction assumption, uniform over all admissible strategies, does not hold. We propose milder sufficient regularity conditions, allowing strategies that give rise with probability one to any number of simultaneous stages. We give sufficient conditions for the existence of equilibrium and ∈-equilibrium stationary strategies in the sense of Nash. In the two-player zero-sum case, when an equilibrium strategy exists, the value of the game is the unique fixed point of a specific functional operator and can be computed by dynamic programming.     

On stochastic games with additive reward and transition structure

In this paper, we introduce a new class of two-person stochastic games with nice properties. For games in this class, the payoffs as well as the transitions in each state consist of a part which depends only on the action of the first player and a part dependent only on the action of the second player.For the zero-sum games in this class, we prove that the orderfield property holds in the infinite-horizon case and that there exist optimal pure stationary strategies for the discounted as well as the undiscounted payoff criterion. For both criteria also, finite algorithms are given to solve the game. An example shows that, for nonzero sum games in this class, there are not necessarily pure stationary equilibria. But, if such a game possesses a stationary equilibrium point, then there also exists a stationary equilibrium point which uses in each state at most two pure actions for each player.     

Separable and low-rank continuous games

In this paper, we study nonzero-sum separable games, which are continuous games whose payoffs take a sum-of-products form. Included in this subclass are all finite games and polynomial games. We investigate the structure of equilibria in separable games. We show that these games admit finitely supported Nash equilibria. Motivated by the bounds on the supports of mixed equilibria in two-player finite games in terms of the ranks of the payoff matrices, we define the notion of the rank of an n-player continuous game and use this to provide bounds on the cardinality of the support of equilibrium strategies. We present a general characterization theorem that states that a continuous game has finite rank if and only if it is separable. Using our rank results, we present an efficient algorithm for computing approximate equilibria of two-player separable games with fixed strategy spaces in time polynomial in the rank of the game. This research was funded in part by National Science Foundation grants DMI-0545910 and ECCS-0621922 and AFOSR MURI subaward 2003-07688-1.     

Existence of Value and Saddle Point in Infinite-Dimensional Differential Games

We study two-player zero-sum differential games of finite duration in a Hilbert space. Following the Berkovitz notion of strategies, we prove the existence of value and saddle-point equilibrium. We characterize the value as the unique viscosity solution of the associated Hamilton–Jacobi–Isaacs equation using dynamic programming inequalities.     

On Nash Equilibrium Strategy of Two-person Zero-sum Games with Trapezoidal Fuzzy Payoffs

In this paper, we investigate Nash equilibrium strategy of two-person zero-sum games with fuzzy payoffs. Based on fuzzy max order, Maeda and Cunlin constructed several models in symmetric triangular and asymmetric triangular fuzzy environment, respectively. We extended their models in trapezoidal fuzzy environment and proposed the existence of equilibrium strategies for these models. We also established the relation between Pareto Nash equilibrium strategy and parametric bi-matrix game. In addition, numerical examples are presented to find Pareto Nash equilibrium strategy and weak Pareto Nash equilibrium strategy from bi-matrix game.     

Correlated equilibria in some classes of two-person games

A correlated equilibrium in a two-person game is “good” if for everyNash equilibrium there is a player who prefers the correlated equilibrium to theNash equilibrium. If a game is “best-response equivalent” to a two-person zero-sum game, then it has no good correlated equilibria. But games which are “almost strictly competitive” or “order equivalent” to a two-person zero-sum game may have good correlated equilibria.     

Games with the total bandwagon property meet the Quint–Shubik conjecture

This paper revisits the total bandwagon property (TBP) introduced by Kandori and Rob (Games Econ Behav 22:30–60, 1998). With this property, we characterize the class of two-player symmetric \(n\times n\) games, showing that a game has TBP if and only if the game has \(2^{n}-1\) symmetric Nash equilibria. We extend this result to bimatrix games by generalizing TBP. This sheds light on the (wrong) conjecture of Quint and Shubik (Int J Game Theory 26:353–359, 1997) that any nondegenerate \(n\times n\) bimatrix game has at most \(2^{n}-1\) Nash equilibria. We also provide an equilibrium selection criterion to two subclasses of games with TBP.     

Open Loop Saddle Point on Linear Quadratic Stochastic Differential Games

In this paper, we deal with one kind of two-player zero-sum linear quadratic stochastic differential game problem. We give the existence of an open loop saddle point if and only if the lower and upper values exist.     

Op en Lo op Saddle Point on Linear Quadratic Sto chastic Differential Games

In this paper, we deal with one kind of two-player zero-sum linear quadratic stochastic differential game problem. We give the existence of an open loop saddle point if and only if the lower and upper values exist.     

Zero-sum risk-sensitive stochastic games for continuous time Markov chains

We study infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled continuous time Markov chains on a countable state space. For the discounted-cost game, we prove the existence of value and saddle-point equilibrium in the class of Markov strategies under nominal conditions. For the ergodic-cost game, we prove the existence of values and saddle point equilibrium by studying the corresponding Hamilton-Jacobi-Isaacs equation under a certain Lyapunov condition.     

Two-player stochastic games I: A reduction

This paper is the first step in the proof of existence of equilibrium payoffs for two-player stochastic games with finite state and action sets. It reduces the existence problem to the class of so-called positive absorbing recursive games. The existence problem for this class is solved in a subsequent paper.     

连续区间策略下二人零和对策问题研究

考虑连续区间策略下的二人零和对策问题,研究其均衡策略的存在性。首先分析了完全信息下的二人零和对策问题,证明了该问题均衡策略的存在性并给出求解方法。然后进一步研究了收益函数不确定的不完全信息二人零和对策问题,在各局中人都认为对方是风险厌恶型的假设下,分析该类对策纯策略均衡的存在性,并通过研究纯策略均衡存在的充要条件给出判断并寻找纯策略均衡解的方法。最后给出一个数值算例,验证本文所提出方法的可行性。     

Equilibrium in a war of attrition with an option to fight decisively

We develop a symmetric incomplete-information continuous-time two-player war-of-attrition game with an option to fight decisively. We show that there exists an essentially unique symmetric Bayesian Nash equilibrium. Under equilibrium, the game does not end immediately, and a costly delay persists even with the availability of the fighting option that ends the game if chosen. In addition, there exists a critical time in which a fight occurs unless a player resigns before that time.     

Dynamic competition over social networks

We propose an analytical approach to the problem of influence maximization in a social network where two players compete by means of dynamic targeting strategies. We formulate the problem as a two-player zero-sum stochastic game. We prove the existence of the uniform value: if the players are sufficiently patient, both can guarantee the same mean-average opinion without knowing the exact length of the game. Furthermore, we put forward some elements for the characterization of equilibrium strategies. In general, players must implement a trade-off between a forward-looking perspective, according to which they aim to maximize the future spread of their opinion in the network, and a backward-looking perspective, according to which they aim to counteract their opponent’s previous actions. When the influence potential of players is small, we describe an equilibrium through a one-shot game based on eigenvector centrality.     

Sensitive equilibria for ergodic stochastic games with countable state spaces

We consider stochastic games with countable state spaces and unbounded immediate payoff functions. Our assumptions on the transition structure of the game are based on a recent work by Meyn and Tweedie [19] on computable bounds for geometric convergence rates of Markov chains. The main results in this paper concern the existence of sensitive optimal strategies in some classes of zero-sum stochastic games. By sensitive optimality we mean overtaking or 1-optimality. We also provide a new Nash equilibrium theorem for a class of ergodic nonzero-sum stochastic games with denumerable state spaces.