Stochastic games with non-observable actions

We examine n-player stochastic games. These are dynamic games where a play evolves in stages along a finite set of states; at each stage players independently have to choose actions in the present state and these choices determine a stage payoff to each player as well as a transition to a new state where actions have to be chosen at the next stage. For each player the infinite sequence of his stage payoffs is evaluated by taking the limiting average. Normally stochastic games are examined under the condition of full monitoring, i.e. at any stage each player observes the present state and the actions chosen by all players. This paper is a first attempt towards understanding under what circumstances equilibria could exist in n-player stochastic games without full monitoring. We demonstrate the non-existence of -equilibria in n-player stochastic games, with respect to the average reward, when at each stage each player is able to observe the present state, his own action, his own payoff, and the payoffs of the other players, but is unable to observe the actions of them. For this purpose, we present and examine a counterexample with 3 players. If we further drop the assumption that the players can observe the payoffs of the others, then counterexamples already exist in games with only 2 players.     

Zero-sum constrained stochastic games with independent state processes

We consider a zero-sum stochastic game with side constraints for both players with a special structure. There are two independent controlled Markov chains, one for each player. The transition probabilities of the chain associated with a player as well as the related side constraints depend only on the actions of the corresponding player; the side constraints also depend on the player’s controlled chain. The global cost that player 1 wishes to minimize and that player 2 wishes to maximize, depend however on the actions and Markov chains of both players. We obtain a linear programming formulations that allows to compute the value and saddle point policies for this problem. We illustrate the theoretical results through a zero-sum stochastic game in wireless networks in which each player has power constraints     

Constrained cost-coupled stochastic games with independent state processes

We study non-cooperative constrained stochastic games in which each player controls its own Markov chain based on its own state and actions. Interactions between players occur through their costs and constraints which depend on the state and actions of all players. We provide an example from wireless communications.     

Stochastic games without perfect monitoring

A two-person zero-sum stochastic game with finitely many states and actions is considered. The classical assumption of perfect monitoring is relaxed. Instead of being informed of the previous action of his opponent, each player receives a random signal, the law of which depending on both previous actions and on the previous state. We prove the existence of the max-min and dually of the min-max, thus extending both the result of Mertens-Neyman about the existence of the value in case of perfect monitoring and a theorem obtained by the author on a subclass of stochastic games: the absorbing games. It is a pleasure to thank J. Filar and V. Gaitsgory for their friendly hospitality at the School of Mathematics, University of South-Australia, where this work was initiated. There, the author was supported by a grant from the Australian Research Council no. A69703141. I would like also to thank S. Sorin for his keen interest and finally an anonymous referee for his careful reading of the proof.     

Fictitious play in stochastic games

In this paper we examine an extension of the fictitious play process for bimatrix games to stochastic games. We show that the fictitious play process does not necessarily converge, not even in the 2 × 2 × 2 case with a unique equilibrium in stationary strategies. Here 2 × 2 × 2 stands for 2 players, 2 states, 2 actions for each player in each state.     

The Value of Information Structures in Zero-sum Games with Lack of Information on One Side

Two players are engaged in a zero-sum game with lack of information on one side, in which player 1 (the informed player) receives some stochastic signal about the state of nature. I consider the value of the game as a function of player 1’s information structure, and study the properties of this function. It turns out that these properties reflect the fact that in zero sum situation the value of information for each player is positive.     

Egalitarian allocation and players of certain type

In cooperative game theory the Shapley value is different from the egalitarian value, the latter of which allocates payoffs equally. The null player property and the nullifying player property assign zero payoff to each null player and each nullifying player, respectively. It is known that if the null player property for characterizing the Shapley value is replaced by the nullifying player property, then the egalitarian value is determined uniquely. We propose several properties to replace the nullifying player property to characterize the egalitarian value. Roughly speaking, the results in this note hint that equal division for players of certain types may lead to the egalitarian allocation.     

Pure subgame-perfect equilibria in free transition games

We consider a class of stochastic games, where each state is identified with a player. At any moment during play, one of the players is called active. The active player can terminate the game, or he can announce any player, who then becomes the active player. There is a non-negative payoff for each player upon termination of the game, which depends only on the player who decided to terminate. We give a combinatorial proof of the existence of subgame-perfect equilibria in pure strategies for the games in our class.     

A BSDE approach to stochastic differential games with incomplete information

We consider a two-player zero-sum stochastic differential game in which one of the players has a private information on the game. Both players observe each other, so that the non-informed player can try to guess his missing information. Our aim is to quantify the amount of information the informed player has to reveal in order to play optimally: to do so, we show that the value function of this zero-sum game can be rewritten as a minimization problem over some martingale measures with a payoff given by the solution of a backward stochastic differential equation.     

Stochastic Discrete-Time Nash Games with Constrained State Estimators

In this paper, we consider stochastic linear-quadratic discrete-time Nash games in which two players have access only to noise-corrupted output measurements. We assume that each player is constrained to use a linear Kalman filter-like state estimator to implement his optimal strategies. Two information structures available to the players in their state estimators are investigated. The first has access to one-step delayed output and a one-step delayed control input of the player. The second has access to the current output and a one-step delayed control input of the player. In both cases, statistics of the process and statistics of the measurements of each player are known to both players. A simple example of a two-zone energy trading system is considered to illustrate the developed Nash strategies. In this example, the Nash strategies are calculated for the two cases of unlimited and limited transmission capacity constraints.     

Repeated proximity games

We consider repeated games with complete information and imperfect monitoring, where each player is assigned a fixed subset of players and only observes the moves chosen by the players in this subset. This structure is naturally represented by a directed graph. We prove that a generalized folk theorem holds for any payoff function if and only if the graph is 2-connected, and then extend this result to the context of finitely repeated games. Received June 1997/Revised version March 1998     

The modified stochastic game

We present a new tool for the study of multiplayer stochastic games, namely the modified game, which is a normal-form game that depends on the discount factor, the initial state, and for every player a partition of the set of states and a vector that assigns a real number to each element of the partition. We study properties of the modified game, like its equilibria, min–max value, and max–min value. We then show how this tool can be used to prove the existence of a uniform equilibrium in a certain class of multiplayer stochastic games.     

Layers of noncooperative games

We model and analyze classes of antagonistic stochastic games of two players. The actions of the players are formalized by marked point processes recording the cumulative damage to the players at any moment of time. The processes evolve until one of the processes crosses its fixed preassigned threshold of tolerance. Once the threshold is reached or exceeded at some point of the time (exit time), the associated player is ruined. Both stochastic processes are being “observed” by a third party point stochastic process, over which the information regarding the status of both players is obtained. We succeed in these goals by arriving at closed form joint functionals of the named elements and processes. Furthermore, we also look into the game more closely by introducing an intermediate threshold (see a layer), which a losing player is to cross prior to his ruin, in order to analyze the game more scrupulously and see what makes the player lose the game.     

Multilayers in a modulated stochastic game

We are concerned with an antagonistic stochastic game between two players A and B which finds applications in economics and warfare. The actions of the players are manifested by a series of strikes of random magnitudes at random times exerted by each player against his opponent. Each of the assaults inflicts a random damage to enemy's vital areas. In contrast with traditional games, in our setting, each player can endure multiple strikes before perishing. Predicting the ruin time (exit) of player A, along with the total amount of casualties to both players at the exit is a main objective of this work. In contrast to the time sensitive analysis (earlier developed to refine the information on the game) we insert auxiliary control levels, which both players will cross in due game before the ruin of A. This gives A (and also B) an additional opportunity to reevaluate his strategy and change the course of the game. We formalize such a game and also allow the real time information about the game to be randomly delayed. The delayed exit time, cumulative casualties to both players, and prior crossings are all obtained in a closed-form joint functional.     

Layers of noncooperative games

We model and analyze classes of antagonistic stochastic games of two players. The actions of the players are formalized by marked point processes recording the cumulative damage to the players at any moment of time. The processes evolve until one of the processes crosses its fixed preassigned threshold of tolerance. Once the threshold is reached or exceeded at some point of the time (exit time), the associated player is ruined. Both stochastic processes are being “observed” by a third party point stochastic process, over which the information regarding the status of both players is obtained. We succeed in these goals by arriving at closed form joint functionals of the named elements and processes. Furthermore, we also look into the game more closely by introducing an intermediate threshold (see a layer), which a losing player is to cross prior to his ruin, in order to analyze the game more scrupulously and see what makes the player lose the game.     

A link between sequential semi-anonymous nonatomic games and their large finite counterparts

We show that obtainable equilibria of a multi-period nonatomic game can be used by players in its large finite counterparts to achieve near-equilibrium payoffs. Such equilibria in the form of random state-to-action rules are parsimonious in form and easy to execute, as they are both oblivious of past history and blind to other players’ present states. Our transient results can be extended to a stationary case, where the finite multi-period games are special discounted stochastic games. In both nonatomic and finite games, players’ states influence their payoffs along with actions they take; also, the random evolution of one particular player’s state is driven by all players’ states as well as actions. The finite games can model diverse situations such as dynamic price competition. But they are notoriously difficult to analyze. Our results thus suggest ways to tackle these problems approximately.     

Algorithms for discounted stochastic games

In this paper, a two-person zero-sum discounted stochastic game with a finite state space is considered. The movement of the game from state to state is jointly controlled by the two players with a finite number of alternatives available to each player in each of the states. We present two convergent algorithms for arriving at minimax strategies for the players and the value of the game. The two algorithms are compared with respect to computational efficiency. Finally, a possible extension to nonzero sum stochastic game is suggested.This research was supported in part by funds allocated to the Department of Operations Research, School of Management, Case Western Reserve University under Contract No. DAHC 19-68-C-0007 (Project Themis) with the U.S. Army Research Office, Durham, North Carolina. The authors thank the referees for their valuable suggestions.     

Zero-Sum Stochastic Games with Partial Information

We study a zero-sum stochastic game on a Borel state space where the state of the game is not known to the players. Both players take their decisions based on an observation process. We transform this into an equivalent problem with complete information. Then, we establish the existence of a value and optimal strategies for both players.     

Emergence and nonemergence of alternating offers in bilateral bargaining

We study in what circumstance players alternate offers in bilateral bargaining. To examine this question, we suppose that players choose whether to take the initiative in each period. The player who tries to take the initiative is able to make an offer only when the other player does not. The probability that a player tries to take the initiative is referred to as the frequency of initiative taking. We assume that this is conditioned on mutually observable states and is, once chosen, unchangeable. When players make their frequency of initiative taking dependent on the identity of the latest proposer, the players alternate their offers (possibly with some stochastic delay). In contrast, when players always use the same frequency of initiative taking, or when players only distinguish odd-numbered from even-numbered periods for the frequency of initiative taking, both players constantly try to take the initiative. Consequently, an impasse arises.     

基于时隙ALOHA协议的数据传输二人随机博弈模型

在一个给定的拓扑网络中研究关于数据传输的二人随机博弈模型.两个局中人（源节点）试图通过一个公共节点向目的节点传输随机数据包,这些数据包被分为重要的数据包和不重要的数据包两类,假设每个局中人都有一个用于存储数据包的有限容量的缓冲器.通过构造数据传输的成本分摊和奖励体系,把这种动态的冲突控制过程建模为具有有限状态集合的随机博弈,研究局中人在这种随机博弈模型下的非合作以及合作行为.在非合作情形下,给出纳什均衡的求解算法;在合作情形下,选择Shapley值作为局中人支付总和的分配方案,并讨论其子博弈一致性,提出使得Shapley值为子博弈一致的分配补偿程序.