Remarks on sensitive equilibria in stochastic games with additive reward and transition structure

A class of stochastic games with additive reward and transition structure is studied. For zero-sum games under some ergodicity assumptions 1-equilibria are shown to exist. They correspond to so-called sensitive optimal policies in dynamic programming. For a class of nonzero-sum stochastic games with nonatomic transitions nonrandomized Nash equilibrium points with respect to the average payoff criterion are also obtained. Included examples show that the results of this paper can not be extented to more general payoff or transition structure.     

An operator solution of stochastic games

A class of zero-sum, two-person stochastic games is shown to have a value which can be calculated by transfinite iteration of an operator. The games considered have a countable state space, finite action spaces for each player, and a payoff sufficiently general to include classical stochastic games as well as Blackwell’s infiniteG _δ games of imperfect information. Research supported by National Science Foundation Grants DMS-8801085 and DMS-8911548.     

Zero-Sum Ergodic Semi-Markov Games with Weakly Continuous Transition Probabilities

Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities and lower semicontinuous, possibly unbounded, payoff functions are studied. Two payoff criteria are considered: the ratio average and the time average. The main result concerns the existence of a lower semicontinuous solution to the optimality equation and its proof is based on a fixed-point argument. Moreover, it is shown that the ratio average as well as the time average payoff stochastic games have the same value. In addition, one player possesses an ε-optimal stationary strategy (ε>0), whereas the other has an optimal stationary strategy. A. Jaśkiewicz is on leave from Institute of Mathematics and Computer Science, Wrocław University of Technology. This work is supported by MNiSW Grant 1 P03A 01030.     

Multicriteria Goal Games

In this paper, we deal with multicriteria matrix games. Different solution concepts have been proposed to cope with these games. Recently, the concept of Pareto-optimal security strategy which assures the property of security in the individual criteria against an opponent's deviation in strategy has been introduced. However, the idea of security behind this concept is based on expected values, so that this security might be violated by mixed strategies when replications are not allowed. To avoid this inconvenience, we propose in this paper a new concept of solution for these games: the G-goal security strategy, which includes as part of the solution the probability of obtaining prespecified values in the payoff functions. Thus, attitude toward risk together with payoff values are considered jointly in the solution analysis.     

Characterization of optimal strategies in matrix games with convexity properties

This paper gives a full characterization of matrices with rows and columns having properties closely related to the (quasi-) convexity-concavity of functions. The matrix games described by such payoff matrices well approximate continuous games on the unit square with payoff functions F (x, y) concave in x for each y, and convex in y for each x. It is shown that the optimal strategies in such matrix games have a very simple structure and a search-procedure is given. The results have a very close relationship with the known theorem of Debreu and Glicksberg about the existence of a pure Nash equilibrium in n-person games. Received: May 1997/Final version: August 1999     

Convexity properties for interior operator games

Interior operator games arose by abstracting some properties of several types of cooperative games (for instance: peer group games, big boss games, clan games and information market games). This reason allow us to focus on different problems in the same way. We introduced these games in Bilbao et al. (Ann. Oper. Res. 137:141–160, 2005) by a set system with structure of antimatroid, that determines the feasible coalitions, and a non-negative vector, that represents a payoff distribution over the players. These games, in general, are not convex games. The main goal of this paper is to study under which conditions an interior operator game verifies other convexity properties: 1-convexity, k-convexity (k≥2 ) or semiconvexity. But, we will study these properties over structures more general than antimatroids: the interior operator structures. In every case, several characterizations in terms of the gap function and the initial vector are obtained. We also find the family of interior operator structures (particularly antimatroids) where every interior operator game satisfies one of these properties.     

Games with finite resources

Games with Finite Resources as defined by Gale (1957) are two-person zero-sum N-stage games in which each player has N resources and may use each resource once and only once in the N stages. Gale's theorem on these games is generalized in several directions. First the payoff is allowed to be any symmetric function of the stage payoffs. Second, the players are allowed some latitude in choosing which game is being played. Applications are given to some open questions in the area of Inspection Games. Finally the payoff is allowed to be random, thus incorporating a result of Ross (1972) on Goofspiel. Application is made to a game-theoretic version of the Generalized House Selling Problem. Received August 1999/revised version March 2000     

Potential theory for mean payoff games

The paper presents an O(mn2ⁿ log Z) deterministic algorithm for solving the mean payoff game problem, m and n being the numbers of arcs and vertices, respectively, in the game graph, and Z being the maximum weight (the weights are assumed to be integers). The theoretical basis for the algorithm is the potential theory for mean payoff games. This theory allows one to restate the problem in terms of solving systems of algebraic equations with minima and maxima. Also, in order to solve the mean payoff game problem, the arc reweighting technique is used. To this end, simple modifications, which do not change the set of winning strategies, are applied to the game graph; in the end, a trivial instance of the problem is obtained. It is shown that any game graph can be simplified by n reweightings. Bibliography: 16 titles. __________ Translated from Zapiski Nauchnykh Seminarov POMI, Vol. 340, 2006, pp. 61–75.     

Probabilities of Pure Nash Equilibria in Matrix Games when the Payoff Entries of One Player Are Randomly Selected

The Nash equilibrium in pure strategies represents an important solution concept in nonzero sum matrix games. Existence of Nash equilibria in games with known and with randomly selected payoff entries have been studied extensively. In many real games, however, a player may know his own payoff entries but not the payoff entries of the other player. In this paper, we consider nonzero sum matrix games where the payoff entries of one player are known, but the payoff entries of the other player are assumed to be randomly selected. We are interested in determining the probabilities of existence of pure Nash equilibria in such games. We characterize these probabilities by first determining the finite space of ordinal matrix games that corresponds to the infinite space of matrix games with random entries for only one player. We then partition this space into mutually exclusive spaces that correspond to games with no Nash equilibria and with r Nash equilibria. In order to effectively compute the sizes of these spaces, we introduce the concept of top-rated preferences minimal ordinal games. We then present a theorem which provides a mechanism for computing the number of games in each of these mutually exclusive spaces, which then can be used to determine the probabilities. Finally, we summarize the results by deriving the probabilities of existence of unique, nonunique, and no Nash equilibria, and we present an illustrative example.     

Cone Concavity and Multiple-Payoff Constrained n-Person Games

This paper investigates special cases of abstract economies, i.e., n-person games with multiple payoff functions. Dominances with certain convex cones and interactive strategies are introduced in such game settings. Gradients of payoff functions are involved to establish certain Lagrange or Kuhn–Tucker conditions which may lead to some algorithms to actually compute an equilibrium. Sufficient and necessary conditions for such multiple payoff constrained n-person games are obtained.     

Independent mistakes in large games

Economic models usually assume that agents play precise best responses to others' actions. It is sometimes argued that this is a good approximation when there are many agents in the game, because if their mistakes are independent, aggregate uncertainty is small. We study a class of games in which players' payoffs depend solely on their individual actions and on the aggregate of all players' actions. We investigate whether their equilibria are affected by mistakes when the number of players becomes large. Indeed, in generic games with continuous payoff functions, independent mistakes wash out in the limit. This may not be the case if payoffs are discontinuous. As a counter-example we present the n players Nash bargaining game, as well as a large class of “free-rider games.” Received: November 1997/Final version: December 1999     

S-modular games,with queueing applications

The notion ofS-modularity was developed by Glasserman and Yao [9] in the context of optimal control of queueing networks.S-modularity allows the objective function to be supermodular in some variables and submodular in others. It models both compatible and conflicting incentives, and hence conveniently accommodates a wide variety of applications. In this paper, we introduceS-modularity into the context ofn-player noncooperative games. This generalizes the well-known supermodular games of Topkis [22], where each player maximizes a supermodular payoff function (or equivalently, minimizes a submodular payoff function). We illustrate the theory through a variety of applications in queueing systems.Supported in part by NSF Grant MSS-92-16490, and by Columbia's Center for Telecommunications Research.     

Sequentially compatible payoffs and the core in TU-games

In the context of cooperative TU-games, we introduce a recursive procedure to distribute the surplus of cooperation when there is an exogenous ordering among the set of players N. In each step of the process, using a given notion of reduced games, an upper and a lower bound for the payoff to the player at issue are required. Sequentially compatible payoffs are defined as those allocation vectors that meet these recursive bounds. For a family of reduction operations, the behavior of this new solution concept is analyzed. For any ordering of N, the core of the game turns out to be the set of sequentially compatible payoffs when the Davis–Maschler reduced games are used. Finally, we study which reduction operations give an advantage to the first player in the ordering.     

Quitting games – An example

Quitting games are multi-player sequential games in which, at every stage, each player has the choice between continuing and quitting. The game ends as soon as at least one player chooses to quit; each player i then receives a payoff r _S ⁱ, which depends on the set S of players that did choose to quit. If the game never ends, the payoff to each player is zero.? We exhibit a four-player quitting game, where the “simplest” equilibrium is periodic with period two. We argue that this implies that all known methods to prove existence of an equilibrium payoff in multi-player stochastic games are therefore bound to fail in general, and provide some geometric intuition for this phenomenon. Received: October 2001     

基于图对策的收益分配

基于具有交流结构的合作对策,即图对策,对平均树解拓展形式的特征进行刻画,提出此解满足可加性公理。进一步地,分析了对于无圈图对策此解是分支有效的。并且当连通分支中两个局中人相关联的边删掉后,此连通分支的收益变化情况可用平均树解表示。这一性质是Shapley值和Myerson值所不具有的。最后,我们给出了模糊联盟图对策中模糊平均树解的可加性和分支有效性。     

A New Condition and Approach for Zero-Sum Stochastic Games with Average Payoffs

Abstract

This article deals with discrete-time two-person zero-sum stochastic games with Borel state and action spaces. The optimality criterion to be studied is the long-run expected average payoff criterion, and the (immediate) payoff function may have neither upper nor lower bounds. We first replace the optimality equation widely used in the previous literature with two so-called optimality inequalities, and give a new set of conditions for the existence of solutions to the optimality inequalities. Then, from the optimality inequalities we ensure the existence of a pair of average optimal stationary strategies. Our new condition is slightly weaker than those in the previous literature, and as a byproduct some interesting results such as the convergence of a value iteration scheme to the value of the discounted payoff game is obtained. Finally, we first apply the main results in this article to generalized inventory systems, and then further provide an example of controlled population processes for which all of our conditions are satisfied, while some of conditions in some of previous literature fail to hold.     

Stochastic games with non-observable actions

We examine n-player stochastic games. These are dynamic games where a play evolves in stages along a finite set of states; at each stage players independently have to choose actions in the present state and these choices determine a stage payoff to each player as well as a transition to a new state where actions have to be chosen at the next stage. For each player the infinite sequence of his stage payoffs is evaluated by taking the limiting average. Normally stochastic games are examined under the condition of full monitoring, i.e. at any stage each player observes the present state and the actions chosen by all players. This paper is a first attempt towards understanding under what circumstances equilibria could exist in n-player stochastic games without full monitoring. We demonstrate the non-existence of -equilibria in n-player stochastic games, with respect to the average reward, when at each stage each player is able to observe the present state, his own action, his own payoff, and the payoffs of the other players, but is unable to observe the actions of them. For this purpose, we present and examine a counterexample with 3 players. If we further drop the assumption that the players can observe the payoffs of the others, then counterexamples already exist in games with only 2 players.     

An operator approach to zero-sum repeated games

We consider two person zero-sum stochastic games. The recursive formula for the valuesvλ (resp.v _n) of the discounted (resp. finitely repeated) version can be written in terms of a single basic operator Φ(α,f) where α is the weight on the present payoff andf the future payoff. We give sufficient conditions in terms of Φ(α,f) and its derivative at 0 for limv _n and limvλ to exist and to be equal. We apply these results to obtain such convergence properties for absorbing games with compact action spaces and incomplete information games.     

The nucleolus and kernel for simple games or special valid inequalities for 0–1 linear integer programs

An equivalence between simplen-person cooperative games and linear integer programs in 0–1 variables is presented and in particular the nucleolus and kernel are shown to be special valid inequalities of the corresponding 0–1 program. In the special case of weighted majority games, corresponding to knapsack inequalities, we show a further class of games for which the nucleolus is a representation of the game, and develop a single test to show when payoff vectors giving identical amounts or zero to each player are in the kernel. Finally we give an algorithm for computing the nucleolus which has been used successfully on weighted majority games with over twenty players.