On noncooperative hybrid stochastic games

The present article models and analyzes a noncooperative hybrid stochastic game of two players. The main phase (prime hybrid mode) of the game is preceded by “unprovoked” hostile actions by one of the players (during antecedent hybrid mode) that at some time transforms into a large scale conflict between two players. The game lasts until one of the players gets ruined. The latter occurs when the cumulative damage to the losing player exceeds a fixed threshold. Both hybrid modes are formalized by marked point stochastic processes and the theory of fluctuations is utilized as one of the chief techniques to arrive at a closed form functional describing the status of both players at the ruin time.     

Nonzero-sum stochastic games with unbounded costs: Discounted and average cost cases

We treat non-cooperative stochastic games with countable state space and with finitely many players each having finitely many moves available in a given state. As a function of the current state and move vector, each player incurs a nonnegative cost. Assumptions are given for the expected discounted cost game to have a Nash equilibrium randomized stationary strategy. These conditions hold for bounded costs, thereby generalizing Parthasarathy (1973) and Federgruen (1978). Assumptions are given for the long-run average expected cost game to have a Nash equilibrium randomized stationary strategy, under which each player has constant average cost. A flow control example illustrates the results. This paper complements the treatment of the zero-sum case in Sennott (1993a).     

Zero-sum stochastic games with unbounded costs: Discounted and average cost cases

Zero-sum stochastic games with countable state space and with finitely many moves available to each player in a given state are treated. As a function of the current state and the moves chosen, player I incurs a nonnegative cost and player II receives this as a reward. For both the discounted and average cost cases, assumptions are given for the game to have a finite value and for the existence of an optimal randomized stationary strategy pair. In the average cost case, the assumptions generalize those given in Sennott (1993) for the case of a Markov decision chain. Theorems of Hoffman and Karp (1966) and Nowak (1992) are obtained as corollaries. Sufficient conditions are given for the assumptions to hold. A flow control example illustrates the results.     

Cores in noncooperative games

Noncooperative games in normal form and in characteristic function form are considered. The supergame of the noncooperative game is defined as an infinite sequence of plays of the original game. The notions of strong Pareto equilibrium point (s.p.e.p.) and essential core are introduced. A relationship between the essential core of a noncooperative game and the set of s.p.e.p. of its supergame is asserted. This result is similar to that ofAumann for cooperative games without side payments.     

Layers of noncooperative games

We model and analyze classes of antagonistic stochastic games of two players. The actions of the players are formalized by marked point processes recording the cumulative damage to the players at any moment of time. The processes evolve until one of the processes crosses its fixed preassigned threshold of tolerance. Once the threshold is reached or exceeded at some point of the time (exit time), the associated player is ruined. Both stochastic processes are being “observed” by a third party point stochastic process, over which the information regarding the status of both players is obtained. We succeed in these goals by arriving at closed form joint functionals of the named elements and processes. Furthermore, we also look into the game more closely by introducing an intermediate threshold (see a layer), which a losing player is to cross prior to his ruin, in order to analyze the game more scrupulously and see what makes the player lose the game.     

Layers of noncooperative games

We model and analyze classes of antagonistic stochastic games of two players. The actions of the players are formalized by marked point processes recording the cumulative damage to the players at any moment of time. The processes evolve until one of the processes crosses its fixed preassigned threshold of tolerance. Once the threshold is reached or exceeded at some point of the time (exit time), the associated player is ruined. Both stochastic processes are being “observed” by a third party point stochastic process, over which the information regarding the status of both players is obtained. We succeed in these goals by arriving at closed form joint functionals of the named elements and processes. Furthermore, we also look into the game more closely by introducing an intermediate threshold (see a layer), which a losing player is to cross prior to his ruin, in order to analyze the game more scrupulously and see what makes the player lose the game.     

Nonrandomized strategy equilibria in noncooperative stochastic games with additive transition and reward structure

In this paper, we study a discounted noncooperative stochastic game with an abstract measurable state space, compact metric action spaces of players, and additive transition and reward structure in the sense of Himmelberget al. (Ref. 1) and Parthasarathy (Ref. 2). We also assume that the transition law of the game is absolutely continuous with respect to some probability distributionp of the initial state and together with the reward functions of players satisfies certain continuity conditions. We prove that such a game has an equilibrium stationary point, which extends a result of Parthasarathy from Ref. 2, where the action spaces of players are assumed to be finite sets. Moreover, we show that our game has a nonrandomized (- )-equilibrium stationary point for each >0, provided that the probability distributionp is nonatomic. The latter result is a new existence theorem.     

Reductions of interval noncooperative games

Noncooperative games of a finite number of persons with interval-valued payoff functions are considered. The concept of an equilibrium situation is introduced. A reduction of such games to deterministic noncooperative games is proposed. Properties of the reduced games are discussed. Interval antagonistic and bimatrix games are examined, and illustrative examples are considered.     

A solution for noncooperative games

In this paper, we study solutions of strict noncooperative games that are played just once. The players are not allowed to communicate with each other. The main ingredient of our theory is the concept of rationalizing a set of strategies for each player of a game. We state an axiom based on this concept that every solution of a noncooperative game is required to satisfy. Strong Nash solvability is shown to be a sufficient condition for the rationalizing set to exist, but it is not necessary. Also, Nash solvability is neither necessary nor sufficient for the existence of the rationalizing set of a game. For a game with no solution (in our sense), a player is assumed to recourse to a standard of behavior. Some standards of behavior are examined and discussed.This work was sponsored by the United States Army under Contract No. DAAG29-75-C-0024 and by the National Science Foundation under Grant No. MCS-75-17385-A01. The author is grateful to J. C. Harsanyi for his comments and to S. M. Robinson for suggesting the problem.     

Infinite horizon noncooperative differential games

For a noncooperative differential game, the value functions of the various players satisfy a system of Hamilton-Jacobi equations. In the present paper, we consider a class of infinite horizon games with nonlinear costs exponentially discounted in time. By the analysis of the value functions, we establish the existence of Nash equilibrium solutions in feedback form and provide results and counterexamples on their uniqueness and stability.     

Complete stability of noncooperative games

This paper is concerned with a class of noncooperative games ofn players that are defined byn reward functions which depend continuously on the action variables of the players. This framework provides a realistic model of many interactive situations, including many common models in economics, sociology, engineering, and political science. The concept of Nash equilibrium is a suitable companion to such models.A variety of different sufficient conditions for existence, uniqueness, and stability of a Nash equilibrium point have been previously proposed. By sharpening the noncooperative aspect of the framework (which is really only implicit in the original framework), this paper attempts to isolate one set of natural conditions that are sufficient for existence, uniqueness, and stability. It is argued thatl quasicontraction is such a natural condition. The concept of complete stability is introduced to reflect the full character of noncooperation. It is then shown that, in the linear case, the condition ofl quasicontraction is both necessary and sufficient for complete stability.This research was supported by the Air Force Office of Scientific Research under Grant No. AFOSR 77-3141 and by the National Science Foundation under Grant No. GK18748.     

Existence of equilibrium stationary strategies in discounted noncooperative stochastic games with uncountable state space

This paper considers discounted noncooperative stochastic games with uncountable state space and compact metric action spaces. We assume that the transition law is absolutely continuous with respect to some probability measure defined on the state space. We prove, under certain additional continuity and integrability conditions, that such games have -equilibrium stationary strategies for each >0. To prove this fact, we provide a method for approximating the original game by a sequence of finite or countable state games. The main result of this paper answers partially a question raised by Parthasarathy in Ref. 1.     

A solution concept forn-person noncooperative games

The paper describes a solution concept forn-person noncooperative games, developed jointly by the author and Reinhard Selten. Its purpose is to select one specific perfect equilibrium points=s (G) as the solution of any given noncooperative gameG. The solution is constructed by an inductive procedure. In defining the solutions (G) of gameG, we use the solutionss (G ^*) of the component gamesG ^* (if any) ofG; and in defining the solutions (G^*) of any such component gameG ^*, we use the solutionss (G ^**) of its own component gamesG ^** (if any), etc. This inductive procedure is well-defined because it always comes to an end after a finite number of steps. At each level, the solution of a game (or of a component game) is defined in two steps. First, aprior subjectiveprobability distribution p _i is assigned to the pure strategies of each playeri, meant to represent the other players' initial expectations about playeri's likely strategy choice. Then, a mathematical procedure, called thetracing procedure, is used to define the solution on the basis of these prior probability distributionsp _i . The tracing procedure is meant to provide a mathematical representation for thesolution process by which rational players manage to coordinate their strategy plans and their expectations, and make them converge to one specific equilibrium point as solution for the game     

Distributed consensus in noncooperative inventory games

This paper deals with repeated nonsymmetric congestion games in which the players cannot observe their payoffs at each stage. Examples of applications come from sharing facilities by multiple users. We show that these games present a unique Pareto optimal Nash equilibrium that dominates all other Nash equilibria and consequently it is also the social optimum among all equilibria, as it minimizes the sum of all the players’ costs. We assume that the players adopt a best response strategy. At each stage, they construct their belief concerning others probable behavior, and then, simultaneously make a decision by optimizing their payoff based on their beliefs. Within this context, we provide a consensus protocol that allows the convergence of the players’ strategies to the Pareto optimal Nash equilibrium. The protocol allows each player to construct its belief by exchanging only some aggregate but sufficient information with a restricted number of neighbor players. Such a networked information structure has the advantages of being scalable to systems with a large number of players and of reducing each player’s data exposure to the competitors.     

Stochastic dominance equilibria in two-person noncooperative games

Two-person noncooperative games with finitely many pure strategies are considered, in which the players have linear orderings over sure outcomes but incomplete preferences over probability distributions resulting from mixed strategies. These probability distributions are evaluated according to t-degree stochastic dominance. A t-best reply is a strategy that induces a t-degree stochastically undominated distribution, and a t-equilibrium is a pair of t-best replies. The paper provides a characterization and an existence proof of t-equilibria in terms of representing utility functions, and shows that for large t behavior converges to a form of max–min play. Specifically, increased aversion to bad outcomes makes each player put all weight on a strategy that maximizes the worst outcome for the opponent, within the supports of the strategies in the limiting sequence of t-equilibria.The paper has benefitted from the comments of four referees and an associate editor.     

On stochastic games,II

In this paper, we consider positive stochastic games, when the state and action spaces are all infinite. We prove that, under certain conditions, the positive stochastic game has a value and that the maximizing player has an -optimal stationary strategy and the minimizing player has an optimal stationary strategy.The authors are grateful to Professor David Blackwell and the referee for some useful comments.     

Nonzero-sum differential games (the noncooperative variant)

One presents the fundamental results of the theory of noncooperative differential games: necessary and sufficient equilibrium conditions, existence theorems, properties of equilibrium solutions, numerical methods. One indicates applications to concrete problems in economics, the mechanics of controlled motions, and to strategic games.Translated from Itogi Nauki i Tekhniki, Matematicheskii Analiz, Vol. 15, pp. 199–266, 1977.The authors are grateful to É. M. Vaisbord, A. F. Koaonenko, V. S. Molostvov, V. P. Patsyukov, V. A. Plotnikov, V. V. Podinovskii, R. A. Polyak, and R. T. Yanushevskii for the problems in Sec. 11 and for the remarks which have been taken into account in the final editing of the survey.