Nonzero-sum stochastic games with unbounded costs: Discounted and average cost cases

We treat non-cooperative stochastic games with countable state space and with finitely many players each having finitely many moves available in a given state. As a function of the current state and move vector, each player incurs a nonnegative cost. Assumptions are given for the expected discounted cost game to have a Nash equilibrium randomized stationary strategy. These conditions hold for bounded costs, thereby generalizing Parthasarathy (1973) and Federgruen (1978). Assumptions are given for the long-run average expected cost game to have a Nash equilibrium randomized stationary strategy, under which each player has constant average cost. A flow control example illustrates the results. This paper complements the treatment of the zero-sum case in Sennott (1993a).     

A Penalty Approach for Generalized Nash Equilibrium Problem

The generalized Nash equilibrium problem (GNEP) is a generalization of the standard Nash equilibrium problem (NEP),in which both the utility function and the strategy space of each player depend on the strategies chosen by all other players.This problem has been used to model various problems in applications.However,the convergent solution algorithms are extremely scare in the literature.In this paper,we present an incremental penalty method for the GNEP,and show that a solution of the GNEP can be found by solving a sequence of smooth NEPs.We then apply the semismooth Newton method with Armijo line search to solve latter problems and provide some results of numerical experiments to illustrate the proposed approach.     

Limiting distributions of the number of pure strategy Nash equilibria in n-person games

We study the number of pure strategy Nash equilibria in a “random” n-person non-cooperative game in which all players have a countable number of strategies. We consider both the cases where all players have strictly and weakly ordinal preferences over their outcomes. For both cases, we show that the distribution of the number of pure strategy Nash equilibria approaches the Poisson distribution with mean 1 as the numbers of strategies of two or more players go to infinity. We also find, for each case, the distribution of the number of pure strategy Nash equilibria when the number of strategies of one player goes to infinity, while those of the other players remain finite.     

A solution for noncooperative games

In this paper, we study solutions of strict noncooperative games that are played just once. The players are not allowed to communicate with each other. The main ingredient of our theory is the concept of rationalizing a set of strategies for each player of a game. We state an axiom based on this concept that every solution of a noncooperative game is required to satisfy. Strong Nash solvability is shown to be a sufficient condition for the rationalizing set to exist, but it is not necessary. Also, Nash solvability is neither necessary nor sufficient for the existence of the rationalizing set of a game. For a game with no solution (in our sense), a player is assumed to recourse to a standard of behavior. Some standards of behavior are examined and discussed.This work was sponsored by the United States Army under Contract No. DAAG29-75-C-0024 and by the National Science Foundation under Grant No. MCS-75-17385-A01. The author is grateful to J. C. Harsanyi for his comments and to S. M. Robinson for suggesting the problem.     

Probabilities of Pure Nash Equilibria in Matrix Games when the Payoff Entries of One Player Are Randomly Selected

The Nash equilibrium in pure strategies represents an important solution concept in nonzero sum matrix games. Existence of Nash equilibria in games with known and with randomly selected payoff entries have been studied extensively. In many real games, however, a player may know his own payoff entries but not the payoff entries of the other player. In this paper, we consider nonzero sum matrix games where the payoff entries of one player are known, but the payoff entries of the other player are assumed to be randomly selected. We are interested in determining the probabilities of existence of pure Nash equilibria in such games. We characterize these probabilities by first determining the finite space of ordinal matrix games that corresponds to the infinite space of matrix games with random entries for only one player. We then partition this space into mutually exclusive spaces that correspond to games with no Nash equilibria and with r Nash equilibria. In order to effectively compute the sizes of these spaces, we introduce the concept of top-rated preferences minimal ordinal games. We then present a theorem which provides a mechanism for computing the number of games in each of these mutually exclusive spaces, which then can be used to determine the probabilities. Finally, we summarize the results by deriving the probabilities of existence of unique, nonunique, and no Nash equilibria, and we present an illustrative example.     

Advertising coordination games of a manufacturer and a retailer while introducing a new product

Two kinds of vertical cooperative advertising program are considered in a distribution channel constituted by a manufacturer and a retailer, where the manufacturer pays part of the retailer’s advertising costs. In the first participation scheme, the manufacturer chooses his/her advertising participation rate in the retailer’s advertising effort and then each player determines the advertising effort that maximizes his/her profit. In the second scheme, the retailer chooses the manufacturer’s participation rate and then the manufacturer determines the advertising efforts of both players with the objective of maximizing the manufacturer’s profit. Each participation scheme corresponds to a special Stackelberg game: the manufacturer is the leader of the first, while the retailer is the leader of the second. The Stackelberg equilibrium advertising efforts and participation rate in both games are provided. Then the equilibrium strategies of the two players in the analyzed scenarios are compared with the Nash equilibrium in the competitive framework. Finally, the conditions which suggest a special kind of agreement to a player are analyzed. This work was supported by the Italian Ministry of University and Research and the University of Padua.     

Nash Equilibrium Seeking in Quadratic Noncooperative Games Under Two Delayed Information-Sharing Schemes

In this paper, we propose non-model-based strategies for locally stable convergence to Nash equilibrium in quadratic noncooperative games where acquisition of information (of two different types) incurs delays. Two sets of results are introduced: (a) one, which we call cooperative scenario, where each player employs the knowledge of the functional form of his payoff and knowledge of other players’ actions, but with delays; and (b) the second one, which we term the noncooperative scenario, where the players have access only to their own payoff values, again with delay. Both approaches are based on the extremum seeking perspective, which has previously been reported for real-time optimization problems by exploring sinusoidal excitation signals to estimate the Gradient (first derivative) and Hessian (second derivative) of unknown quadratic functions. In order to compensate distinct delays in the inputs of the players, we have employed predictor feedback. We apply a small-gain analysis as well as averaging theory in infinite dimensions, due to the infinite-dimensional state of the time delays, in order to obtain local convergence results for the unknown quadratic payoffs to a small neighborhood of the Nash equilibrium. We quantify the size of these residual sets and corroborate the theoretical results numerically on an example of a two-player game with delays.

     

An improved two-step method for solving generalized Nash equilibrium problems

The generalized Nash equilibrium problem (GNEP) is a noncooperative game in which the strategy set of each player, as well as his payoff function, depend on the rival players strategies. As a generalization of the standard Nash equilibrium problem (NEP), the GNEP has recently drawn much attention due to its capability of modeling a number of interesting conflict situations in, for example, an electricity market and an international pollution control. In this paper, we propose an improved two-step (a prediction step and a correction step) method for solving the quasi-variational inequality (QVI) formulation of the GNEP. Per iteration, we first do a projection onto the feasible set defined by the current iterate (prediction) to get a trial point; then, we perform another projection step (correction) to obtain the new iterate. Under certain assumptions, we prove the global convergence of the new algorithm. We also present some numerical results to illustrate the ability of our method, which indicate that our method outperforms the most recent projection-like methods of Zhang et al. (2010).     

Constrained cost-coupled stochastic games with independent state processes

We study non-cooperative constrained stochastic games in which each player controls its own Markov chain based on its own state and actions. Interactions between players occur through their costs and constraints which depend on the state and actions of all players. We provide an example from wireless communications.     

Iterative computation of noncooperative equilibria in nonzero-sum differential games with weakly coupled players

We study the Nash equilibria of a class of two-person nonlinear, deterministic differential games where the players are weakly coupled through the state equation and their objective functionals. The weak coupling is characterized in terms of a small perturbation parameter . With =0, the problem decomposes into two independent standard optimal control problems, while for 0, even though it is possible to derive the necessary and sufficient conditions to be satisfied by a Nash equilibrium solution, it is not always possible to construct such a solution. In this paper, we develop an iterative scheme to obtain an approximate Nash solution when lies in a small interval around zero. Further, after requiring strong time consistency and/or robustness of the Nash equilibrium solution when at least one of the players uses dynamic information, we address the issues of existence and uniqueness of these solutions for the cases when both players use the same information, either closed loop or open loop, and when one player uses open-loop information and the other player uses closed-loop information. We also show that, even though the original problem is nonlinear, the higher (than zero) order terms in the Nash equilibria can be obtained as solutions to LQ optimal control problems or static quadratic optimization problems.This research was supported in part by the US Department of Energy under Grant DE-FG-02-88-ER-13939.Paper presented at the 29th IEEE Conference on Decision and Control, Honolulu, Hawaii, 1990.     

A dynamic solution in N-person cooperative game theory

We formulate a cooperative game as an extended form game in which each player in turn proposes payoffs to a coalition over M steps. Payoffs at time t are discounted by a penalty function f(t). If all players in a coalition agree to their payoffs, they receive them. Under a convergence hypothesis verified by computer for three players in many cases, we compute the payoffs resulting from a coalition pattern and give necessary conditions for particular patterns. The resulting solution is related to the Nash bargaining solution and the competitive solution.     

Zero-sum stochastic games with unbounded costs: Discounted and average cost cases

Zero-sum stochastic games with countable state space and with finitely many moves available to each player in a given state are treated. As a function of the current state and the moves chosen, player I incurs a nonnegative cost and player II receives this as a reward. For both the discounted and average cost cases, assumptions are given for the game to have a finite value and for the existence of an optimal randomized stationary strategy pair. In the average cost case, the assumptions generalize those given in Sennott (1993) for the case of a Markov decision chain. Theorems of Hoffman and Karp (1966) and Nowak (1992) are obtained as corollaries. Sufficient conditions are given for the assumptions to hold. A flow control example illustrates the results.     

Nonzero-sum non-stationary discounted Markov game model

The goal of this paper is provide a theory of K-person non-stationary Markov games with unbounded rewards, for a countable state space and action spaces. We investigate both the finite and infinite horizon problems. We define the concept of strong Nash equilibrium and present conditions for both problems for which strong Nash or Nash equilibrium strategies exist for all players within the Markov strategies, and show that the rewards in equilibrium satisfy the optimality equations.     

Learning algorithms for repeated bimatrix Nash games with incomplete information

The purpose of this paper is to study a particular recursive scheme for updating the actions of two players involved in a Nash game, who do not know the parameters of the game, so that the resulting costs and strategies converge to (or approach a neighborhood of) those that could be calculated in the known parameter case. We study this problem in the context of a matrix Nash game, where the elements of the matrices are unknown to both players. The essence of the contribution of this paper is twofold. On the one hand, it shows that learning algorithms which are known to work for zero-sum games or team problems can also perform well for Nash games. On the other hand, it shows that, if two players act without even knowing that they are involved in a game, but merely thinking that they try to maximize their output using the learning algorithm proposed, they end up being in Nash equilibrium.This research was supported in part by NSF Grant No. ECS-87-14777.     

基于时隙ALOHA协议的数据传输二人随机博弈模型

在一个给定的拓扑网络中研究关于数据传输的二人随机博弈模型.两个局中人（源节点）试图通过一个公共节点向目的节点传输随机数据包,这些数据包被分为重要的数据包和不重要的数据包两类,假设每个局中人都有一个用于存储数据包的有限容量的缓冲器.通过构造数据传输的成本分摊和奖励体系,把这种动态的冲突控制过程建模为具有有限状态集合的随机博弈,研究局中人在这种随机博弈模型下的非合作以及合作行为.在非合作情形下,给出纳什均衡的求解算法;在合作情形下,选择Shapley值作为局中人支付总和的分配方案,并讨论其子博弈一致性,提出使得Shapley值为子博弈一致的分配补偿程序.     

Performance versus informativeness in linear-quadratic Gaussian noncooperative games

In this paper, we study the impact of informativeness on the performance of linear quadratic Gaussian Nash and Stackelberg games. We first show that, in two-person static Nash games, if one of the players acquires more information, then this extra information is beneficial to him, provided that it is orthogonal to both players' information. A special case is that when one of the players is informationally stronger than the other, then any new information is beneficial to him. We then show that a similar result holds for dynamic Nash games. In the dynamic games, the players use strategies that are linear functions of the current estimates of the state, generated by two Kalman filters. The same properties are proved to hold in static and feedback Stackelberg games as well.This work was partially supported by the US Air Force Office of Scientific Research under Grant No. AFOSR-82-0174.     

Stay-in-a-set games

There exists a Nash equilibrium (ε-Nash equilibrium) for every n-person stochastic game with a finite (countable) state space and finite action sets for the players if the payoff to each player i is one when the process of states remains in a given set of states G _i and is zero otherwise. Received: December 2000     

An Approach to Fuzzy Noncooperative Nash Games

Systems that involve more than one decision maker are often optimized using the theory of games. In the traditional game theory, it is assumed that each player has a well-defined quantitative utility function over a set of the player decision space. Each player attempts to maximize/minimize his/her own expected utility and each is assumed to know the extensive game in full. At present, it cannot be claimed that the first assumption has been shown to be true in a wide variety of situations involving complex problems in economics, engineering, social and political sciences due to the difficulty inherent in defining an adequate utility function for each player in these types of problems. On the other hand, in many of such complex problems, each player has a heuristic knowledge of the desires of the other players and a heuristic knowledge of the control choices that they will make in order to meet their ends.In this paper, we utilize fuzzy set theory in order to incorporate the players' heuristic knowledge of decision making into the framework of conventional game theory or ordinal game theory. We define a new approach to N-person static fuzzy noncooperative games and develop a solution concept such as Nash for these types of games. We show that this general formulation of fuzzy noncooperative games can be applied to solve multidecision-making problems where no objective function is specified. The computational procedure is illustrated via application to a multiagent optimization problem dealing with the design and operation of future military operations.     

A non-zero-zum War of Attrition

The division of a cake by two players is modelled by means of a game of timing in which the players have a probability of learning when their opponent acts. It is shown that the game has a unique Nash equilibrium when both players are non-noisy but that there are many Nash equilibria including pure ones when at least one of the players is noisy. Explicit expressions for the strategies used in these Nash equilibria are obtained.This work was carried out while Dr. Garnaev was visiting the University of Southampton on a Postdoctoral Fellowship of The Royal Society of London.     

Variational inequality formulation for the games with random payoffs

We consider an n-player non-cooperative game with random payoffs and continuous strategy set for each player. The random payoffs of each player are defined using a finite dimensional random vector. We formulate this problem as a chance-constrained game by defining the payoff function of each player using a chance constraint. We first consider the case where the continuous strategy set of each player does not depend on the strategies of other players. If a random vector defining the payoffs of each player follows a multivariate elliptically symmetric distribution, we show that there exists a Nash equilibrium. We characterize the set of Nash equilibria using the solution set of a variational inequality (VI) problem. Next, we consider the case where the continuous strategy set of each player is defined by a shared constraint set. In this case, we show that there exists a generalized Nash equilibrium for elliptically symmetric distributed payoffs. Under certain conditions, we characterize the set of a generalized Nash equilibria using the solution set of a VI problem. As an application, the random payoff games arising from electricity market are studied under chance-constrained game framework.