Markov games with incomplete information

We consider zero-sum Markov games with incomplete information. Here, the second player is never informed about the current state of the underlying Markov chain. The existence of a value and of optimal strategies for both players is shown. In particular, we present finite algorithms for computing optimal strategies for the informed and uninformed player. The algorithms are based on linear programming results.     

Forecast Horizon for a Class of Dynamic Games

In theory, a Markov perfect equilibrium of an infinite-horizon nonstationary dynamic game requires from the players the ability to forecast an infinite amount of data. In this paper, we prove that early strategic decisions are decoupled effectively from the tail game in nonstationary dynamic games with discounting and uniformly bounded rewards. This decoupling is formalized by the notion of a forecast horizon. In words, the first-period equilibrium strategies are invariant with respect to changes in the game parameters for periods beyond the forecast horizon. We illustrate our results in the context of dynamic games of exploitation of a common pool resource and make use of the rather natural monotonicity properties of finite-horizon equilibria.     

Markov Perfect Equilibrium Existence for a Class of Undiscounted Infinite-Horizon Dynamic Games

We prove the existence of Markov perfect equilibria (MPE) for nonstationary undiscounted infinite-horizon dynamic games with alternating moves. A suitable finite-horizon equilibrium relaxation, the ending state constrained MPE, captures the relevant features of an infinite-horizon MPE for a long enough horizon, under a uniformly bounded reachability assumption.     

Existence of equilibrium stationary strategies in discounted noncooperative stochastic games with uncountable state space

This paper considers discounted noncooperative stochastic games with uncountable state space and compact metric action spaces. We assume that the transition law is absolutely continuous with respect to some probability measure defined on the state space. We prove, under certain additional continuity and integrability conditions, that such games have -equilibrium stationary strategies for each >0. To prove this fact, we provide a method for approximating the original game by a sequence of finite or countable state games. The main result of this paper answers partially a question raised by Parthasarathy in Ref. 1.     

Zero-sum risk-sensitive stochastic games for continuous time Markov chains

We study infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled continuous time Markov chains on a countable state space. For the discounted-cost game, we prove the existence of value and saddle-point equilibrium in the class of Markov strategies under nominal conditions. For the ergodic-cost game, we prove the existence of values and saddle point equilibrium by studying the corresponding Hamilton-Jacobi-Isaacs equation under a certain Lyapunov condition.     

Representation Formulas for Solutions of the HJI Equations with Discontinuous Coefficients and Existence of Value in Differential Games

In this paper, we study the Hamilton-Jacobi-Isaacs equation of zerosum differential games with discontinuous running cost. For such class of equations, the uniqueness of the solutions is not guaranteed in general. We prove principles of optimality for viscosity solutions where one of the players can play either causal strategies or only a subset of continuous strategies. This allows us to obtain nonstandard representation formulas for the minimal and maximal viscosity solutions and prove that a weak form of the existence of value is always satisfied. We state also an explicit uniqueness result for the HJI equations for piecewise continuous coefficients, in which case the usual statement on the existence of value holds.     

On differential games with Markov perfect triggering

This paper is an attempt to throw some light on the issue of whether defining history-dependent state variables in a differential game does allow for a dynamic play of precommitment equilibria. We suggest the application of trigger strategies in a state-feedback context. Based on a punishment mode in Markov perfect strategies and being able to detect even past deviations followed by histories returning to equilibrium, these subgame-perfect strategies lead to the enforcement of certain outcomes by means of dynamic rules of strategic interaction. The last part of our exposition is devoted to specific game structures under which a trigger equilibrium can be used as well as a punishment mode.     

Existence of optimal strategies in Markov games with incomplete information

The existence of a value and optimal strategies is proved for the class of two-person repeated games where the state follows a Markov chain independently of players’ actions and at the beginning of each stage only Player 1 is informed about the state. The results apply to the case of standard signaling where players’ stage actions are observable, as well as to the model with general signals provided that Player 1 has a nonrevealing repeated game strategy. The proofs reduce the analysis of these repeated games to that of classical repeated games with incomplete information on one side. This research was supported in part by Israeli Science Foundation grants 382/98, 263/03, and 1123/06, and by the Zvi Hermann Shapira Research Fund.     

Randomized stopping games and Markov market games

We study nonzero-sum stopping games with randomized stopping strategies. The existence of Nash equilibrium and ɛ-equilibrium strategies are discussed under various assumptions on players random payoffs and utility functions dependent on the observed discrete time Markov process. Then we will present a model of a market game in which randomized stopping times are involved. The model is a mixture of a stochastic game and stopping game. Research supported by grant PBZ-KBN-016/P03/99.     

Equilibrium Pricing and Advertising Strategies in a Marketing Channel

This paper is concerned with conflict and coordination in a two-member channel of distribution. We propose a differential game model that includes carryover effects of advertising, expressed by a retailer-specific stock of advertising goodwill. Pricing and advertising strategies for both firms are identified under channel conflict as well as coordination. Dynamic advertising policies are designed as stationary Markov perfect strategies. In a symmetric case, these strategies can be determined in closed form, taking into consideration explicitly nonnegativity constraints on advertising rates. We establish a global result for the relationship between the advertising strategies of the two firms under conflict and coordination.     

Moderate deviations for Markov chains with atom

We obtain in this paper moderate deviations for functional empirical processes of general state space valued Markov chains with atom under weak conditions: a tail condition on the first time of return to the atom, and usual conditions on the class of functions. Our proofs rely on the regeneration method and sharp conditions issued of moderate deviations of independent random variables. We prove our result in the nonseparable case for additive and unbounded functionals of Markov chains, extending the work of de Acosta and Chen (J. Theoret. Probab. (1998) 75–110) and Wu (Ann. Probab. (1995) 420–445). One may regard it as the analog for the Markov chains of the beautiful characterization of moderate deviations for i.i.d. case of Ledoux 1992. Some applications to Markov chains with a countable state space are considered.