A two-person zero-sum Markov game with a stopped set

We consider a two-person zero-sum Markov game with continuous time up to the time that the game process goes into a fixed subset of a countable state space, this subset is called a stopped set of the game. We show that such a game with a discount factor has optimal value function and both players will have their optimal stationary strategies. The same result is proved for the case of a nondiscounted Markov game under some additional conditions, that is a reward rate function is nonnegative and the first time τ (entrance time) of the game process going to the stopped set is finite with probability one (i.e., p(τ < ∞) = 1). It is remarkable that in the case of a nondiscounted Markov game, if the expectation of the entrance time is bounded, and the reward rate function need not be nonnegative, then the same result holds.     

On an N-person noncooperative Markov game with a metric state space

We consider a noncooperative N-person discounted Markov game with a metric state space, and define the total expected discounted gain. Under some conditions imposed on the objects in the game system, we prove that our game system has an equilibrium point and each player has his equilibrium strategy. Moreover in the case of a nondiscounted game, the total expected gain up to a finite time can be obtained, and we define the long-run expected average gain. Thus if we impose a further assumption for the objects besides the conditions in the case of the discounted game, then it is proved that the equilibrium point exists in the nondiscounted Markov game. The technique for proving the nondiscounted case is essentially to modify the objects of the game so that they become objects of a modified Markov game with a discounted factor which has an equilibrium point in addition to the equilibrium point of the discounted game.     

Successive approximations for average reward Markov games

This paper considers two-person zero-sum Markov games with finitely many states and actions with the criterion of average reward per unit time. Two special situations are treated and it is shown that in both cases the method of successive approximations yields anε-band for the value of the game as well as stationaryε-optimal strategies. In the first case all underlying Markov chains of pure stationary optimal strategies are assumed to be unichained. In the second case it is assumed that the functional equation Uv=v+ge has a solution.     

Non-cooperative n-person game with a stopped set

A continuous time non-cooperative n-person Markov game with a stopped set is studied in this paper. We prove that, in the game process with or without discount factor, there exists an optimal stationary point of strategies, called the equilibrium point, and each player has his equilibrium stationary strategy, such that the total expected discounted or non-discounted gain are maximums.     

Generalized Gambler’s Ruin Problem: Explicit Formulas via Siegmund Duality

We give explicit formulas for ruin probabilities in a multidimensional Generalized Gambler’s ruin problem. The generalization is best interpreted as a game of one player against d other players, allowing arbitrary winning and losing probabilities (including ties) depending on the current fortune with particular player. It includes many previous other generalizations as special cases. Instead of usually utilized first-step-like analysis we involve dualities between Markov chains. We give general procedure for solving ruin-like problems utilizing Siegmund duality in Markov chains for partially ordered state spaces studied recently in context of Möbius monotonicity.     

Generalizations of bold play in red and black

The strategy of bold play in the game of red and black leads to a number of interesting mathematical properties: the player's fortune follows a deterministic map, before the transition that ends the game; the bold strategy can be “re-scaled” to produce new strategies with the same win probability; the win probability is a continuous function of the initial fortune, and in the fair case, equals the initial fortune. We consider several Markov chains in more general settings and study the extent to which the properties are preserved. In particular, we study two “k-player” models.     

Zero-sum risk-sensitive stochastic games on a countable state space

Infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled Markov chains with countably many states are analyzed. Upper and lower values for these games are established. The existence of value and saddle-point equilibria in the class of Markov strategies is proved for the discounted-cost game. The existence of value and saddle-point equilibria in the class of stationary strategies is proved under the uniform ergodicity condition for the ergodic-cost game. The value of the ergodic-cost game happens to be the product of the inverse of the risk-sensitivity factor and the logarithm of the common Perron–Frobenius eigenvalue of the associated controlled nonlinear kernels.     

随机游动在足球福彩中的应用

通过对一个随机游戏的分析，提出“近似马氏稳态时间”定理并加以证明，而后利用马氏链模型和对策论建立解决方案的最佳模型，并利用此模型预测足球比赛的胜、负的概率。     

Cyclic Markov equilibria in stochastic games

We examine a three-person stochastic game where the only existing equilibria consist of cyclic Markov strategies. Unlike in two-person games of a similar type, stationary ε-equilibria (ε > 0) do not exist for this game. Besides we characterize the set of feasible equilibrium rewards.     

Zero-sum risk-sensitive stochastic games for continuous time Markov chains

We study infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled continuous time Markov chains on a countable state space. For the discounted-cost game, we prove the existence of value and saddle-point equilibrium in the class of Markov strategies under nominal conditions. For the ergodic-cost game, we prove the existence of values and saddle point equilibrium by studying the corresponding Hamilton-Jacobi-Isaacs equation under a certain Lyapunov condition.     

Zero-sum constrained stochastic games with independent state processes

We consider a zero-sum stochastic game with side constraints for both players with a special structure. There are two independent controlled Markov chains, one for each player. The transition probabilities of the chain associated with a player as well as the related side constraints depend only on the actions of the corresponding player; the side constraints also depend on the player’s controlled chain. The global cost that player 1 wishes to minimize and that player 2 wishes to maximize, depend however on the actions and Markov chains of both players. We obtain a linear programming formulations that allows to compute the value and saddle point policies for this problem. We illustrate the theoretical results through a zero-sum stochastic game in wireless networks in which each player has power constraints     

Explicit optimal value for Dynkin's stopping game

Under the One-step Look Ahead rule of Dynamic Programming, an explicit game value of Dynkin's stopping problem for a Markov chain is obtained by using a potential operator. The condition on the One-step rule could be extended to the k-step and infinity-step rule. We shall also decompose the game value as the sum of two explicit functions under these rules.     

Systemic Risk and Interbank Lending

We propose a simple model of the banking system incorporating a game feature, where the evolution of monetary reserve is modeled as a system of coupled Feller diffusions. The optimization reflects the desire of each bank to borrow from or lend to a central bank through manipulating its lending preference and the intention of each bank to deposit in the central bank in order to control the reserve and the corresponding volatility for cost minimization. The Markov Nash equilibrium for finite many players generated by minimizing the linear quadratic cost subject to Cox–Ingersoll–Ross type processes creates liquidity and deposit rate. The adding liquidity leads to a flocking effect implying stability or systemic risk depending on the level of the growth rate, but the deposit rate diminishes the growth of the total monetary reserve causing a large number of bank defaults. The central bank acts as a central deposit corporation. In addition, the corresponding mean field game in the case of the number of banks N large and the infinite time horizon stochastic game with the discount factor are also discussed.     

Randomized stopping games and Markov market games

We study nonzero-sum stopping games with randomized stopping strategies. The existence of Nash equilibrium and ɛ-equilibrium strategies are discussed under various assumptions on players random payoffs and utility functions dependent on the observed discrete time Markov process. Then we will present a model of a market game in which randomized stopping times are involved. The model is a mixture of a stochastic game and stopping game. Research supported by grant PBZ-KBN-016/P03/99.     

A BSDE approach to risk-based asset allocation of pension funds with regime switching

An asset allocation problem of a member of a defined contribution (DC) pension fund is discussed in a hidden, Markov regime-switching, economy using backward stochastic differential equations, (BSDEs). A risk-based approach is considered, where the member selects an optimal asset mix with a view to minimizing the risk described by a convex risk measure of his/her terminal wealth. Firstly, filtering theory is adopted to transform the hidden, Markov regime-switching, economy into one with complete observations and to develop, (robust), filters for the hidden Markov chain. Then the optimal asset allocation problem of the member is formulated as a two-person, zero-sum stochastic differential game between the member and the market in the economy with complete observations. The BSDE approach is then used to solve the game problem and to characterize the saddle point of the game problem. An explicit expression for the optimal asset mix is obtained in the case of a convex risk measure with quadratic penalty and it can be considered a generalized version of the Merton ratio. An explicit expression for the optimal strategy of the market is also obtained, which leads to a risk-neutral wealth dynamic and may provide some insights into asset pricing in the economy with inflation risk and regime-switching risk. Numerical examples are provided to illustrate financial implications of the BSDE solution.     

Computing p-values in conditional independence models for a contingency table

We present a Markov chain Monte Carlo (MCMC) method for generating Markov chains using Markov bases for conditional independence models for a four-way contingency table. We then describe a Markov basis characterized by Markov properties associated with a given conditional independence model and show how to use the Markov basis to generate random tables of a Markov chain. The estimates of exact p-values can be obtained from random tables generated by the MCMC method. Numerical experiments examine the performance of the proposed MCMC method in comparison with the χ ² approximation using large sparse contingency tables.     

Sequential games as stochastic processes

This paper examines the stochastic processes generated by sequential games that involve repeated play of a specific game. Such sequential games are viewed as adaptive decision-making processes over time wherein each player updates his “state” after every play. This revision may involve one's strategy or one's prior distribution on the competitor's strategies. It is shown that results from the theory of discrete time Markov processes can be applied to gain insight into the asymptotic behavior of the game. This is illustrated with a duopoly game in economics.     

On risk minimizing portfolios under a Markovian regime-switching Black-Scholes economy

We consider a risk minimization problem in a continuous-time Markovian regime-switching financial model modulated by a continuous-time, observable and finite-state Markov chain whose states represent different market regimes. We adopt a particular form of convex risk measure, which includes the entropic risk measure as a particular case, as a measure of risk. The risk-minimization problem is formulated as a Markovian regime-switching version of a two-player, zero-sum stochastic differential game. One important feature of our model is to allow the flexibility of controlling both the diffusion process representing the financial risk and the Markov chain representing macro-economic risk. This is novel and interesting from both the perspectives of stochastic differential game and stochastic control. A verification theorem for the Hamilton-Jacobi-Bellman (HJB) solution of the game is provided and some particular cases are discussed.     

Continuous-time zero-sum stochastic game with stopping and control

We consider a zero-sum stochastic game for continuous-time Markov chain with countable state space and unbounded transition and pay-off rates. The additional feature of the game is that the controllers together with taking actions are also allowed to stop the process. Under suitable hypothesis we show that the game has a value and it is the unique solution of certain dynamic programming inequalities with bilateral constraints. In the process we also prescribe a saddle point equilibrium.     

Cohomological dimension of Markov compacta

We rephrase Gromov's definition of Markov compacta, introduce a subclass of Markov compacta defined by one building block and study cohomological dimensions of these compacta. We show that for a Markov compactum X, dimZ(p)X=dimQX for all but finitely many primes p where Z(p) is the localization of Z at p. We construct Markov compacta of arbitrarily large dimension having dimQX=1 as well as Markov compacta of arbitrary large rational dimension with dimZpX=1 for a given p.