Maximum Principle for Stochastic Differential Games with Partial Information

In this paper, we first deal with the problem of optimal control for zero-sum stochastic differential games. We give a necessary and sufficient maximum principle for that problem with partial information. Then, we use the result to solve a problem in finance. Finally, we extend our approach to general stochastic games (nonzero-sum), and obtain an equilibrium point of such game.     

Zero-Sum Stochastic Games with Partial Information and Average Payoff

We consider a discrete time partially observable zero-sum stochastic game with average payoff criterion. We study the game using an equivalent completely observable game. We show that the game has a value and also we present a pair of optimal strategies for both the players.     

Some Classes of Imperfect Information Finite State-Space Stochastic Games with Finite-Dimensional Solutions

Stochastic games under imperfect information are typically computationally intractable even in the discrete-time/discrete-state case considered here. We consider a problem where one player has perfect information. A function of a conditional probability distribution is proposed as an information state. In the problem form here, the payoff is only a function of the terminal state of the system, and the initial information state is either linear or a sum of max-plus delta functions. When the initial information state belongs to these classes, its propagation is finite-dimensional. The state feedback value function is also finite-dimensional, and obtained via dynamic programming, but has a nonstandard form due to the necessity of an expanded state variable. Under a saddle point assumption, Certainty Equivalence is obtained and the proposed function is indeed an information state.     

一类部分信息的随机控制问题的极值原理

在本文中,我们证明了一类部分信息的随机控制问题的极值原理的一个充分条件和一个必要条件.其中,随机控制问题的控制系统是一个由鞅和Brown运动趋动的随机偏微分方程.     

Stochastic Differential Games with Asymmetric Information

We investigate a two-player zero-sum stochastic differential game in which the players have an asymmetric information on the random payoff. We prove that the game has a value and characterize this value in terms of dual viscosity solutions of some second order Hamilton-Jacobi equation.     

Repeated Games with Bonuses

This paper deals with 2-player zero-sum repeated games in which player 1 receives a bonus at stage t if he repeats the action he played at stage t−1. We investigate the optimality of simple strategies for player 1. A simple strategy for player 1 consists of playing the same mixed action at every stage, irrespective of past play. Furthermore, for games in which player 1 has a simple optimal strategy, we characterize the set of stationary optimal strategies for player 2.     

Stochastic pursuit-evasion differential games in 3D

A stochastic version of Isaacs's homicidal chauffeur game in the (x, y, z)-space is considered. This is used to solve a pursuit-evasion problem in the (x, y, z)-space in which the pursuer has incomplete information on the evader motion. Optimal feedback strategies for the game, and optimal feedback guidance laws for the pursuer, which uses only the measurements available to the pursuer, are computed. A simple suboptimal guidance law for the pursuer is suggested.     

Total Reward Stochastic Games and Sensitive Average Reward Strategies

In this paper, total reward stochastic games are surveyed. Total reward games are motivated as a refinement of average reward games. The total reward is defined as the limiting average of the partial sums of the stream of payoffs. It is shown that total reward games with finite state space are strategically equivalent to a class of average reward games with an infinite countable state space. The role of stationary strategies in total reward games is investigated in detail. Further, it is outlined that, for total reward games with average reward value 0 and where additionally both players possess average reward optimal stationary strategies, it holds that the total reward value exists.     

Infinite-Horizon Stochastic Differential Games with Branching Payoffs

In this paper, we consider infinite-horizon stochastic differential games with an autonomous structure and steady branching payoffs. While the introduction of additional stochastic elements via branching payoffs offers a fruitful alternative to modeling game situations under uncertainty, the solution to such a problem is not known. A theorem on the characterization of a Nash equilibrium solution for this kind of games is presented. An application in renewable resource extraction is provided to illustrate the solution mechanism.     

Zero-Sum Ergodic Semi-Markov Games with Weakly Continuous Transition Probabilities

Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities and lower semicontinuous, possibly unbounded, payoff functions are studied. Two payoff criteria are considered: the ratio average and the time average. The main result concerns the existence of a lower semicontinuous solution to the optimality equation and its proof is based on a fixed-point argument. Moreover, it is shown that the ratio average as well as the time average payoff stochastic games have the same value. In addition, one player possesses an ε-optimal stationary strategy (ε>0), whereas the other has an optimal stationary strategy. A. Jaśkiewicz is on leave from Institute of Mathematics and Computer Science, Wrocław University of Technology. This work is supported by MNiSW Grant 1 P03A 01030.     

Stochastic Discrete-Time Nash Games with Constrained State Estimators

In this paper, we consider stochastic linear-quadratic discrete-time Nash games in which two players have access only to noise-corrupted output measurements. We assume that each player is constrained to use a linear Kalman filter-like state estimator to implement his optimal strategies. Two information structures available to the players in their state estimators are investigated. The first has access to one-step delayed output and a one-step delayed control input of the player. The second has access to the current output and a one-step delayed control input of the player. In both cases, statistics of the process and statistics of the measurements of each player are known to both players. A simple example of a two-zone energy trading system is considered to illustrate the developed Nash strategies. In this example, the Nash strategies are calculated for the two cases of unlimited and limited transmission capacity constraints.     

Some Remark on Optimal Stochastic Control with Partial Information

Abstract

We are interested in the control problem of a partially observable diffusion process, which is initialized at a fixed point of ? ⁿ , and we want to characterize the associated value function. To resort to the theory of viscosity solutions depends on the possibility to translate such a problem on Hilbert spaces like L ²(? ⁿ ), and so it can not be used here. Nevertheless, a result of N. Bouleau and F. Hirsch allows us to introduce a broadened problem which fulfills the condition. The fact remains to link these two control problems.     

A Maximum Principle for Stochastic Control with Partial Information

Abstract

We study the problem of optimal control of a jump diffusion, that is, a process which is the solution of a stochastic differential equation driven by Lévy processes. It is required that the control process is adapted to a given subfiltration of the filtration generated by the underlying Lévy processes. We prove two maximum principles (one sufficient and one necessary) for this type of partial information control. The results are applied to a partial information mean-variance portfolio selection problem in finance.     

Harnack's Inequality for p-Harmonic Functions via Stochastic Games

We give a proof of asymptotic Lipschitz continuity of p-harmonious functions, that are tug-of-war game analogies of ordinary p-harmonic functions. This result is used to obtain a new proof of Lipschitz continuity and Harnack's inequality for p-harmonic functions in the case p > 2. The proof avoids classical techniques like Moser iteration, but instead relies on suitable choices of strategies for the stochastic tug-of-war game.     

Stochastic Games for N Players

The objective of this paper is to present a useful application of the theory of regularity of systems of nonlinear partial differential equations to the solution of stochastic differential games with N players. It is particularly interesting to notice that the structure of games fits perfectly with what is requested to prove the regularity property which is needed.     

Zero-Sum Differential Games Involving Hybrid Controls

We study a zero-sum differential game with hybrid controls in which both players are allowed to use continuous as well as discrete controls. Discrete controls act on the system at a given set interface. The state of the system is changed discontinuously when the trajectory hits predefined sets, an autonomous jump set A or a controlled jump set C, where one controller can choose to jump or not. At each jump, the trajectory can move to a different Euclidean space. One player uses all the three types of controls, namely, continuous controls, autonomous jumps, and controlled jumps; the other player uses continuous controls and autonomous jumps. We prove the continuity of the associated lower and upper value functions V⁻ and V⁺. Using the dynamic programming principle satisfied by V⁻ and V⁺, we derive lower and upper quasivariational inequalities satisfied in the viscosity sense. We characterize the lower and upper value functions as the unique viscosity solutions of the corresponding quasivariational inequalities. Lastly, we state an Isaacs like condition for the game to have a value This work was partially supported by Grants DRDO 508 and ISRO 050 to the Non-linear Studies Group, Indian Institute of Science. The first author is a University Grant Commission Research Fellow and the financial support is gratefully acknowledged. The authors thank Prof. M.K. Ghosh, Department of Mathematics, Indian Institute of Science, for introducing the problem and thank the referee for useful suggestions.     

Subgame Consistent Cooperative Solutions in Stochastic Differential Games

Subgame consistency is a fundamental element in the solution of cooperative stochastic differential games. In particular, it ensures that: (i) the extension of the solution policy to a later starting time and to any possible state brought about by the prior optimal behavior of the players would remain optimal; (ii) all players do not have incentive to deviate from the initial plan. In this paper, we develop a mechanism for the derivation of the payoff distribution procedures of subgame consistent solutions in stochastic differential games with transferable payoffs. The payoff distribution procedure of the subgame consistent solution can be identified analytically under different optimality principles. Demonstration of the use of the technique for specific optimality principles is shown with an explicitly solvable game. For the first time, analytically tractable solutions of cooperative stochastic differential games with subgame consistency are derived.     

Stochastic Partial Differential Equations on Manifolds~II. Nonlinear Filtering

The conditional law of an unobservable component x(t) of a diffusion (x(t),y(t)) given the observations {y(s):s[0,t]} is investigated when x(t) lives on a submanifold of . The existence of the conditional density with respect to a given measure on is shown under fairly general conditions, and the analytical properties of this density are characterized in terms of the Sobolev spaces used in the first part of this series.     

On a duel with time lag and equal accuracy functions

This paper deals with a duel with time lag that has the following structure: Each of two players I and II has a gun with one bullet and he can fire his bullet at any time in [0, 1], aiming at this opponent. The gun of player I is silent and the gun of player II is noisy with time lagt (i.e., if player II fires at timex, then player I knows it at timex+t). They both have equal accuracy functions. Furthermore, if player I hits player II without being hit himself before, the payoff is +1; if player I is hit by player II without hitting player II before, the payoff is –1; if they hit each other at the same time or both survive, the payoff is 0.This paper gives the optimal strategy for each player, the game value, and some examples.