Existence and Uniqueness of Fixed Points for Markov Operators and Markov Processes

This paper concerns a Markov operator T on a space L₁, and aMarkov process P which defines a Markov operator on a spaceM of finite signed measures. For T, the paper presents necessaryand sufficient conditions for:

a the existence of invariant probabilitydensities (IPDs)

b the existence of strictly positive IPDs,and

c the existence and uniqueness of IPDs.

Similar resultson invariant probability measures for P are presented. The basicapproach is to pose a fixed-point problem as the problem ofsolving a certain linear equation in a suitable Banach space,and then obtain necessary and sufficient conditions for thisequation to have a solution. 1991 Mathematics Subject Classification:60J05, 47B65, 47N30.     

Adaptive control of Markov processes with incomplete state information and unknown parameters

Recent results for parameter-adaptive Markov decision processes (MDP's) are extended to partially observed MDP's depending on unknown parameters. These results include approximations converging uniformly to the optimal reward function and asymptotically optimal adaptive policies.This research was supported in part by the Consejo del Sistema Nacional de Educación Tecnologica (COSNET) under Grant 178/84, in part by the Air Force Office of Scientific Research under Grant AFOSR-84-0089, in part by the National Science Foundation under Grant ECS-84-12100, and in part by the Joint Services Electronics Program under Contract F49602-82-C-0033.