A partially observed control problem for Markov chains

A finite state, continuous time Markov chain is considered and the solution to the filtering problem given when the observation process counts the total number of jumps. The Zakai equation for the unnormalized conditional distribution is obtained and the control problem discussed in separated form with this as the state. A new feature is that, because of the correlation between the state and observation process, the control parameter appears in the diffusion coefficient which multiplies the Poisson noise in the Zakai equation. By introducing a Gâteaux derivative the minimum principle, satisfied by an optimal control, is derived. If the optimal control is Markov, a stochastic integrand can be obtained more explicitly and new forward and backward equations satisfied by the adjoint process are obtained.This research was partially supported by NSERC Grant A7964, the Air Force Office of Scientific Research, United States Air Force, under Contract AFOSR-86-0332, and the U.S. Army Research Office under Contract DAAL03-87-0102.