Optimal control of random evolutions

We consider a stochastic control problem for a random evolution. We study the Bellman equation of the problem and we prove the existence of an optimal stochastic control which is Markovian. This problem enables us to approximate the general problem of the optimal control of solutions of stochastic differential equations.