Numerical solution of a long-term average control problem for singular stochastic processes

This paper analyzes numerically a long-term average stochastic control problem involving a controlled diffusion on a bounded region. The solution technique takes advantage of an infinite-dimensional linear programming formulation for the problem which relates the stationary measures to the generators of the diffusion. The restriction of the diffusion to an interval is accomplished through reflection at one end point and a jump operator acting singularly in time at the other end point. Different approximations of the linear program are obtained using finite differences for the differential operators (a Markov chain approximation to the diffusion) and using a finite element method to approximate the stationary density. The numerical results are compared with each other and with dynamic programming. This research has been supported in part by the U.S. National Security Agency under Grant Agreement Number H98230-05-1-0062. The United States Government is authorized to reproduce and distribute reprints notwithstanding any copyright notation herein.