Numerical approximations for nonlinear stochastic systems with delays

We extend the numerical methods of Kushner, H.J. and Dupuis, P., 1992 Kushner, H.J. and Dupuis, P. 2001. Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd ed., Berlin and New York: Springer-Verlag. Crossref] , Google Scholar], Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd ed., 2001 (Berlin and New York: Springer Verlag], known as the Markov chain approximation methods, to controlled general nonlinear delayed reflected diffusion models. Both the path and the control can be delayed. For the no-delay case, the method covers virtually all models of current interest. The method is robust, the approximations have physical interpretations as control problems closely related to the original one, and there are many effective methods for getting the approximations, and for solving the Bellman equation for low-dimensional problems. These advantages carry over to the delay problem. It is shown how to adapt the methods for getting the approximations, and the convergence proofs are outlined for the discounted cost function. Extensions to all of the cost functions of current interest as well as to models with Poisson jump terms are possible. The paper is particularly concerned with representations of the state and algorithms that minimize the memory requirements.