Truncated policy iteration methods

Policy iteration methods are important but often computationally expensive approaches for solving certain stochastic optimization problems. Modified policy iteration methods have been proposed to reduce the storage and computational burden. The asymptotic speed-of-convergence of such methods is, however, not well understood. In this paper we show how modified policy iteration methods may be constructed to achieve a preassigned rate-of-convergence. Our analysis provides a framework for analyzing the local behavior of such methods and provides impetus for perhaps more computationally efficient procedures than currently exist.