Optimization of a special case of continuous-time Markov decision processes with compact action set |
| |
Authors: | Tang Hao Zhou Lei Arai Tamio |
| |
Institution: | 1. School of Computer and Information, Hefei University of Technology, Anhui 230009, PR China;2. Department of Precision Engineering, The University of Tokyo, Bunkyo-ku, Tokyo 113-8656, Japan |
| |
Abstract: | Performance optimization is considered for average-cost multichain Markov decision processes (MDPs) with compact action set. Since, for a general compact multichain model, the optimality equation system may have no solution, and also a policy iteration algorithm may yield a suboptimal policy rather than an optimal one, we concentrate only on a special case of multichain models in this paper, where we assume that the classifications of states are fixed identically rather than varying with policies. By using the concept of performance potentials, the existence of solutions to the optimality equation system is established, and then a potential-based policy iteration algorithm is supposed to solve this system. In addition, the optimality convergence, for recurrent classes, of the algorithm has been proved. Finally, a numerical example is provided. |
| |
Keywords: | Markov processes Compact action set Performance potential Policy iteration |
本文献已被 ScienceDirect 等数据库收录! |
|