连续时间马尔可夫决策过程的折扣模型

本文考虑的是转移速率族任意且费用率函数可能无界的连续时间马尔可夫决策过程的折扣模型．放弃了传统的要求相应于每个策略的　Ｑ　－过程唯一等条件,而首次考虑相应每个策略的　Ｑ　－过程不一定唯一,　转移速率族也不一定保守,　费用率函数可能无界,　且允许行动空间非空任意的情形．　本文首次用"α－折扣费用最优不等式"更新了传统的α－折扣费用最优方程,并用"最优不等式"和新的方法,不仅证明了传统的主要结果即最优平稳策略的存在性,　而且还进一步探讨了（　∈＞０　　）－最优平稳策略,具有单调性质的最优平稳策略,　以及（∈≥０）　－最优决策过程的存在性,　得到了一些有意义的新结果．　最后,　提供了一个迁移率受控的生灭系统例子,　它满足本文的所有条件,　而传统的假设（见文献［１－１４］）均不成立．

The Unbounded Cost Discounted Model for Coutinuous Time Markov Decision

This paper deals with the discounted cost model for coutinuous time Markov decision processes with arbitrary transition rates and possible unbounded costs. Getting rid of the traditional condition that the Q -process deduced from a given policy is needed to be unique, we first consider the case of the Q-processes being not unique, of the costs possibly being unbounded, and of the action space being arbitrary. By first using the "α-discounted cost optimality inequality"instead of the traditional optimality equation, we not only prove the existence of optimal stationary policies, but also derive the existence of (∈>0) -optimal stationary policies, of the monotone optimal stationary policies, and of the (∈≥0) -optimal decision processes. Moreover, some new results are obtained. Finally, a substantial applied example of birth and death processes witn controlled migration rates is used to show that our assumptions here are satisfied, whereas the conditions in 1-14] fail to hold.