Optimal Control of Markov Processes with Age-Dependent Transition Rates

We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterise the value function and optimal controls for both discounted and average cost cases.     

Transition Probabilities for Symmetric Jump Processes

We consider symmetric Markov chains on the integer lattice in dimensions, where and the conductance between and is comparable to . We establish upper and lower bounds for the transition probabilities that are sharp up to constants.

     

�лָ������ɷ�֧���̵�Feller����

首先, 当$Q$是一个拟单调的q矩阵的时候, 我们找出最小的$Q$函数是一个Feller的转移函数的准则. 然后我们把这个结论应用于生成分支q矩阵并得到相应的生成分支过程的Feller准则. 特别地, 设$\theta$是分支q矩阵中的非线性数, 总是存在一个分点$\theta_0$满足$1\leq\theta_0\leq2$或$\theta_0<+\infty$使得 生成分支过程是否是Feller的要依据$\theta<\theta_0$或者$\theta>\theta_0$.     

非负费用折扣半马氏决策过程

本文考虑可数状态非负费用的折扣半马氏决策过程.首先在给定半马氏决策核和策略下构造一个连续时间半马氏决策过程,然后用最小非负解方法证明值函数满足最优方程和存在ε-最优平稳策略,并进一步给出最优策略的存在性条件及其一些性质.最后,给出了值迭代算法和一个数值算例.     

独立平稳随机环境中的P—S—D分枝过程

本文研究了独立同分布的随机环境中的Ｐ－Ｓ－Ｄ分枝过程,获得了有关过程的渐近性态以及灭 绝概率的一些结果．     

Some Reward Paths in Semi-Markov Models with Stochastic Selection of the Transition Probabilities

In the present paper, the reward paths in non homogeneous semi-Markov systems in discrete time are examined with stochastic selection of the transition probabilities. The mean entrance probabilities and the mean rewards in the course of time are evaluated. Then the rate of the total reward for the homogeneous case is examined and the mean total reward is evaluated by means of p.g.f’s.      

关于瞬时转移速率qij的一个简化证明

本文利用推广的全概率公式对连续时间齐次马尔可夫链的标准转移概率函数pij（t）的两个重要极限qj和qij给出了一个简化证明，直观且便于理解。     

关于平稳随机过程的采样定理的一致收敛速度

改进和推广了平稳随机过程x(t)的采样定理求出了它的一致收敛速度及误差估计,并讨论了x(t)的均方导数及均方积分的采样定理.     

静态迁徙下依赖于年龄的随机环境中分枝过程

引进了静态迁徙下依赖于年龄的随机环境中分枝过程的模型,给出了该模型的条件母函数的更新方程并考虑了特殊情形下的随机Kolmogorov方程.与此同时,通过研究更新方程得到了分枝过程的各阶矩,考虑了简单情形下的灭绝概率.最后给出了一个开问题.     

Central Limit Theorem for Nonlinear Semi-Markov Reward Processes

Abstract

In this work, we obtain a central limit theorem for reward processes defined on a finite state space semi-Markov process, when reward functions assumed to have general forms and are not of constant rates. Martingale theory is the main tool which have been used for establishing the convergence of scaled and shifted reward process to a zero mean Brownian motion. The striking point in this article is considering general forms for the reward functions which are realistic in applications. The conditions needed for these results are existence of variances for sojourn times in each state and second order integrability of reward functions with respect to sojourn times distributions.     

Stochastic Comparison of Age-Dependent Block Replacement Policies

The age-dependent block replacement policy is a modified block replacement policy with an age limit for preventive replacements. Under this policy, any failed component is repaired, but only the components whose ages exceed a fixed age limit are replaced preventively at the scheduled maintenance times. Using the compensator method, we compare stochastically the failure counting processes of the age-dependent block replacement policies with different parameters, and show that the age-dependent block replacement policy, although cost effective, leads to more failures than the age and block replacement policies. AMS 2000 Subject Classification 60K10     

随机过程增量不等式的推广

本文将推广在［３］中由Ｅ．Ｃｓａｋｉ及Ｍ．Ｃｓｏｒｇｏ所引入的关于随机过程不等式，并把它应用到某些随机过程中，从而得到这些随机过程的一些极限定理．     

Bisexual Galton-Watson Branching Processes in Random Environments

In this paper, we consider a bisexual Galton-Watson branching process whose offspring probability distribution is controlled by a random environment proccss. Some results for the probability generating functions associated with the process are obtained and sufficient conditions for certain extinction and for non-certain extinction are established.     

Nonparametric density estimation for nonmixing approximable Stochastic Processes

The author deals with nonparametric density estimation for stochastic processes which satisfy the L _∞-approximability property. He considers a Parzen–Rosenblatt estimator of the density for general stationary L _∞-approximable processes. He states conditions under which it is consistent and investigates its rate of convergence. Finally, he applies his results to general nonmixing linear processes and nonmixing nonlinear autoregressive processes.     

Estimates on the Transition Densities of Girsanov Transforms of Symmetric Stable Processes

In this paper, we first study a purely discontinuous Girsanov transform which is more general than that studied in Chen and Song [(2003), J. Funct. Anal. 201, 262–281]. Then we show that the transition density of any purely discontinuous Girsanov transform of a symmetric stable process is comparable to the transition density of the symmetric stable process. The same is true for the Girsanov transform introduced in Chen and Zhang [(2002), Ann. Inst. Henri poincaré 38, 475–505]. As an application of these results, we show that the Green function of Feynman–Kac type transforms of symmetric stable processes by continuous additive functionals of zero energy, when exists, is comparable to that of the symmetric stable process.      

具有突变率的广义生-灭过程——随机单调性、Feller性及可配称性

本文给出了具有突变率的广义生－灭过程的随机单调性、Feller性及可配称性的充要条件。     

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

Abstract

In this article, we study continuous-time Markov decision processes in Polish spaces. The optimality criterion to be maximized is the expected discounted criterion. The transition rates may be unbounded, and the reward rates may have neither upper nor lower bounds. We provide conditions on the controlled system's primitive data under which we prove that the transition functions of possibly non-homogeneous continuous-time Markov processes are regular by using Feller's construction approach to such transition functions. Then, under continuity and compactness conditions we prove the existence of optimal stationary policies by using the technique of extended infinitesimal operators associated with the transition functions of possibly non-homogeneous continuous-time Markov processes, and also provide a recursive way to compute (or at least to approximate) the optimal reward values. The conditions provided in this paper are different from those used in the previous literature, and they are illustrated with an example.     

Study of Dependence for Some Stochastic Processes

Abstract

This article is concerned with studying the following problem: Consider a multivariate stochastic process whose law is characterized in terms of some infinitesimal characteristics, such as the infinitesimal generator in case of finite Markov chains. Under what conditions imposed on these infinitesimal characteristics of this multivariate process, the univariate components of the process agree in law with given univariate stochastic processes. Thus, in a sense, we study a stochastic processe' counterpart of the stochastic dependence problem, which in case of real valued random variables is solved in terms of Sklar's theorem.     

The Euler Scheme for Feller Processes

We consider the Euler scheme for stochastic differential equations with jumps, whose intensity might be infinite and the jump structure may depend on the position. This general type of SDE is explicitly given for Feller processes and a general convergence condition is presented.

In particular, the characteristic functions of the increments of the Euler scheme are calculated in terms of the symbol of the Feller process in a closed form. These increments are increments of Lévy processes and, thus, the Euler scheme can be used for simulation by applying standard techniques from Lévy processes.     

带干扰双Poisson风险模型生存概率的Feller表示

研究了带干扰双Poisson风险模型,运用鞅论的方法给出了该模型生存概率的Feller表示式.