On ergodicity conditions in a polling model with Markov modulated input and state-dependent routing

A polling system with switchover times and state-dependent server routing is studied. Input flows are modulated by a random external environment. Input flows are ordinary Poisson flows in each state of the environment, with intensities determined by the environment state. Service and switchover durations have exponential laws of probability distribution. A continuous-time Markov chain is introduced to describe the dynamics of the server, the sizes of the queues and the states of the environment. By means of the iterative-dominating method a sufficient condition for ergodicity of the system is obtained for the continuous-time Markov chain. This condition also ensures the existence of a stationary probability distribution of the embedded Markov chain at instants of jumps. The customers sojourn cost during the period of unloading the stable queueing system is chosen as a performance metric. Numerical study in case of two input flows and a class of priority and threshold routing algorithms is conducted. It is demonstrated that in case of light inputs a priority routing rule doesn’t seem to be quasi-optimal.     

Exponential martingale for compound Poisson process with latent variable and its applications

In this article, we construct an exponential martingale for the compound Poisson process with latent variableWith the help of this exponential martingale, we provide an asymptotic behavior of the coherent entropic risk measure for the compound Poisson process and a deviation inequality for the ruin probability of the partly shifted risk process.     

Semi-Markov modulated Poisson process: probabilistic and statistical analysis

We consider a Poisson process that is modulated in such a way that the arrival rate at any time depends on the state of a semi-Markov process. This presents an interesting generalization of Poisson processes with important implications in real life applications. Our analysis concentrates on the transient as well as the long term behaviour of the arrival count and the arrival time processes. We discuss probabilistic as well as statistical issues related to various quantities of interest.     

Removing logarithms in Poisson approximation to the partial sum process of a Markov chain

He and Xia (1997, Stochastic Processes Appl. 68, pp. 101–111) gave some error bounds for a Wasserstein distance between the distributions of the partial sum process of a Markov chain and a Poisson point process on the positive half-line. However, all these bounds increase logarithmically with the mean of the Poisson point process. In this paper, using the coupling method and a general deep result for estimating the errors of Poisson process approximation in Brown and Xia (2001, Ann. Probab. 29, pp. 1373–1403), we give a new error bound for the above Wasserstein distance. In contrast to the previous results of He and Xia (1997), our new error bound has no logarithm anymore and it is bounded and asymptotically remains constant as the mean increases.     

Backward stochastic differential equations associated to jump Markov processes and applications

In this paper we study backward stochastic differential equations (BSDEs) driven by the compensated random measure associated to a given pure jump Markov process X$X$ on a general state space K$K$. We apply these results to prove well-posedness of a class of nonlinear parabolic differential equations on K$K$, that generalize the Kolmogorov equation of X$X$. Finally we formulate and solve optimal control problems for Markov jump processes, relating the value function and the optimal control law to an appropriate BSDE that also allows to construct probabilistically the unique solution to the Hamilton–Jacobi–Bellman equation and to identify it with the value function.     

A class of path-valued Markov processes and its applications to superprocesses

Summary Let ( _s ) be a continuous Markov process satisfying certain regularity assumptions. We introduce a path-valued strong Markov process associated with ( _s ), which is closely related to the so-called superprocess with spatial motion ( _s ). In particular, a subsetH of the state space of ( _s ) intersects the range of the superprocess if and only if the set of paths that hitH is not polar for the path-valued process. The latter property can be investigated using the tools of the potential theory of symmetric Markov processes: A set is not polar if and only if it supports a measure of finite energy. The same approach can be applied to study sets that are polar for the graph of the superprocess. In the special case when ( _s ) is a diffusion process, we recover certain results recently obtained by Dynkin.     

Galerkin projections for state-dependent delay differential equations with applications to drilling

A Galerkin projection scheme to obtain low dimensional approximations of delay differential equations (DDEs) involving state-dependent delays is developed. The current scheme is an extension of a similar, recently proposed scheme for DDEs with constant delays in the publication by P. Wahi, A. Chatterjee 2005. The resulting ordinary differential equations (ODEs) from the Galerkin scheme are easier to integrate using commercial ODE solvers, and are amenable to stability and bifurcation analysis using standard techniques. First, the application of the formulation is demonstrated through a scalar delay differential equation, and the performance of the formulation is assessed. Next, the scheme is applied to a two degrees-of-freedom model describing the coupled axial and torsional vibrations of oil well drill-strings. In both cases, the Galerkin approximations show an excellent agreements with the direct numerical simulations of the original systems.     

一个推广的Markov过程模型及其应用

在一个开放性系统中,主体不同的行为对主体不同的目标具有不同的重要性,因而,对行动与目标关联性的量化分析对于主体的决策具有重要作用.我们将尝试用Markov过程模型来从一个侧面完成以上刻画在Markov过程模型中引入多行动项,将其看作过程状态改变的原因,并把行动与目标之间的关联解释为行动项与目标状态之间的概率可达性,最后,简单讨论这种模型的一些理论和应用意义.     

Sobolev spaces of Banach-valued functions associated with a Markov process

Summary We discuss Sobolev spaces of Banach-valued functions. They are extensions of Sobolev spaces of scalar functions. We use a gamma transform of a semigroup associated with a Markov process. A typical example is the Ornstein-Uhlenbeck process on the Wiener space.     

Markov processes associated with L p -resolvents and applications to stochastic differential equations on Hilbert space

We give general conditions on a generator of a C₀-semigroup (resp. of a C₀-resolvent) on L^p(E,μ), p ≥ 1, where E is an arbitrary (Lusin) topological space and μ a σ-finite measure on its Borel σ-algebra, so that it generates a sufficiently regular Markov process on E. We present a general method how these conditions can be checked in many situations. Applications to solve stochastic differential equations on Hilbert space in the sense of a martingale problem are given. Dedicated to Giuseppe Da Prato on the occasion of his 70th birthday     

Mixing times with applications to perturbed Markov chains

A measure of the “mixing time” or “time to stationarity” in a finite irreducible discrete time Markov chain is considered. The statistic , where {π_j} is the stationary distribution and m_ij is the mean first passage time from state i to state j of the Markov chain, is shown to be independent of the initial state i (so that η_i = η for all i), is minimal in the case of a periodic chain, yet can be arbitrarily large in a variety of situations. An application considering the effects perturbations of the transition probabilities have on the stationary distributions of Markov chains leads to a new bound, involving η, for the 1-norm of the difference between the stationary probability vectors of the original and the perturbed chain. When η is large the stationary distribution of the Markov chain is very sensitive to perturbations of the transition probabilities.     

Busy-period and blocking behavior of finite queues with state-dependent Markov renewal arrivals

The busy-period length distributions and blocking probabilities are considered for finiteG/G/1/K queues with state-dependent Markov renewal arrivals. The Laplace-Stieltjes transforms of the distributions and blocking probabilities are given for the non-preemptive and last-come-first-served preemptive resume (or repeat) service disciplines. For Erlangian (or deterministic) service times in particular, it is proved that the busy-period length (the number of blocked customers) for the non-preemptive discipline is smaller (larger) than for the preemptive resume discipline.     

Markov decision processes with state-dependent discount factors and unbounded rewards/costs

This paper deals with discrete-time Markov decision processes with state-dependent discount factors and unbounded rewards/costs. Under general conditions, we develop an iteration algorithm for computing the optimal value function, and also prove the existence of optimal stationary policies. Furthermore, we illustrate our results with a cash-balance model.     

On the Asymptotic Behavior of the Storage Process Fed by a Markov Modulated Brownian Motion

Consider a storage model fed by a Markov modulated Brownian motion. We prove that the stationary distribution of the model exits and that the running maximum of the storage process over the interval [0, t] grows asymptotically like log t as t→∞.     

Using a continuous-time Markov model with Poisson arrivals to describe the movements of geriatric patients

The population of geriatrics in a given hospital district is relatively stable and therefore we may model the movement of geriatric patients by considering both their stays in hospital and subsequent releases back into the community. The care of the elderly in departments of geriatric medicine may be generally classified into two forms of clinical care, acute/rehabilitative and long stay. Our paper describes the movement of pateints through departments of geriatric medicine and subsequent stays in the community by a four-stage continuous-time Markov model, where the stages represent acute/rehabilitative patients, long-stay patients, ex-patients in the community and former patients who are now dead, respectively. Admissions are modelled as a Poisson stream and expressions are calculated for the distribution, mean and variance of numbers of patients in each compartment at any time. Using these expressions the model is then fitted to a large data set of hospital spells containing over 10 000 admissions. © 1998 John Wiley & Sons, Ltd.     

Markov process with creation and annihilation

Summary A simple way is given to construct probabilistically a strong Markov process (y _i, P _x, of an extended sense such that the expectation u = E_x[f(y _t)] provides the solution of (1.1) for a Borel measurable function c (i.e. with both terms of creation and annihilation of mass), where and stand for the dates of birth and death of a particle, respectively. Actually the (non-probability) measure P _x is given as the sum of induced measures of probability measure (of Brownian motion with age) by a set of mappings. The probability measure is obtained by making killed processes of Brownian motion and piecing them together in a specific way.