Invariance principles for renewal processes when only moments of low order exist

Starting from recent strong and weak approximations to the partial sums of i.i.d. random vectors (cf. U. Einmahl, Ann. Probab., 15 1419–1440), some corresponding invariance principles are developed for associated renewal processes and random sums. Optimality of the approximation is proved in the case when only two moments exist. Among other applications, a Darling-Erdös type extreme value theorem for renewal processes will be derived.     

A Markov Renewal Based Model for Wireless Networks

We propose a queueing network model which can be used for the integration of the mobility and teletraffic aspects that are characteristic of wireless networks. In the general case, the model is an open network of infinite server queues where customers arrive according to a non-homogeneous Poisson process. The movement of a customer in the network is described by a Markov renewal process. Moreover, customers have attributes, such as a teletraffic state, that are driven by continuous time Markov chains and, therefore, change as they move through the network. We investigate the transient and limit number of customers in disjoint sets of nodes and attributes. These turn out to be independent Poisson random variables. We also calculate the covariances of the number of customers in two sets of nodes and attributes at different time epochs. Moreover, we conclude that the arrival process per attribute to a node is the sum of independent Poisson cluster processes and derive its univariate probability generating function. In addition, the arrival process to an outside node of the network is a non-homogeneous Poisson process. We illustrate the applications of the queueing network model and the results derived in a particular wireless network.     

On Hitting Times and Fastest Strong Stationary Times for Skip-Free and More General Chains

An (upward) skip-free Markov chain with the set of nonnegative integers as state space is a chain for which upward jumps may be only of unit size; there is no restriction on downward jumps. In a 1987 paper, Brown and Shao determined, for an irreducible continuous-time skip-free chain and any d, the passage time distribution from state 0 to state d. When the nonzero eigenvalues ν _j of the generator on {0,…,d}, with d made absorbing, are all real, their result states that the passage time is distributed as the sum of d independent exponential random variables with rates ν _j . We give another proof of their theorem. In the case of birth-and-death chains, our proof leads to an explicit representation of the passage time as a sum of independent exponential random variables. Diaconis and Miclo recently obtained the first such representation, but our construction is much simpler. We obtain similar (and new) results for a fastest strong stationary time T of an ergodic continuous-time skip-free chain with stochastically monotone time-reversal started in state 0, and we also obtain discrete-time analogs of all our results. In the paper’s final section we present extensions of our results to more general chains. Research supported by NSF grant DMS–0406104, and by The Johns Hopkins University’s Acheson J. Duncan Fund for the Advancement of Research in Statistics.     

Identities for stopped markov chains and their applications

In this paper we obtain identities for some stopped Markov chains. These identities give a unified approach to many problems in optimal stopping of a Markovian sequence, extinction probability of a Markovian branching process and martingale theory.     

Strong Approximations of Martingale Vectors and Their Applications in Markov-Chain Adaptive Designs

The strong approximations of a class of R~d-valued martingales are considered.The conditions usedin this paper are easier to check than those used in [3] and [9].As an application,the strong approximation ofa class of non-homogenous Markov chains is established,and the asymptotic properties are established for themulti-treatment Markov chain adaptive designs in clinical trials.     

First Exit Times for Ordinary and Compound Poisson Processes with Non-linear Boundaries

Distributions of the first-exit times from a region with concave upper boundary are discussed for ordinary and compound Poisson processes. Explicit formulae are developed for the case of ordinary Poisson processes. Recursive formulae are given for the compound Poisson case, where the jumps are positive, having discrete or continuous distributions with finite means. Applications to sequential point estimation and insurance are illustrated.      

On Times to Quasi-stationarity for Birth and Death Processes

The purpose of this paper is to present a probabilistic proof of the well-known result stating that the time needed by a continuous-time finite birth and death process for going from the left end to the right end of its state space is a sum of independent exponential variables whose parameters are the negatives of the eigenvalues of the underlying generator when the right end is treated as an absorbing state. The exponential variables appear as fastest strong quasi-stationary times for successive dual processes associated to the original absorbed process. As an aftermath, we get an interesting probabilistic representation of the time marginal laws of the process in terms of “local equilibria.”      

The cutoff phenomenon for random birth and death chains

For any distribution π on , we study elements drawn at random from the set of tridiagonal stochastic matrices K satisfying for all . These matrices correspond to birth and death chains with stationary distribution π. We analyze an algorithm for sampling from and use results from this analysis to draw conclusions about the Markov chains corresponding to typical elements of . Our main interest is in determining when certain sequences of random birth and death chains exhibit the cutoff phenomenon. © 2016 Wiley Periodicals, Inc. Random Struct. Alg., 50, 287–321, 2017     

二重非齐次马氏链的遍历性及其应用

给出二重非齐次马氏链的强遍历性,绝对平均强遍历性,Cesaro平均收敛的概念．利用二维马氏链的遍历性和C-K方程,建立了二维马氏链与二重非齐次马氏链遍历性的关系．并讨论了齐次二重马氏链绝对平均强遍历与强遍历的等价性．最后给出Cesaro平均收敛在马氏决策过程和信息论中应用．     

Controlled Markov Chains with Weak and Strong Interactions: Asymptotic Optimality and Applications to Manufacturing

This paper deals with the asymptotic optimality of a stochastic dynamic system driven by a singularly perturbed Markov chain with finite state space. The states of the Markov chain belong to several groups such that transitions among the states within each group occur much more frequently than transitions among the states in different groups. Aggregating the states of the Markov chain leads to a limit control problem, which is obtained by replacing the states in each group by the corresponding average distribution. The limit control problem is simpler to solve as compared with the original one. A nearly-optimal solution for the original problem is constructed by using the optimal solution to the limit problem. To demonstrate, the suggested approach of asymptotic optimal control is applied to examples of manufacturing systems of production planning.     

The steady‐state probabilities for regenerative semi‐Markov processes with application to prevention and screening

There is a growing interest in planning and implementing broad‐scale clinical trials with a focus on prevention and screening. Often, the data‐generating mechanism for such experiments can be viewed as a semi‐Markov process. In this communication, we develop general expressions for the steady‐state probabilities for regenerative semi‐Markov processes. Hence, the probability of being in a certain state at the time of recruitment to a clinical trial can be calculated. An application to breast cancer prevention is demonstrated. Copyright © 1999 John Wiley & Sons, Ltd.