Separable solutions for Markov processes in random environments

In this paper we address the problem of efficiently deriving the steady-state distribution for a continuous time Markov chain (CTMC) S evolving in a random environment E. The process underlying E is also a CTMC. S is called Markov modulated process. Markov modulated processes have been widely studied in literature since they are applicable when an environment influences the behaviour of a system. For instance, this is the case of a wireless link, whose quality may depend on the state of some random factors such as the intensity of the noise in the environment. In this paper we study the class of Markov modulated processes which exhibits separable, product-form stationary distribution. We show that several models that have been proposed in literature can be studied applying the Extended Reversed Compound Agent Theorem (ERCAT), and also new product-forms are derived. We also address the problem of the necessity of ERCAT for product-forms and show a meaningful example of product-form not derivable via ERCAT.     

Moduli of continuity of local times of strongly symmetric Markov processes via Gaussian processes

LetX be a strongly symmetric standard Markov process on a locally compact metric spaceS with 1-potential densityu ¹(x, y). Let {L _t ^y , (t, y)R ⁺×S} denote the local times ofX and letG={G(y), yS} be a mean zero Gaussian process with covarianceu ¹(x, y). In this paper results about the moduli of continuity ofG are carried over to give similar moduli of continuity results aboutL _t ^y considered as a function ofy. Several examples are given with particular attention paid to symmetric Lévy processes.The research of both authors was supported in part by a grant from the National Science Foundation. In addition the research of Professor Rosen was also supported in part by a PSC-CUNY research grant. Professor Rosen would like to thank the Israel Institute of Technology, where he spent the academic year 1989–90 and was supported, in part, by the United States-Israel Binational Science Foundation. Professor Marcus was a faculty member at Texas A&M University while some of this research was carried out.     

Strongly self-interacting processes on the circle

ABSTRACT

The purpose of this paper is to investigate the long-time behaviour for a self-interacting diffusion and a self-interacting velocity jump process. While the diffusion case has already been studied for some particular potential function, the second one, which belongs to the family of piecewise deterministic processes, is new. Depending on the underlying potential function's shape, we prove either the almost sure convergence or the recurrence for a natural extended process given by a change a variable.     

Exponential and Strong Ergodicity for Markov Processes with an Application to Queues

For an ergodic continuous-time Markov process with a particular state in its space,the authors provide the necessary and sufficient conditions for exponential and strong ergodicity in terms of the moments of the first hitting time on the state.An application to the queue length process of M/G/1 queue with multiple vacations is given.     

Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes

We obtain sufficient criteria for central limit theorems (CLTs) for ergodic continuous-time Markov chains (CTMCs). We apply the results to establish CLTs for continuous-time single birth processes. Moreover, we present an explicit expression of the time average variance constant for a single birth process whenever a CLT exists. Several examples are given to illustrate these results.     

On infinitesimal σ-fields generated by random processes

It is proved that the infinitesimal look-ahead and look-back σ-fields of a random process disagree at atmost countably many time instants.     

Large deviations for conditional Volterra processes

In this article we investigate a problem of large deviations for continuous Gaussian Volterra processes, conditioned to follow a fixed trajectory up to a fixed time T > 0, in order to establish the behavior of the process in the near future after T and to give an asymptotic estimate of the exit probability of its bridge. Some examples are considered.     

Regarch模型的几何遍历性及伴随几何遍历性

本文引入随机环境干扰,将非线性时间序列GARCH模型推广为REGARCH模型,并讨论REGARCH模型的几何遍历性及伴随几何遍历性。     

On Dynkin's Markov property of random fields associated with symmetric processes

Let p(t, x, y) be a symmetric transition density with respect to a σ-finite measure m on (E, $\text{E}$), g(x,y)=∫p(t,x,y)dt, and $\text{M}\text{={σ-finite measures μ?0:∫g(x,y)μ(}\text{d}\text{x)μ(}\text{d}\text{y)<∞}}$. There exists a Gaussian random field $\text{Φ={?}{}_{\mathrm{\mu }}\text{:μ?}\text{M}\text{}}$ with mean 0 and covariance $\text{E}\text{?}{}_{\mathrm{\mu }}\text{?}{}_{\mathrm{\nu }}\text{=∫g(x,y)μ(}\text{d}\text{x)ν(}\text{d}\text{y)}$. Letting $\text{F}\text{(B)=σ{?}{}_{\mathrm{\mu }}\text{:μ(B}{}^{\mathrm{c}}\text{)=0}}$ we consider necessary and sufficient conditions for the Markov property (MP) on sets B, C: $\text{F}$(B), $\text{F}$(C) c.i. given $\text{F}$(B ∩ C). Of crucial importance is the following, proved by Dynkin: $\text{E}\text{{?}{}_{\mathrm{\mu }}\text{∣}\text{F}\text{(B)}=?}{}_{\mathrm{\mu B}}$, where μ_B is the hitting distribution of the process corresponding to p, m with initial law μ. Another important fact is that ?_μ=?_ν iff μ, ν have the same potential. Putting these together with an additional transience assumption, we present a potential theoretic proof of the following necessary and sufficient condition for (MP) on sets B, C: For every x?E, T_B∩C=T_B+T_C∮ θ_TB=T_C+T_B∮θ_TC a.s. P^x where, for D ? $\text{E}$, T_D is the hitting time of D for the process associated with p, m. This implies a necessary condition proved by Dynkin in a recent preprint for the case where B∪C=E and B, C are finely closed.     

Comparison of hypothesis testing techniques for markov processes estimated from micro versus macro data

We estimate the parameters of a Markov chain model using two types of simulated data: micro, or actual interstate transition counts, and macro aggregate frequency. We compare, by means of Monte Carlo experiments, the validity and power for micro likelihood ratio tests with their macro counterparts, previously developed by the authors to complement standard least-squares point estimates. We consider five specific null hypotheses, including parameter stationarity, entity homogeneity, a zero-order process, a specified probability value, and equal diagonal probabilities. The results from these micro-macro comparisons should help to indicate whether micro panel data collection is justified over the use of simpler state frequency counts.     

Thresholds of moving average of stationary processes for given on target significant levels

A Markov chain is associated to a finite order one-sided moving average of a discrete time stationary Gaussian process. A method is developed to specify thresholds for given on target significant levels in the sense that in the long run the probability that the moving average process lies in [L _i , L _i+1), will be π _i , i = 0,. . . ,m. Special inputs, AR(1) and MA(1) are treated in details. This article extends the work of Soltani et al. in (Commun Stat Theory Methods 36(14):2595–2606) where the inputs were assumed to be i.i.d.; and a single threshold was considered. This research was supported by Kuwait University, Research Administration, Research Grant No.[SS08/06].     

Logarithmic Sobolev inequality and strong ergodicity for birth-death processes

We give two examples to show that the strong ergodicity and the logarithmic Sobolev inequality are incomparable for ergodic birth-death processes.     

Weak ergodicity of stationary pairwise independent processes

It is proven that a stationary process of pairwise independent random variables with values in a separable metric space is weakly ergodic, i.e. each random variable is independent of the system of invariant sets of the process. An example shows that a process of identically distributed pairwise independent random variables is in general, however, not weakly ergodic.

     

A useful technique for piecewise deterministic Markov decision processes

This note presents a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary PDMDP from the original one. The auxiliary PDMDP possesses certain desired properties, which may not be possessed by the original PDMDP. We apply this technique to risk-sensitive PDMDPs with total cost criteria, and comment on its connection with the uniformization technique.     

Central Limit Theorems for Level Functionals of Stationary Gaussian Processes and Fields

We introduce a general method, which combines the one developed by authors in 1997 and one derived from the work of Malevich,⁽¹⁷⁾ Cuzick⁽⁷⁾ and mainly Berman,⁽³⁾ to provide in an easy way a CLT for level functionals of a Gaussian process, as well as a CLT for the length of a level curve of a Gaussian field.     

Densities for infinitely divisible random processes

Let {ξ_j(t), t ∈ [0, T]} j = 1, 2 be infinitely divisible processes with distinct Poisson components and no Gaussian components. Let X be the set of all real-valued functions on [0, T] which are not identically zero, and $\text{B}$ be the σ-ring generated by the cylinder sets of ξ_j(t), j = 1, 2. Let μ_j be the measure on $\text{B}$ induced by ξ_j(t).Necessary and sufficient conditions on the projective limits of the Levy-Khinchine spectral measures of the processes are found to make μ₂ ? μ₁, and a representation for the density $\text{dμ}{}_2\text{dμ}{}_1$ is obtained.     

Properties of Hida processes on . 2. Prediction and interpolation problems for processes on

If E is an ordered set, we study the processes Y_t, t E, for which the vectorial spaces _t generated by all the conditional expectations E(Y_sβ _t) for s ≥ t have finite dimensions d(t) ≤ N. ( _t is some convenient filtration.) We first develop a geometrical approach in the general situation and give a “Goursat's representation” Y_t = Σf_i(t)M_i(t), where the M_i(t) are martingales. We then restrict us to the cases E = or E = ² and give representations of the processes by the mean of stochastic integrals of “Goursat's kernels.” The special case when Y_t is the solution of a differential equation is considered.     

On the law of the iterated logarithm for Gaussian processes

We present some optimal conditions for the compact law of the iterated logarithm of a sequence of jointly Gaussian processes in different situations. We also discuss the local law of the iterated logarithm for Gaussian processes indexed by arbitrary index sets, in particular for self-similar Gaussian processes. We apply these results to obtain the law of the iterated logarithm for compositions of Gaussian processes. Research partially supported by NSF Grant DMS-93-02583.     

随机环境中的一类非线性时间序列的伴随几何遍历性分析

本文将AR(m)-ARCH(m)模型推广为REAR(m)-ARCH(m)模型,并给出了REAR(m)-ARCH(m)具有伴随几何遍历性的一个充分条件。     

Moments of first hitting times for birth-death processes on trees

An explicit and recursive representation is presented for moments of the first hitting times of birth-death processes on trees. Based on that, the criteria on ergodicity, strong ergodicity, and l-ergodicity of the processes as well as a necessary condition for exponential ergodicity are obtained.