Geometric ergodicity for classes of homogeneous Markov chains

The paper deals with non asymptotic computable bounds for the geometric convergence rate of homogeneous ergodic Markov processes. Some sufficient conditions are stated for simultaneous geometric ergodicity of Markov chain classes. This property is applied to nonparametric estimation in ergodic diffusion processes.     

Asymptotically optimal pointwise and minimax quickest change-point detection for dependent data

We consider the quickest change-point detection problem in pointwise and minimax settings for general dependent data models. Two new classes of sequential detection procedures associated with the maximal “local” probability of a false alarm within a period of some fixed length are introduced. For these classes of detection procedures, we consider two popular risks: the expected positive part of the delay to detection and the conditional delay to detection. Under very general conditions for the observations, we show that the popular Shiryaev–Roberts procedure is asymptotically optimal, as the local probability of false alarm goes to zero, with respect to both these risks pointwise (uniformly for every possible point of change) and in the minimax sense (with respect to maximal over point of change expected detection delays). The conditions are formulated in terms of the rate of convergence in the strong law of large numbers for the log-likelihood ratios between the “change” and “no-change” hypotheses, specifically as a uniform complete convergence of the normalized log-likelihood ratio to a positive and finite number. We also develop tools and a set of sufficient conditions for verification of the uniform complete convergence for a large class of Markov processes. These tools are based on concentration inequalities for functions of Markov processes and the Meyn–Tweedie geometric ergodic theory. Finally, we check these sufficient conditions for a number of challenging examples (time series) frequently arising in applications, such as autoregression, autoregressive GARCH, etc.     

Sensitive equilibria for ergodic stochastic games with countable state spaces

We consider stochastic games with countable state spaces and unbounded immediate payoff functions. Our assumptions on the transition structure of the game are based on a recent work by Meyn and Tweedie [19] on computable bounds for geometric convergence rates of Markov chains. The main results in this paper concern the existence of sensitive optimal strategies in some classes of zero-sum stochastic games. By sensitive optimality we mean overtaking or 1-optimality. We also provide a new Nash equilibrium theorem for a class of ergodic nonzero-sum stochastic games with denumerable state spaces.     

Speed of stability for birth-death processes

This paper is a continuation of the study on the stability speed for Markov processes. It extends the previous study of the ergodic convergence speed to the non-ergodic one, in which the processes are even allowed to be explosive or to have general killings. At the beginning stage, this paper is concentrated on the birth-death processes. According to the classification of the boundaries, there are four cases plus one more having general killings. In each case, some dual variational formulas for the convergence rate are presented, from which, the criterion for the positivity of the rate and an approximating procedure of estimating the rate are deduced. As the first step of the approximation, the ratio of the resulting bounds is usually no more than 2. The criteria as well as basic estimates for more general types of stability are also presented. Even though the paper contributes mainly to the non-ergodic case, there are some improvements in the ergodic one. To illustrate the power of the results, a large number of examples are included.     

The tail empirical process of regularly varying functions of geometrically ergodic Markov chains

We consider a stationary regularly varying time series which can be expressed as a function of a geometrically ergodic Markov chain. We obtain practical conditions for the weak convergence of the tail array sums and feasible estimators of cluster statistics. These conditions include the so-called geometric drift or Foster–Lyapunov condition and can be easily checked for most usual time series models with a Markovian structure. We illustrate these conditions on several models and statistical applications. A counterexample is given to show a different limiting behavior when the geometric drift condition is not fulfilled.     

二重非齐次马氏链的遍历性及其应用

给出二重非齐次马氏链的强遍历性,绝对平均强遍历性,Cesaro平均收敛的概念．利用二维马氏链的遍历性和C-K方程,建立了二维马氏链与二重非齐次马氏链遍历性的关系．并讨论了齐次二重马氏链绝对平均强遍历与强遍历的等价性．最后给出Cesaro平均收敛在马氏决策过程和信息论中应用．     

连续时间非时齐马氏过程的广义Dobrushin系数的估计

本文研究了非时齐马氏过程的广义Dobrushin系数的估计问题.在将经典Dobrushin遍历系数推广为加权的遍历系数的基础上,利用了矩阵拆分的方法,得到了对这种广义遍历系数的估计方法,推广了时齐马氏过程关于遍历系数的估计结果,借此可进一步得到有关遍历性的判定结论.     

随机环境中的马氏链的不变测度与遍历性

本文考虑了一类特殊的随机环境的马氏链。假设随机“Doeblin”条件成立，我们证明了随机环境的马氏链的不变测度存在，且任何初始分布以指数收敛速度到些不变测度。进一步的，存在关于绕积算子遍历的不变测度。最后，我们得到了随机马氏链的强大数定律。     

On the class of Markov chains with finite convergence time

We study the necessary and sufficient conditions for a finite ergodic Markov chain to converge in a finite number of transitions to its stationary distribution. Using this result, we describe the class of Markov chains which attain the stationary distribution in a finite number of steps, independent of the initial distribution. We then exhibit a queueing model that has a Markov chain embedded at the points of regeneration that falls within this class. Finally, we examine the class of continuous time Markov processes whose embedded Markov chain possesses the property of rapid convergence, and find that, in the case where the distribution of sojourn times is independent of the state, we can compute the distribution of the system at time t in the form of a simple closed expression.     

Markov branching processes with killing and resurrection

In this paper, we consider Markov branching processes with killing and resurrection. We first show that the Markov branching process with killing and stable resurrection is just the Feller minimum process which is honest and thus unique. We then further show that this honest Feller minimum process is not only positive recurrent but also strongly ergodic. The generating function of the important stationary distribution is explicitly expressed. For the interest of comparison and completeness, the results of the Markov branching processes with killing and instantaneous resurrection are also briefly stated. A new result regarding strong ergodicity of this difficult case is presented. The birth and death process with killing and resurrection together with another example is also analyzed.     

An optimal portfolio model with stochastic volatility and stochastic interest rate

We consider a portfolio optimization problem under stochastic volatility as well as stochastic interest rate on an infinite time horizon. It is assumed that risky asset prices follow geometric Brownian motion and both volatility and interest rate vary according to ergodic Markov diffusion processes and are correlated with risky asset price. We use an asymptotic method to obtain an optimal consumption and investment policy and find some characteristics of the policy depending upon the correlation between the underlying risky asset price and the stochastic interest rate.     

Geometric ergodicity of Harris recurrent Marcov chains with applications to renewal theory

Let (X_n) be a positive recurrent Harris chain on a general state space, with invariant probability measure π. We give necessary and sufficient conditions for the geometric convergence of λPⁿf towards its limit π(f), and show that when such convergence happens it is, in fact, uniform over f and in L¹(π)-norm. As a corollary we obtain that, when (Xn) is geometrically ergodic, ∝ π(dx)6Pⁿ(x,·)-π6 converges to zero geometrically fast. We also characterize the geometric ergodicity of (X_n) in terms of hitting time distributions. We show that here the so-called small sets act like individual points of a countable state space chain. We give a test function criterion for geometric ergodicity and apply it to random walks on the positive half line. We apply these results to non-singular renewal processes on [0,∞) providing a probabilistic approach to the exponencial convergence of renewal measures.     

Shift ergodicity for stationary Markov processes

In this paper shift ergodicity and related topics are studied for certain stationary processes. We first present a simple proof of the conclusion that every stationary Markov process is a generalized convex combination of stationary ergodic Markov processes. A direct consequence is that a stationary distribution of a Markov process is extremal if and only if the corresponding stationary Markov process is time ergodic and every stationary distribution is a generalized convex combination of such extremal ones. We then consider space ergodicity for spin flip particle systems. We prove space shift ergodicity and mixing for certain extremal invariant measures for a class of spin systems, in which most of the typical models, such as the Voter Models and the Contact Models, are included. As a consequence of these results we see that for such systems, under each of those extremal invariant measures, the space and time means of an observable coincide, an important phenomenon in statistical physics. Our results provide partial answers to certain interesting problems in spin systems.     

Asymptotic normality of fluctuations of the procedure of stochastic approximation with diffusive perturbation in a Markov medium

We consider the asymptotic normality of a continuous procedure of stochastic approximation in the case where the regression function contains a singularly perturbed term depending on the external medium described by a uniformly ergodic Markov process. Within the framework of the scheme of diffusion approximation, we formulate sufficient conditions for asymptotic normality in terms of the existence of a Lyapunov function for the corresponding averaged equation. __________ Translated from Ukrains’kyi Matematychnyi Zhurnal, Vol. 58, No. 12, pp. 1686–1692, December, 2006.     

Renewal theory for extremal Markov sequences of Kendall type

The paper deals with renewal theory for a class of extremal Markov sequences connected with the Kendall convolution. We consider here some particular cases of the Wold processes associated with generalized convolutions. We prove an analogue of the Fredholm theorem for all regular generalized convolutions algebras. Using regularly varying functions we prove a Blackwell theorem and a limit theorem for renewal processes defined by Kendall random walks.Our results set new research hypotheses for other generalized convolution algebras to investigate renewal processes constructed by Markov processes with respect to generalized convolutions.     

Coupling for Markov renewal processes and the rate of convergence in ergodic theorems for processes with semi-Markov switchings

Coupling procedures for Markov renewal processes are described. Applications to ergodic theorems for processes with semi-Markov switchings are considered.This paper was partly prepared with the support of NFR Grant F-UP 10257-300.     

The limiting distribution for the infinitely deep dam with a Markovian input

The paper outlines a case for taking greater interest in the bottomless, or infinitely deep, dam model in Hydrology. It then shows that for such a model with unit withdrawals and an ergodic Markov chain input process the limiting distribution of depletion, when this exists, is a zero modified geometric distribution. This result generalises the well known result for independent inputs. The technical conditions required for the proof are satisfied for finite state space input processes and are shown to be satisfied by certain infinite state space input processes. These include as special cases examples which have a negative binomial limiting input distribution.     

Ruin probabilities for a risk model with two classes of claims

In this paper we consider a risk model with two kinds of claims, whose claims number processes are Poisson process and ordinary renewal process respectively. For this model, the surplus process is not Markovian, however, it can be Markovianized by introducing a supplementary process, We prove the Markov property of the related vector processes. Because such obtained processes belong to the class of the so-called piecewise-deterministic Markov process, the extended infinitesimal generator is derived, exponential martingale for the risk process is studied. The exponential bound of ruin probability in iafinite time horizon is obtained.     

Law of the iterated logarithm for the periodogram

We consider the almost sure asymptotic behavior of the periodogram of stationary and ergodic sequences. Under mild conditions we establish that the limsup of the periodogram properly normalized identifies almost surely the spectral density function associated with the stationary process. Results for a specified frequency are also given. Our results also lead to the law of the iterated logarithm for the real and imaginary parts of the discrete Fourier transform. The proofs rely on martingale approximations combined with results from harmonic analysis and techniques from ergodic theory. Several applications to linear processes and their functionals, iterated random functions, mixing structures and Markov chains are also presented.     

Asymptotic properties of nonlinear autoregressive Markov processes with state-dependent switching

In this paper, we consider a class of nonlinear autoregressive (AR) processes with state-dependent switching, which are two-component Markov processes. The state-dependent switching model is a nontrivial generalization of Markovian switching formulation and it includes the Markovian switching as a special case. We prove the Feller and strong Feller continuity by means of introducing auxiliary processes and making use of the Radon-Nikodym derivatives. Then, we investigate the geometric ergodicity by the Foster-Lyapunov inequality. Moreover, we establish the V-uniform ergodicity by means of introducing additional auxiliary processes and by virtue of constructing certain order-preserving couplings of the original as well as the auxiliary processes. In addition, illustrative examples are provided for demonstration.