A copula-based approximation to Markov chains

The Markov chain is well studied and widely applied in many areas.For some Markov chains,it is infeasible to obtain the explicit expressions of their corresponding finite-dimensional distributions and sometimes it is time-consuming for computation.In this paper,we propose an approximation method for Markov chains by applying the copula theory.For this purpose,we first discuss the checkerboard copula-based Markov chain,which is the Markov chain generated by the family of checkerboard copulas.This Markov chain has some appealing properties,such as self-similarity in copulas and having explicit forms of finite-dimensional distributions.Then we prove that each Markov chain can be approximated by a sequence of checkerboard copula-based Markov chains,and the error bounds of the approximate distributions are provided.Employing the checkerboard copula-based approximation method,we propose a sufficient condition for the geometric β-mixing of copula-based Markov chains.This condition allows copulas of Markov chains to be asymmetric.Finally,by applying the approximation method,analytical recurrence formulas are also derived for computing approximate distributions of both the first passage time and the occupation time of a Markov chain,and numerical results are listed to show the approximation errors.     

Mixing times with applications to perturbed Markov chains

A measure of the “mixing time” or “time to stationarity” in a finite irreducible discrete time Markov chain is considered. The statistic , where {π_j} is the stationary distribution and m_ij is the mean first passage time from state i to state j of the Markov chain, is shown to be independent of the initial state i (so that η_i = η for all i), is minimal in the case of a periodic chain, yet can be arbitrarily large in a variety of situations. An application considering the effects perturbations of the transition probabilities have on the stationary distributions of Markov chains leads to a new bound, involving η, for the 1-norm of the difference between the stationary probability vectors of the original and the perturbed chain. When η is large the stationary distribution of the Markov chain is very sensitive to perturbations of the transition probabilities.     

Performance analysis of a multiplexer with the threshold based overload control in ATM networks

In this paper we analyze the performance of a statistical ATM multiplexer with bursty input traffic and two thresholds in the buffer by using queueing model. Two priority levels are considered for source traffic, which is modeled by Markov Modulated Poisson Process to represent the bursty characteristics. Service time distributions of two priority sources are assumed to be same and deterministic for ATM environment. The partial buffer sharing scheme with one threshold may experience a sensitive state change around the threshold. But the proposed multiplexer with two thresholds avoids this hysterical phenominon to improve the system operation.     

Strong approximation theorems for density dependent Markov chains

A variety of continuous parameter Markov chains arising in applied probability (e.g. epidemic and chemical reaction models) can be obtained as solutions of equations of the form

$\text{X}{}_{\mathrm{N}}\text{(t)=x}{}_0\text{+∑}\text{1}\text{N}\text{lY}{}_1\text{N ∫}{}^{\mathrm{t}}{}_0\text{f}{}_1\text{(X}{}_{\mathrm{N}}\text{(s))ds}$

where $\text{l∈}\text{Z}{}^{\mathrm{t}}$, the Y₁ are independent Poisson processes, and N is a parameter with a natural interpretation (e.g. total population size or volume of a reacting solution).The corresponding deterministic model, satisfies

$\text{X(t)=x}{}_0\text{+ ∫}{}^{\mathrm{t}}{}_0\text{∑ lf}{}_1\text{(X(s))ds}$

Under very general conditions lim_N→∞X_N(t)=X(t) a.s. The process X_N(t) is compared to the diffusion processes given by

$\text{Z}{}_{\mathrm{N}}\text{(t)=x}{}_0\text{+∑}\text{1}\text{N}\text{lB}{}_1\text{N∫}{}^{\mathrm{t}}{}_0\text{ft(Z}{}_{\mathrm{N}}\text{(s))ds}$

and

$\text{V(t)=∑ l∫}{}^{\mathrm{t}}{}_0\text{f}{}_1\text{(X(s))}\text{d}\text{W}\text{?}{}_1\text{+∫}{}^{\mathrm{t}}{}_0\text{?F(X(s))·V(s)}\text{ds.}$

Under conditions satisfied by most of the applied probability models, it is shown that X_N,Z_N and V can be constructed on the same sample space in such a way that

$\text{X}{}_{\mathrm{N}}\text{(t)=Z}{}_{\mathrm{N}}\text{(t)+O}\text{logN}\text{N}$

and

$\text{N}\text{(X}{}_{\mathrm{N}}\text{(t)?X(t))=V(t)+O}\text{log N}\text{N}$

     

Perturbation results for nearly uncoupled Markov chains with applications to iterative methods

Summary The standard perturbation theory for linear equations states that nearly uncoupled Markov chains (NUMCs) are very sensitive to small changes in the elements. Indeed, some algorithms, such as standard Gaussian elimination, will obtain poor results for such problems. A structured perturbation theory is given that shows that NUMCs usually lead to well conditioned problems. It is shown that with appropriate stopping, criteria, iterative aggregation/disaggregation algorithms will achieve these structured error bounds. A variant of Gaussian elimination due to Grassman, Taksar and Heyman was recently shown by O'Cinneide to achieve such bounds.Supported by the National Science Foundation under grant CCR-9000526 and its renewal, grant CCR-9201692. This research was done in part, during the author's visit to the Institute for Mathematics and its Applications, 514 Vincent Hall, 206 Church St. S.E., University of Minnesota, Minneapolis, MN 55455, USA     

The stochastic shortest-path problem for Markov chains with infinite state space with applications to nearest-neighbor lattice chains

The aim of this paper is to solve the basic stochastic shortest-path problem (SSPP) for Markov chains (MCs) with countable state space and then apply the results to a class of nearest-neighbor MCs on the lattice state space $\mathbb Z \times \mathbb Z $ whose only moves are one step up, down, to the right or to the left. The objective is to control the MC, by suppressing certain moves, so as to minimize the expected time to reach a certain given target state. We characterize the optimal policies for SSPPs for general MCs with countably infinite state space, the main tool being a verification theorem for the value function, and give an algorithmic construction. Then we apply the results to a large class of examples: nearest-neighbor MCs for which the state space $\mathbb Z \times \mathbb Z $ is split by a vertical line into two regions inside which the transition probabilities are the same for every state. We give a necessary and sufficient condition for the so-called distance-diminishing policy to be optimal. For the general case in which this condition does not hold we develop an explicit finite construction of an optimal policy.     

Interconnections of Markov chains and quasi-reversible queuing networks

A quasi-reversible queue can be associated with certain type of transactions of a Markov chain. It is shown that if Markov chains are coupled in a certain way, then to the resulting chain be associated a queuing network which is itself quasi-reversible and the stationary distribution of the chain takes the product form. The product form for mixed networks is derived from the result for open networks.     

A central limit theorem for Markov chains and applications to hypergroups

Let be a homogeneous Markov chain on an unbounded Borel subset of with a drift function which tends to a limit at infinity. Under a very simple hypothesis on the chain we prove that converges in distribution to a normal law where the variance depends on the asymptotic behaviour of . When goes to zero quickly enough and , the random centering may be replaced by These results are applied to the case of random walks on some hypergroups.

     

Transportation inequalities for hidden Markov chains and applications

We prove some transportation inequalities for hidden Markov chains, generalize the results proved by Kontorovich and Ramanan in two directions and give some applications to log-likelihood functions and hypothesis testing.     

On the rational approximation of Markov functions,with applications to the computation of Markov functions of Toeplitz matrices

Numerical Algorithms - We investigate the problem of approximating the matrix function f(A) by r(A), with f a Markov function, r a rational interpolant of f, and A a symmetric Toeplitz matrix. In a...     

Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes

We obtain sufficient criteria for central limit theorems (CLTs) for ergodic continuous-time Markov chains (CTMCs). We apply the results to establish CLTs for continuous-time single birth processes. Moreover, we present an explicit expression of the time average variance constant for a single birth process whenever a CLT exists. Several examples are given to illustrate these results.     

Markov chains with replacement

The following modification of a general state space discrete-time Markov chain is considered: certain transitions are supposed “forbidden” and the chain evolves until there is such a transition. At this instant the value of the chain is “replaced” according to a given rule, and, starting from the new value, the chain evolves normally until there is a forbidden transition again; the cycle is then repeated. The relationship of this modified process to the original one is studied in general terms, with particular emphasis being given to invariant measures. Examples are given which illustrate the results obtained.     

Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio management

In this paper we extend standard dynamic programming results for the risk sensitive optimal control of discrete time Markov chains to a new class of models. The state space is only finite, but now the assumptions about the Markov transition matrix are much less restrictive. Our results are then applied to the financial problem of managing a portfolio of assets which are affected by Markovian microeconomic and macroeconomic factors and where the investor seeks to maximize the portfolio's risk adjusted growth rate.     

Reduced systems in Markov chains and their applications in queueing theory

A reduced system is a smaller system derived in the process of analyzing a larger system. While solving for steady-state probabilities of a Markov chain, generally the solution can be found by first solving a reduced system of equations which is obtained by appropriately partitioning the transition probability matrix. In this paper, we catagorize reduced systems as standard and nonstandard and explore the existence of reduced systems and their properties relative to the original system. We also discuss first passage probabilities and means for the standard reduced system relative to the original system. These properties are illustrated while determining the steady-state probabilities and first passage time characteristics of a queueing system.     

A martingale decomposition for quadratic forms of Markov chains (with applications)

We develop a martingale-based decomposition for a general class of quadratic forms of Markov chains, which resembles the well-known Hoeffding decomposition of U$U$-statistics of i.i.d. data up to a reminder term. To illustrate the applicability of our results, we discuss how this decomposition may be used to studying the large-sample properties of certain statistics in two problems: (i) we examine the asymptotic behavior of lag-window estimators in time series, and (ii) we derive an asymptotic linear representation and limiting distribution of U$U$-statistics with varying kernels in time series. We also discuss simplified examples of interest in statistics and econometrics.