Exponential rate of convergence for some Markov operators

The exponential rate of convergence for Markov operators is established. The operators correspond to continuous iterated function systems which are a very useful tool in some cell cycle models.     

Exponential convergence rate in entropy

The exponential convergence rate in entroy is studied for symmetric forms, with a special attention to the Markov chain with a state space having two points only. Some upper and lower bounds of the rate are obtained and five examples with precise or qualitatively exact estimates are presented.      

Exponential bounds for discrete-time singularly perturbed Markov chains

This paper develops exponential type upper bounds for scaled occupation measures of singularly perturbed Markov chains in discrete time. By considering two-time scale in the Markov chains, asymptotic analysis is carried out. The cases of the fast changing transition probability matrix is irreducible and that are divisible into l ergodic classes are examined first; the upper bounds of a sequence of scaled occupation measures are derived. Then extensions to Markov chains involving transient states and/or nonhomogeneous transition probabilities are dealt with. The results enable us to further our understanding of the underlying Markov chains and related dynamic systems, which is essential for solving many control and optimization problems.     

Exponential convergence rate in Boltzmann-Shannon entropy

This paper deals with the optimal exponential convergence rate β to the equilibrium state in Boltzmann-Shannon entropy for general Markov semigroups. We prove a variational formula of β, and then discuss the relation among β, spectral gap λ and logarithmic Sobolev constant α, which is read as λ≥β≥α.     

On approximation of smoothing probabilities for hidden Markov models

We consider the smoothing probabilities of hidden Markov model (HMM). We show that under fairly general conditions for HMM, the exponential forgetting still holds, and the smoothing probabilities can be well approximated with the ones of double-sided HMM. This makes it possible to use ergodic theorems. As an application we consider the pointwise maximum a posteriori segmentation, and show that the corresponding risks converge.     

A correction on approximation of smoothing probabilities for hidden Markov models

In this note, we correct a mistake concerning Theorem 2.1 in Lember (2011a).     

Bayesian Monte Carlo estimation for profile hidden Markov models

Hidden Markov models are used as tools for pattern recognition in a number of areas, ranging from speech processing to biological sequence analysis. Profile hidden Markov models represent a class of so-called “left–right” models that have an architecture that is specifically relevant to classification of proteins into structural families based on their amino acid sequences. Standard learning methods for such models employ a variety of heuristics applied to the expectation-maximization implementation of the maximum likelihood estimation procedure in order to find the global maximum of the likelihood function. Here, we compare maximum likelihood estimation to fully Bayesian estimation of parameters for profile hidden Markov models with a small number of parameters. We find that, relative to maximum likelihood methods, Bayesian methods assign higher scores to data sequences that are distantly related to the pattern consensus, show better performance in classifying these sequences correctly, and continue to perform robustly with regard to misspecification of the number of model parameters. Though our study is limited in scope, we expect our results to remain relevant for models with a large number of parameters and other types of left–right hidden Markov models.     

Leroux's method for general hidden Markov models

The method introduced by Leroux [Maximum likelihood estimation for hidden Markov models, Stochastic Process Appl. 40 (1992) 127–143] to study the exact likelihood of hidden Markov models is extended to the case where the state variable evolves in an open interval of the real line. Under rather minimal assumptions, we obtain the convergence of the normalized log-likelihood function to a limit that we identify at the true value of the parameter. The method is illustrated in full details on the Kalman filter model.     

Risk-sensitive and risk-neutral control for continuous-time hidden Markov models

In this paper the optimal control of a continuous-time hidden Markov model is discussed. The risk-sensitive problem involves a cost function which has an exponential form and a risk parameter, and is solved by defining an appropriate information state and dynamic programming. As the risk parameter tends to zero, the classical risk-neutral optimal control problem is recovered. The limits are proved using viscosity solution methods.The first author wishes to acknowledge the funding of the activities of the Cooperative Research Centre for Robust and Adaptive Systems by the Australian Commonwealth Government under the Cooperative Research Centers Program. The support of NSERC Grant A7964 is acknowledged by the second author, as is the hospitality of the Department of Systems Engineering and the Cooperative Research Centre for Robust and Adaptive Systems, Australian National University, in July 1993.     

Wasserstein convergence rates for random bit approximations of continuous Markov processes

We determine the convergence speed of a numerical scheme for approximating one-dimensional continuous strong Markov processes. The scheme is based on the construction of certain Markov chains whose laws can be embedded into the process with a sequence of stopping times. Under a mild condition on the process' speed measure we prove that the approximating Markov chains converge at fixed times at the rate of 1/4 with respect to every p-th Wasserstein distance. For the convergence of paths, we prove any rate strictly smaller than 1/4. Our results apply, in particular, to processes with irregular behavior such as solutions of SDEs with irregular coefficients and processes with sticky points.     

On geometric convergence rate of Markov search towards the fat target

Let $f:A\to \mathbb{R}$ be a continuous function with the minimal value $f^{?}$, where $A$ is the compact metric space. Let ${\left\{X_t\right\}}_{t\in \mathbb{N}}$ be a Markov chain which represents the global optimization process on $A$. We present sufficient conditions for very strong, geometric convergence mode of the form $Ef\left(X_t\right)?f^1\le c^t?(Ef\left(X_0\right)?f^1)$, where $c\in (0,1)$ is some constant. This convergence mode is natural if the set of global minima is fat.     

Optimal filters for a hidden Markov random field model

A Markov random field (MRF) is a useful technical tool for modeling dynamics systems exhibiting some type of spatio-temporal variability. In this paper, we propose optimal filters for the states of a partially observed temporal Markov random field. We also discuss parameters estimation. This generalizes an earlier work by Elliott and Aggoun [1].     

Optimal convergence rate for Yosida approximations of bounded holomorphic semigroups

In this paper, we obtain optimal bounds for convergence rate for Yosida approximations of bounded holomorphic semigroups. We also provide asymptotic expansions for semigroups in terms of Yosida approximations and obtain optimal error bounds for these expansions.     

Exponential Bounds for Ruin Probability in Two Moving Average Risk Models with Constant Interest Rate

The authors consider two discrete-time insurance risk models. Two moving average risk models are introduced to model the surplus process, and the probabilities of ruin are examined in models with a constant interest force. Exponential bounds for ruin probabilities of an infinite time horizon are derived by the martingale method.     

Spectral gap and convergence rate for discrete-time Markov chains

Let P be a transition matrix which is symmetric with respect to a measure π.The spectral gap of P in L2(π)-space,denoted by gap(P),is defined as the distance between 1 and the rest of the spectrum of P.In this paper,we study the relationship between gap(P) and the convergence rate of Pn.When P is transient,the convergence rate of P n is equal to 1 gap(P).When P is ergodic,we give the explicit upper and lower bounds for the convergence rate of Pn in terms of gap(P).These results are extended to L∞(π)-space.     

Switching diffusion logistic models involving singularly perturbed Markov chains: Weak convergence and stochastic permanence

Focusing on stochastic dynamics involve continuous states as well as discrete events, this article investigates stochastic logistic model with regime switching modulated by a singular Markov chain involving a small parameter. This Markov chain undergoes weak and strong interactions, where the small parameter is used to reflect rapid rate of regime switching among each state class. Two-time-scale formulation is used to reduce the complexity. We obtain weak convergence of the underlying system so that the limit has much simpler structure. Then we utilize the structure of limit system as a bridge, to invest stochastic permanence of original system driving by a singular Markov chain with a large number of states. Sufficient conditions for stochastic permanence are obtained. A couple of examples and numerical simulations are given to illustrate our results.     

Interactive hidden Markov models and their applications

^** Email: wching{at}hkusua.hku.hk In this paper, we propose an Interactive hidden Markov model(IHMM). In a traditional HMM, the observable states are affecteddirectly by the hidden states, but not vice versa. In the proposedIHMM, the transitions of hidden states depend on the observablestates. We also develop an efficient estimation method for themodel parameters. Numerical examples on the sales demand dataand economic data are given to demonstrate the applicabilityof the model.     

Universally consistent conditionalU-statistics for absolutely regular processes and its applications for hidden Markov models

A general class of conditionalU-statistics was introduced by W. Stute as a generalization of the Nadaraya-Watson estimates of a regression function. It was shown that such statistics are universally consistent. Also, universal consistentcies of the window andk _n -nearest neighbor estimators (as two special cases of the conditionalU-statistics) were proved. In this paper, we extend these results from the independent case to dependent case. The result is applied to verify the Bayes risk consistency of the corresponding discrimination rules. Research supported by the Office of Naval Research Contract N00014-91-J-1020.     

The rate of convergence for backwards products of a convergent sequence of finite Markov matrices

Recent papers have shown that Π^∞_{k = 1}P(k) = lim_m→∞ (P(m) ? P(1)) exists whenever the sequence of stochastic matrices {P(k)}^∞_{k = 1} exhibits convergence to an aperiodic matrix P with a single subchain (closed, irreducible set of states). We show how the limit matrix depends upon P(1).In addition, we prove that lim_m→∞lim_n→∞ (P(n + m) ? P(m + 1)) exists and equals the invariant probability matrix associated with P. The convergence rate is determined by the rate of convergence of {P(k)}^∞_{k = 1} towards P.Examples are given which show how these results break down in case the limiting matrix P has multiple subchains, with {P(k)}^∞_{k = 1} approaching the latter at a less than geometric rate.