A higher order Markov model for analyzing covariate dependence

During the recent past, there has been a renewed interest in Markov chain for its attractive properties for analyzing real life data emerging from time series or longitudinal data in various fields. The models were proposed for fitting first or higher order Markov chains. However, there is a serious lack of realistic methods for linking covariate dependence with transition probabilities in order to analyze the factors associated with such transitions especially for higher order Markov chains. L.R. Muenz and L.V. Rubinstein [Markov models for covariate dependence of binary sequences, Biometrics 41 (1985) 91–101] employed logistic regression models to analyze the transition probabilities for a first order Markov model. The methodology is still far from generalization in terms of formulating a model for higher order Markov chains. In this study, it is aimed to provide a comprehensive covariate-dependent Markov model for higher order. The proposed model generalizes the estimation procedure for Markov models for any order. The proposed models and inference procedures are simple and the covariate dependence of the transition probabilities of any order can be examined without making the underlying model complex. An example from rainfall data is illustrated in this paper that shows the utility of the proposed model for analyzing complex real life problems. The application of the proposed method indicates that the higher order covariate dependent Markov models can be conveniently employed in a very useful manner and the results can provide in-depth insights to both the researchers and policymakers to resolve complex problems of underlying factors attributing to different types of transitions, reverse transitions and repeated transitions. The estimation and test procedures can be employed for any order of Markov model without making the theory and interpretation difficult for the common users.     

高阶马尔可夫链的降阶模型及其应用

本文通过对传统高阶马尔可夫链模型的状态空间进行阶数重构,导出一个在重构状态空间上的降阶马尔可夫模型。理论分析证明,降阶马尔可夫链模型不但可以描述传统高阶马尔可夫链模型的全部性态,更能表达较传统模型细微的随机结构。然后,应用降阶模型对我国股票指数的动态变化进行实证分析,讨论了阶数的选取和高阶马尔可夫性检验,分析了股票市场内在波动结构。最后,对股指序列作出短期与长期的预测分析。     

Compatibility of multi-wave panel data and the continuous-time homogeneous Markov chain. An analysis of a continuous-time process by means of discrete-time longitudinal observations

Multi-wave panel data are observations at two or more points in time on a continuously changing attribute of interest (e.g. behaviour). In this paper, the adequacy of the continuous-time homogeneous Markov chain (CTHMC) model is assessed for describing the process of change underlying such data. In the case of equidistant observational times, it may happen that the maximum-likelihood estimate of the transition probability matrix between successive observational times from these data cannot arise from a CTHMC. It is investigated whether this event can be ascribed to chance through the introduction of an hypothesis test. © 1998 John Wiley & Sons, Ltd.     

基于改进的GM(1,1)-Markov链组合模型广东省单位GDP能耗预测

本文以灰色系统理论的GM(1,1)模型和随机过程理论的Markov链模型为基础构建了一个动态GM(1,1)-Markov链组合预测模型。该模型同时利用了GM(1,1)模型对序列趋势因素良好的拟合能力和Markov链模型对残差序列信息的提取能力。为进一步提高该模型的预测精度,用泰勒(Taylor)近似方法和新信息优先的思想对该模型进行了改进。最后,以1991-2014年广东省单位GDP能耗数据实证了该模型的预测效果。     

基于改进灰色GM(1,1)马尔柯夫模型的煤矿事故百万吨死亡率预测

煤矿安全事故预防和控制是煤矿安全评价和决策的基础.灰色预测适合于时间短、数据量少和波动不大的系统对象,而马尔可夫链理论适用于预测随机波动大的动态过程.结合灰色预测GM(1,1)模型和马尔可夫链理论的优点,提出了一种改进的灰色马尔可夫GM(1,1)模型.利用改进的GM(1,1)模型进一步拟合煤矿人因失误事故的发展变化趋势,并以此为基础进行马尔柯夫预测,提高预测效果.以2000-2010年全国煤矿事故百万吨死亡率为例进行了预测分析,结果表明模型既能揭示煤矿人因失误事故百万吨死亡率变化的总体趋势,又能克服随机波动性数据对预测精度的影响,具有较强的工程实用性,并对煤矿人因失误安全事故的预测和控制有一定的实际意义.     

A different construction of Gaussian fields from Markov chains: Dirichlet covariances

We study a class of Gaussian random fields with negative correlations. These fields are easy to simulate. They are defined in a natural way from a Markov chain that has the index space of the Gaussian field as its state space. In parallel with Dynkin's investigation of Gaussian fields having covariance given by the Green's function of a Markov process, we develop connections between the occupation times of the Markov chain and the prediction properties of the Gaussian field. Our interest in such fields was initiated by their appearance in random matrix theory.     

Maintenance Decision Support: Analysing Crankcase Lubricant Condition by Markov Process Modelling

The probabilistic changes in certain measurable physical parameters of a lubricant are used to develop a mathematical model of the physical-chemical deterioration process taking place in the crankcase of locomotive diesel engines. A seven-state Markov chain is formulated with actual data taken from the year 1985. Statistical inference is used to establish that the chain is of the first order and that it is applicable across the fleet of locomotives in the study. The Markov model is shown to have the twin advantages of enhancing one's understanding of the deterioration process and of providing a means for predicting future changes. The model also lends itself to the specification of input data.     

A Probabilistic Model of the Fatigue Life of Composite Materials for Fatigue-Curve Approximations

A simple mathematical model for the fatigue life of composite materials in which the basic load is taken up by parallel rigid components (fiber strands) is offered. Based on the Markov chain theory, the model allows us to obtain an S-type curve for the growth of internal stresses and to calculate the relationship between the parameters of static strength distribution and the S-N fatigue curve (Wholer curve). The model is too simple and does not provide a numerical coincidence with experimental fatigue-test data, but, when used with additional parameters, it agrees with experiments and can be applied to the model of nonlinear regression in describing the fatigue curve and its variation as the parameters of static strength change.     

A Retrial BMAP/PH/N System

A multi-server retrial queueing model with Batch Markovian Arrival Process and phase-type service time distribution is analyzed. The continuous-time multi-dimensional Markov chain describing the behavior of the system is investigated by means of reducing it to the corresponding discrete-time multi-dimensional Markov chain. The latter belongs to the class of multi-dimensional quasi-Toeplitz Markov chains in the case of a constant retrial rate and to the class of multi-dimensional asymptotically quasi-Toeplitz Markov chains in the case of an infinitely increasing retrial rate. It allows to obtain the existence conditions for the stationary distribution and to elaborate the algorithms for calculating the stationary state probabilities.     

Generalization bounds of ERM algorithm with Markov chain samples

One of the main goals of machine learning is to study the generalization performance of learning algorithms. The previous main results describing the generalization ability of learning algorithms are usually based on independent and identically distributed （i.i.d.） samples. However, independence is a very restrictive concept for both theory and real-world applications. In this paper we go far beyond this classical framework by establishing the bounds on the rate of relative uniform convergence for the Empirical Risk Minimization （ERM） algorithm with uniformly ergodic Markov chain samples. We not only obtain generalization bounds of ERM algorithm, but also show that the ERM algorithm with uniformly ergodic Markov chain samples is consistent. The established theory underlies application of ERM type of learning algorithms.     

Markov modulated Bernoulli process

We consider a Bernoulli process where the success probability changes with respect to a Markov chain. Such a model represents an interesting application of stochastic processes where the parameters are not constants; rather, they are stochastic processes themselves due to their dependence on a randomly changing environment. The model operates in a random environment depicted by a Markov chain so that the probability of success at each trial depends on the state of the environment. We will concentrate, in particular, on applications in reliability theory to motivate our model. The analysis will focus on transient as well as long-term behaviour of various processes involved.     

A comparison of some criteria for states selection in the latent Markov model for longitudinal data

We compare different selection criteria to choose the number of latent states of a multivariate latent Markov model for longitudinal data. This model is based on an underlying Markov chain to represent the evolution of a latent characteristic of a group of individuals over time. Then, the response variables observed at different occasions are assumed to be conditionally independent given this chain. Maximum likelihood estimation of the model is carried out through an Expectation–Maximization algorithm based on forward–backward recursions which are well known in the hidden Markov literature for time series. The selection criteria we consider are based on penalized versions of the maximum log-likelihood or on the posterior probabilities of belonging to each latent state, that is, the conditional probability of the latent state given the observed data. Among the latter criteria, we propose an appropriate entropy measure tailored for the latent Markov models. We show the results of a Monte Carlo simulation study aimed at comparing the performance of the above states selection criteria on the basis of a wide set of model specifications.     

Modelling the profitability of credit cards by Markov decision processes

This paper derives a Markov decision process model for the profitability of credit cards, which allows lenders to find an optimal dynamic credit limit policy. The states of the system are based on the borrower’s behavioural score and the decisions are what credit limit to give the borrower each period. In determining which Markov chain best describes the borrower’s performance, second order as well as first order Markov chains are considered and estimation procedures developed that deal with the low default levels that may exist in the data. A case study is given in which the optimal credit limit is derived and the results compared with the actual outcomes.     

证券组合的Markov链模型

本文将证券价格时间序列分解成趋势变动序列和 Markov链 ,建立了证券组合的 Markov链模型 ,应用 Markov链理论对此模型进行了分析 ,给出了充分大的一个时间内的收益率 ,风险和切点组合的计算公式     

Pricing annuity guarantees under a double regime-switching model

This paper is concerned with the valuation of equity-linked annuities with mortality risk under a double regime-switching model, which provides a way to endogenously determine the regime-switching risk. The model parameters and the reference investment fund price level are modulated by a continuous-time, finite-time, observable Markov chain. In particular, the risk-free interest rate, the appreciation rate, the volatility and the martingale describing the jump component of the reference investment fund are related to the modulating Markov chain. Two approaches, namely, the regime-switching Esscher transform and the minimal martingale measure, are used to select pricing kernels for the fair valuation. Analytical pricing formulas for the embedded options underlying these products are derived using the inverse Fourier transform. The fast Fourier transform approach is then used to numerically evaluate the embedded options. Numerical examples are provided to illustrate our approach.     

Controlling jumps in correlated processes of Poisson counts

Processes of autocorrelated Poisson counts can often be modelled by a Poisson INAR(1) model, which proved to apply well to typical tasks of SPC. Statistical properties of this model are briefly reviewed. Based on these properties, we propose a new control chart: the combined jumps chart. It monitors the counts and jumps of a Poisson INAR(1) process simultaneously. As the bivariate process of counts and jumps is a homogeneous Markov chain, average run lengths (ARLs) can be computed exactly with the well‐known Markov chain approach. Based on an investigation of such ARLs, we derive design recommendations and show that a properly designed chart can be applied nearly universally. This is also demonstrated by a real‐data example from the insurance field. Copyright © 2008 John Wiley & Sons, Ltd.     

Spatial Regression Models for Extremes

Meteorological data are often recorded at a number of spatial locations. This gives rise to the possibility of pooling data through a spatial model to overcome some of the limitations imposed on an extreme value analysis by a lack of information. In this paper we develop a spatial model for extremes based on a standard representation for site-wise extremal behavior, combined with a spatial latent process for parameter variation over the region. A smooth, but possibly non-linear, spatial structure is an intrinsic feature of the model, and difficulties in computation are solved using Markov chain Monte Carlo inference. A simulation study is carried out to illustrate the potential gain in efficiency achieved by the spatial model. Finally, the model is applied to data generated from a climatological model in order to characterize the hurricane climate of the Gulf and Atlantic coasts of the United States.     

Testing the Order of Markov Dependence in DNA Sequences

DNA or protein sequences are usually modeled as probabilistic phenomena. The simplest model is created on the assumption that the nucleotides at the various sites are independently distributed. Usually the type of nucleotide at some site depends on the type at another site and therefore the DNA sequence is modeled as a Markov chain of random variables taking on the values A, G, C and T corresponding to the four nucleotides. First order or higher order Markov models provide better fit to a DNA sequence. Based on this remark, the aim of this paper is to present and study a family of test statistics for testing order Markov dependence in DNA sequences. This new family includes as a particular case the classical likelihood ratio test. A simulation study is presented in order to find test statistics, in this family, with a better behaviour than the likelihood ratio test.     

Regularity criterion for a mathematical model of the laser

There exists a deep relationship between the nonexplosion conditions for Markov evolution in classical and quantum probability theories. Both of these conditions are equivalent to the preservation of the unit operator (total probability) by a minimal Markov semigroup. In this work, we study the Heisenberg evolution describing an interaction between the chain ofN two-level atoms andn-mode external Bose field, which was considered recently by J. Alli and J. Sewell. The unbounded generator of the Markov evolution of observables acts in the von Neumann algebra. For the generator of a Markov semigroup, we prove a nonexplosion condition, which is the operator analog of a similar condition suggested by R. Z. Khas’minski and later by T. Taniguchi for classical stochastic processes. For the operator algebra situation, this condition ensures the uniqueness and conservativity of the quantum dynamical semigroup describing the Markov evolution of a quantum system. In the regular case, the nonexplosion condition establishes a one-to-one relation between the formal generator and the infinitesimal operator of the Markov semigroup. Translated fromMatematicheskie Zemetki, Vol. 67, No. 5, pp. 788–796, May, 2000.     

Discrete-time analysis of the GI/G/1 system with Bernoulli retrials: An algorithmic approach

In this paper, we show that the discrete GI/G/1 system with Bernoulli retrials can be analyzed as a level-dependent QBD process with infinite blocks; these blocks are finite when both the inter-arrival and service times have finite supports. The resulting QBD has a special structure which makes it convenient to analyze by the Matrix-analytic method (MAM). By representing both the inter-arrival and service times using a Markov chain based approach we are able to use the tools for phase type distributions in our model. Secondly, the resulting phase type distributions have additional structures which we exploit in the development of the algorithmic approach. The final working model approximates the level-dependent Markov chain with a level independent Markov chain that has a large set of boundaries. This allows us to use the modified matrix-geometric method to analyze the problem. A key task is selecting the level at which this level independence should begin. A procedure for this selection process is presented and then the distribution of the number of jobs in the orbit is obtained. Numerical examples are presented to demonstrate how this method works.