A higher order Markov model for analyzing covariate dependence

During the recent past, there has been a renewed interest in Markov chain for its attractive properties for analyzing real life data emerging from time series or longitudinal data in various fields. The models were proposed for fitting first or higher order Markov chains. However, there is a serious lack of realistic methods for linking covariate dependence with transition probabilities in order to analyze the factors associated with such transitions especially for higher order Markov chains. L.R. Muenz and L.V. Rubinstein [Markov models for covariate dependence of binary sequences, Biometrics 41 (1985) 91–101] employed logistic regression models to analyze the transition probabilities for a first order Markov model. The methodology is still far from generalization in terms of formulating a model for higher order Markov chains. In this study, it is aimed to provide a comprehensive covariate-dependent Markov model for higher order. The proposed model generalizes the estimation procedure for Markov models for any order. The proposed models and inference procedures are simple and the covariate dependence of the transition probabilities of any order can be examined without making the underlying model complex. An example from rainfall data is illustrated in this paper that shows the utility of the proposed model for analyzing complex real life problems. The application of the proposed method indicates that the higher order covariate dependent Markov models can be conveniently employed in a very useful manner and the results can provide in-depth insights to both the researchers and policymakers to resolve complex problems of underlying factors attributing to different types of transitions, reverse transitions and repeated transitions. The estimation and test procedures can be employed for any order of Markov model without making the theory and interpretation difficult for the common users.     

基于输入输出线性化方法的一类非线性系统观测器设计

研究一类非线性系统的观测器设计问题.基于输入输出线性化方法提出了一类非线性系统的观测器设计.并且此非线性系统具有多输入多输出的特点,证明了在适当条件下,提出的观测器保证了观测误差渐近趋于零.仿真例表明了所得结果的有效性.     

Parameter estimation for partially observable systems subject to random failure

In this paper, we present a parameter estimation procedure for a condition‐based maintenance model under partial observations. Systems can be in a healthy or unhealthy operational state, or in a failure state. System deterioration is driven by a continuous time homogeneous Markov chain and the system state is unobservable, except the failure state. Vector information that is stochastically related to the system state is obtained through condition monitoring at equidistant sampling times. Two types of data histories are available — data histories that end with observable failure, and censored data histories that end when the system has been suspended from operation but has not failed. The state and observation processes are modeled in the hidden Markov framework and the model parameters are estimated using the expectation–maximization algorithm. We show that both the pseudolikelihood function and the parameter updates in each iteration of the expectation–maximization algorithm have explicit formulas. A numerical example is developed using real multivariate spectrometric oil data coming from the failing transmission units of 240‐ton heavy hauler trucks used in the Athabasca oil sands of Alberta, Canada. Copyright © 2012 John Wiley & Sons, Ltd.     

Realization of multi-input/multi-output switched linear systems from Markov parameters

This paper presents a four-stage algorithm for the realization of multi-input/multi-output (MIMO) switched linear systems (SLSs) from Markov parameters. In the first stage, a linear time-varying (LTV) realization that is topologically equivalent to the true SLS is derived from the Markov parameters assuming that the discrete states have a common MacMillan degree and a mild condition on their dwell times holds. In the second stage, stationary point set of a Hankel matrix with fixed dimensions built from the Markov parameters is examined. Splitting of this set into disjoint intervals and complements reveals linear time-invariant dynamics prevailing on these intervals. Clustering over a feature space permits recovery of the discrete states up to similarity transformations which is complete if a unimodality assumption holds and the discrete states satisfy a residence requirement. In the third stage, the switching sequence is estimated by three schemes. The first scheme is non-iterative in time. The second scheme is based on matching the estimated and the true Markov parameters of the SLS system over segments. The third scheme works also on the same principle, but it is a discrete optimization/hypothesis testing algorithm. The three schemes operate on different dwell time and model structure requirements, but the dwell time requirements are weaker than that needed to recover the discrete states. In the fourth stage, the discrete state estimates are brought to a common basis by a novel basis transformation which is necessary for predicting outputs to prescribed inputs. Robustness of the four-stage algorithm to amplitude bounded noise is studied and it is shown that small perturbations may only produce small deviations in the estimates vanishing as noise amplitude diminishes. Time complexities of the stages are also studied. A numerical example illustrates the derived results.     

Computational issues in parameter estimation for stationary hidden Markov models

The parameters of a hidden Markov model (HMM) can be estimated by numerical maximization of the log-likelihood function or, more popularly, using the expectation–maximization (EM) algorithm. In its standard implementation the latter is unsuitable for fitting stationary hidden Markov models (HMMs). We show how it can be modified to achieve this. We propose a hybrid algorithm that is designed to combine the advantageous features of the two algorithms and compare the performance of the three algorithms using simulated data from a designed experiment, and a real data set. The properties investigated are speed of convergence, stability, dependence on initial values, different parameterizations. We also describe the results of an experiment to assess the true coverage probability of bootstrap-based confidence intervals for the parameters.     

Parameter cascades and profiling in functional data analysis

A data smoothing method is described where the roughness penalty depends on a parameter that must be estimated from the data. Three levels of parameters are involved in this situation: Local parameters are the coefficients of the basis function expansion defining the smooth, global parameters define low-dimensional trend and the roughness penalty, and a complexity parameter controls the amount of roughness in the smooth. By defining local parameters as regularized functions of global parameters, and global parameters in turn as functions of complexity parameter, we define a parameter cascade, and show that the accompanying multi-criterion optimization problem leads to good estimates of all levels of parameters and their precisions. The approach is illustrated with real and simulated data, and this application is a prototype for a wide range of problems involving nuisance or local parameters.     

Optimal Bayesian fault prediction scheme for a partially observable system subject to random failure

A new method for predicting failures of a partially observable system is presented. System deterioration is modeled as a hidden, 3-state continuous time homogeneous Markov process. States 0 and 1, which are not observable, represent good and warning conditions, respectively. Only the failure state 2 is assumed to be observable. The system is subject to condition monitoring at equidistant, discrete time epochs. The vector observation process is stochastically related to the system state. The objective is to develop a method for optimally predicting impending system failures. Model parameters are estimated using EM algorithm and a cost-optimal Bayesian fault prediction scheme is proposed. The method is illustrated using real data obtained from spectrometric analysis of oil samples collected at regular time epochs from transmission units of heavy hauler trucks used in mining industry. A comparison with other methods is given, which illustrates effectiveness of our approach.     

Computational Bayesian Analysis of Hidden Markov Models

Abstract

Versions of the Gibbs Sampler are derived for the analysis of data from hidden Markov chains and hidden Markov random fields. The principal new development is to use the pseudolikelihood function associated with the underlying Markov process in place of the likelihood, which is intractable in the case of a Markov random field, in the simulation step for the parameters in the Markov process. Theoretical aspects are discussed and a numerical study is reported.     

Moderate deviations for time-varying dynamic systems driven by non-homogeneous Markov chains with Two-time Scales

Motivated by problems arising in time-dependent queues and dynamic systems with random environment, this work develops moderate deviations principles for dynamic systems driven by a fast-varying non-homogeneous Markov chain in continuous time. A distinct feature is that the Markov chain is time dependent or inhomogeneous, so are the dynamic systems. Under irreducibility of the non-homogeneous Markov chain, moderate deviations of a non-homogeneous functional are established first. With the help of a martingale problem formulation and a functional central limit theorem for the two timescale system, both upper and lower bounds of moderate deviations are obtained for the rapidly fluctuating Markovian systems. Then applications to queueing systems and dynamic systems modulated by a fast-varying Markov chain are examined.     

Bayesian Monte Carlo estimation for profile hidden Markov models

Hidden Markov models are used as tools for pattern recognition in a number of areas, ranging from speech processing to biological sequence analysis. Profile hidden Markov models represent a class of so-called “left–right” models that have an architecture that is specifically relevant to classification of proteins into structural families based on their amino acid sequences. Standard learning methods for such models employ a variety of heuristics applied to the expectation-maximization implementation of the maximum likelihood estimation procedure in order to find the global maximum of the likelihood function. Here, we compare maximum likelihood estimation to fully Bayesian estimation of parameters for profile hidden Markov models with a small number of parameters. We find that, relative to maximum likelihood methods, Bayesian methods assign higher scores to data sequences that are distantly related to the pattern consensus, show better performance in classifying these sequences correctly, and continue to perform robustly with regard to misspecification of the number of model parameters. Though our study is limited in scope, we expect our results to remain relevant for models with a large number of parameters and other types of left–right hidden Markov models.     

Bayesian Registration of Functions With a Gaussian Process Prior

We present a Bayesian framework for registration of real-valued functional data. At the core of our approach is a series of transformations of the data and functional parameters, developed under a differential geometric framework. We aim to avoid discretization of functional objects for as long as possible, thus minimizing the potential pitfalls associated with high-dimensional Bayesian inference. Approximate draws from the posterior distribution are obtained using a novel Markov chain Monte Carlo (MCMC) algorithm, which is well suited for estimation of functions. We illustrate our approach via pairwise and multiple functional data registration, using both simulated and real datasets. Supplementary material for this article is available online.     

Non-commutative Markov chains and multi-analytic operators

We study a model of repeated interaction between quantum systems which can be thought of as a non-commutative Markov chain. It is shown that there exists an outgoing Cuntz scattering system associated to this model which induces an input-output formalism with a transfer function corresponding to a multi-analytic operator, in the sense of multivariate operator theory. Finally we show that observability for this system is closely related to the scattering theory of non-commutative Markov chains.     

Nonstationary Markov decision problems with converging parameters

This paper considers the solution of Markov decision problems whose parameters can be obtained only via approximating schemes, or where it is computationally preferable to approximate the parameters, rather than employing exact algorithms for their computation.Various models are presented in which this situation occurs. Furthermore, it is shown that a modified value-iteration method may be employed, both for the discounted version and for the undiscounted version of the model, in order to solve the optimality equation and to find optimal policies. In both cases, the convergence rate is determined.As a side result, we characterize the asymptotic behavior of backward products of a geometrically convergent sequence of Markov matrices.     

Further results on adaptive full-order and reduced-order observers for Lur’e differential inclusions

This paper is concerned with the adaptive observer design of Lur’e differential inclusions with unknown parameters. Under a relaxed assumption on nonlinear perturbation functions, a sufficient condition for the existence of an adaptive full-order observer is established. Comparing with results in the literature, the present conditions are complemented with a numerically reliable computational approach, which can be checked by means of linear matrix inequalities. Furthermore, it is shown that, under the sufficient condition, the existence of a reduced-order observer is guaranteed. Also, the reduced-order observer is designed. The effectiveness of the proposed design is illustrated via a simulation example.     

Fuzzy Markov Chains and Decision-Making

Decision-making in an environment of uncertainty and imprecision for real-world problems is a complex task. In this paper it is introduced general finite state fuzzy Markov chains that have a finite convergence to a stationary (may be periodic) solution. The Cesaro average and the -potential for fuzzy Markov chains are defined, then it is shown that the relationship between them corresponds to the Blackwell formula in the classical theory of Markov decision processes. Furthermore, it is pointed out that recurrency does not necessarily imply ergodicity. However, if a fuzzy Markov chain is ergodic, then the rows of its ergodic projection equal the greatest eigen fuzzy set of the transition matrix. Then, the fuzzy Markov chain is shown to be a robust system with respect to small perturbations of the transition matrix, which is not the case for the classical probabilistic Markov chains. Fuzzy Markov decision processes are finally introduced and discussed.     

Efficient Data Augmentation for Fitting Stochastic Epidemic Models to Prevalence Data

Stochastic epidemic models describe the dynamics of an epidemic as a disease spreads through a population. Typically, only a fraction of cases are observed at a set of discrete times. The absence of complete information about the time evolution of an epidemic gives rise to a complicated latent variable problem in which the state space size of the epidemic grows large as the population size increases. This makes analytically integrating over the missing data infeasible for populations of even moderate size. We present a data augmentation Markov chain Monte Carlo (MCMC) framework for Bayesian estimation of stochastic epidemic model parameters, in which measurements are augmented with subject-level disease histories. In our MCMC algorithm, we propose each new subject-level path, conditional on the data, using a time-inhomogenous continuous-time Markov process with rates determined by the infection histories of other individuals. The method is general, and may be applied to a broad class of epidemic models with only minimal modifications to the model dynamics and/or emission distribution. We present our algorithm in the context of multiple stochastic epidemic models in which the data are binomially sampled prevalence counts, and apply our method to data from an outbreak of influenza in a British boarding school. Supplementary material for this article is available online.     

Lower-order state-space self-tuning control for a stochastic chaotic hybrid system

^** Email: shtsai{at}mail.ncku.edu.tw In this paper, subject to acceptable closed-loop performance,an effective lower-order tuner for a stochastic chaotic hybridsystem is designed using the observer/Kalman filter identification(OKID) method, in which the system state in a general coordinateform is transformed to one in an observer form. The OKID methodis a time-domain technique that identifies a discrete input–outputmap by using known input–output sampled data in the generalcoordinate form, through an extension of the eigensystem realizationalgorithm. Moreover, it provides a lower-order realization ofthe tracker, with computationally effective initialization,for on-line "auto-regressive moving average process with exogenousmodel" -based identification and a lower-order state-space self-tuningcontrol technique. Finally, the chaotic Chen's system is usedas an illustrative example to demonstrate the effectivenessof the proposed methodology.     

Preserving Positive Realness Through Discretization

The importance of positive real transfer functions relies on the fact that they are associated with positive linear systems. Those systems possess the property that their input-output product time-integral, which is a measure of the total enerty, is nonnegative. Such a property can be also formulated in the discrete context. It is shown that a discrete positive real transfer function is obtained from a positive real continuous one of relative order zero being strictly stable poles via discretization by a sampler and zero-order hold device provided that the direct input-output transmission gain is sufficiently large. It is also proved that a discrete positive real transfer function may be obtained from a stable continuous one of relative order zero and high direct input-output gain which posses simple complex conjugate critically stable poles even in the case that this one is not positive real. For that purpose, the use of an appropriate phase-lag or phase lead compensating network for the continuous transfer function may be required to ensure positive realness of the discrete transfer function.     

A semi-global finite-time convergent observer for a class of nonlinear systems with bounded trajectories

This paper considers the problem of designing finite-time convergent observers for a class of lower-triangular nonlinear systems with bounded solution trajectories. Using the homogeneous domination approach, we construct an observer with homogeneous structure and saturation design, whose states will converge to the real states in a finite time by adjusting the observer gain. Several application examples of this finite-time convergent observer are discussed in this paper.     

A new adaptive observer design for a class of nonautonomous complex chaotic systems

This article is concerned with designing of a robust adaptive observer for a class of nonautonomous chaotic system with unknown parameters having unknown bounds. The proposed observer is established from the offered output measurement and robust against model uncertainties and external disturbances. Convergence analysis of the observation error dynamics is realized and proved by Lyapunov stabilization theory. Finally, for verification and demonstration, the proposed method is applied to the Chen as an autonomous chaotic system and the electrostatic transducer as a nonautonomous chaotic system. The numerical simulations illustrate the excellent performance of the proposed scheme. © 2014 Wiley Periodicals, Inc. Complexity 21: 145–153, 2015