Computing the nearest reversible Markov chain

Reversible Markov chains are the basis of many applications. However, computing transition probabilities by a finite sampling of a Markov chain can lead to truncation errors. Even if the original Markov chain is reversible, the approximated Markov chain might be non‐reversible and will lose important properties, like the real‐valued spectrum. In this paper, we show how to find the closest reversible Markov chain to a given transition matrix. It turns out that this matrix can be computed by solving a convex minimization problem. Copyright © 2015 John Wiley & Sons, Ltd.     

Finite axiomatizations for existentially closed posets and semilattices

In this paper we exhibit axiomatizations for the theories of existentially closed posets and existentially closed semilattices. We do this by considering an infinite axiomatization which characterizes these structures in terms of embeddings of finite substructures, an axiomatization which exists for any locally finite universal class with a finite language and with the joint embedding and amalgamation properties. We then find particular finite subsets of these axioms which suffice to axiomatize both classes. Research supported by an NSERC Postdoctoral Fellowship. Research supported by NSERC Grant No. A7256.     

Generalized inverses and their application to applied probability problems

The main aim of this paper is to examine the applicability of generalized inverses to a wide variety of problems in applied probability where a Markov chain is present either directly or indirectly through some form of imbedding. By characterizing all generalized inverses of I—P, where P is the transition matrix of a finite irreducible discrete time Markov chain, we are able to obtain general procedures for finding stationary distributions, moments of the first passage time distributions, and asymptotic forms for the moments of the occupation-time random variables. It is shown that all known explicit methods for examining these problems can be expressed in this generalized inverse framework. More generally, in the context of a Markov renewal process setting the aforementioned problems are also examined using generalized inverses of I—P. As a special case, Markov chains in continuous time are considered, and we show that the generalized inverse technique can be applied directly to the infinitesimal generator of the process, instead of to I—P, where P is the transition matrix of the discrete time jump Markov chain.     

How to count and guess well: Discrete adaptive filters

A discrete state and time Markov chain is observed through a finite state function which is subject to random perturbations. Such a situation is often called a Hidden Markov Model. A general filter is obtained which provides recursive updates of estimates of processes related to the Markov chain given the observations. In the unnormalized, or Zakai, form this provides particularly simple equations. Specializing this result provides recursive estimates and smoothers for the state of the process, for the number of jumps from one state to another, for the occupation time in any state and for a process related to the observations. These results allow a re-estimation of the parameters of the model, so that our procedures are adaptive or self tuning to the data. The main contributions of this paper are the introduction of an equivalent measure under which the observation values are independent and identically distributed, and the use of the idempotent property when the state space of the Markov chain is identified with canonical unit vectors in a Euclidean space.This research was supported by NSERC Grant A7964.     

Random walk centrality and a partition of Kemeny’s constant

We consider an accessibility index for the states of a discrete-time, ergodic, homogeneous Markov chain on a finite state space; this index is naturally associated with the random walk centrality introduced by Noh and Reiger (2004) for a random walk on a connected graph. We observe that the vector of accessibility indices provides a partition of Kemeny’s constant for the Markov chain. We provide three characterizations of this accessibility index: one in terms of the first return time to the state in question, and two in terms of the transition matrix associated with the Markov chain. Several bounds are provided on the accessibility index in terms of the eigenvalues of the transition matrix and the stationary vector, and the bounds are shown to be tight. The behaviour of the accessibility index under perturbation of the transition matrix is investigated, and examples exhibiting some counter-intuitive behaviour are presented. Finally, we characterize the situation in which the accessibility indices for all states coincide.     

On a result for finite Markov chains

In an undergraduate course on stochastic processes, Markov chains are discussed in great detail. Textbooks on stochastic processes provide interesting properties of finite Markov chains. This note discusses one such property regarding the number of steps in which a state is reachable or accessible from another state in a finite Markov chain with M (≥ 2) states.     

Markov chain models for delinquency: Transition matrix estimation and forecasting

A Markov chain is a natural probability model for accounts receivable. For example, accounts that are ‘current’ this month have a probability of moving next month into ‘current’, ‘delinquent’ or ‘paid‐off’ states. If the transition matrix of the Markov chain were known, forecasts could be formed for future months for each state. This paper applies a Markov chain model to subprime loans that appear neither homogeneous nor stationary. Innovative estimation methods for the transition matrix are proposed. Bayes and empirical Bayes estimators are derived where the population is divided into segments or subpopulations whose transition matrices differ in some, but not all entries. Loan‐level models for key transition matrix entries can be constructed where loan‐level covariates capture the non‐stationarity of the transition matrix. Prediction is illustrated on a $7 billion portfolio of subprime fixed first mortgages and the forecasts show good agreement with actual balances in the delinquency states. Copyright © 2010 John Wiley & Sons, Ltd.     

Invariance and efficiency of convex representations

We consider two notions for the representations of convex cones G-representation and lifted-G-representation. The former represents a convex cone as a slice of another; the latter allows in addition, the usage of auxiliary variables in the representation. We first study the basic properties of these representations. We show that some basic properties of convex cones are invariant under one notion of representation but not the other. In particular, we prove that lifted-G-representation is closed under duality when the representing cone is self-dual. We also prove that strict complementarity of a convex optimization problem in conic form is preserved under G-representations. Then we move to study efficiency measures for representations. We evaluate the representations of homogeneous convex cones based on the “smoothness” of the transformations mapping the central path of the representation to the central path of the represented optimization problem. Research of the first author was supported in part by a grant from the Faculty of Mathematics, University of Waterloo and by a Discovery Grant from NSERC. Research of the second author was supported in part by a Discovery Grant from NSERC and a PREA from Ontario, Canada.     

A Tabu Search heuristic procedure in Markov chain bootstrapping

Markov chain theory is proving to be a powerful approach to bootstrap finite states processes, especially where time dependence is non linear. In this work we extend such approach to bootstrap discrete time continuous-valued processes. To this purpose we solve a minimization problem to partition the state space of a continuous-valued process into a finite number of intervals or unions of intervals (i.e. its states) and identify the time lags which provide “memory” to the process. A distance is used as objective function to stimulate the clustering of the states having similar transition probabilities. The problem of the exploding number of alternative partitions in the solution space (which grows with the number of states and the order of the Markov chain) is addressed through a Tabu Search algorithm. The method is applied to bootstrap the series of the German and Spanish electricity prices. The analysis of the results confirms the good consistency properties of the method we propose.     

Exact and limiting distributions of the number of successions in a random permutation

Traditionally the distributions of the number of patterns and successions in a random permutation ofn integers 1,2, ..., andn were studied by combinatorial analysis. In this short article, a simple way based on finite Markov chain imbedding technique is used to obtain the exact distribution of successions on a permutation. This approach also gives a direct proof that the limiting distribution of successions is a Poisson distribution with parameter =1. Furthermore, a direct application of the main result, it also yields the waiting time distribution of a succession.This work was supported in part by the Natural Sciences and Engineering Research Council of Canada under Grant NSERC A-9216, and National Science Council of Republic of China under Grant 85-2121-M-259-003.     

Super-Brownian motions in higher dimensions with absolutely continuous measure states

Continuous super-Brownian motions in two and more dimensions are known to have singular measure states. However, by weakening the branching mechanism in an irregular way they can be forced to have absolutely continuous states. The sufficient conditions we impose are identified in a couple of examples with irregularities in only one coordinate. This includes the case of branching restricted to some densely situated ensemble of hyperplanes.Supported by an NSERC Grant.     

Flux through a Markov chain

In this paper we study the flux through a finite Markov chain of a quantity, that we will call mass, which moves through the states of the chain according to the Markov transition probabilities. Mass is supplied by an external source and accumulates in the absorbing states of the chain. We believe that studying how this conserved quantity evolves through the transient (non-absorbing) states of the chain could be useful for the modelization of open systems whose dynamics has a Markov property.     

A χ2 goodness-of-fit test for Markov renewal processesgoodness-of-fit test for Markov renewal processes

A Markov Renewal Process (M.R.P.) is a process similar to a Markov chain, except that the time required to move from one state to another is not fixed, but is a random variable whose distribution may depend on the two states between which the transition is made. For an M.R.P. ofm (<∞) states we derive a goodness-of-fit test for a hypothetical matrix of transition probabilities. This test is similar to the test Bartlett has derived for Markov chains. We calculate the first two moments of the test statistic and modify it to fit the moments of a standard χ². Finally, we illustrate the above procedure numeerically for a particular case of a two-state M.R.P. Dwight B. Brock is mathematical statistican, Office of Statistical Methods, National Center for Health Statistics, Rockville, Maryland. A. M. Kshisagar is Associate Professor, Department of Statistics, Southern Methodist University. This research was partially supported by Office of Naval Research Contract No. N000 14-68-A-0515, and by NIH Training Grant GM-951, both with Southern Methodist University. This article is partially based on Dwight B. Brock's Ph.D. dissertation accepted by Southern Methodist University.     

Renorms and topological linear contractions on Hilbert spaces

Properties of and the relationships between (topological) proper contractions, (topological) strict contractions and (topological) contractions are investigated. Explicit renorms are constructed so that all operators in a (finite or countable) family or a semigroup simultaneously become proper contractions or strict contractions. Some results are obtained for operator weighted shifts or operator weighted continuous shifts to be topological strict contractions. Project supported by the Chinese University of Hong Kong (Grant No. 2060144) and NSERC of Canada (Grant No. A8096).     

Basic problems and solution methods for two-dimensional continuous 3 × 3 order hidden Markov model

A novel model referred to as two-dimensional continuous 3 × 3 order hidden Markov model is put forward to avoid the disadvantages of the classical hypothesis of two-dimensional continuous hidden Markov model. This paper presents three equivalent definitions of the model, in which the state transition probability relies on not only immediate horizontal and vertical states but also immediate diagonal state, and in which the probability density of the observation relies on not only current state but also immediate horizontal and vertical states. The paper focuses on the three basic problems of the model, namely probability density calculation, parameters estimation and path backtracking. Some algorithms solving the questions are theoretically derived, by exploiting the idea that the sequences of states on rows or columns of the model can be viewed as states of a one-dimensional continuous 1 × 2 order hidden Markov model. Simulation results further demonstrate the performance of the algorithms. Because there are more statistical characteristics in the structure of the proposed new model, it can more accurately describe some practical problems, as compared to two-dimensional continuous hidden Markov model.     

Homotopy self-equivalences of 4-manifolds

We correct an error in the calculation of the braid diagram of Hambleton and Kreck (Math. Z. 248, 147–172, 2004) for 4-manifolds with finite odd order fundamental group. Research partially supported by NSERC Research Grant A4000. The online version of the original article can be found at .     

Analysis of search-extension method for finding multiple solutions of nonlinear problem

For numerical computations of multiple solutions of the nonlinear elliptic problemΔu f(u)=0 inΩ, u=0 onΓ, a search-extension method (SEM) was proposed and systematically studied by the authors. This paper shall complete its theoretical analysis. It is assumed that the nonlinearity is non-convex and its solution is isolated, under some conditions the corresponding linearized problem has a unique solution. By use of the compactness of the solution family and the contradiction argument, in general conditions, the high order regularity of the solution u∈H~(1 α),α>0 is proved. Assume that some initial value searched by suitably many eigenbases is already fallen into the neighborhood of the isolated solution, then the optimal error estimates of its nonlinear finite element approximation are shown by the duality argument and continuation method.     

The coalescent

The n-coalescent is a continuous-time Markov chain on a finite set of states, which describes the family relationships among a sample of n members drawn from a large haploid population. Its transition probabilities can be calculated from a factorization of the chain into two independent components, a pure death process and a discrete-time jump chain. For a deeper study, it is useful to construct a more complicated Markov process in which n-coalescents for all values of n are embedded in a natural way.     

On Finite Markov Chain Imbedding and Its Applications

The finite Markov Chain Imbedding technique has been successfully applied in various fields for finding the exact or approximate distributions of runs and patterns under independent and identically distributed or Markov dependent trials. In this paper, we derive a new recursive equation for distribution of scan statistic using the finite Markov chain imbedding technique. We also address the problem of obtaining transition probabilities of the imbedded Markov chain by introducing a notion termed Double Finite Markov Chain Imbedding where transition probabilities are obtained by using the finite Markov chain imbedding technique again. Applications for random permutation model in chemistry and coupon collector’s problem are given to illustrate our idea.     

Lagrangian duality of concave minimization subject to linear constraints and an additional facial reverse convex constraint

This paper is concerned with the global optimization problem of minimizing a concave function subject to linear constraints and an additional facial reverse convex constraint. Here, the feasible set is the union of some faces of the polyhedron determined by the linear constraints. Several well-known mathematical problems can be written or transformed into the form considered. The paper addresses the Lagrangian duality of the problem. It is shown that, under slight assumptions, the duality gap can be closed with a finite dual multiplier. Finite methods based on solving concave minimization problems are also proposed. We deal with the advantages accrued when outer approximation, cutting plane, or branch-and-bound methods are used for solving these subproblems.This research was supported in part by the Hungarian National Research Foundation, Grant OTKA 2568. The author wishes to thank the Associate Editor and the referees for their valuable comments.