Existence and uniqueness of solutions to stochastic heat equations with Markovian switching and Feller property

Abstract

In this paper, we focus on two-component Markov processes which consist of continuous dynamics and discrete events. Using the classical fixed point theorem for contractions to investigate the existence and uniqueness of solutions of stochastic heat equations with Markovian switching, then developing the corresponding Feller property of the solution.     

Bayesian Analysis of a Two-State Markov Modulated Poisson Process

Abstract

We postulate observations from a Poisson process whose rate parameter modulates between two values determined by an unobserved Markov chain. The theory switches from continuous to discrete time by considering the intervals between observations as a sequence of dependent random variables. A result from hidden Markov models allows us to sample from the posterior distribution of the model parameters given the observed event times using a Gibbs sampler with only two steps per iteration.     

Explicit Solution of a Quorum Queueing System

This paper deals with the queueing system M/G ^r /1 in which customers are served in batches of fixed size r. The author considers the discrete time parameter process, an embedded Markov chain, and the continuous time parameter process, a semi-regenerative process. In both cases, explicit solutions are given for the system size steady-state probabilities in terms of the zeros, inside and/or outside the closed unit ball, of the characteristic equation. Moments are derived and illustrative examples are provided.     

On <Emphasis Type="Italic">g</Emphasis>-measures in symbolic dynamics

Finitary Markov processes are described in G. Morvai and B. Weiss, Prediction for discrete time series, Probability Theory and Related Fields 132 (2005), 1–12. The transition functions of finitary Markov processes are residually locally constant g-functions that can be extended by continuity to their maximal domain of definition. The study of their associated symbolic dynamics leads one to the D-shifts as introduced in W. Krieger, On g-functions for subshifts, Institute of Mathematical Statistics Lecture Notes-Monograph Series, Vol. 48, Dynamics & Stochastics, arXiv:math.DS/0608259, (2006), 306–316, We study the phenomena that can arise in residually locally constant and locally constant maximally defined g-functions on D-shifts, Markov shifts and synchronizing systems with respect to future measures and g-measures     

On Stochastic Stabilization of Discrete‐Time Markovian Jump Systems with Delay in State

Abstract

In this paper, we investigate the stochastic stabilization problem for a class of linear discrete time‐delay systems with Markovian jump parameters. The jump parameters considered here is modeled by a discrete‐time Markov chain. Our attention is focused on the design of linear state feedback memoryless controller such that stochastic stability of the resulting closed‐loop system is guaranteed when the system under consideration is either with or without parameter uncertainties. Sufficient conditions are proposed to solve the above problems, which are in terms of a set of solutions of coupled matrix inequalities.     

Self-Interacting Markov Chains

Abstract

In this article we study a class of self-interacting Markov chain models. We propose a novel theoretical basis based on measure-valued processes and semigroup techniques to analyze its asymptotic behavior as the time parameter tends to infinity. We exhibit different types of decays to equilibrium, depending on the level of interaction. We illustrate these results in a variety of examples, including Gaussian or Poisson self-interacting models. We analyze the long-time behavior of a new class of evolutionary self-interacting chain models. These genetic type algorithms can also be regarded as reinforced stochastic explorations of an environment with obstacles related to a potential function.     

Fisher information in window censored renewal process data and its applications

Suppose we have a renewal process observed over a fixed length of time starting from a random time point and only the times of renewals that occur within the observation window are recorded. Assuming a parametric model for the renewal time distribution with parameter θ, we obtain the likelihood of the observed data and describe the exact and asymptotic behavior of the Fisher information (FI) on θ contained in this window censored renewal process. We illustrate our results with exponential, gamma, and Weibull models for the renewal distribution. We use the FI matrix to determine optimal window length for designing experiments with recurring events when the total time of observation is fixed. Our results are useful in estimating the standard errors of the maximum likelihood estimators and in determining the sample size and duration of clinical trials that involve recurring events associated with diseases such as lupus.     

Some autoregressive moving average processes with generalized Poisson marginal distributions

Some simple models are introduced which may be used for modelling or generating sequences of dependent discrete random variables with generalized Poisson marginal distribution. Our approach for building these models is similar to that of the Poisson ARMA processes considered by Al-Osh and Alzaid (1987,J. Time Ser. Anal.,8, 261–275; 1988,Statist. Hefte,29, 281–300) and McKenzie (1988,Adv. in Appl. Probab.,20, 822–835). The models have the same autocorrelation structure as their counterparts of standard ARMA models. Various properties, such as joint distribution, time reversibility and regression behavior, for each model are investigated.     

Malliavin calculus, geometric mixing, and expansion of diffusion functionals

Under geometric mixing condition, we presented asymptotic expansion of the distribution of an additive functional of a Markov or an ε-Markov process with finite autoregression including Markov type semimartingales and time series models with discrete time parameter. The emphasis is put on the use of the Malliavin calculus in place of the conditional type Cramér condition, whose verification is in most case not easy for continuous time processes without such an infinite dimensional approach. In the second part, by means of the perturbation method and the operational calculus, we proved the geometric mixing property for non-symmetric diffusion processes, and presented a sufficient condition which is easily checked in practice. Accordingly, we obtained asymptotic expansion of diffusion functionals and proved the validity of it under mild conditions, e.g., without the strong contractivity condition. Received: 7 September 1997 / Revised version: 17 March 1999     

Post-Newtonian Approximation of the Vlasov–Nordström System

ABSTRACT

We study the Nordström–Vlasov system, which describes the dynamics of a self-gravitating ensemble of collisionless particles in the framework of the Nordström scalar theory of gravitation. If the speed of light c is considered as a parameter, it is known that in the Newtonian limit c → ∞ the Vlasov–Poisson system is obtained. In this paper we determine a higher approximation and establish a pointwise error estimate of order 𝒪(c ^?4). Such an approximation is usually called a 1.5 post-Newtonian approximation.     

IPA Derivatives for Make-to-Stock Production-Inventory Systems With Backorders Under the (R,r) Policy

This paper addresses Infinitesimal Perturbation Analysis (IPA) in the class of Make-to Stock (MTS) production-inventory systems with backorders under the continuous-review (R,r) policy, where R is the stock-up-to level and r is the reorder point. A system from this class is traditionally modeled as a discrete system with discrete demand arrivals at the inventory facility and discrete replenishment orders placed at the production facility. Here, however, we map an underlying discrete MTS system to a Stochastic Fluid Model (SFM) counterpart in which stochastic fluid-flow rate processes with piecewise constant sample paths replace the corresponding traditional discrete demand arrival and replenishment stochastic processes, under very mild regularity assumptions. The paper then analyzes the SFM counterpart and derives closed-form IPA derivative formulas of the time-averaged inventory level and time-averaged backorder level with respect to the policy parameters, R and r, and shows them to be unbiased. The obtained formulas are comprehensive in the sense that they are computed for any initial inventory state and any time horizon, and are simple and fast to compute. These properties hold the promise of utilizing IPA derivatives as an ingredient of offline design algorithms and online management and control algorithms of the class of systems under study.      

Perishable Inventory System at Service Facilities with N Policy

Abstract

This article presents a perishable stochastic inventory system under continuous review at a service facility in which the waiting hall for customers is of finite size M. The service starts only when the customer level reaches N (< M), once the server has become idle for want of customers. The maximum storage capacity is fixed as S. It is assumed that demand for the commodity is of unit size. The arrivals of customers to the service station form a Poisson process with parameter λ. The individual customer is issued a demanded item after a random service time, which is distributed as negative exponential. The items of inventory have exponential life times. It is also assumed that lead time for the reorders is distributed as exponential and is independent of the service time distribution. The demands that occur during stock out periods are lost.The joint probability distribution of the number of customers in the system and the inventory levels is obtained in steady state case. Some measures of system performance in the steady state are derived. The results are illustrated with numerical examples.     

Departures inGR^{X{n}} /G_n /\infty

Departure processes in infinite server queues with non-Poisson arrivals have not received much attention in the past. In this paper, we try to fill this gap by considering the continuous time departure process in a general infinite server system with a Markov renewal batch arrival process ofM different types of customers. By a conditional approach, analytical results are obtained for the generating functions and binomial moments of the departure process. Special cases are discussed, showing that while Poisson arrival processes generate Poisson departures, departure processes are much more complicated with non-Poisson arrivals.This research has been supported in part by the Natural Science and Engineering Research Council of Canada (Grant A5639).     

Generalized binomial and negative binomial distributions of order<Emphasis Type="Italic">k</Emphasis> by the<Emphasis Type="Italic">l</Emphasis>-overlapping enumeration scheme

In this paper, we investigate the exact distribution of the waiting time for ther-th ℓ-overlapping occurrence of success-runs of a specified length in a sequence of two state Markov dependent trials. The probability generating functions are derived explicitly, and as asymptotic results, relationships of a negative binomial distribution of orderk and an extended Poisson distribution of orderk are discussed. We provide further insights into the run-related problems from the viewpoint of the ℓ-overlapping enumeration scheme. We also study the exact distribution of the number of ℓ-overlapping occurrences of success-runs in a fixed number of trials and derive the probability generating functions. The present work extends several properties of distributions of orderk and leads us a new type of geneses of the discrete distributions.     

How to count and guess well: Discrete adaptive filters

A discrete state and time Markov chain is observed through a finite state function which is subject to random perturbations. Such a situation is often called a Hidden Markov Model. A general filter is obtained which provides recursive updates of estimates of processes related to the Markov chain given the observations. In the unnormalized, or Zakai, form this provides particularly simple equations. Specializing this result provides recursive estimates and smoothers for the state of the process, for the number of jumps from one state to another, for the occupation time in any state and for a process related to the observations. These results allow a re-estimation of the parameters of the model, so that our procedures are adaptive or self tuning to the data. The main contributions of this paper are the introduction of an equivalent measure under which the observation values are independent and identically distributed, and the use of the idempotent property when the state space of the Markov chain is identified with canonical unit vectors in a Euclidean space.This research was supported by NSERC Grant A7964.     

Lumpability and Commutativity of Markov Processes

Abstract

We introduce the concepts of lumpability and commutativity of a continuous time discrete state space Markov process, and provide a necessary and sufficient condition for a lumpable Markov process to be commutative. Under suitable conditions we recover some of the basic quantities of the original Markov process from the jump chain of the lumped Markov process.     

A genetic filtering problem ∗

Members of a population of fixed size N can be in any one of n states. In discrete time the individuals jump from one state to another, independently of each other, and with probabilities described by a homogeneous Markov chain. At each time a sample of size M is withdrawn, (with replacement). Based on these observations, and using the techniques of Hidden Markov Models, recursive estimates for the distribution of the population are obtained     

A Geometrical Look at Iterative Methods for Operators with Fixed Points

ABSTRACT

We distinguish classes of operators T with fixed points on a real Hilbert space by comparing the distances of a point x and its image Tx to the (set of) fixed points of T; this leads to a ranking of those classes, based on a nonnegative parameter. That same parameter also lets us conclude about the sign of and an upper bound for a characteristic inner product result that arises in iterative processes to obtain a common fixed point of a set of operators. We use that parameter as the starting point for a geometrically-inclined study of specific iterative algorithms intended to find a common fixed point of operators belonging to such class.     

Percolation and limit theory for the poisson lilypond model

The lilypond model on a point process in d ‐space is a growth‐maximal system of non‐overlapping balls centred at the points. We establish central limit theorems for the total volume and the number of components of the lilypond model on a sequence of Poisson or binomial point processes on expanding windows. For the lilypond model over a homogeneous Poisson process, we give subexponentially decaying tail bounds for the size of the cluster at the origin. Finally, we consider the enhanced Poisson lilypond model where all the balls are enlarged by a fixed amount (the enhancement parameter), and show that for d > 1 the critical value of this parameter, above which the enhanced model percolates, is strictly positive. © 2012 Wiley Periodicals, Inc. Random Struct. Alg., 2012     

Kitting process in a stochastic assembly system

In small-lot, multi-product, multi-level assembly systems, kitting (or accumulating) components required for assembly plays a crucial role in determining system performance, especially when the system operates in a stochastic environment. This paper analyzes the kitting process of a stochastic assembly system, treating it as an assembly-like queue. If components arrive according to Poisson processes, we show that the output stream departing the kitting operation is a Markov renewal process. The distribution of time between kit completions is also derived. Under the special condition of identical component arrival streams having the same Poisson parameter, we show that the output stream of kits approximates a Poisson process with parameter equal to that of the input stream. This approximately decouples assembly from kitting, allowing the assembly operation to be analyzed separately.