An embedded Markov chain approach to stock rationing

We propose a new method for the analysis of lot-per-lot inventory systems with backorders under rationing. We introduce an embedded Markov chain that approximates the state-transition probabilities. We provide a recursive procedure for generating these probabilities and obtain the steady-state distribution.     

Optimal production and rationing policies of a make-to-stock production system with batch demand and backordering

In this paper, we consider the stock rationing problem of a single-item make-to-stock production/inventory system with multiple demand classes. Demand arrives as a Poisson process with a randomly distributed batch size. It is assumed that the batch demand can be partially satisfied. The facility can produce a batch up to a certain capacity at the same time. Production time follows an exponential distribution. We show that the optimal policy is characterized by multiple rationing levels.     

The determination of quasistationary distributions directly from the transition rates of an absorbing Markov chain

There are many stochastic systems arising in areas as diverse as wildlife management, chemical kinetics and reliability theory, which eventually “die out,” yet appear to be stationary over any reasonable time scale. The notion of a quasistationary distribution has proved to be a potent tool in modelling this behaviour. In finite-state systems the existence of a quasistationary distribution is guaranteed. However, in the infinite-state case this may not always be so and the question of whether or not quasistationary distributions exist requires delicate mathematical analysis. The purpose of this paper is to present simple conditions for the existence of quasistationary distributions for continuous-time Markov chains and to demonstrate how these can be applied in practice.     

关于随机环境中的马尔可夫过程的简介——纪念李国平院士吴新谋教授诞辰100周年

该文系统地介绍随机环境中的马尔可夫过程. 共4章, 第一章介绍依时的随机环境中的马尔可夫链(MCTRE), 包括MCTRE的存在性及等价描述; 状态分类; 遍历理论及不变测度; p-θ 链的中心极限定理和不变原理. 第二章介绍依时的随机环境中的马尔可夫过程(MPTRE), 包括MPTRE的基本概念; 随机环境中的q -过程存在唯一性; 时齐的q -过程;MPTRE的构造及等价性定理.第三章介绍依时的随机环境中的分枝链(MBCRE), 包括有限维的和无穷维的MBCRE的模型和基本概念; 它们的灭绝概念;两极分化; 增殖率等.第四章介绍依时依空的随机环境中的马尔可夫链(MCSTRE), 包括MCSTRE的基本概念、构造; 依时依空的随机环境中的随机徘徊(RWSTRE)的中心极限定理、不变原理.     

An algorithmic approach for a special class of Markov chains

In this paper, we develop an algorithmic method for the evaluation of the steady state probability vector of a special class of finite state Markov chains. For the class of Markov chains considered here, it is assumed that the matrix associated with the set of linear equations for the steady state probabilities possess a special structure, such that it can be rearranged and decomposed as a sum of two matrices, one lower triangular with nonzero diagonal elements, and the other an upper triangular matrix with only very few nonzero columns. Almost all Markov chain models of queueing systems with finite source and/or finite capacity and first-come-first-served or head of the line nonpreemptive priority service discipline belongs to this special class.     

Runs,scans and URN model distributions: A unified Markov chain approach

This paper presents a unified approach for the study of the exact distribution (probability mass function, mean, generating functions) of three types of random variables: (a) variables related to success runs in a sequence of Bernoulli trials (b) scan statistics, i.e. variables enumerating the moving windows in a linearly ordered sequence of binary outcomes (success or failure) which contain prescribed number of successes and (c) success run statistics related to several well known urn models. Our approach is based on a Markov chain imbedding which permits the construction of probability vectors satisfying triangular recurrence relations. The results presented here cover not only the case of identical and independently distributed Bernoulli variables, but the non-identical case as well. An extension to models exhibiting Markov dependence among the successive trials is also discussed in brief.     

On the transition from a Markov chain to a continuous time process

Starting from a real-valued Markov chain X₀,X₁,…,X_n with stationary transition probabilities, a random element {Y(t);t[0, 1]} of the function space D[0, 1] is constructed by letting Y(k/n)=X_k, k= 0,1,…,n, and assuming Y (t) constant in between. Sample tightness criteria for sequences {Y(t);t[0,1]};_n of such random elements in D[0, 1] are then given in terms of the one-step transition probabilities of the underlying Markov chains. Applications are made to Galton-Watson branching processes.     

Markov chain inversion approach to identify the transition rates of ion channels

We consider how to identify the transition rates of ion channels with the underlying scheme which is kinetically modelled as time-homogeneous Markov chain. A Markov chain inversion approach is developed to perform a difficult inversion to identify the transition rates from the parameters characterizing the lifetime distributions at a small number of states, although it is straightforward to derive the lifetime distribution. The general explicit equations relating the parameters of the lifetime distribution to the transition rates are derived and transition rates are then obtained as roots to this system of equations. The concrete solutions are proposed to the basic and regular schemes such as linear, star-graph branch and loop. Useful conclusions and solutions to realistic schemes are also included to show its efficiency.     

An overshoot approach to recurrence and transience of Markov processes

We develop criteria for recurrence and transience of one-dimensional Markov processes which have jumps and oscillate between +∞ and −∞. The conditions are based on a Markov chain which only consists of jumps (overshoots) of the process into complementary parts of the state space.In particular, we show that a stable-like process with generator −(−Δ)^α(x)/2 such that α(x)=α for x<−R and α(x)=β for x>R for some R>0 and α,β∈(0,2) is transient if and only if α+β<2, otherwise it is recurrent.As a special case, this yields a new proof for the recurrence, point recurrence and transience of symmetric α-stable processes.     

An integrated production and inventory model to dampen upstream demand variability in the supply chain

We consider a two-echelon supply chain: a single retailer holds a finished goods inventory to meet an i.i.d. customer demand, and a single manufacturer produces the retailer’s replenishment orders on a make-to-order basis. In this setting the retailer’s order decision has a direct impact on the manufacturer’s production. It is a well known phenomenon that inventory control policies at the retailer level often propagate customer demand variability towards the manufacturer, sometimes even in an amplified form (known as the bullwhip effect). The manufacturer, however, prefers to smooth production, and thus he prefers a smooth order pattern from the retailer. At first sight a decrease in order variability comes at the cost of an increased variance of the retailer’s inventory levels, inflating the retailer’s safety stock requirements. However, integrating the impact of the retailer’s order decision on the manufacturer’s production leads to new insights. A smooth order pattern generates shorter and less variable (production/replenishment) lead times, introducing a compensating effect on the retailer’s safety stock. We show that by including the impact of the order decision on lead times, the order pattern can be smoothed to a considerable extent without increasing stock levels. This leads to a situation where both parties are better off.     

Convergence of a transition probability tensor of a higher‐order Markov chain to the stationary probability vector

In this paper, first we introduce a new tensor product for a transition probability tensor originating from a higher‐order Markov chain. Subsequently, some properties of the new tensor product are explained, and its relationship with the stationary probability vector is studied. Also, similarity between results obtained by this new product and the first‐order case is shown. Furthermore, we prove the convergence of a transition probability tensor to the stationary probability vector. Finally, we show how to achieve a stationary probability vector with some numerical examples and make some comparison between the proposed method and another existing method for obtaining stationary probability vectors. Copyright © 2016 John Wiley & Sons, Ltd.     

A Two-Phase Cyclic Nonhomogeneous Markov Chain Performability Evaluation by Explicit Approximate Inverses Applied to a Replicated Database System

The need for a more accurate modeling of the performance of systems whose functioning mainly dependant on external time parameters such as the number of requests during a particular time phase, led us to a novel approach, taking into account the time parameters involved. This is achieved through the evaluation of a performability indicator modeled by means of a two-phase cyclic nonhomogenous Markov chain considering periodical time-dependant arrival request probabilities and applied to a replicated database system. The computation of the performability indicator modeled by cyclic nonhomogeneous Markov chain requires the use of efficient computational methods by using explicit approximate inverse preconditioning methods. This revised version was published online in July 2006 with corrections to the Cover Date.