关于马氏环境中马氏链的几点注记

讨论了马氏环境中马氏链与马氏双链间的关系,通过两个例子,纠正了有关文献的一些错误结论。     

马氏环境中可数马氏链的Poisson极限率

研究了马氏环境中的可数马氏链,主要证明了过程于小柱集上的回返次数是渐近地服从Poisson分布。为此,引入熵函数h,首先给出了马氏环境中马氏链的Shannon-Mc Millan-Breiman定理,还给出了一个非马氏过程Posson逼近的例子。当环境过程退化为一常数序列时,便得到可数马氏链的Poisson极限定理。这是有限马氏链Pitskel相应结果的拓广。     

马氏环境中的马氏链与马氏双链

本文研究了马氏环境中的马氏链,利用马氏双链的性质,得到了马氏环境中的马氏链回返于小柱集上的概率的若干估计式.     

马氏双链函数的强大数定律及其应用

本文研究了马氏随机环境中马氏双链函数的强大数定律.利用将双链函数进行分段研究的方法,获得了马氏环境中马氏双链函数强大数定律成立的一个充分条件.运用该定律,推导出马氏双链从一个状态到另一个状态转移概率的极限性质,进而推广了马氏双链的极限性质.     

状态可数的马氏环境中马氏链函数的强大数定律

讨论了马氏双链与随机环境中马氏链的关系．在此基础上，研究了具有离散参量的马氏环境中马氏链函数的强大数定律，并且给出了直接加于链和过程样本函数上的充分条件．     

The cross-entropy method with patching for rare-event simulation of large Markov chains

There are various importance sampling schemes to estimate rare event probabilities in Markovian systems such as Markovian reliability models and Jackson networks. In this work, we present a general state-dependent importance sampling method which partitions the state space and applies the cross-entropy method to each partition. We investigate two versions of our algorithm and apply them to several examples of reliability and queueing models. In all these examples we compare our method with other importance sampling schemes. The performance of the importance sampling schemes is measured by the relative error of the estimator and by the efficiency of the algorithm. The results from experiments show considerable improvements both in running time of the algorithm and the variance of the estimator.     

Rumor spreading in random evolving graphs

We analyze randomized broadcast in dynamic networks modeled as edge‐Markovian evolving graphs. The most realistic range of edge‐Markovian parameters yields sparse and disconnected graphs. We prove that, in this setting, the “push” protocol completes with high probability in optimal logarithmic time. © 2015 Wiley Periodicals, Inc. Random Struct. Alg., 48, 290–312, 2016     

Population Size Dependent Generalized Multitype Branching Processes

Abstract

In this paper, we introduce the population size dependent generalized multitype branching process. This is a Markovian model that allows us to study homogeneous multitype branching processes in a unified way. The basic properties for this model, transitions between its states, as well as the existence of a stationary limiting distribution, are investigated. Finally, we apply the obtained results to a new controlled multitype branching process.     

On some applications of Wald's identity to dams

It is known that the main difficulty in applying the Markovian analogue of Wald's Identity is the presence, in the Identity, of the last state variable before the random walk is terminated. In this paper we show that this difficulty can be overcome if the underlying Markov chain has a finite state space. The absorption probabilities thus obtained are used, by employing a duality argument, to derive time-dependent and limiting probabilities for the depletion process of a dam with Markovian inputs.The second problem that is considered here is that of a non-homogeneous but cyclic Markov chain. An analogue of Wald's Identity is obtained for this case, and is used to derive time- dependent and limiting probabilities for the depletion process with inputs forming a non- homogeneous (cyclic) Markov chain.     

The asymptotic equipartition property of Markov chains in single infinite Markovian environment on countable state space

ABSTRACT

The asymptotic equipartition property is a basic theorem in information theory. In this paper, we study the strong law of large numbers of Markov chains in single-infinite Markovian environment on countable state space. As corollary, we obtain the strong laws of large numbers for the frequencies of occurrence of states and ordered couples of states for this process. Finally, we give the asymptotic equipartition property of Markov chains in single-infinite Markovian environment on countable state space.     

A retrial BMAP/SM/1 system with linear repeated requests

A retrial queueing system with the batch Markovian arrival process and semi-Markovian service is investigated. We suppose that the intensity of retrials linearly depends on the number of repeated calls. The distribution of the number of calls in the system is the subject of research. Asymptotically quasi-Toeplitz 2-dimensional Markov chains are introduced into consideration and applied for solving the problem.     

On a new method of Markov chain reduction

The practical usefulness of Markov models and Markovian decision process has been severely limited due to their extremely large dimension. Thus, a reduced model without sacrificing significant accuracy can be very interesting.

The homogeneous finite Markov chain's long-run behaviour is given by the persistent states, obtained after the decomposition in classes of connected states. In this paper we expound a new reduction method for ergodic classes formed by such persistent states. An ergodic class has a steady-state independent of the initial distribution. This class constitutes an irreducible finite ergodic Markov chain, which evolves independently after the capture of the event.

The reduction is made according to the significance of steady-state probabilities. For being treatable by this method, the ergodic chain must have the Two-Time-Scale property.

The presented reduction method is an approximate method. We begin with an arrangement of irreducible Markov chain states, in decreasing order of their steady state probability's size. Furthermore, the Two-Time-Scale property of the chain enables us to make an assumption giving the reduction. Thus, we reduce the ergodic class only to its stronger part, which contains the most important events having also a slower evolution. The reduced system keeps the stochastic property, so it will be a Markov chain     

An integration by parts formula in a Markovian regime switching model and application to sensitivity analysis

We establish an integration by parts formula for the random functionals of a continuous-time Markov chain, based on partial differentiation with respect to jump times. In comparison with existing methods, our approach does not rely on the Girsanov theorem and it imposes less restrictions on the choice of directions of differentiation, while assuming additional continuity conditions on the considered functionals. As an application we compute sensitivities (Greeks) using stochastic weights in an asset price model with Markovian regime switching.     

On a BMAP/G/1 G-queue with setup times and multiple vacations

In this paper, we consider a BMAP/G/1 G-queue with setup times and multiple vacations. Arrivals of positive customers and negative customers follow a batch Markovian arrival process (BMAP) and Markovian arrival process (MAP) respectively. The arrival of a negative customer removes all the customers in the system when the server is working. The server leaves for a vacation as soon as the system empties and is allowed to take repeated (multiple) vacations. By using the supplementary variables method and the censoring technique, we obtain the queue length distributions. We also obtain the mean of the busy period based on the renewal theory.     

Importance Sampling Simulations of Markovian Reliability Systems Using Cross-Entropy

This paper reports simulation experiments, applying the cross entropy method such as the importance sampling algorithm for efficient estimation of rare event probabilities in Markovian reliability systems. The method is compared to various failure biasing schemes that have been proved to give estimators with bounded relative errors. The results from the experiments indicate a considerable improvement of the performance of the importance sampling estimators, where performance is measured by the relative error of the estimate, by the relative error of the estimator, and by the gain of the importance sampling simulation to the normal simulation.     

Controlled diffusion processes with Markovian switchings for modeling dynamical engineering systems

In this paper the intrinsic complex nature of engineering systems under control is treated by introducing an approach based on Controlled Stochastic Differential Equations with Markovian Switchings (in short CSDEMS). Technical conditions for the existence and uniqueness of the solutions of the CSDEMS are provided. In this context it is not unusual to deal with non-linear CSDEMS that cannot be solved analytically. Therefore, we develop a new two-step, predictor–corrector method for finding numerical approximations to solutions of CSDEMS. This method utilizes the Euler–Maruyama method. An illustrative application to the biochemical engineering area is presented to highlight the usefulness of our approach as a simulation tool.     

税源预测的数学模型

税源预测是财税部门的重要工作。本文通过实例提出税源预测中常用的折扣最小二乘法、一元线性回归、税收弹性及马尔柯夫预测方法等数学模型，使预测税源工作定量化、科学化。     

随机环境中马氏链的大数定律和强大数定律

本文主要研究了单无限马氏环境中马氏链的几类特殊函数的大数定律及强大数定律,并且给出了加于链和过程特殊样本函数上的充分条件.     

Markov Chains with finite convergence time

We study the properties of finite ergodic Markov Chains whose transition probability matrix P is singular. The results establish bounds on the convergence time of P^m to a matrix where all the rows are equal to the stationary distribution of P. The results suggest a simple rule for identifying the singular matrices which do not have a finite convergence time. We next study finite convergence to the stationary distribution independent of the initial distribution. The results establish the connection between the convergence time and the order of the minimal polynomial of the transition probability matrix. A queuing problem and a maintenance Markovian decision problem which possess the property of rapid convergence are presented.     

Nonhomogeneous markov chains with desired properties - the nearest markovian model

A new class of operators performing an optimization (optimization operators or, simply, optimators) which generate transition matrices with required properties such as ergodicity, recurrence etc., is considered and their fundamental features are described. Some criteria for comparing such operators by taking into account their strenght are given and sufficient conditions for both weak and strong ergodicity are derived. The nearest Markovian model with respect to a given set of observed probability vectors is then defined as a sequence of transition matrices satisfying certain constraints that express our prior knowledge about the system. Finally, sufficient conditions for the existence of such a model are given and the related algorithm is illustrated by an example.