Filtering with Discrete State Observations

The problem of estimating a finite state Markov chain observed via a process on the same state space is discussed. Optimal solutions are given for both the ``weak' and ``strong' formulations of the problem. The ``weak' formulation proceeds using a reference probability and a measure change for the Markov chain. The ``strong' formulation considers an observation process related to perturbations of the counting processes associated with the Markov chain. In this case the ``small noise' convergence is investigated. Accepted 7 April 1998     

Measure preserving transformations similar to Markov shifts

Similarity, that is, the existence of joint common extensions, defines an interesting equivalence relation for infinite measure preserving transformations T. We provide a sufficient condition, given in terms of return processes to reference sets of finite measure, for T to be similar to a Markov shift. This is then shown to apply to various piecewise smooth dynamical systems, including weakly hyperbolic transformations with indifferent periodic points or flat critical points, and discrete random walks driven by (weakly) hyperbolic maps.     

On a Unique Ergodicity of Some Markov Processes

It is proved that the sufficient condition for the uniqueness of an invariant measure for Markov processes with the strong asymptotic Feller property formulated by Hairer and Mattingly (Ann Math 164(3):993–1032, 2006) entails the existence of at most one invariant measure for e-processes as well. Some application to time-homogeneous Markov processes associated with a nonlinear heat equation driven by an impulsive noise is also given.     

具有Bernoulli反馈且带有负顾客的PH/M/1排队系统

在PH/M/1排队模型中,引入了负顾客和Bernoulli反馈,并讨论了服务台容量为有限和无限两类模型,其中,模型一为服务台容量为无限的PH/M/1排队模型,利用拟生灭过程和矩阵几何解法得到了系统的转移速率矩阵,给出了系统正常返的充要条件,并得到了系统的稳态队长、忙期长度的拉普拉斯变换,以及系统的其它相关性能指标.模型二为服务台容量为有限的PH/M/1/N排队模型,同样使用拟生灭过程给出了马尔科夫过程的转移速率矩阵,并利用矩阵分析法进行求解,得到了该系统的稳态解和其它相关指标.     

Existence of mild solutions for stochastic differential equations and semilinear equations with non-Gaussian Lévy noise

Existence and uniqueness of the mild solutions for stochastic differential equations for Hilbert valued stochastic processes are discussed, with the multiplicative noise term given by an integral with respect to a general compensated Poisson random measure. Parts of the results allow for coefficients which can depend on the entire past path of the solution process. In the Markov case Yosida approximations are also discussed, as well as continuous dependence on initial data, and coefficients. The case of coefficients that besides the dependence on the solution process have also an additional random dependence is also included in our treatment. All results are proven for processes with values in separable Hilbert spaces. Differentiable dependence on the initial condition is proven by adapting a method of S. Cerrai.     

Qualitative behaviour of stochastic delay equations with a bounded memory

A dichotomy is proved concerning recurrence properties of the solution of certain stochastic delay equations. If the solution process is recurrent, there exists an invariant measure π on the state space C which is unique (up to a multiplicative constant) and the tail-field is trivial. If π happens to be a probability measure, then for every initial condition, the distribution of the process converges to it as t→∞. We will formulate a sufficient condition for the existence of an invariant probability measure (ipm) in icrnia of Lyapunov junctionals and give two examples, one Heing the stochastic-delay version of the famous logistic equation of population growth. Finally we study approximations of delay equations by Markov chains.     

Déviations modérées pour la moyennisation d'une EDS avec petite diffusion

We prove in this Note the moderate deviation principle (MDP) for the averaging principle of a stochastic differential equation (SDE) in a fast random environment, modelized by an exponentially ergodic Markov process independent of the Wiener process driving the SDE. The main tools will be the method of Puhalskii for exponential tightness and a MDP for inhomogeneous functionals of Markov processes established in [5].     

On Markov diffusion processes with delayed reflection from boundaries of a segment

A continuous semi-Markov process with values in a closed interval is considered. This process coincides with a Markov diffusion process inside the interval. Thus, violation of the Markov property is only possible at the boundary of the interval. We prove a sufficient condition under which a semi-Markov process is Markov. We show that, in addition to Markov processes with instantaneous reflection from the boundary of the interval. there exists a class of Markov processes with delayed reflection from the boundary. Such a process has a positive average measure of time at which its trajectory belongs to the boundaries. This gives a different proof of a similar result by Gikhman and Skorokhod of 1968. Bibliography: 5 titles.     

The age of a Markov process

The concept of a limiting conditional age distribution of a continuous time Markov process whose state space is the set of non-negative integers and for which {0} is absorbing is defined as the weak limit as t→∞ of the last time before t an associated “return” Markov process exited from {0} conditional on the state, j, of this process at t. It is shown that this limit exists and is non-defective if the return process is ρ-recurrent and satisfies the strong ratio limit property. As a preliminary to the proof of the main results some general results are established on the representation of the ρ-invariant measure and function of a Markov process. The conditions of the main results are shown to be satisfied by the return process constructed from a Markov branching process and by birth and death processes. Finally, a number of limit theorems for the limiting age as j→∞ are given.     

Asymptotically homogeneous iterated random functions with applications to the HARCH process

We show that a certain class of Markov chains satisfies the general Foster–Lyapunov condition under nonrestrictive assumptions. In particular, if a chain belongs to this class, the existence of an invariant measure can be established by means of the theorem provided. As an application of the theorem, the HARCH process is considered.     

The Transformation Method for Continuous-Time Markov Decision Processes

In this paper, we show that a discounted continuous-time Markov decision process in Borel spaces with randomized history-dependent policies, arbitrarily unbounded transition rates and a non-negative reward rate is equivalent to a discrete-time Markov decision process. Based on a completely new proof, which does not involve Kolmogorov??s forward equation, it is shown that the value function for both models is given by the minimal non-negative solution to the same Bellman equation. A verifiable necessary and sufficient condition for the finiteness of this value function is given, which induces a new condition for the non-explosion of the underlying controlled process.     

随机环境中的马氏链的不变测度与遍历性

本文考虑了一类特殊的随机环境的马氏链。假设随机“Doeblin”条件成立，我们证明了随机环境的马氏链的不变测度存在，且任何初始分布以指数收敛速度到些不变测度。进一步的，存在关于绕积算子遍历的不变测度。最后，我们得到了随机马氏链的强大数定律。     

Gaussian OS-positive processes

Summary Gaussian processes satisfying Osterwalder-Schrader positivity are studied. A representation of the (generalized) covariance function of an OS-positive process as the Laplace transform of an operator-valued probability measure is given. It is shown that every Gaussian OS-positive process has a unique Gaussian canonical Markov extension. An explicit application is made to the generalized free Euclidean fields.Partially supported by the National Science Foundation under grant MCS-76 06332     

Markov Uniqueness and Rademacher Theorem for Smooth Measures on an Infinite-Dimensional Space under Successful-Filtration Condition

For a smooth measure on an infinite-dimensional space, a “successful-filtration” condition is introduced and the Markov uniqueness and Rademacher theorem for measures satisfying this condition are proved. Some sufficient conditions, such as the well-known Hoegh-Krohn condition, are also considered. Examples demonstrating connections between these conditions and applications to convex measures are given.__________Published in Ukrains’kyi Matematychnyi Zhurnal, Vol. 57, No. 2, pp. 170–186, February, 2005.     

Asymptotic Uniqueness of Best Rational Approximants of Given Degree to Markov Functions in L 2 of the Circle

We improve over a sufficient condition given in [8] for uniqueness of a nondegenerate critical point in best rational approximation of prescribed degree over the conjugate-symmetric Hardy space of the complement of the disk. The improved condition connects to error estimates in AAK approximation, and is necessary and sufficient when the function to be approximated is of Markov type. For Markov functions whose defining measure satisfies the Szego condition, we combine what precedes with sharp asymptotics in multipoint Padé approximation from [43], [40] in order to prove uniqueness of a critical point when the degree of the approximant goes large. This lends perspective to the uniqueness issue for more general classes of functions defined through Cauchy integrals.     

Markov chain approximations to filtering equations for reflecting diffusion processes

Herein, we consider direct Markov chain approximations to the Duncan–Mortensen–Zakai equations for nonlinear filtering problems on regular, bounded domains. For clarity of presentation, we restrict our attention to reflecting diffusion signals with symmetrizable generators. Our Markov chains are constructed by employing a wide band observation noise approximation, dividing the signal state space into cells, and utilizing an empirical measure process estimation. The upshot of our approximation is an efficient, effective algorithm for implementing such filtering problems. We prove that our approximations converge to the desired conditional distribution of the signal given the observation. Moreover, we use simulations to compare computational efficiency of this new method to the previously developed branching particle filter and interacting particle filter methods. This Markov chain method is demonstrated to outperform the two-particle filter methods on our simulated test problem, which is motivated by the fish farming industry.     

Large Deviations from a Stationary Measure for a Class of Dissipative PDEs with Random Kicks

We study a class of dissipative PDEs perturbed by a bounded random kick force. It is assumed that the random force is nondegenerate, so that the Markov process obtained by the restriction of solutions to integer times has a unique stationary measure. The main result of the paper is a large deviations principle for occupation measures of the Markov process in question. The proof is based on Kifer's large‐deviation criterion, a coupling argument for Markov processes, and an abstract result on large‐time asymptotic for generalized Markov semigroups.© 2015 Wiley Periodicals, Inc.     

Partially observed optimal stopping problem for discrete-time Markov processes

This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization procedure based on optimal quantization. First, we discretize the state space of the unobserved variable by quantizing an underlying reference measure. Then we jointly discretize the resulting approximate filter and the observation process. We obtain a fully computable approximation of the value function with explicit error bounds for its convergence towards the true value function.     

Conservative markov processes on a topological space

A Markov operator preservingC(X) is known to induce a decomposition of the locally compact spaceX to conservative and dissipative parts. Two notions of ergodicity are defined and the existence of subprocesses is studied. A sufficient condition for the existence of a conservative subprocess is given, and then the process is assumed to be conservative. When it has no subprocesses, sufficient conditions for the existence of a σ-finite invariant measure are given, and are extended to continuous-time processes. When the invariant measure is unique, ratio limit theorems are proved for the discrete and continuous time processes. Examples show that some combinations of conservative processes are not necessarily conservative. This paper is a part of the authors’s Ph.D. thesis prepared at the Hebrew University under the direction of Professor S. R. Foguel, to whom the author is grateful for his helpful advice and kind encouragement.