State estimation for partially observed Markov chains

Recursive equations are derived for the conditional distribution of the state of a Markov chain, given observations of a function of the state. Mainly continuous time chains are considered. The equations for the conditional distribution are given in matrix form and in differential equation form. The conditional distribution itself forms a Markov process. Special cases considered are doubly stochastic Poisson processes with a Markovian intensity, Markov chains with a random time, and Markovian approximations of semi-Markov processes. Further the results are used to compute the Radon-Nikodym derivative for two probability measures for a Markov chain, when a function of the state is observed.     

On likelihood estimation for a discretely observed jump process

We consider the parameter estimation problem for a Markov jump process sampled at periodic epochs with a constant step. Unlike the diffusion case where a closed form of the likelihood function is usually unavailable, we provide here an explicit expression of the likelihood function of the sampled chain. Moreover under suitable ergodicity condition on the jump process, we establish the consistency and the asymptotic normality of the likelihood estimator as the observation period tends to infinity. To cite this article: D. Dehay, J.-f. Yao, C. R. Acad. Sci. Paris, Ser. I 342 (2006).     

Optimal stopping for partially observed piecewise-deterministic Markov processes

This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of the partially observed optimal stopping problem. Then, we propose a numerical method, based on the quantization of the discrete-time filter process and the inter-jump times, to approximate the value function and to compute an ?$?$-optimal stopping time. We prove the convergence of the algorithms and bound the rates of convergence.     

Parameter estimation of partially observed continuous time stochastic processes via the EM algorithm

An algorithm is presented for the problem of maximum likelihood (ML) estimation of parameters of partially observed continuous time random processes. This algorithm is an extension of the EM algorithm [3] used in the time series literature, and preserves its main features. It is then applied to the problem of parameter estimation of continuous time, finite state or infinite state (diffusions) Markov processes observed via a noisy sensor. The algorithm in general involves iterations of non-linear smoothing with known parameters and then a non-stochastic maximization. For special cases, including linear models and AR/ARMA processes observed in white noise, each iteration is easily performed with finite dimensional filters. Finally, the algorithm is applied to parameter estimation of “randomly slowly varying” linear systems observed in white noise, and explicit results are derived.     

Consistent parameter estimation for partially observed diffusions with small noise

In this paper we provide a consistency result for the MLE for partially observed diffusion processes with small noise intensities. We prove that if the underlying deterministic system enjoys an identifiability property, then any MLE is close to the true parameter if the noise intensities are small enough. The proof uses large deviations limits obtained by PDE vanishing viscosity methods. A deterministic method of parameter estimation is formulated. We also specialize our results to a binary detection problem, and compare deterministic and stochastic notions of identifiability.This research was supported: by Systems Research Center, University of Maryland through NSF Grant CDR-85-00108 and AFOSR-URI Grant 87-0073; by Lefschetz Center for Dynamical Systems, Division of Applied Mathematics, Brown University, under ARO/MIT Grant DAAL-03-86-K-0171; by INRIA Sophia Antipolis, under ERO/INRIA Grant DAJA45-90-C-0008, and by the CNRS-GRAutomatique.     

partially observed control of markov processes,I

It is shown that to each partially observed control problem corresponds a completely observed separated problem, in the sense that the corresponding value functions are equal. This result, established in certain cases previously by Bismut and Fleming, is established here in general, in particular for diffusions. It is also shown that the corresponding measure-valued martingale problem is well posed for constant controls     

Asymptotic normality for discretely observed Markov jump processes with an absorbing state

For a continuous-time Markov process, occasionally, only discrete-time observations are available. For a simple sample of homogeneous Markov jump processes with an absorbing state, observed each on a stochastic grid of time points, we establish asymptotic normality of the maximum likelihood estimator and close the gap in Kremer and Weißbach (2013). By showing that the solution of the Kolmogorov backward equation system is continuous differentiable, we can apply results for M-estimators.     

A survey of algorithmic methods for partially observed Markov decision processes

A partially observed Markov decision process (POMDP) is a generalization of a Markov decision process that allows for incomplete information regarding the state of the system. The significant applied potential for such processes remains largely unrealized, due to an historical lack of tractable solution methodologies. This paper reviews some of the current algorithmic alternatives for solving discrete-time, finite POMDPs over both finite and infinite horizons. The major impediment to exact solution is that, even with a finite set of internal system states, the set of possible information states is uncountably infinite. Finite algorithms are theoretically available for exact solution of the finite horizon problem, but these are computationally intractable for even modest-sized problems. Several approximation methodologies are reviewed that have the potential to generate computationally feasible, high precision solutions.     

Unbiased truncated quadratic variation for volatility estimation in jump diffusion processes

The problem of integrated volatility estimation for an Ito semimartingale is considered under discrete high-frequency observations in short time horizon. We provide an asymptotic expansion for the integrated volatility that gives us, in detail, the contribution deriving from the jump part. The knowledge of such a contribution allows us to build an unbiased version of the truncated quadratic variation, in which the bias is visibly reduced. In earlier results to have the original truncated realized volatility well-performed the condition $\beta >\frac{1}{2(2-\alpha )}$ on $\beta$ (that is such that ${\left(\frac{1}{n}\right)}^{\beta }$ is the threshold of the truncated quadratic variation) and on the degree of jump activity $\alpha$ was needed (see Mancini, 2011; Jacod, 2008). In this paper we theoretically relax this condition and we show that our unbiased estimator achieves excellent numerical results for any couple ($\alpha$, $\beta$).     

Equations of optimal nonlinear filtration and interpolation for partially observed Markov processes

Recurrence relations are obtained for problems of optimal filtration and interpolation of partially observed discrete Markov chains. We present the system of differential equations for problems of optimal nonlinear filtration for Markov processes with continuous time and the system of inverse differential equations for problems of optimal nonlinear interpolation.Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 46, No. 8, pp. 971–976, August, 1994.     

State estimation for cox processes on general spaces

Let N be an observable Cox process on a locally compact space E directed by an unobservable random measure M. Techniques are presented for estimation of M, using the observations of N to calculate conditional expectations of the form E [M]|$\text{F}$_A], where $\text{F}$_A is the σ–algebra generated by the restriction of N to A. We introduce a random measure whose distribution depends on N_A, from which we obtain both exact estimates and a recursive method for updating them as further observations become available. Application is made to the specific cases of estimation of an unknown, random scalar multiplier of a known measure, of a symmetrically distributed directing measure M and of a Markov–directed Cox process on $\text{R}$. By means of a Poisson cluster representation, the results are extended to treat the situation where N is conditionally additive and infinitely divisible given M.     

State estimation for cox processes with unknown probability law

Let N_i, i?1, be i.i.d. observable Cox processes on a compact metric space E, directed by unobservable random measures M_i. Assume that the probability law of the M_i is completely unknown. Techniques are developed for approximation of state estimators using data from the processes N₁,…,N_n to estimate necessary attributes of the unknown probability law of the time M_i. The techniques are based on representation of the state estimators in terms of reduced Palm distributions of the N_i and on estimation of these Palm distributions. Estimators of Palm distributions are shown to be strongly consistent and asymptotically normal. The difference between the true and the pseudo-state estimators converges to zero in L² at rate n^{?$\text{1}\text{4+δ}$} for each δ > 0.     

Adaptive nonparametric estimation for Lévy processes observed at low frequency

This article deals with adaptive nonparametric estimation for Lévy processes observed at low frequency. For general linear functionals of the Lévy measure, we construct kernel estimators, provide upper risk bounds and derive rates of convergence under regularity assumptions.     

Nonparametric estimation for pure jump Lévy processes based on high frequency data

In this paper, we study nonparametric estimation of the Lévy density for pure jump Lévy processes. We consider n$n$ discrete time observations with step Δ$\Delta$. The asymptotic framework is: n$n$ tends to infinity, Δ=_Δn$\Delta ={\Delta }_n$ tends to zero while n_Δn$n{\Delta }_n$ tends to infinity. First, we use a Fourier approach (“frequency domain”): this allows us to construct an adaptive nonparametric estimator and to provide a bound for the global L²${\mathbb{L}}^2$-risk. Second, we use a direct approach (“time domain”) which allows us to construct an estimator on a given compact interval. We provide a bound for L²${\mathbb{L}}^2$-risk restricted to the compact interval. We discuss rates of convergence and give examples and simulation results for processes fitting in our framework.