Filtering partially observable diffusions up to the exit time from a domain

We consider a two-component diffusion process with the second component treated as the observations of the first one. The observations are available only until the first exit time of the first component from a fixed domain. We derive filtering equations for an unnormalized conditional distribution of the first component before it hits the boundary and give a formula for the conditional distribution of the first component at the first time it hits the boundary.     

Control of partially observed diffusions

The optimal control of a partially observed diffusion is discussed when the control parameter is present in both the drift and diffusion coefficients. Using a differentiation result of Blagovescenskii and Freidlin, and adapting techniques of Bensoussan, we obtain a stochastic minimum principle.This research was partially supported by NSERC Grant A7964, by the US Air Force Office of Scientific Research Contract AFOSR-86-0332, and by the US Army Research Office Contract DAAL03-87-K-0102.     

Controlled partially observed diffusions with correlated noise

This work is concerned with separated control problems for optimal stochastic controls under partial observations. Continuity properties of the unnormalized conditional distribution measure are found, and the Nisio nonlinear semigroup is formed in the case when a functionh(X _t ,Y _t ,U _t ) of stateX _t observationY _t , and controlU _t plus correlated additive white noise is observed.     

Entanglement measures based on observable correlations

By regarding quantum states as communication channels and using observable correlations quantitatively expressed by mutual information, we introduce a hierarchy of entanglement measures that includes the entanglement of formation as a particular instance. We compare the maximal and minimal measures and indicate the conceptual advantages of the minimal measure over the entanglement of formation. We reveal a curious feature of the entanglement of formation by showing that it can exceed the quantum mutual information, which is usually regarded as a theoretical measure of total correlations. This places the entanglement of formation in a broader scenario, highlights its peculiarity in relation to pure-state ensembles, and introduces a competing definition with intrinsic informational significance. __________ Translated from Teoreticheskaya i Matematicheskaya Fizika, Vol. 155, No. 3, pp. 453–462, June, 2008.     

Control limits for two-state partially observable Markov decision processes

A production process can be in either a GOOD or a BAD state. The true state is unknown and can only be inferred from observations. If the state is good during one period it may deteriorate and become bad during the next period. Two actions are available: CONTINUE or REPLACE (for a fixed cost). The objective is to maximize the expected discounted value of the total future profits.     

Risk measurement and risk-averse control of partially observable discrete-time Markov systems

We consider risk measurement in controlled partially observable Markov processes in discrete time. We introduce a new concept of conditional stochastic time consistency and we derive the structure of risk measures enjoying this property. We prove that they can be represented by a collection of static law invariant risk measures on the space of function of the observable part of the state. We also derive the corresponding dynamic programming equations. Finally we illustrate the results on a machine deterioration problem.     

Consistent parameter estimation for partially observed diffusions with small noise

In this paper we provide a consistency result for the MLE for partially observed diffusion processes with small noise intensities. We prove that if the underlying deterministic system enjoys an identifiability property, then any MLE is close to the true parameter if the noise intensities are small enough. The proof uses large deviations limits obtained by PDE vanishing viscosity methods. A deterministic method of parameter estimation is formulated. We also specialize our results to a binary detection problem, and compare deterministic and stochastic notions of identifiability.This research was supported: by Systems Research Center, University of Maryland through NSF Grant CDR-85-00108 and AFOSR-URI Grant 87-0073; by Lefschetz Center for Dynamical Systems, Division of Applied Mathematics, Brown University, under ARO/MIT Grant DAAL-03-86-K-0171; by INRIA Sophia Antipolis, under ERO/INRIA Grant DAJA45-90-C-0008, and by the CNRS-GRAutomatique.     

Estimation of parameters for Hilbert space-valued partially observable stochastic processes

We give the asymptotic statistical theory (strong consistency and asymptotic normality) of a modified least-square-estimator for the parameters of a linear time discrete Kalman-filter-system. The method of proof uses a strong law of large numbers for martingale difference and ergodic sequences and a central limit theorem for q-dependent stationary processes.     

Invariant measures of stochastic partial differential equations and conditioned diffusions

This work establishes and exploits a connection between the invariant measure of stochastic partial differential equations (SPDEs) and the law of bridge processes. Namely, it is shown that the invariant measure of $u_t=u_{xx}+f(u)+\sqrt{2?}\eta (x\text{,}t)$, where $\eta (x\text{,}t)$ is a space–time white-noise, is identical to the law of the bridge process associated to $\mathrm{d}U=a(U)\mathrm{d}x+\sqrt{?}\mathrm{d}W(x)$, provided that a and f are related by $?a^{″}(u)+2a^{\prime }(u)a(u)=?2f(u)$, $u\in \mathbb{R}$. Some consequences of this connection are investigated, including the existence and properties of the invariant measure for the SPDE on the line, $x\in \mathbb{R}$. To cite this article: M.G. Reznikoff, E. Vanden-Eijnden, C. R. Acad. Sci. Paris, Ser. I 340 (2005).     

Optimal non-linear filtering of a class of partially observable Markov processes

For a class of partially observed Markov processes a representation for the optimal non-linear filter is obtained in which stochastic integrals are absent.     

Transformation of partially observable Markov decision processes into piecewise linear ones

This paper considers how partially observable Markov decision processes may be transformed into piecewise linear ones, which have many advantages in that they are easily represented in a computer. Also we specify how to find the products of simple partitions on which cost functions are piecewise linear.     

A partially observable decision problem under a shifted likelihood ratio ordering

In this paper, we discuss a partially observable sequential decision problem under a shifted likelihood ratio ordering. Since we employ the Bayes' theorem for the learning procedure, we treat this problem under several assumptions. Under these assumptions, we obtain some fundamental results about the relation between prior and posterior information. We also consider an optimal stopping problem for this partially observable Markov decision process.     

The problem of optimal stopping in a partially observable Markov chain

The optimal-stopping problem in a partially observable Markov chain is considered, and this is formulated as a Markov decision process. We treat a multiple stopping problem in this paper. Unlike the classical stopping problem, the current state of the chain is not known directly. Information about the current state is always available from an information process. Several properties about the value and the optimal policy are given. For example, if we add another stop action to thek-stop problem, the increment of the value is decreasing ink.The author wishes to thank Professor M. Sakaguchi of Osaka University for his encouragement and guidance. He also thanks the referees for their careful readings and helpful comments.     

Estimation and control for linear,partially observable systems with non-gaussian initial distribution

The nonlinear filtering problem of estimating the state of a linear stochastic system from noisy observations is solved for a broad class of probability distributions of the initial state. It is shown that the conditional density of the present state, given the past observations, is a mixture of Gaussian distributions, and is parametrically determined by two sets of sufficient statistics which satisfy stochastic DEs; this result leads to a generalization of the Kalman–Bucy filter to a structure with a conditional mean vector, and additional sufficient statistics that obey nonlinear equations, and determine a generalized (random) Kalman gain. The theory is used to solve explicitly a control problem with quadratic running and terminal costs, and bounded controls.