Filtering partially observable diffusions up to the exit time from a domain

We consider a two-component diffusion process with the second component treated as the observations of the first one. The observations are available only until the first exit time of the first component from a fixed domain. We derive filtering equations for an unnormalized conditional distribution of the first component before it hits the boundary and give a formula for the conditional distribution of the first component at the first time it hits the boundary.     

Control of partially observed diffusions

The optimal control of a partially observed diffusion is discussed when the control parameter is present in both the drift and diffusion coefficients. Using a differentiation result of Blagovescenskii and Freidlin, and adapting techniques of Bensoussan, we obtain a stochastic minimum principle.This research was partially supported by NSERC Grant A7964, by the US Air Force Office of Scientific Research Contract AFOSR-86-0332, and by the US Army Research Office Contract DAAL03-87-K-0102.     

Controlled partially observed diffusions with correlated noise

This work is concerned with separated control problems for optimal stochastic controls under partial observations. Continuity properties of the unnormalized conditional distribution measure are found, and the Nisio nonlinear semigroup is formed in the case when a functionh(X _t ,Y _t ,U _t ) of stateX _t observationY _t , and controlU _t plus correlated additive white noise is observed.     

Maximal integral over observable measures

Let f be a continuous transformation on a compact,finite-dimensional manifold M,andψa continuous function on M.This paper establishes the following formula:ess sup lim sup n→∞ 1/n ψ_n(x)=sup{∫ψdμ︱μ∈O_f}≤lim sup n→∞ 1/n ess sup ψ_n(x),where ess sup denotes the essential supremum taken against the Lebesgue measure,ψ_n(x)=n-1 ∑ i=0 ψ(f~ix) and O_f is the set of observable measures.Examples are provided to illustrate that the inequality could be an equality or strict.Moreover,if μ is the unique maximizing observable measure for ψ,it is weakly statistical stable.     

Entanglement measures based on observable correlations

By regarding quantum states as communication channels and using observable correlations quantitatively expressed by mutual information, we introduce a hierarchy of entanglement measures that includes the entanglement of formation as a particular instance. We compare the maximal and minimal measures and indicate the conceptual advantages of the minimal measure over the entanglement of formation. We reveal a curious feature of the entanglement of formation by showing that it can exceed the quantum mutual information, which is usually regarded as a theoretical measure of total correlations. This places the entanglement of formation in a broader scenario, highlights its peculiarity in relation to pure-state ensembles, and introduces a competing definition with intrinsic informational significance. __________ Translated from Teoreticheskaya i Matematicheskaya Fizika, Vol. 155, No. 3, pp. 453–462, June, 2008.     

Control limits for two-state partially observable Markov decision processes

A production process can be in either a GOOD or a BAD state. The true state is unknown and can only be inferred from observations. If the state is good during one period it may deteriorate and become bad during the next period. Two actions are available: CONTINUE or REPLACE (for a fixed cost). The objective is to maximize the expected discounted value of the total future profits.     

Risk measurement and risk-averse control of partially observable discrete-time Markov systems

We consider risk measurement in controlled partially observable Markov processes in discrete time. We introduce a new concept of conditional stochastic time consistency and we derive the structure of risk measures enjoying this property. We prove that they can be represented by a collection of static law invariant risk measures on the space of function of the observable part of the state. We also derive the corresponding dynamic programming equations. Finally we illustrate the results on a machine deterioration problem.     

Consistent parameter estimation for partially observed diffusions with small noise

In this paper we provide a consistency result for the MLE for partially observed diffusion processes with small noise intensities. We prove that if the underlying deterministic system enjoys an identifiability property, then any MLE is close to the true parameter if the noise intensities are small enough. The proof uses large deviations limits obtained by PDE vanishing viscosity methods. A deterministic method of parameter estimation is formulated. We also specialize our results to a binary detection problem, and compare deterministic and stochastic notions of identifiability.This research was supported: by Systems Research Center, University of Maryland through NSF Grant CDR-85-00108 and AFOSR-URI Grant 87-0073; by Lefschetz Center for Dynamical Systems, Division of Applied Mathematics, Brown University, under ARO/MIT Grant DAAL-03-86-K-0171; by INRIA Sophia Antipolis, under ERO/INRIA Grant DAJA45-90-C-0008, and by the CNRS-GRAutomatique.     

Estimation of parameters for Hilbert space-valued partially observable stochastic processes

We give the asymptotic statistical theory (strong consistency and asymptotic normality) of a modified least-square-estimator for the parameters of a linear time discrete Kalman-filter-system. The method of proof uses a strong law of large numbers for martingale difference and ergodic sequences and a central limit theorem for q-dependent stationary processes.     

Parameter estimation for partially observable systems subject to random failure

In this paper, we present a parameter estimation procedure for a condition‐based maintenance model under partial observations. Systems can be in a healthy or unhealthy operational state, or in a failure state. System deterioration is driven by a continuous time homogeneous Markov chain and the system state is unobservable, except the failure state. Vector information that is stochastically related to the system state is obtained through condition monitoring at equidistant sampling times. Two types of data histories are available — data histories that end with observable failure, and censored data histories that end when the system has been suspended from operation but has not failed. The state and observation processes are modeled in the hidden Markov framework and the model parameters are estimated using the expectation–maximization algorithm. We show that both the pseudolikelihood function and the parameter updates in each iteration of the expectation–maximization algorithm have explicit formulas. A numerical example is developed using real multivariate spectrometric oil data coming from the failing transmission units of 240‐ton heavy hauler trucks used in the Athabasca oil sands of Alberta, Canada. Copyright © 2012 John Wiley & Sons, Ltd.     

Invariant measures of stochastic partial differential equations and conditioned diffusions

This work establishes and exploits a connection between the invariant measure of stochastic partial differential equations (SPDEs) and the law of bridge processes. Namely, it is shown that the invariant measure of $u_t=u_{xx}+f(u)+\sqrt{2?}\eta (x\text{,}t)$, where $\eta (x\text{,}t)$ is a space–time white-noise, is identical to the law of the bridge process associated to $\mathrm{d}U=a(U)\mathrm{d}x+\sqrt{?}\mathrm{d}W(x)$, provided that a and f are related by $?a^{″}(u)+2a^{\prime }(u)a(u)=?2f(u)$, $u\in \mathbb{R}$. Some consequences of this connection are investigated, including the existence and properties of the invariant measure for the SPDE on the line, $x\in \mathbb{R}$. To cite this article: M.G. Reznikoff, E. Vanden-Eijnden, C. R. Acad. Sci. Paris, Ser. I 340 (2005).     

Transformation of partially observable Markov decision processes into piecewise linear ones

This paper considers how partially observable Markov decision processes may be transformed into piecewise linear ones, which have many advantages in that they are easily represented in a computer. Also we specify how to find the products of simple partitions on which cost functions are piecewise linear.     

Optimal non-linear filtering of a class of partially observable Markov processes

For a class of partially observed Markov processes a representation for the optimal non-linear filter is obtained in which stochastic integrals are absent.     

A partially observable decision problem under a shifted likelihood ratio ordering

In this paper, we discuss a partially observable sequential decision problem under a shifted likelihood ratio ordering. Since we employ the Bayes' theorem for the learning procedure, we treat this problem under several assumptions. Under these assumptions, we obtain some fundamental results about the relation between prior and posterior information. We also consider an optimal stopping problem for this partially observable Markov decision process.     

The problem of optimal stopping in a partially observable Markov chain

The optimal-stopping problem in a partially observable Markov chain is considered, and this is formulated as a Markov decision process. We treat a multiple stopping problem in this paper. Unlike the classical stopping problem, the current state of the chain is not known directly. Information about the current state is always available from an information process. Several properties about the value and the optimal policy are given. For example, if we add another stop action to thek-stop problem, the increment of the value is decreasing ink.The author wishes to thank Professor M. Sakaguchi of Osaka University for his encouragement and guidance. He also thanks the referees for their careful readings and helpful comments.