On the computation of the optimal cost function for discrete time Markov models with partial observations

We consider several applications of two state, finite action, infinite horizon, discrete-time Markov decision processes with partial observations, for two special cases of observation quality, and show that in each of these cases the optimal cost function is piecewise linear. This in turn allows us to obtain either explicit formulas or simplified algorithms to compute the optimal cost function and the associated optimal control policy. Several examples are presented.Research supported in part by the Air Force Office of Scientific Research under Grant AFOSR-86-0029, in part by the National Science Foundation under Grant ECS-8617860, in part by the Advanced Technology Program of the State of Texas, and in part by the DoD Joint Services Electronics Program through the Air Force Office of Scientific Research (AFSC) Contract F49620-86-C-0045.     

薄透镜焦距测定的改进及误差分析

本文通过对薄透镜距测量的改进，并以薄透镜焦距测量为例进行误差分析及讨论．恰当地运用数学工具对观测数据做出科学的分析处理，获得正确的结论，对提高实验能力和运用误差理论来处理实验数据起到很大作用．     

Statistical isotropy of the cosmic microwave background

The breakdown of statistical homogeneity and isotropy of cosmic perturbations is a generic feature of ultra-large scale structure of the cosmos, in particular, of non-trivial cosmic topology. The statistical isotropy (SI) of the cosmic microwave background temperature fluctuations (CMB anisotropy) is sensitive to this breakdown on the largest scales comparable to, and even beyond the cosmic horizon. We propose a set of measures,K _l (l = 1, 2,3,...) which for non-zero values indicate and quantify statistical isotropy violations in a CMB map. We numerically compute the predictedK _l spectra for CMB anisotropy in flat torus universe models. Characteristic signatures of different models in theK _l spectrum are noted.     

Estimation of IIR Systems with Binary-Valued Observations*

Estimation and control problems with binary-valued observations exist widely in practical systems. However, most of the related works are devoted to finite impulse response (FIR for short) systems, and the theoretical problem of infinite impulse response (IIR for short) systems has been less explored. To study the estimation problems of IIR systems with binary-valued observations, the authors introduce a projected recursive estimation algorithm and analyse its global convergence properties, by using the stochastic Lyapunov function methods and the limit theory on double array martingales. It is shown that the estimation algorithm has similar convergence results as those for FIR systems under a weakest possible non-persistent excitation condition. Moreover, the upper bound for the accumulated regret of adaptive prediction is also established without resorting to any excitation condition.     

Stochastic stabilization of hybrid neural networks by periodically intermittent control based on discrete-time state observations

This paper is concerned with stabilization of hybrid neural networks by intermittent control based on continuous or discrete-time state observations. By means of exponential martingale inequality and the ergodic property of the Markov chain, we establish a sufficient stability criterion on hybrid neural networks by intermittent control based on continuous-time state observations. Meantime, by M-matrix theory and comparison method, we show that hybrid neural networks can be stabilized by intermittent control based on discrete-time state observations. Finally, two examples are presented to illustrate our theory.     

An alternative approach to Privault's discrete-time chaotic calculus

In this paper, we present an alternative approach to Privault's discrete-time chaotic calculus. Let Z be an appropriate stochastic process indexed by N (the set of nonnegative integers) and l²(Γ) the space of square summable functions defined on Γ (the finite power set of N). First we introduce a stochastic integral operator J with respect to Z, which, unlike discrete multiple Wiener integral operators, acts on l²(Γ). And then we show how to define the gradient and divergence by means of the operator J and the combinatorial properties of l²(Γ). We also prove in our setting the main results of the discrete-time chaotic calculus like the Clark formula, the integration by parts formula, etc. Finally we show an application of the gradient and divergence operators to quantum probability.     

On the discrete-time Geo/G/1 queue with randomized vacations and at most J vacations

This paper examines a discrete-time Geo/G/1 queue, where the server may take at most J − 1 vacations after the essential vacation. In this system, messages arrive according to Bernoulli process and receive corresponding service immediately if the server is available upon arrival. When the server is busy or on vacation, arriving messages have to wait in the queue. After the messages in the queue are served exhaustively, the server leaves for the essential vacation. At the end of essential vacation, the server activates immediately to serve if there are messages waiting in the queue. Alternatively, the server may take another vacation with probability p or go into idle state with probability (1 − p) until the next message arrives. Such pattern continues until the number of vacations taken reaches J. This queueing system has potential applications in the packet-switched networks. By applying the generating function technique, some important performance measures are derived, which may be useful for network and software system engineers. A cost model, developed to determine the optimum values of p and J at a minimum cost, is also studied.     

Generalized Markov Models of Infectious Disease Spread: A Novel Framework for Developing Dynamic Health Policies

We propose a class of mathematical models for the transmission of infectious diseases in large populations. This class of models, which generalizes the existing discrete-time Markov chain models of infectious diseases, is compatible with efficient dynamic optimization techniques to assist real-time selection and modification of public health interventions in response to evolving epidemiological situations and changing availability of information and medical resources. While retaining the strength of existing classes of mathematical models in their ability to represent the within-host natural history of disease and between-host transmission dynamics, the proposed models possess two advantages over previous models: (1) these models can be used to generate optimal dynamic health policies for controlling spreads of infectious diseases, and (2) these models are able to approximate the spread of the disease in relatively large populations with a limited state space size and computation time.     

Cost‐efficient monitoring of continuous‐time stochastic processes based on discrete observations

Planning a cost‐efficient monitoring policy of stochastic processes arises from many industrial problems. We formulate a simple discrete‐time monitoring problem of continuous‐time stochastic processes with its applications to several industrial problems. A key in our model is a doubling trick of the variables, with which we can construct an algorithm to solve the problem. The cost‐efficient monitoring policy balancing between the observation cost and information loss is governed by an optimality equation of a fixed point type, which is solvable with an iterative algorithm based on the Feynman‐Kac formula. This is a new linkage between monitoring problems and mathematical sciences. We show regularity results of the optimization problem and present a numerical algorithm for its approximation. A problem having model ambiguity is presented as well. The presented model is applied to problems of environment, ecology, and energy, having qualitatively different target stochastic processes with each other.     

Integrating noisy data

Suppose the parametric form of a curve is not known, but only a set of observations. Quadrature formulae can be used to integrate a function only known from a set of data points. However, the results will be unreliable if the data contains measurement errors (noise). The method presented here fits an even degree piecewise polynomial to the data where all the data points are being used as knot points and the smoothing parameter is optimal for the indefinite integral of the curve which happens to be a smoothing spline. After the smoothing parameter has been chosen, this approach is less computationally expensive than fitting a smoothing spline and integrating.