Stochastic adaptive control for continuous-time linear systems with quadratic cost

An adaptive control problem is formulated and solved for a completely observed, continuous-time, linear stochastic system with an ergodic quadratic cost criterion. The linear transformationsA of the state,B of the control, andC of the noise are assumed to be unknown. Assuming only thatA is stable and that the pair (A, C) is controllable and using a diminishing excitation control that is asymptotically negligible for an ergodic, quadratic cost criterion it is shown that a family of least-squares estimates is strongly consistent. Furthermore, an adaptive control is given using switchings that is self-optimizing for an ergodic, quadratic cost criterion.This research was partially supported b y NSF Grants ECS-9102714, ECS-9113029, and DMS-9305936.     

Discrete-time stochastic adaptive control with small observation noise

A discrete-time stochastic system depending on an unknown parameter and with small observation noise is considered in this paper. A quadratic variation test is proposed to detect the unknown system parameter. Then, based on the test result, asymptotic filters and computable adaptive asymptotically optimal controls are constructed. Finally, numerical experiments are undertaken regarding the above aspects.This work was supported by NSERC under Grant 8051.     

Stochastic piecewise affine control with application to pitch control of helicopter

In this paper, at first the stability condition which gives an upper stochastic bound for a class of Stochastic Hybrid Systems (SHS) with deterministic jumps is derived. Here, additive noise signals are considered that do not vanish at equilibrium points. The presented theorem gives an upper bound for the second stochastic moment or variance of the system trajectories. Then, the linear case of SHS is investigated to show the application of the theorem. For the linear case of such stochastic hybrid systems, the stability criterion is obtained in terms of Linear Matrix Inequality (LMI) and an upper bound on state covariance is obtained for them. Then utilizing the stability theorem, an output feedback controller design procedure is proposed which requires the Bilinear Matrix Inequalities (BMI) to be solved. Next, the pitch dynamics of a helicopter is approximated with a set of linear stochastic systems, and the proposed controller is designed for the approximated model and implemented on the main nonlinear system to demonstrate the effectiveness of the proposed theorem and the control design method.     

Estimation and control for linear,partially observable systems with non-gaussian initial distribution

The nonlinear filtering problem of estimating the state of a linear stochastic system from noisy observations is solved for a broad class of probability distributions of the initial state. It is shown that the conditional density of the present state, given the past observations, is a mixture of Gaussian distributions, and is parametrically determined by two sets of sufficient statistics which satisfy stochastic DEs; this result leads to a generalization of the Kalman–Bucy filter to a structure with a conditional mean vector, and additional sufficient statistics that obey nonlinear equations, and determine a generalized (random) Kalman gain. The theory is used to solve explicitly a control problem with quadratic running and terminal costs, and bounded controls.     

Stochastic Partial Differential Equations on Manifolds~II. Nonlinear Filtering

The conditional law of an unobservable component x(t) of a diffusion (x(t),y(t)) given the observations {y(s):s[0,t]} is investigated when x(t) lives on a submanifold of . The existence of the conditional density with respect to a given measure on is shown under fairly general conditions, and the analytical properties of this density are characterized in terms of the Sobolev spaces used in the first part of this series.     

Stochastic recursive inclusions with non-additive iterate-dependent Markov noise

In this paper we study the asymptotic behaviour of stochastic approximation schemes with set-valued drift function and non-additive iterate-dependent Markov noise. We show that a linearly interpolated trajectory of such a recursion is an asymptotic pseudotrajectory for the flow of a limiting differential inclusion obtained by averaging the set-valued drift function of the recursion w.r.t. the stationary distributions of the Markov noise. The limit set theorem by Benaim is then used to characterize the limit sets of the recursion in terms of the dynamics of the limiting differential inclusion. We then state two variants of the Markov noise assumption under which the analysis of the recursion is similar to the one presented in this paper. Scenarios where our recursion naturally appears are presented as applications. These include controlled stochastic approximation, subgradient descent, approximate drift problem and analysis of discontinuous dynamics all in the presence of non-additive iterate-dependent Markov noise.     

Stochastic partial differential equations with unbounded and degenerate coefficients

In this article, using DiPerna-Lions theory (DiPerna and Lions, 1989) [1], we investigate linear second order stochastic partial differential equations with unbounded and degenerate non-smooth coefficients, and obtain several conditions for existence and uniqueness. Moreover, we also prove the L¹-integrability and a general maximal principle for generalized solutions of SPDEs. As applications, we study nonlinear filtering problem and also obtain the existence and uniqueness of generalized solutions for a degenerate nonlinear SPDE.     

Stochastic Detectability and Mean Bounded Error Covariance of the Recursive Kalman Filter with Markov Jump Parameters

In this article, we study the error covariance of the recursive Kalman filter when the parameters of the filter are driven by a Markov chain taking values in a countably infinite set. We do not assume ergodicity nor require the existence of limiting probabilities for the Markov chain. The error covariance matrix of the filter depends on the Markov state realizations, and hence forms a stochastic process. We show in a rather direct and comprehensive manner that this error covariance process is mean bounded under the standard stochastic detectability concept. Illustrative examples are included.     

Stochastic approximation with dependent noise

In this work we derive the usual limit laws (weak and strong convergence, central limit theorem, invariance principle) for stochastic approximation with stationary noise. The idea is to introduce an artificial sequence, related to the SA scheme, but which clearly obeys the desired limit law. This sequence is subtracted from the SA scheme and the remainder, which behaves more or less deterministically, is shown to vanish using simple limit arguments.     

反射的椭圆随机偏微分方程的网格逼近

研究了由可乘噪声驱动的反射的椭圆随机偏微分方程网格近似解的收敛性,其中考虑区域D:=(0,1)~d,d=1,2,3.此外,还研究确定的椭圆障碍问题离散格式的解存在唯一性,并得到解关于障碍函数的连续依赖性和收敛性.     

Stochastic control of funding systems

This paper is concerned with funding systems, i.e. systems which accumulate funds for the future payment of financial obligations. Commonly, such funding requires a balance between (1) the desire to minimise the contributions that need to be diverted from other use to the support of the Fund, and (2) the need to maintain reasonable solvency in the Fund.Such funding is discussed here in a general framework. Applications are numerous. The specific applications mentioned in the paper are:

• Defined benefit retirement funding,

• Maintenance of a prudential margin by a non-life insurer,

• Dividend payment strategy.

The paper applies stochastic optimal control theory to determine how rates of contribution to the Fund and allocation of its assets by asset sector should respond to changing solvency. These results are obtainable from a particular differential equation, which may be solved numerically. Detailed numerical examples are provided.     

Estimation of systems with statistically-constrained inputs

This paper discusses the estimation of a class of discrete-time linear stochastic systems with statistically-constrained unknown inputs (UI), which can represent an arbitrary combination of a class of un-modeled dynamics, random UI with unknown covariance matrix and deterministic UI. In filter design, an upper bound filter is explored to compute, recursively and adaptively, the upper bounds of covariance matrices of the state prediction error, innovation and state estimate error. Furthermore, the minimum upper bound filter (MUBF) is obtained via online scalar parameter convex optimization in pursuit of the minimum upper bounds. Two examples, a system with multiple piecewise UIs and a continuous stirred tank reactor (CSTR), are used to illustrate the proposed MUBF scheme and verify its performance.     

Stochastic Evolution Equations Driven by a Fractional White Noise

Abstract

In this paper we study stochastic evolution equations driven by a fractional white noise with arbitrary Hurst parameter in infinite dimension. We establish the existence and uniqueness of a mild solution for a nonlinear equation with multiplicative noise under Lipschitz condition by using a fixed point argument in an appropriate inductive limit space. In the linear case with additive noise, a strong solution is obtained. Those results are applied to stochastic parabolic partial differential equations perturbed by a fractional white noise.     

The Filtering Equations of Forward-Backward Stochastic Systems with Random Jumps and Applications to Partial Information Stochastic Optimal Control

In this article, we consider a filtering problem for forward-backward stochastic systems that are driven by Brownian motions and Poisson processes. This kind of filtering problem arises from the study of partially observable stochastic linear-quadratic control problems. Combining forward-backward stochastic differential equation theory with certain classical filtering techniques, the desired filtering equation is established. To illustrate the filtering theory, the theoretical result is applied to solve a partially observable linear-quadratic control problem, where an explicit observable optimal control is determined by the optimal filtering estimation.     

The nonlinear Schrödinger equation with white noise dispersion

Under certain scaling the nonlinear Schrödinger equation with random dispersion converges to the nonlinear Schrödinger equation with white noise dispersion. The aim of this work is to prove that this latter equation is globally well posed in L² or H¹. The main ingredient is the generalization of the classical Strichartz estimates. Additionally, we justify rigorously the formal limit described above.     

Stochastic control with exit time and constraints,application to small time attainability of sets

We study the existence of a solution of controlled stochastic differential equations remaining in a given set of constraints at any time smaller than the exit time of a given open set. We also investigate the small time attainability of a given closed set K, i.e., the property that, for all arbitrary small time horizon T and for all initial condition in a sufficiently small neighborhood of K, there exists a solution to the controlled stochastic differential equation which reaches K before T.     

Stochastic stability of FitzHugh-Nagumo systems in infinite lattice perturbed by Gaussian white noise

The current paper is devoted to the study of stochastic stability of FitzHugh-Nagumo systems in infinite lattice perturbed by Gaussian white noise. We first study the dynamics of stochastic FitzHugh-Nagumo systems, then prove the existence and uniqueness of their equilibriums, which mix exponentially. Finally, we investigate asymptotic behavior of equilibriums when the size of noise gets to zero.     

Nonlinear estimation applying an unscented transformation in systems with correlated uncertain observations

An approximation to the least squares filter is proposed for discrete signals whose evolution is governed by nonlinear functions, when the estimation is based on nonlinear observations with additive noise which can consist only of random noise; this uncertainty in the observation process is modelled by Bernoulli random variables which are correlated at consecutive time instants and are otherwise independent. The proposed recursive approximation is based on the unscented principle; successive applications of the unscented transformation to a suitable augmented state vector enable us to approximate the one-stage state and observation predictors from the state filter at the previous time instant. The performance of the proposed algorithm is compared with that of an extended algorithm in a numerical simulation example.