Tracking the output of a poorly known markov process

Tracking the output of an unknown Markov process with unknown generator and unknown output function is considered. It is assumed the unknown quantities have a known prior probability distribution. It is shown that the optimal control is a linear feedback in the tracking error plus the conditional expectation of a quantity involving the unknown generator and output function of the Markov process. The results also have application to Bayesian identification of hidden Markov models     

Quantized stabilization of stochastic systems with multiplicative noise under Markovian switching

A problem of quantized state feedback quadratic mean-square stabilization of discrete-time stochastic processes under Markovian switching and multiplicative noise is considered. A static quantizer is used in the feedback channel and the jump Markovian switching is modeled by a discrete-time Markov chain. The control input is simultaneously applied to both the rate vector and the diffusion term. It is shown that the coarsest quantization density that permits quadratic mean-square stabilization of this system is achieved with the use of a logarithmic quantizer, and the coarsest quantization density is determined by an algebraic Riccati equation, which is also the solution to a special linear stochastic Markovian switching control system. Also, sufficient conditions for exponential mean-square stabilization of such systems are also explored. An example is given to demonstrate the obtained results.     

Hybrid Impulsive Control of Stochastic Systems with Multiplicative Noise Under Markovian Switching

A problem of state output feedback stabilization of discrete-time stochastic systems with multiplicative noise under Markovian switching is considered. Under some appropriate assumptions, the stability of this system under pure impulsive control is given. Further under hybrid impulsive control, the output feedback stabilization problem is investigated. The hybrid control action is formulated as a combination of the regular control along with an impulsive control action. The jump Markovian switching is modeled by a discrete-time Markov chain. The control input is simultaneously applied to both the stochastic and the deterministic terms. Sufficient conditions based on stochastic semi-definite programming and linear matrix inequalities (LMIs) for both stochastic stability and stabilization are obtained. Such a nonconvex problem is solved using the existing optimization algorithms and the nonconvex CVX package. The robustness of the stability and stabilization concepts against all admissible uncertainties are also investigated. The parameter uncertainties we consider here are norm bounded. Two examples are given to demonstrate the obtained results.     

Robust Control of Markovian Switching Stochastic Systems with Noise Dependent States and Inputs

A problem of state feedback stabilization of discrete-time stochastic processes under Markovian switching and random diffusion (noise) is considered. The jump Markovian switching is modeled by a discrete-time Markov chain. The control input is simultaneously applied to both the rate vector and the diffusion term. Sufficient conditions based on linear matrix inequalities (LMI's) for stochastic stability is obtained. The robustness results of such stability concept against all admissible uncertainties are also investigated. An example is given to demonstrate the obtained results.     

A method of stochastic subgradients with complete feedback stepsize rule for convex stochastic approximation problems

A stochastic subgradient algorithm for solving convex stochastic approximation problems is considered. In the algorithm, the stepsize coefficients are controlled on-line on the basis of information gathered in the course of computations according to a new, complete feedback rule derived from the concept of regularized improvement function. Convergence with probability 1 of the method is established.This work was supported by Project No. CPBP/02.15.     

Smoothing algorithms for nonlinear finite-dimensional systems

Systems are considered where the state evolves either as a diffusion process or as a finitestate Markov process, and the measurement process consists either of a nonlinear function of the state with additive white noise or as a counting process with intensity dependent on the state. Fixed interval smooting is considered, and the first main result obtained expresses a smoothing probability or a probability density symmetrically in terms of forward filtered, reverse-time filterd and unfiltered quantities; an associated result replaces the unfiltered and reverse-time filtered qauantities by a likelihood function. Then stochastic differential equationsare obtained for the evolution of the reverse-time filtered probability or probability density and the reverse-time likelihood function. Lastly, a partial differential equation is obtained linking smoothed and forward filterd probabilities or probability densities; in all instances considered, this equation is not driven by any measurement process. The different approaches are also linked to known techniques applicable in the linear-Gaussian case.     

Forward–backward stochastic differential equations with monotone functionals and mean field games with common noise

We consider a system of forward–backward stochastic differential equations (FBSDEs) with monotone functionals. We show that such a system is well-posed by the method of continuation similarly to Peng and Wu (1999) for classical FBSDEs. As applications, we prove the well-posedness result for a mean field FBSDE with conditional law and show the existence of a decoupling function. Lastly, we show that mean field games with common noise are uniquely solvable under a linear-convex setting and weak-monotone cost functions and prove that the optimal control is in a feedback form depending only on the current state and conditional law.     

A vector measure approach to state feedback control in Banach resolution spaces I

An abstract version of the linear regulator-quadratic cost problem is considered for a dynamical system S, where input and output are elements of various Banach resolution spaces. Our main result is the representation of the optimal control in memoryless state feedback form. This representation is obtained as an integral with respect to a vector measure defined on the state space of S.     

On high-order discrete derivatives of stochastic variables

We derive an explicit expression for the probability density function of the mth numerical derivative of a stochastic variable. It is shown that the proposed statistics can analytically be obtained based on the original probability characteristics of the observed signal in a simple manner. We argue that this allows estimating the statistical parameters of the original distribution and further, to simulate the noise contribution in the original stochastic process so that the noise component is statistically indistinguishable from the true contribution of the noise in the originally observed data signal.     

Borgonovo moment independent global sensitivity analysis by Gaussian radial basis function meta-model

Moment independent sensitivity index is widely concerned and used since it can reflect the influence of model input uncertainty on the entire distribution of model output instead of a specific moment. In this paper, a novel analytical expression to estimate the Borgonovo moment independent sensitivity index is derived by use of the Gaussian radial basis function and the Edgeworth expansion. Firstly, the analytical expressions of the unconditional and conditional first four-order moments are established by the training points and the widths of the Gaussian radial basis function. Secondly, the Edgeworth expansion is used to express the unconditional and conditional probability density functions of model output by the unconditional and conditional first four-order moments, respectively. Finally, the index can be readily computed by measuring the shifts between the obtained unconditional and conditional probability density functions of model output, where this process doesn't need any extra calls of model evaluation. The computational cost of the proposed method is independent of the dimensionality of model inputs and it only depends on the training points and the widths which are involved in the Gaussian radial basis function meta-model. Results of several case studies demonstrate the effectiveness of the proposed method.     

Witnessed k-Distance

Distance functions to compact sets play a central role in several areas of computational geometry. Methods that rely on them are robust to the perturbations of the data by the Hausdorff noise, but fail in the presence of outliers. The recently introduced distance to a measure offers a solution by extending the distance function framework to reasoning about the geometry of probability measures, while maintaining theoretical guarantees about the quality of the inferred information. A combinatorial explosion hinders working with distance to a measure as an ordinary power distance function. In this paper, we analyze an approximation scheme that keeps the representation linear in the size of the input, while maintaining the guarantees on the inference quality close to those for the exact but costly representation.     

极大极小随机规划逼近问题最优解集和最优值的稳定性

研究了特殊的二层极大极小随机规划逼近收敛问题. 首先将下层初始随机规划最优解集拓展到非单点集情形, 且可行集正则的条件下, 讨论了下层随机规划逼近问题最优解集关于上层决策变量参数的上半收敛性和最优值函数的连续性. 然后把下层随机规划的epsilon-最优解向量函数反馈到上层随机规划的目标函数中, 得到了上层随机规划逼近问题的最优解集关于最小信息概率度量收敛的上半收敛性和最优值的连续性.     

Stochastic stabilization of first-passage failure of Rayleigh oscillator under Gaussian White-Noise parametric excitations

Stochastic stabilization of first-passage failure of Rayleigh oscillator under Gaussian White-Noise parametric excitation is studied. The equation of motion of the system is first reduced to an averaged Itô stochastic differential equation by using the stochastic averaging method. Then, a backward Kolmogorov equation governing the conditional reliability function of first-passage failure is established. The conditional reliability function, and the conditional probability density are obtained by solving the backward Kolmogorov equation with boundary conditions. Finally, the cost function and optimal control forces are determined by the requirements of stabilizing the system by evaluating the maximal Lyapunov exponent. The numerical results show that the procedure is effective and efficiency.     

加性二值噪声激励下Duffing系统的随机分岔

研究了Duffing系统在加性二值噪声作用下的随机分岔现象.首先,根据二值噪声的统计特性,推导得到二值噪声状态间的跃迁概率,据此对二值噪声进行了数值模拟.其次,利用四阶Runge-Kutta(龙格-库塔)数值算法得到该系统位移和速率的稳态联合概率密度及位移的稳态概率密度.然后,通过对位移稳态概率密度单双峰结构变化的研究,发现加性二值噪声的状态和强度能够诱导系统产生随机分岔现象.最后,观察到随着系统非对称参数的逐渐变化,系统同样产生了随机分岔现象.     

Stochastic Discrete-Time Nash Games with Constrained State Estimators

In this paper, we consider stochastic linear-quadratic discrete-time Nash games in which two players have access only to noise-corrupted output measurements. We assume that each player is constrained to use a linear Kalman filter-like state estimator to implement his optimal strategies. Two information structures available to the players in their state estimators are investigated. The first has access to one-step delayed output and a one-step delayed control input of the player. The second has access to the current output and a one-step delayed control input of the player. In both cases, statistics of the process and statistics of the measurements of each player are known to both players. A simple example of a two-zone energy trading system is considered to illustrate the developed Nash strategies. In this example, the Nash strategies are calculated for the two cases of unlimited and limited transmission capacity constraints.     

Estimation and control for linear,partially observable systems with non-gaussian initial distribution

The nonlinear filtering problem of estimating the state of a linear stochastic system from noisy observations is solved for a broad class of probability distributions of the initial state. It is shown that the conditional density of the present state, given the past observations, is a mixture of Gaussian distributions, and is parametrically determined by two sets of sufficient statistics which satisfy stochastic DEs; this result leads to a generalization of the Kalman–Bucy filter to a structure with a conditional mean vector, and additional sufficient statistics that obey nonlinear equations, and determine a generalized (random) Kalman gain. The theory is used to solve explicitly a control problem with quadratic running and terminal costs, and bounded controls.     

Particle methods for stochastic optimal control problems

When dealing with numerical solution of stochastic optimal control problems, stochastic dynamic programming is the natural framework. In order to try to overcome the so-called curse of dimensionality, the stochastic programming school promoted another approach based on scenario trees which can be seen as the combination of Monte Carlo sampling ideas on the one hand, and of a heuristic technique to handle causality (or nonanticipativeness) constraints on the other hand. However, if one considers that the solution of a stochastic optimal control problem is a feedback law which relates control to state variables, the numerical resolution of the optimization problem over a scenario tree should be completed by a feedback synthesis stage in which, at each time step of the scenario tree, control values at nodes are plotted against corresponding state values to provide a first discrete shape of this feedback law from which a continuous function can be finally inferred. From this point of view, the scenario tree approach faces an important difficulty: at the first time stages (close to the tree root), there are a few nodes (or Monte-Carlo particles), and therefore a relatively scarce amount of information to guess a feedback law, but this information is generally of a good quality (that is, viewed as a set of control value estimates for some particular state values, it has a small variance because the future of those nodes is rich enough); on the contrary, at the final time stages (near the tree leaves), the number of nodes increases but the variance gets large because the future of each node gets poor (and sometimes even deterministic). After this dilemma has been confirmed by numerical experiments, we have tried to derive new variational approaches. First of all, two different formulations of the essential constraint of nonanticipativeness are considered: one is called algebraic and the other one is called functional. Next, in both settings, we obtain optimality conditions for the corresponding optimal control problem. For the numerical resolution of those optimality conditions, an adaptive mesh discretization method is used in the state space in order to provide information for feedback synthesis. This mesh is naturally derived from a bunch of sample noise trajectories which need not to be put into the form of a tree prior to numerical resolution. In particular, an important consequence of this discrepancy with the scenario tree approach is that the same number of nodes (or points) are available from the beginning to the end of the time horizon. And this will be obtained without sacrifying the quality of the results (that is, the variance of the estimates). Results of experiments with a hydro-electric dam production management problem will be presented and will demonstrate the claimed improvements. A more realistic problem will also be presented in order to demonstrate the effectiveness of the method for high dimensional problems.     

Nonparametric multi-step prediction in nonlinear state space dynamic systems

Filtering and smoothing of stochastic state space dynamic systems have benefited from several generations of estimation approaches since the seminal works of Kalman in the sixties. A set of global analytical or numerical methods are now available, such as the well-known sequential Monte Carlo particle methods which offer some theoretical convergence results for both types of problems. However except in the case of linear Gaussian systems, objectives of the third kind i.e. prediction objectives, which aim at estimating k time steps ahead the anticipated probability density function of the system state variables, conditional on past and present system output observations, still raise theoretical and practical difficulties. The aim of this paper is to propose a nonparametric particle multi-step prediction method able to consistently estimate such anticipated conditional pdf of the state variables as well as their expectations.     

Robust finite-time H∞ control of singular stochastic systems via static output feedback

This paper addresses the problem of robust finite-time stabilization of singular stochastic systems via static output feedback. Firstly, sufficient conditions of singular stochastic finite-time boundedness on static output feedback are obtained for the family of singular stochastic systems with parametric uncertainties and time-varying norm-bounded disturbance. Then the results are extended to singular stochastic H_∞ finite-time boundedness for the class of singular stochastic systems. Designed algorithm for static output feedback controller is provided to guarantee that the underlying closed-loop singular stochastic system is singular stochastic H_∞ finite-time boundedness in terms of strict linear matrix equalities with a fixed parameter. Finally, an illustrative example is presented to show the validity of the developed methodology.     

Direction selection in stochastic directional distance functions

Researchers rely on the distance function to model multiple product production using multiple inputs. A stochastic directional distance function (SDDF) allows for noise in potentially all input and output variables. Yet, when estimated, the direction selected will affect the functional estimates because deviations from the estimated function are minimized in the specified direction. Specifically, the parameters of the parametric SDDF are point identified when the direction is specified; we show that the parameters of the parametric SDDF are set identified when multiple directions are considered. Further, the set of identified parameters can be narrowed via data-driven approaches to restrict the directions considered. We demonstrate a similar narrowing of the identified parameter set for a shape constrained nonparametric method, where the shape constraints impose standard features of a cost function such as monotonicity and convexity.Our Monte Carlo simulation studies reveal significant improvements, as measured by out of sample radial mean squared error, in functional estimates when we use a directional distance function with an appropriately selected direction and the errors are uncorrelated across variables. We show that these benefits increase as the correlation in error terms across variables increase. This correlation is a type of endogeneity that is common in production settings. From our Monte Carlo simulations we conclude that selecting a direction that is approximately orthogonal to the estimated function in the central region of the data gives significantly better estimates relative to the directions commonly used in the literature. For practitioners, our results imply that selecting a direction vector that has non-zero components for all variables that may have measurement error provides a significant improvement in the estimator’s performance. We illustrate these results using cost and production data from samples of approximately 500 US hospitals per year operating in 2007, 2008, and 2009, respectively, and find that the shape constrained nonparametric methods provide a significant increase in flexibility over second order local approximation parametric methods.