Necessary conditions for optimal singular stochastic control problems

In this paper, necessary conditions of optimality, in the form of a maximum principle, are obtained for singular stochastic control problems. This maximum principle is derived for a state process satisfying a general stochastic differential equation where the coefficient associated to the control process can be dependent on the state, extending earlier results of the literature.     

Dynamic programming for multidimensional stochastic control problems

In this paper we study a general multidimensional diffusion-type stochastic control problem. Our model contains the usual regular control problem, singular control problem and impulse control problem as special cases. Using a unified treatment of dynamic programming, we show that the value function of the problem is a viscosity solution of certain Hamilton-Jacobi-Bellman (HJB) quasivariational inequality. The uniqueness of such a quasi-variational inequality is proved. Supported in part by USA Office of Naval Research grant #N00014-96-1-0262. Supported in part by the NSFC Grant #79790130, the National Distinguished Youth Science Foundation of China Grant #19725106 and the Chinese Education Ministry Science Foundation.     

A method for solving stochastic eigenvalue problems

We present a novel approach for calculating stochastic eigenvalues of differential and integral equations as well as for random matrices. Five examples based on very different types of problem have been analysed and detailed numerical results obtained. It would seem that the method has considerable promise. The essence of the method is to replace the stochastic eigenvalue problem λ(ξ)?(ξ)=A(ξ)?(ξ), where ξ is a set of random variables, by the introduction of an auxiliary equation in which . This changes the problem from an eigenvalue one to an initial value problem in the new pseudo-time variable t. The new linear time-dependent equation may then be solved by a polynomial chaos expansion (PCE) and the stochastic eigenvalue and its moments recovered by a limiting process. This technique has the advantage of avoiding the non-linear terms in the conventional method of stochastic eigenvalue calculation by PCE, but it does introduce an additional, ‘pseudo-time’, independent variable t. The paper illustrates the viability of this approach by application to several examples based on realistic problems.     

Solution of a class of stochastic linear-convex control problems using deterministic equivalents

In this paper, we identify a new class of stochastic linearconvex optimal control problems, whose solution can be obtained by solving appropriate equivalent deterministic optimal control problems. The term linear-convex is meant to imply that the dynamics is linear and the cost function is convex in the state variables, linear in the control variables, and separable. Moreover, some of the coefficients in the dynamics are allowed to be random and the expectations of the control variables are allowed to be constrained. For any stochastic linear-convex problem, the equivalent deterministic problem is obtained. Furthermore, it is shown that the optimal feedback policy of the stochastic problem is affine in its current state, where the affine transformation depends explicitly on the optimal solution of the equivalent deterministic problem in a simple way. The result is illustrated by its application to a simple stochastic inventory control problem.This research was supported in part by NSERC Grant A4617, by SSHRC Grant 410-83-0888, and by an INRIA Post-Doctoral Fellowship.     

A scenario aggregation algorithm for the solution of stochastic dynamic minimax problems

In this paper, we present a scenario aggregation algorithm for the solution of the dynamic minimax problem in stochastic programming. We consider the case where the joint probability distribution has a known finite support. The algorithm applies the Alternating Direction of Multipliers Method on a reformulation of the minimax problem using a double duality framework. The problem is solved by decomposition into scenario sub-problems, which are deterministic multi-period problems. Convergence properties are deduced from the Alternating Direction of Multipliers. The resulting algorithm can be seen as an extension of Rockafellar and Wets Progressive Hedging algorithm to the dynamic minimax context.     

Turnpike properties for a class of piecewise deterministic systems arising in manufacturing flow control

This paper deals with a general class of piecewise deterministic control systems that encompasses FMS flow control models. One uses the Markov renewal decision process formalism to characterize optimal policies via a discrete event dynamic programming approach. A family of control problems with a random stopping time is associated with these optimality conditions. These problems can be reformulated as infinite horizon deterministic control problems. It is then shown how the so-calledturnpike property should hold for these deterministic control problems under classical convexity assumptions. These turnpikes have the same generic properties as the attractors obtained via a problem specific approach in FMS flow control models and production planning and are calledhedging points in this literature.This research has been supported by NSERC-Canada, Grants No. A4952 by FCAR-Québec, Grant No. 88EQ3528, Actions Structurantes, MESS-Québec, Grant No. 6.1/7.4(28), and FNRS-Switzerland.     

Efficient dynamic programming implementations of Newton's method for unconstrained optimal control problems

Naive implementations of Newton's method for unconstrainedN-stage discrete-time optimal control problems with Bolza objective functions tend to increase in cost likeN ³ asN increases. However, if the inherent recursive structure of the Bolza problem is properly exploited, the cost of computing a Newton step will increase only linearly withN. The efficient Newton implementation scheme proposed here is similar to Mayne's DDP (differential dynamic programming) method but produces the Newton step exactly, even when the dynamical equations are nonlinear. The proposed scheme is also related to a Riccati treatment of the linear, two-point boundary-value problems that characterize optimal solutions. For discrete-time problems, the dynamic programming approach and the Riccati substitution differ in an interesting way; however, these differences essentially vanish in the continuous-time limit.This work was supported by the National Science Foundation, Grant No. DMS-85-03746.     

Low-order self-tuner for fault-tolerant control of a class of unknown nonlinear stochastic sampled-data systems

Based on the modified state-space self-tuning control (STC) via the observer/Kalman filter identification (OKID) method, an effective low-order tuner for fault-tolerant control of a class of unknown nonlinear stochastic sampled-data systems is proposed in this paper. The OKID method is a time-domain technique that identifies a discrete input–output map by using known input–output sampled data in the general coordinate form, through an extension of the eigensystem realization algorithm (ERA). Then, the above identified model in a general coordinate form is transformed to an observer form to provide a computationally effective initialization for a low-order on-line “auto-regressive moving average process with exogenous (ARMAX) model”-based identification. Furthermore, the proposed approach uses a modified Kalman filter estimate algorithm and the current-output-based observer to repair the drawback of the system multiple failures. Thus, the fault-tolerant control (FTC) performance can be significantly improved. As a result, a low-order state-space self-tuning control (STC) is constructed. Finally, the method is applied for a three-tank system with various faults to demonstrate the effectiveness of the proposed methodology.     

Finite element approximations of stochastic optimal control problems constrained by stochastic elliptic PDEs

In this paper we study mathematically and computationally optimal control problems for stochastic elliptic partial differential equations. The control objective is to minimize the expectation of a tracking cost functional, and the control is of the deterministic, distributed type. The main analytical tool is the Wiener-Itô chaos or the Karhunen-Loève expansion. Mathematically, we prove the existence of an optimal solution; we establish the validity of the Lagrange multiplier rule and obtain a stochastic optimality system of equations; we represent the input data in their Wiener-Itô chaos expansions and deduce the deterministic optimality system of equations. Computationally, we approximate the optimality system through the discretizations of the probability space and the spatial space by the finite element method; we also derive error estimates in terms of both types of discretizations.     

A method of stochastic subgradients with complete feedback stepsize rule for convex stochastic approximation problems

A stochastic subgradient algorithm for solving convex stochastic approximation problems is considered. In the algorithm, the stepsize coefficients are controlled on-line on the basis of information gathered in the course of computations according to a new, complete feedback rule derived from the concept of regularized improvement function. Convergence with probability 1 of the method is established.This work was supported by Project No. CPBP/02.15.