Solution of a class of stochastic linear-convex control problems using deterministic equivalents

In this paper, we identify a new class of stochastic linearconvex optimal control problems, whose solution can be obtained by solving appropriate equivalent deterministic optimal control problems. The term linear-convex is meant to imply that the dynamics is linear and the cost function is convex in the state variables, linear in the control variables, and separable. Moreover, some of the coefficients in the dynamics are allowed to be random and the expectations of the control variables are allowed to be constrained. For any stochastic linear-convex problem, the equivalent deterministic problem is obtained. Furthermore, it is shown that the optimal feedback policy of the stochastic problem is affine in its current state, where the affine transformation depends explicitly on the optimal solution of the equivalent deterministic problem in a simple way. The result is illustrated by its application to a simple stochastic inventory control problem.This research was supported in part by NSERC Grant A4617, by SSHRC Grant 410-83-0888, and by an INRIA Post-Doctoral Fellowship.     

Pricing problems with a continuum of customers as stochastic Stackelberg games

The pricing problem where a company sells a certain kind of product to a continuum of customers is considered. It is formulated as a stochastic Stackelberg game with nonnested information structure. The inducible region concept, recently developed for deterministic Stackelberg games, is extended to treat the stochastic pricing problem. Necessary and sufficient conditions for a pricing scheme to be optimal are derived, and the pricing problem is solved by first delineating its inducible region, and then solving a constrained optimal control problem.The research work reported here as supported in part by the National Science Foundation under Grant ECS-81-05984, Grant ECS-82-10673, and by the Air Force Office of Scientific Research under AFOSR Grant 80-0098.     

A finite-state,finite-memory minimum principle

A class of finite-state, finite-memory stochastic control problems is considered. A minimum principle is derived. Signaling strategies are defined and related to the necessary conditions of the minimum principle. Min-H algorithms for the problem are described.This research has been conducted in the MIT Electronic Systems Laboratory with support from AFOSR Grant No. 72-2273, NASA/AMES Grant No. NGL-22-009(124), and NSF Grant No. GK-25781.     

An optimal control problem with a random stopping time

This paper deals with a stochastic optimal control problem where the randomness is essentially concentrated in the stopping time terminating the process. If the stopping time is characterized by an intensity depending on the state and control variables, one can reformulate the problem equivalently as an infinite-horizon optimal control problem. Applying dynamic programming and minimum principle techniques to this associated deterministic control problem yields specific optimality conditions for the original stochastic control problem. It is also possible to characterize extremal steady states. The model is illustrated by an example related to the economics of technological innovation.This research has been supported by NSERC-Canada, Grants 36444 and A4952; by FCAR-Québec, Grant 88EQ3528, Actions Structurantes; and by MESS-Québec, Grant 6.1/7.4(28).     

Hierarchical production policies in stochastic two-machine flowshops with finite buffers

This paper concerns production planning in manufacturing systems with two unreliable machines in tandem. The problem is formulated as a stochastic control problem in which the objective is to minimize the expected total cost of production, inventories, and backlogs. Since the sizes of the internal and external buffers are finite, the problem is one with state constraints. As the optimal solutions to this problem are extremely difficult to obtain due to the uncertainty in machine capacities as well as the presence of state constraints, a deterministic limting problem in which the stochastic machine capacities are replaced by their mean capacities is considered instead. The weak Lipschitz property of the value functions for the original and limiting problems is introduced and proved; a constraint domain approximation approach is developed to show that the value function of the original problem converges to that of the limiting problem as the rate of change in machine states approaches infinity. Asymptotic optimal production policies for the orginal problem are constructed explicity from the near-optimal policies of the limiting problem, and the error estimate for the policies constructed is obtained. Algorithms for constructing these policies are presented.This work was partly supported by CUHK Direct Grant 220500660, RGC Earmarked Grant CUHK 249/94E, and RGC Earmarked Grant CUHK 489/95E.     

Measurement scheduling for recursive team estimation

We consider a decentralized LQG measurement scheduling problem in which every measurement is costly, no communication between observers is permitted, and the observers' estimation errors are coupled quadratically. This setup, motivated by considerations from organization theory, models measurement scheduling problems in which cost, bandwidth, or security constraints necessitate that estimates be decentralized, although their errors are coupled. We show that, unlike the centralized case, in the decentralized case the problem of optimizing the time integral of the measurement cost and the quadratic estimation error is fundamentally stochastic, and we characterize the -optimal open-loop schedules as chattering solutions of a deterministic Lagrange optimal control problem. Using a numerical example, we describe also how this deterministic optimal control problem can be solved by nonlinear programming.This research was supported in part by ARPA Grant N00174-91-C-0116 and NSF Grant NCR-92-04419.     

Dynamic programming for multidimensional stochastic control problems

In this paper we study a general multidimensional diffusion-type stochastic control problem. Our model contains the usual regular control problem, singular control problem and impulse control problem as special cases. Using a unified treatment of dynamic programming, we show that the value function of the problem is a viscosity solution of certain Hamilton-Jacobi-Bellman (HJB) quasivariational inequality. The uniqueness of such a quasi-variational inequality is proved. Supported in part by USA Office of Naval Research grant #N00014-96-1-0262. Supported in part by the NSFC Grant #79790130, the National Distinguished Youth Science Foundation of China Grant #19725106 and the Chinese Education Ministry Science Foundation.     

A maximum principle for optimal control problem of fully coupled forward-backward stochastic systems with partial information

The paper is concerned with a stochastic optimal control problem in which the controlled system is described by a fully coupled nonlinear forward-backward stochastic differential equation driven by a Brownian motion. It is required that all admissible control processes are adapted to a given subfiltration of the filtration generated by the underlying Brownian motion. For this type of partial information control, one sufficient (a verification theorem) and one necessary conditions of optimality are proved. The control domain need to be convex and the forward diffusion coefficient of the system can contain the control variable. This work was partially supported by Basic Research Program of China (Grant No. 2007CB814904), National Natural Science Foundation of China (Grant No. 10325101) and Natural Science Foundation of Zhejiang Province (Grant No. Y605478, Y606667)     

Optimal control with a cost on changing control

In this paper, we consider a class of optimal control problems in which the cost functional is the sum of the terminal cost, the integral cost, and the full variation of control. The term involving the full variation of control is to measure the changes on the control action. A computational method based on the control parametrization technique is developed for solving this class of optimal control problems. This computational method is supported by a convergence analysis. For illustration, two numerical examples are solved using the proposed method.This project was partially supported by an Australian Research Grant.This paper is dedicated to Professor L. Cesari on the occasion of his 80th birthday.     

A Deterministic Approach To Stochastic Optimal Control With Application To Anticipative Control

Using the decomposition of solution of SDE, we consider the stochastic optimal control problem with anticipative controls as a family of deterministic control problems parametrized by the paths of the driving Wiener process and of a newly introduced Lagrange multiplier stochastic process (nonanticipativity equality constraint). It is shown that the value function of these problems is the unique global solution of a robust equation (random partial differential equation) associated to a linear backward Hamilton-Jacobi-Bellman stochastic partial differential equation (HJB SPDE). This appears as limiting SPDE for a sequence of random HJB PDE's when linear interpolation approximation of the Wiener process is used. Our approach extends the Wong-Zakai type results [20] from SDE to the stochastic dynamic programming equation by showing how this arises as average of the limit of a sequence of deterministic dynamic programming equations. The stochastic characteristics method of Kunita [13] is used to represent the value function. By choosing the Lagrange multiplier equal to its nonanticipative constraint value the usual stochastic (nonanticipative) optimal control and optimal cost are recovered. This suggests a method for solving the anticipative control problems by almost sure deterministic optimal control. We obtain a PDE for the “cost of perfect information” the difference between the cost function of the nonanticipative control problem and the cost of the anticipative problem which satisfies a nonlinear backward HJB SPDE. Poisson bracket conditions are found ensuring this has a global solution. The cost of perfect information is shown to be zero when a Lagrangian submanifold is invariant for the stochastic characteristics. The LQG problem and a nonlinear anticipative control problem are considered as examples in this framework     

Bounded state problem for hereditary control problems

In this paper, we characterize optimal pairs for a hereditary control problem where the state is constrained. We use relaxed controls and the technique of penalization.This research was supported by NSF Grant No. HRD-91-54077.     

Distributed computing viewed as a team problem with constraints

The optimal distribution of the workload in a system of interconnected computer units is considered. Formulated as a team decision problem with a singular cost criterion and with equality and inequality constraints, it is shown that the problem admits always a unique piecewise linear strategy which is globally optimal. Some interesting particular cases are studied.The research reported in this paper was made possible through support from the Office of Naval Research under the Joint Services Electronics Program by Contract No. N00014-75-C-0648 and Contract No. N00014-77-C-0531 and by the National Science Foundation, Grant No. ENG-76-11824.     

Approximation of nonlinear functional differential equation control systems

We develop a general approximation framework for use in optimal control problems governed by nonlinear functional differential equations. Our approach entails only the use of linear semigroup approximation results, while the nonlinearities are treated as perturbations of a linear system. Numerical results are presented for several simple nonlinear optimal control problem examples.This research was supported in part by the US Air Force under Contract No. AF-AFOSR-76-3092 and in part by the National Science Foundation under Grant No. NSF-GP-28931x3.     

随机递归最优控制和混合最优控制问题

对随机递归最优控制问题即代价函数由特定倒向随机微分方程解来描述和递归混合最优控制问题即控制者还需 决定最优停止时刻, 得到了最优控制的存在性结果. 在一类等价概率测度集中,还给出了递归最优值函数的最小和最大数学期望.     

Existence of an optimal control for a coupled FBSDE with a non degenerate diffusion coefficient

We consider a controlled system driven by a coupled forward–backward stochastic differential equation with a non degenerate diffusion matrix. The cost functional is defined by the solution of the controlled backward stochastic differential equation, at the initial time. Our goal is to find an optimal control which minimizes the cost functional. The method consists to construct a sequence of approximating controlled systems for which we show the existence of a sequence of feedback optimal controls. By passing to the limit, we establish the existence of a relaxed optimal control to the initial problem. The existence of a strict control follows from the Filippov convexity condition.     

随机线性系统保成本控制的线性矩阵不等式方法

考虑具有二次成本函数的随机线性系统,研究了状态反馈控制的保证成本控制问题.依据线性矩阵不等式得到了保证成本控制器存在的充分条件,最后得到了随机线性闭环系统保证成本最小的最优保证成本控制律的表达式.     

Optimal Guaranteed Cost Control of Stochastic Discrete-Time Systems with States and Input Dependent Noise Under Markovian Switching

A problem of robust guaranteed cost control of stochastic discrete-time systems with parametric uncertainties under Markovian switching is considered. The control is simultaneously applied to both the random and the deterministic components of the system. The noise (the random) term depends on both the states and the control input. The jump Markovian switching is modeled by a discrete-time Markov chain and the noise or stochastic environmental disturbance is modeled by a sequence of identically independently normally distributed random variables. Using linear matrix inequalities (LMIs) approach, the robust quadratic stochastic stability is obtained. The proposed control law for this quadratic stochastic stabilization result depended on the mode of the system. This control law is developed such that the closed-loop system with a cost function has an upper bound under all admissible parameter uncertainties. The upper bound for the cost function is obtained as a minimization problem. Two numerical examples are given to demonstrate the potential of the proposed techniques and obtained results.     

һ��"��-ͣ"�������������ģ�͵��ƹ�

??This paper extends a class of discount problem of singular
stochastic control with stopping time. We extend the state process and cost function
to general case. By stochastic analysis and optimal control theory, the "fail-stop"
control strategy is its optimal control. The conditions of the "fail-stop" strategy
and optimal control function and control method are given. The conclusion in this
paper has a fairly deep application.     

Limiting behavior of bang-bang controls for Sobolev problems

We consider the limiting behavior of optimal bang-bang controls as a family of Sobolev equations formally converges to a wave equation. The weak-starlimit of the sequence of bang-bang controls is an optimal control for the wave equation problem. The associated optimal states converge strongly and, for the optimal time problem, the optimal times converge to the optimal time for the wave equation.This work was supported in part by the National Science Foundation, Grant No. MCS-79-02037.     

一类具漂移的“跳-停”奇异型随机控制

以随机分析和最优控制理论为基础,讨论了一类带停时的奇异型随机控制问题.在原模型状态过程的基础上添加了漂移因子,并将原模型中的控制费用函数推广为一般的费用函数.在某些条件下,得到"跳一停"策略是其最优控制策略,并给出了"跳一停"策略存在的条件以及控制方法,所得的结论在实际中有较深的应用背景.