Solution of a class of stochastic linear-convex control problems using deterministic equivalents

In this paper, we identify a new class of stochastic linearconvex optimal control problems, whose solution can be obtained by solving appropriate equivalent deterministic optimal control problems. The term linear-convex is meant to imply that the dynamics is linear and the cost function is convex in the state variables, linear in the control variables, and separable. Moreover, some of the coefficients in the dynamics are allowed to be random and the expectations of the control variables are allowed to be constrained. For any stochastic linear-convex problem, the equivalent deterministic problem is obtained. Furthermore, it is shown that the optimal feedback policy of the stochastic problem is affine in its current state, where the affine transformation depends explicitly on the optimal solution of the equivalent deterministic problem in a simple way. The result is illustrated by its application to a simple stochastic inventory control problem.This research was supported in part by NSERC Grant A4617, by SSHRC Grant 410-83-0888, and by an INRIA Post-Doctoral Fellowship.     

Deterministic mean-variance-optimal consumption and investment

In dynamic optimal consumption–investment problems one typically aims to find an optimal control from the set of adapted processes. This is also the natural starting point in case of a mean-variance objective. In contrast, we solve the optimization problem with the special feature that the consumption rate and the investment proportion are constrained to be deterministic processes. As a result we get rid of a series of unwanted features of the stochastic solution including diffusive consumption, satisfaction points and consistency problems. Deterministic strategies typically appear in unit-linked life insurance contracts, where the life-cycle investment strategy is age dependent but wealth independent. We explain how optimal deterministic strategies can be found numerically and present an example from life insurance where we compare the optimal solution with suboptimal deterministic strategies derived from the stochastic solution.     

随机递归最优控制和混合最优控制问题

对随机递归最优控制问题即代价函数由特定倒向随机微分方程解来描述和递归混合最优控制问题即控制者还需 决定最优停止时刻, 得到了最优控制的存在性结果. 在一类等价概率测度集中,还给出了递归最优值函数的最小和最大数学期望.     

Forward-backward Stochastic Differential Equations and Backward Linear Quadratic Stochastic Optimal Control Problem

In this paper, we use the solutions of forward-backward stochastic differential equations to get the optimal control for backward stochastic linear quadratic optimal control problem. And we also give the linear feedback regulator for the optimal control problem by using the solutions of a group of Riccati equations.     

一类新的线性二次随机最优控制器的设计

给出一类正倒向随机微分方程解的存在唯一性结果,应用这个结果研究了一类新的推广的随机线性二次最优控制器的设计问题,得到了由正倒向随机微分方程解所表示的唯一最优控制器的显式结构;在推广的Riccati方程系统基础上,得到最优控制器精确的线性反馈形式.最后,给出了随机线性二次最优控制器的设计算法.     

The efficient solution of the (quietly constrained) noisy, linear regulator problem

In a previous paper we gave a new, natural extension of the calculus of variations/optimal control theory to a (strong) stochastic setting. We now extend the theory of this most fundamental chapter of optimal control in several directions. Most importantly we present a new method of stochastic control, adding Brownian motion which makes the problem “noisy.” Secondly, we show how to obtain efficient solutions: direct stochastic integration for simpler problems and/or efficient and accurate numerical methods with a global a priori error of O(h3/2) for more complex problems. Finally, we include “quiet” constraints, i.e. deterministic relationships between the state and control variables. Our theory and results can be immediately restricted to the non “noisy” (deterministic) case yielding efficient, numerical solution techniques and an a priori error of O(h²). In this event we obtain the most efficient method of solving the (constrained) classical Linear Regulator Problem. Our methods are different from the standard theory of stochastic control. In some cases the solutions coincide or at least are closely related. However, our methods have many advantages including those mentioned above. In addition, our methods more directly follow the motivation and theory of classical (deterministic) optimization which is perhaps the most important area of physical and engineering science. Our results follow from related ideas in the deterministic theory. Thus, our approximation methods follow by guessing at an algorithm, but the proof of global convergence uses stochastic techniques because our trajectories are not differentiable. Along these lines, a general drift term in the trajectory equation is properly viewed as an added constraint and extends ideas given in the deterministic case by the first author.     

A numerical method to approximate optimal production and maintenance plan in a flexible manufacturing system

The simultaneous planning of the production and the maintenance in a flexible manufacturing system is considered in this paper. The manufacturing system is composed of one machine that produces a single product. There is a preventive maintenance plan to reduce the failure rate of the machine. This paper is different from the previous researches in this area in two separate ways. First, the failure rate of the machine is supposed to be a function of its age. Second, we assume that the demand of the manufacturing product is time dependent and its rate depends on the level of advertisement on that product. The objective is to maximize the expected discounted total profit of the firm over an infinite time horizon. In the process of finding a solution to the problem, we first characterize an optimal control by introducing a set of Hamilton–Jacobi–Bellman partial differential equations. Then we realize that under practical assumptions, this set of equations can not be solved analytically. Thus to find a suboptimal control, we approximate the original stochastic optimal control model by a discrete-time deterministic optimal control problem. Then proposing a numerical method to solve the steady state Riccati equation, we approximate a suboptimal solution to the problem.     

由常规故障和临界人为错误引起系统故障的可修复系统稳态解的最优控制

针对由由常规故障和临界人为错误引起系统故障的可修复系统的模型,以范数指标泛函作为衡量系统可控性的标准,利用Banach空间理论讨论系统稳态解达到预期概率分布的最优控制问题,给出了其最优解存在唯一性.     

一类最优投资随机控制模型的马氏链算法

现代金融经济中的很多问题可以构建成随机控制模型,而随机控制的求解却存在一定的困难.马氏链算法应该是一种有效的求解随机控制问题的数值方法.本文以Claus Munk的工作为基础,针对一类最优投资模型,具体确定了马氏链的转移矩阵并证明其满足算法收敛条件,并用MATLAB语言编成一个程序实现.     

最优投资组合模型研究

本文研究了在完备金融市场上 ,投资者最优投资组合的随机模型。在模型参数为常系数 ,效用函数为 (0 ,T],B[0 ,T])上的有界可测函数的情形下 ,得出其最大效用值函数是随机控制问题对应的 HJB方程的平滑解 ;最优策略被证明是存在的 ,并用反馈形式给出了最优投资组合策略。     

动力学系统参数辨识问题最优控制解的理论与方法(Ⅱ)——随机系统参数辨识及应用实例

基于文(Ⅰ)(《应用数学和力学》,1998,20(2))的内容和随机最优控制理论,本文首先介绍了随机动力学系统参数辨识问题最优控制解的概念.然后讨论了建立参数辨识问题HJB方程的过程以及参数辨识的算法.最后给出了一个应用实例:解决动力学系统局部非线性参数辨识问题的方法.     

可修复人机储备系统稳态解的最优控制

针对可修复人机储备系统的模型,以范数指标泛函作为衡量系统可控性的标准,利用Banach空间理论讨论系统稳态解达到预期概率分布的最优控制问题,给出了其最优解存在唯一性.     

OPTIMAL GRAZING MANAGEMENT RULES IN SEMI‐ARID RANGELANDS WITH UNCERTAIN RAINFALL

Abstract. We study optimal adaptive grazing management under uncertain rainfall in a discrete‐time model. As in each year actual rainfall can be observed during the short rainy season, and grazing management can be adapted accordingly for the growing season; the closed‐loop solution of the stochastic optimal control problem does not only depend on the state variable, but also on the realization of the random rainfall. This distinguishes optimal grazing management from the optimal use of most other natural resources under uncertainty, where the closed‐loop solution of the stochastic optimal control problem depends only on the state variables. Solving this unusual stochastic optimization problem allows us to critically contribute to a long‐standing controversy over how to optimally manage semi‐arid rangelands by simple rules of thumb.     

Optimal control problems constrained by the stochastic Navier–Stokes equations with multiplicative Lévy noise

We consider the controlled stochastic Navier–Stokes equations in a bounded multidimensional domain, where the noise term allows jumps. In order to prove existence and uniqueness of an optimal control w.r.t. a given control problem, we first need to show the existence and uniqueness of a local mild solution of the considered controlled stochastic Navier–Stokes equations. We then discuss the control problem, where the related cost functional includes stopping times dependent on controls. Based on the continuity of the cost functional, we can apply existence and uniqueness results provided in [4], which enables us to show that a unique optimal control exists.     

Stability,stabilizability and detectability for Markov jump discrete-time linear systems with multiplicative noise in Hilbert spaces

In this article we discuss stability, stabilizability and detectability problems for Markov-jump discrete-time linear systems (MJDLSs) with multiplicative noise (MN) and countably infinite state space of the Markov chain. On the basis of a new solution representation formula, we give new deterministic characterizations of the stability and the detectability properties of MJDLSs with MN. These results are obtained using an operatorial approach and the properties of certain positive evolution operators defined on ordered Banach spaces of sequences of nuclear operators. Assuming detectability conditions and avoiding stochastic proofs, we prove that any global, nonnegative and bounded solution of the Riccati equation of control is stabilizing for the MJDLSs with MN and control. Finally, we apply our results to solve a linear quadratic optimal control problem. The theory is illustrated by an example.     

含有随机波动的非线性的最优投资和消费模型

在连续时间模型假设下,研究风险资产价格服从一个带有随机波动的几何布朗运动的最优消费和投资问题．首先建立了最优消费和投资同题随机最优控制数学模型;然后运用随机最优控制理论,得到了最优投资和消费随机最优控制问题的值函数所满足的线性抛物线偏微分方程和非线性抛物线偏微分方程．     

一类线性奇异摄动最优控制问题的渐近解

研究了一类线性奇异摄动最优控制问题的空间对照结构,讨论了初始点固定,终端自由的情形.首先根据变分法得到了一阶最优性条件,其次运用退化最优控制问题的解证明了异宿轨道的存在性,从而结合奇异摄动理论证明了原问题空间对照结构解的存在性.进一步根据解的结构,利用边界层函数法构造了奇异摄动最优控制问题一致有效的形式渐近解.最后,通...     

An optimal control problem with a random stopping time

This paper deals with a stochastic optimal control problem where the randomness is essentially concentrated in the stopping time terminating the process. If the stopping time is characterized by an intensity depending on the state and control variables, one can reformulate the problem equivalently as an infinite-horizon optimal control problem. Applying dynamic programming and minimum principle techniques to this associated deterministic control problem yields specific optimality conditions for the original stochastic control problem. It is also possible to characterize extremal steady states. The model is illustrated by an example related to the economics of technological innovation.This research has been supported by NSERC-Canada, Grants 36444 and A4952; by FCAR-Québec, Grant 88EQ3528, Actions Structurantes; and by MESS-Québec, Grant 6.1/7.4(28).     

Maximum Principle for Risk-Sensitive Stochastic Optimal Control Problem and Applications to Finance

This article is concerned with a risk-sensitive stochastic optimal control problem motivated by a kind of optimal portfolio choice problem in the financial market. The maximum principle for this kind of problem is obtained, which is similar in form to its risk-neutral counterpart. But the adjoint equations and maximum condition heavily depend on the risk-sensitive parameter. This result is used to solve a kind of optimal portfolio choice problem and the optimal portfolio choice strategy is obtained. Computational results and figures explicitly illustrate the optimal solution and the sensitivity to the volatility rate parameter.     

一类部分信息的随机控制问题的极值原理

在本文中,我们证明了一类部分信息的随机控制问题的极值原理的一个充分条件和一个必要条件.其中,随机控制问题的控制系统是一个由鞅和Brown运动趋动的随机偏微分方程.