Convex duality for finite-fuel problems in singular stochastic control

Upon introducing a finite-fuel constraint in a stochastic control system, the convex duality formulation can be set up to represent the original singular control problem as a minimization problem over the space of vector measures at each level of available fuel. This minimization problem is imbedded tightly into a related weak problem, which is actually a mathematical programming problem over a convex,w*-compact space of vector-valued Radon measures. Then, through the Fenchel duality principle, the dual for the finite-fuel control problems is to seek the maximum of smooth subsolutions to a dynamic programming variational inequality. The approach is basically in the spirit of Fleming and Vermes, and the results of this paper extend those of Vinter and Lewis in deterministic control problems to the finite-fuel problems in singular stochastic control. Meanwhile, we also obtain the characterization of the value function as a solution to the dynamic programming variational inequality in the sense of the Schwartz distribution.The author is much indebted to Professor Wendell H. Fleming for his constant support and many helpful discussions during the preparation of this paper.     

On singular stochastic control and optimal stopping of spectrally negative jump diffusions

We consider a broad class of singular stochastic control problems of spectrally negative jump diffusions in the presence of potentially nonlinear state-dependent exercise payoffs. We analyse these problems by relying on associated variational inequalities and state a set of sufficient conditions under which the value of the considered problems can be explicitly derived in terms of the increasing minimal r-harmonic map. We also present a set of inequalities bounding the value of the optimal policy and prove that increased policy flexibility increases both the value of the optimal strategy as well as the rate at which this value grows.     

Optimal stopping and stochastic control differential games for jump diffusions

We study stochastic differential games of jump diffusions driven by Brownian motions and compensated Poisson random measures, where one of the players can choose the stochastic control and the other player can decide when to stop the system. We prove a verification theorem for such games in terms of a Hamilton–Jacobi–Bellman variational inequality. The results are applied to study some specific examples, including optimal resource extraction in a worst-case scenario, and risk minimizing optimal portfolio and stopping.     

Opticon: An algorithm for the optimal control of nonlinear stochastic models

In this paper we describe the algorithm OPTCON which has been developed for the optimal control of nonlinear stochastic models. It can be applied to obtain approximate numerical solutions of control problems where the objective function is quadratic and the dynamic system is nonlinear. In addition to the usual additive uncertainty, some or all of the parameters of the model may be stochastic variables. The optimal values of the control variables are computed in an iterative fashion: First, the time-invariant nonlinear system is linearized around a reference path and approximated by a time-varying linear system. Second, this new problem is solved by applying Bellman's principle of optimality. The resulting feedback equations are used to project expected optimal state and control variables. These projections then serve as a new reference path, and the two steps are repeated until convergence is reached. The algorithm has been implemented in the statistical programming system GAUSS. We derive some mathematical results needed for the algorithm and give an overview of the structure of OPTCON. Moreover, we report on some tentative applications of OPTCON to two small macroeconometric models for Austria.     

Imprecise parameters for near-optimal control of stochastic SIV epidemic model

The change of parameters may influence the dynamic behaviors of epidemic diseases. Biological system parameters can also be changed due to diverse uncertainties such as lack of data and errors in the statistical approach. The problem of how to define and decide the optimal-control strategies of epidemic diseases with imprecise parameters deserves further researches. The paper presents a stochastic susceptible, infected, and vaccinated (SIV) system that includes imprecise parameters. Firstly, we give the method of parameter estimates of the SIV model. Then, by using Ekeland's principle and Hamiltonian function, we obtain the sufficient conditions and necessary conditions of near-optimal control of the SIV epidemic model with imprecise parameters. At last, numerical examples prove our theoretical results.     

Monotonicity and bounds for convex stochastic control models

We consider a general convex stochastic control model. Our main interest concerns monotonicity results and bounds for the value functions and for optimal policies. In particular, we show how the value functions depend on the transition kernels and we present conditions for a lower bound of an optimal policy. Our approach is based on convex stochastic orderings of probability measures. We derive several interesting sufficient conditions of these ordering concepts, where we make also use of the Blackwell ordering. The structural results are illustrated by partially observed control models and Bayesian information models.     

An inverse optimal problem in discrete-time stochastic control

In this paper, we study an inverse optimal problem in discrete-time stochastic control. We give necessary and sufficient conditions for a solution to a system of stochastic difference equations to be the solution of a certain optimal control problem. Our results extend to the stochastic case the work of Dechert. In particular, we present a stochastic version of an important principle in welfare economics.     

带停时的奇异型随机控制问题在金融投资模型中的应用

以随机分析的知识和最优控制理论为基础,讨论了一类带停时的奇异型随机控制的折扣费用问题在金融投资模型中的应用,将该带停时的奇异型随机控制模型的受控状态过程和费用函数结构都推广到了最一般的形式,使该模型的应用范围更加广泛.通过讨论一组相应的变分不等式的解,分别对退化和非退化两种情况给出了此随机控制问题的最优策略,相应得出了投资模型中的最佳决策,并且证明了变分不等式的解即为最优费用函数.与以往不同的是,所得的相关结论应用到了金融投资模型中,从而解决了一类金融投资问题.     

LP-stabilization problem for linear stochastic control systems with multiplicative noise

For the deterministic case, a linear controlled system is alwayspth order stable as long as we use the control obtained as the solution of the so-called LQ-problem. For the stochastic case, however, a linear controlled system with multiplicative noise is not alwayspth mean stable for largep, even if we use the LQ-optimal control. Hence, it is meaningful to solve the LP-optimal control problem (i.e., linear system,pth order cost functional) for eachp. In this paper, we define the LP-optimal control problem and completely solve it for the scalar case. For the multidimensional case, we get some results, but the general solution of this problem seems to be impossible. So, we consider thepth mean stabilization problem more intensively and give a sufficient condition for the existence of apth mean stabilizing control by using the contraction mapping method in a Hilbert space. Some examples are also given.This research was conducted while the author was a visitor at the Forschungsschwerpunkt Dynamische Systeme, Universität Bremen, Bremen, West Germany. The author is grateful to Professor L. Arnold for providing interesting seminars and excellent working conditions during his stay. The financial assistance given by the Alexander von Humboldt Foundation during the author's stay is also gratefully acknowledged.     

A maximum principle for optimal control problem of fully coupled forward-backward stochastic systems with partial information

The paper is concerned with a stochastic optimal control problem in which the controlled system is described by a fully coupled nonlinear forward-backward stochastic differential equation driven by a Brownian motion. It is required that all admissible control processes are adapted to a given subfiltration of the filtration generated by the underlying Brownian motion. For this type of partial information control, one sufficient (a verification theorem) and one necessary conditions of optimality are proved. The control domain need to be convex and the forward diffusion coefficient of the system can contain the control variable. This work was partially supported by Basic Research Program of China (Grant No. 2007CB814904), National Natural Science Foundation of China (Grant No. 10325101) and Natural Science Foundation of Zhejiang Province (Grant No. Y605478, Y606667)     

随机非线性物价模型的最优控制

将政府对价格系统的宏观调控作为外部控制力,建立受控的随机非线性物价模型;利用拟Hamilton系统随机平均法和随机动态规划原理的非线性随机控制策略对系统实施最优控制,控制目标是实现系统的稳定性变大;并通过对比控制前后的Lyapunov指教值说明了控制的有效性.     

Optimization model to start harvesting in stochastic aquaculture system

Establishment of cost‐effective management strategy of aquaculture is one of the most important issues in fishery science, which can be addressed with bio‐economic mathematical modeling. This paper deals with the aforementioned issue using a stochastic process model for aquacultured non‐renewable fishery resources from the viewpoint of an optimal stopping (timing) problem. The goal of operating the model is to find the optimal criteria to start harvesting the resources under stochastic environment, which turns out to be determined from the Bellman equation (BE). The BE has a separation of variables type structure and can be simplified to a reduced BE with a fewer degrees of freedom. Dependence of solutions to the original and reduced BEs on parameters and independent variables is analyzed from both analytical and numerical standpoints. Implications of the analysis results to management of aquaculture systems are presented as well. Numerical simulation focusing on aquacultured Plecoglossus altivelis in Japan validates the mathematical analysis results. Copyright © 2017 John Wiley & Sons, Ltd.     

Risk-sensitive and risk-neutral control for continuous-time hidden Markov models

In this paper the optimal control of a continuous-time hidden Markov model is discussed. The risk-sensitive problem involves a cost function which has an exponential form and a risk parameter, and is solved by defining an appropriate information state and dynamic programming. As the risk parameter tends to zero, the classical risk-neutral optimal control problem is recovered. The limits are proved using viscosity solution methods.The first author wishes to acknowledge the funding of the activities of the Cooperative Research Centre for Robust and Adaptive Systems by the Australian Commonwealth Government under the Cooperative Research Centers Program. The support of NSERC Grant A7964 is acknowledged by the second author, as is the hospitality of the Department of Systems Engineering and the Cooperative Research Centre for Robust and Adaptive Systems, Australian National University, in July 1993.     

一类状态受限的随机延迟最优控制问题的最大值原理

本文考虑一类状态受限的随机延迟最优控制问题,其中控制域为凸集且扩散项系数中含有控制变量.控制域可以是无界集合.用最大值原理方法建立了最优控制满足的必要条件.也给出了充分最优性条件,从而有助于找到最优控制.     

Computation of optimal controls for a nonlinear stochastic third-order system

This paper deals with the computation of optimal feedback control laws for a nonlinear stochastic third-order system in which the nonlinear element is not completely specified. It is shown that, due to the structure of the system, the optimal feedback control law, whenever it exists, is not unique. Also, it is shown that, in order to implement an optimal feedback control law, a nonlinear partial differential equation has to be solved. A finite-difference algorithm for the solution of this equation is suggested, and its efficiency and applicability are demonstrated with examples.