A HJB-POD feedback synthesis approach for the wave equation

We propose a computational approach for the solution of an optimal control problem governed by the wave equation. We aim at obtaining approximate feedback laws by means of the application of the dynamic programming principle. Since this methodology is only applicable for low-dimensional dynamical systems, we first introduce a reduced-order model for the wave equation by means of Proper Orthogonal Decomposition. The coupling between the reduced-order model and the related dynamic programming equation allows to obtain the desired approximation of the feedback law. We discuss numerical aspects of the feedback synthesis and providenumerical tests illustrating this approach.     

Randomized filtering and Bellman equation in Wasserstein space for partial observation control problem

We study a stochastic optimal control problem for a partially observed diffusion. By using the control randomization method in Bandini et al. (2018), we prove a corresponding randomized dynamic programming principle (DPP) for the value function, which is obtained from a flow property of an associated filter process. This DPP is the key step towards our main result: a characterization of the value function of the partial observation control problem as the unique viscosity solution to the corresponding dynamic programming Hamilton–Jacobi–Bellman (HJB) equation. The latter is formulated as a new, fully non linear partial differential equation on the Wasserstein space of probability measures. An important feature of our approach is that it does not require any non-degeneracy condition on the diffusion coefficient, and no condition is imposed to guarantee existence of a density for the filter process solution to the controlled Zakai equation. Finally, we give an explicit solution to our HJB equation in the case of a partially observed non Gaussian linear–quadratic model.     

Portfolio optimization in a Lévy market with intertemporal substitution and transaction costs

We investigate an infinite horizon investment-consumption model in which a single agent consumes and distributes her wealth between a risk-free asset (bank account) and several risky assets (stocks) whose prices are governed by Lévy (jump-diffusion) processes. We suppose that transactions between the assets incur a transaction cost proportional to the size of the transaction. The problem is to maximize the total utility of consumption under Hindy-Huang-Kreps intertemporal preferences. This portfolio optimisation problem is formulated as a singular stochastic control problem and is solved using dynamic programming and the theory of viscosity solutions. The associated dynamic programming equation is a second order degenerate elliptic integro-differential variational inequality subject to a state constraint boundary condition. The main result is a characterization of the value function as the unique constrained viscosity solution of the dynamic programming equation. Emphasis is put on providing a framework that allows for a general class of Lévy processes. Owing to the complexity of our investment-consumption model, it is not possible to derive closed form solutions for the value function. Hence, the optimal policies cannot be obtained in closed form from the first order conditions for the dynamic programming equation. Therefore, we have to resort to numerical methods for computing the value function as well as the associated optimal policies. In view of the viscosity solution theory, the analysis found in this paper will ensure the convergence of a large class of numerical methods for the investment-consumption model in question.     

Smooth Solutions to Optimal Investment Models with Stochastic Volatilities and Portfolio Constraints

Abstract. This paper deals with an extension of Merton's optimal investment problem to a multidimensional model with stochastic volatility and portfolio constraints. The classical dynamic programming approach leads to a characterization of the value function as a viscosity solution of the highly nonlinear associated Bellman equation. A logarithmic transformation expresses the value function in terms of the solution to a semilinear parabolic equation with quadratic growth on the derivative term. Using a stochastic control representation and some approximations, we prove the existence of a smooth solution to this semilinear equation. An optimal portfolio is shown to exist, and is expressed in terms of the classical solution to this semilinear equation. This reduction is useful for studying numerical schemes for both the value function and the optimal portfolio. We illustrate our results with several examples of stochastic volatility models popular in the financial literature.     

Optimal stopping of switching diffusions with state dependent switching rates

This paper is concerned with a continuous-time and infinite-horizon optimal stopping problem in switching diffusion models. In contrast to the assumption commonly made in the literature that the regime-switching is modeled by an independent Markov chain, we consider in this paper the case of state-dependent regime-switching. The Hamilton–Jacobi–Bellman (HJB) equation associated with the optimal stopping problem is given by a system of coupled variational inequalities. By means of the dynamic programming (DP) principle, we prove that the value function is the unique viscosity solution of the HJB system. As an interesting application in mathematical finance, we examine the problem of pricing perpetual American put options with state-dependent regime-switching. A numerical procedure is developed based on the DP approach and an efficient discrete tree approximation of the continuous asset price process. Numerical results are reported.     

Smooth Solutions to Optimal Investment Models with Stochastic Volatilities and Portfolio Constraints

   Abstract. This paper deals with an extension of Merton's optimal investment problem to a multidimensional model with stochastic volatility and portfolio constraints. The classical dynamic programming approach leads to a characterization of the value function as a viscosity solution of the highly nonlinear associated Bellman equation. A logarithmic transformation expresses the value function in terms of the solution to a semilinear parabolic equation with quadratic growth on the derivative term. Using a stochastic control representation and some approximations, we prove the existence of a smooth solution to this semilinear equation. An optimal portfolio is shown to exist, and is expressed in terms of the classical solution to this semilinear equation. This reduction is useful for studying numerical schemes for both the value function and the optimal portfolio. We illustrate our results with several examples of stochastic volatility models popular in the financial literature.     

Multi-period portfolio optimization for asset–liability management with bankrupt control

In this paper, we investigate a multi-period portfolio optimization problem for asset–liability management of an investor who intends to control the probability of bankruptcy before reaching the end of an investment horizon. We formulate the problem as a generalized mean–variance model that incorporates bankrupt control over intermediate periods. Based on the Lagrangian multiplier method, the embedding technique, the dynamic programming approach and the Lagrangian duality theory, we propose a method to solve the model. A numerical example is given to demonstrate our method and show the impact of bankrupt control and market parameters on the optimal portfolio strategy.     

Utility-based indifference pricing in regime-switching models

In this paper, we study utility-based indifference pricing and hedging of a contingent claim in a continuous-time, Markov, regime-switching model. The market in this model is incomplete, so there is more than one price kernel. We specify the parametric form of price kernels so that both market risk and economic risk are taken into account. The pricing and hedging problem is formulated as a stochastic optimal control problem and is discussed using the dynamic programming approach. A verification theorem for the Hamilton-Jacobi-Bellman (HJB) solution to the problem is given. An issuer’s price kernel is obtained from a solution of a system of linear programming problems and an optimal hedged portfolio is determined.     

Existence of Risk-Sensitive Optimal Stationary Policies for Controlled Markov Processes

In this paper we are concerned with the existence of optimal stationary policies for infinite-horizon risk-sensitive Markov control processes with denumerable state space, unbounded cost function, and long-run average cost. Introducing a discounted cost dynamic game, we prove that its value function satisfies an Isaacs equation, and its relationship with the risk-sensitive control problem is studied. Using the vanishing discount approach, we prove that the risk-sensitive dynamic programming inequality holds, and derive an optimal stationary policy. Accepted 1 October 1997     

Viscosity Characterization of the Value Function of an Investment-Consumption Problem in Presence of an Illiquid Asset

We study a problem of optimal investment/consumption over an infinite horizon in a market consisting of a liquid and an illiquid asset. The liquid asset is observed and can be traded continuously, while the illiquid one can only be traded and observed at discrete random times corresponding to the jumps of a Poisson process. The problem is a nonstandard mixed discrete/continuous optimal control problem, which we face by the dynamic programming approach. The main goal of the paper is the characterization of the value function as unique viscosity solution of an associated Hamilton–Jacobi–Bellman equation. We then use such a result to build a numerical algorithm, allowing one to approximate the value function and so to measure the cost of illiquidity.     

Multidimensional Dual Dynamic Programming

In this paper, we develop a dual approach to the dynamic programming for the optimal control problem in a multidimensional case. The idea of our method consists in defining, instead of the value function, a new function which satisfies a dual first-order partial differential equation of dynamic programming. We then prove a suitable verification theorem and introduce the concept of a dual feedback control. The sufficient optimality conditions thus obtained are analogous to their one-dimensional counterparts.     

单输入多时滞离散系统的线性二次调节问题

研究了单输入多时滞的离散时间系统的线性二次调节问题(LQR问题),给出了求解最优控制输入序列的一种简单有效而又新颖的方法.将该动态的离散时滞系统的LQR最优控制问题最终转化成了一个静态的、不带时滞的数学规划模型——带等式线性约束的严格凸二次规划问题,并利用两种方法解这个二次规划问题,均成功地导出了系统的最优控制输入序列.仿真结果验证了我们的方法的正确有效性.     

Maximum principle for optimal distributed control of viscous weakly dispersive Degasperis–Procesi equation

This paper is concerned with the optimal distributed control of the viscous weakly dispersive Degasperis–Procesi equation in nonlinear shallow water dynamics. It is well known that the Pontryagin maximum principle, which unifies calculus of variations and control theory of ordinary differential equations, sets up the theoretical basis of the modern optimal control theory along with the Bellman dynamic programming principle. In this paper, we commit ourselves to infinite dimensional generalizations of the maximum principle and aim at the optimal control theory of partial differential equations. In contrast to the finite dimensional setting, the maximum principle for the infinite dimensional system does not generally hold as a necessary condition for optimal control. By the Dubovitskii and Milyutin functional analytical approach, we prove the Pontryagin maximum principle of the controlled viscous weakly dispersive Degasperis–Procesi equation. The necessary optimality condition is established for the problem in fixed final horizon case. Finally, a remark on how to utilize the obtained results is also made. Copyright © 2015 John Wiley & Sons, Ltd.     

动态投入产出最优控制模型

本文建立了一个新的具有上下限约束的投入产出问题的最优控制模型 ,并把最优控制问题转化为动态规划问题 ,利用动态最优化的方法给出了该问题的求解方法     

Properties of ε-value functions for a two person differential game

We consider the problem of approximate minimax for the Bolza problem of optimal control. Starting from the method of dynamic programming (Bellman) we define the ?-value function to be the approximation for the value function being a solution to the Hamilton–Jacobi equation.     

A stochastic control model of investment,production, and consumption on a finite horizon

In this paper, we consider a stochastic control problem on a finite time horizon. The unit price of capital obeys a logarithmic Brownian motion, and the income from production is also subject to the random Brownian fluctuations. The goal is to choose optimal investment and consumption policies to maximize the finite horizon expected discounted hyperbolic absolute risk aversion utility of consumption. A dynamic programming principle is used to derive a time‐dependent Hamilton–Jacobi–Bellman equation. The Leray–Schauder fixed point theorem is used to obtain existence of solution of the HJB equation. At last, we derive the optimal investment and consumption policies by the verification theorem. The main contribution in this paper is the use of PDE technique to the finite time problem for obtaining optimal polices. Copyright © 2014 John Wiley & Sons, Ltd.     

Optimal selection of observation times in the linear-quadratic Gaussian control problem

Feedback control is sometimes applied to a dynamic system operating in a stochastic environment under circumstances where only a limited number of observations can be made. The optimal positioning of ana priori fixed number of observations is considered. A direct approach to this problem yields a dynamic programming functional equation, while a second approach involving an ancillary observation cost leads to a rapid and practical numerical solution.     

An optimistic decision-making in fuzzy environment

Bellman and Zadeh have originated three systems of multistage decision processes in a fuzzy environment: deterministic, stochastic and fuzzy systems. In this article, we consider an optimization problem with an optimistic criterion on a fuzzy system. By making use of minimization–maximization expectation in a fuzzy environment, we derive a recursive equation for the fuzzy decision process through invariant imbedding approach. By illustrating a three-state, two-decision and two-stage model, we give an optimal solution through dynamic programming. The optimal solution is also verified by the method of multistage fuzzy decision tree-table.     

Optimal investment and reinsurance for an insurer under Markov-modulated financial market

This study examines optimal investment and reinsurance policies for an insurer with the classical surplus process. It assumes that the financial market is driven by a drifted Brownian motion with coefficients modulated by an external Markov process specified by the solution to a stochastic differential equation. The goal of the insurer is to maximize the expected terminal utility. This paper derives the Hamilton–Jacobi–Bellman (HJB) equation associated with the control problem using a dynamic programming method. When the insurer admits an exponential utility function, we prove that there exists a unique and smooth solution to the HJB equation. We derive the explicit optimal investment policy by solving the HJB equation. We can also find that the optimal reinsurance policy optimizes a deterministic function. We also obtain the upper bound for ruin probability in finite time for the insurer when the insurer adopts optimal policies.     

A Deterministic Approach To Stochastic Optimal Control With Application To Anticipative Control

Using the decomposition of solution of SDE, we consider the stochastic optimal control problem with anticipative controls as a family of deterministic control problems parametrized by the paths of the driving Wiener process and of a newly introduced Lagrange multiplier stochastic process (nonanticipativity equality constraint). It is shown that the value function of these problems is the unique global solution of a robust equation (random partial differential equation) associated to a linear backward Hamilton-Jacobi-Bellman stochastic partial differential equation (HJB SPDE). This appears as limiting SPDE for a sequence of random HJB PDE's when linear interpolation approximation of the Wiener process is used. Our approach extends the Wong-Zakai type results [20] from SDE to the stochastic dynamic programming equation by showing how this arises as average of the limit of a sequence of deterministic dynamic programming equations. The stochastic characteristics method of Kunita [13] is used to represent the value function. By choosing the Lagrange multiplier equal to its nonanticipative constraint value the usual stochastic (nonanticipative) optimal control and optimal cost are recovered. This suggests a method for solving the anticipative control problems by almost sure deterministic optimal control. We obtain a PDE for the “cost of perfect information” the difference between the cost function of the nonanticipative control problem and the cost of the anticipative problem which satisfies a nonlinear backward HJB SPDE. Poisson bracket conditions are found ensuring this has a global solution. The cost of perfect information is shown to be zero when a Lagrangian submanifold is invariant for the stochastic characteristics. The LQG problem and a nonlinear anticipative control problem are considered as examples in this framework