Optimal control of diffusion processes and hamilton–jacobi–bellman equations part 2 : viscosity solutions and uniqueness

We consider general optimal stochastic control problems and the associated Hamilton–Jacobi–Bellman equations. We develop a general notion of week solutions – called viscosity solutions – of the amilton–Jocobi–Bellman equations that is stable and we show that the optimal cost functions of the control problems are always solutions in that sense of the Hamilton–Jacobi–Bellman equations. We then prove general uniqueness results for viscosity solutions of the Hamilton–Jacobi–Bellman equations.     

Large deviations for stochastic PDE with Lévy noise

We prove a large deviation principle result for solutions of abstract stochastic evolution equations perturbed by small Lévy noise. We use general large deviations theorems of Varadhan and Bryc coupled with the techniques of Feng and Kurtz (2006) [15], viscosity solutions of integro-partial differential equations in Hilbert spaces, and deterministic optimal control methods. The Laplace limit is identified as a viscosity solution of a Hamilton-Jacobi-Bellman equation of an associated control problem. We also establish exponential moment estimates for solutions of stochastic evolution equations driven by Lévy noise. General results are applied to stochastic hyperbolic equations perturbed by subordinated Wiener process.     

Lyapunov Stabilizability of Controlled Diffusions via a Superoptimality Principle for Viscosity Solutions

We prove optimality principles for semicontinuous bounded viscosity solutions of Hamilton-Jacobi-Bellman equations. In particular, we provide a representation formula for viscosity supersolutions as value functions of suitable obstacle control problems. This result is applied to extend the Lyapunov direct method for stability to controlled Ito stochastic differential equations. We define the appropriate concept of the Lyapunov function to study stochastic open loop stabilizability in probability and local and global asymptotic stabilizability (or asymptotic controllability). Finally, we illustrate the theory with some examples.     

On the Hamilton-Jacobi-Bellman equations

We consider general problems of optimal stochastic control and the associated Hamilton-Jacobi-Bellman equations. We recall first the usual derivation of the Hamilton-Jacobi-Bellman equations from the Dynamic Programming Principle. We then show and explain various results, including (i) continuity results for the optimal cost function, (ii) characterizations of the optimal cost function as the maximum subsolution, (iii) regularity results, and (iv) uniqueness results. We also develop the recent notion of viscosity solutions of Hamilton-Jacobi-Bellman equations.     

On Aronsson Equation and Deterministic Optimal Control

When Hamiltonians are nonsmooth, we define viscosity solutions of the Aronsson equation and prove that value functions of the corresponding deterministic optimal control problems are solutions if they are bilateral viscosity solutions of the Hamilton-Jacobi-Bellman equation. We characterize such a property in several ways, in particular it follows that a value function which is an absolute minimizer is a bilateral viscosity solution of the HJB equation and these two properties are often equivalent. We also determine that bilateral solutions of HJB equations are unique among absolute minimizers with prescribed boundary conditions. This research was partially supported by MIUR-Prin project “Metodi di viscosità, metrici e di teoria del controllo in equazioni alle derivate parziali nonlineari”.     

Integro-PDE in Hilbert Spaces: Existence of Viscosity Solutions

Existence of a viscosity solution to a non-local Hamilton-Jacobi-Bellman equation in a Hilbert space is established. We prove that the value function of an associated stochastic control problem is a viscosity solution. We provide a complete proof of the Dynamic Programming Principle for the stochastic control problem. We also illustrate the theory with Bellman equations associated to a controlled wave equation and controlled Musiela equation of mathematical finance both perturbed by Lévy processes.     

Dynamic programming in stochastic control of systems with delay

We consider optimal control problems for systems described by stochastic differential equations with delay (SDDE). We prove a version of Bellman's principle of optimality (the dynamic programming principle) for a general class of such problems. That the class in general means that both the dynamics and the cost depends on the past in a general way. As an application, we study systems where the value function depends on the past only through some weighted average. For such systems we obtain a Hamilton-Jacobi-Bellman partial differential equation that the value function must solve if it is smooth enough. The weak uniqueness of the SDDEs we consider is our main tool in proving the result. Notions of strong and weak uniqueness for SDDEs are introduced, and we prove that strong uniqueness implies weak uniqueness, just as for ordinary stochastic differential equations.     

Numerical solution of a minimax ergodic optimal control problem

In this work we consider an L^∞ minimax ergodic optimal control problem with cumulative cost. We approximate the cost function as a limit of evolutions problems. We present the associated Hamilton-Jacobi-Bellman equation and we prove that it has a unique solution in the viscosity sense. As this HJB equation is consistent with a numerical procedure, we use this discretization to obtain a procedure for the primitive problem. For the numerical solution of the ergodic version we need a perturbation of the instantaneous cost function. We give an appropriate selection of the discretization and penalization parameters to obtain discrete solutions that converge to the optimal cost. We present numerical results. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Distributed control for a class of non-Newtonian fluids

We consider control problems with a general cost functional where the state equations are the stationary, incompressible Navier-Stokes equations with shear-dependent viscosity. The equations are quasi-linear. The control function is given as the inhomogeneity of the momentum equation. In this paper, we study a general class of viscosity functions which correspond to shear-thinning or shear-thickening behavior. The basic results concerning existence, uniqueness, boundedness, and regularity of the solutions of the state equations are reviewed. The main topic of the paper is the proof of Gâteaux differentiability, which extends known results. It is shown that the derivative is the unique solution to a linearized equation. Moreover, necessary first-order optimality conditions are stated, and the existence of a solution of a class of control problems is shown.     

Stability estimates for the solutions of control problems for the stationary magnetohydrodynamic equations

We study control problems for the stationary magnetohydrodynamic equations. In these problems, one has to find an unknown vector function occurring in the boundary condition for the magnetic field and the solution of the boundary value problem in question by minimizing a performance functional depending on the velocity and pressure. We derive new a priori estimates for the solutions of the original boundary value problem and the extremal problem and prove theorems on the local uniqueness and stability of solutions for specific performance functionals.     

Optimality principles and uniqueness for Bellman equations of unbounded control problems with discontinuous running cost

We study viscosity solutions of Hamilton-Jacobi equations that arise in optimal control problems with unbounded controls and discontinuous Lagrangian. In our assumptions, the comparison principle will not hold, in general. We prove optimality principles that extend the scope of the results of [23] under very general assumptions, allowing unbounded controls. In particular, our results apply to calculus of variations problems under Tonelli type coercivity conditions. Optimality principles can be applied to obtain necessary and sufficient conditions for uniqueness in boundary value problems, and to characterize minimal and maximal solutions when uniqueness fails. We give examples of applications of our results in this direction.     

Representation Formulas for Solutions of the HJI Equations with Discontinuous Coefficients and Existence of Value in Differential Games

In this paper, we study the Hamilton-Jacobi-Isaacs equation of zerosum differential games with discontinuous running cost. For such class of equations, the uniqueness of the solutions is not guaranteed in general. We prove principles of optimality for viscosity solutions where one of the players can play either causal strategies or only a subset of continuous strategies. This allows us to obtain nonstandard representation formulas for the minimal and maximal viscosity solutions and prove that a weak form of the existence of value is always satisfied. We state also an explicit uniqueness result for the HJI equations for piecewise continuous coefficients, in which case the usual statement on the existence of value holds.     

Generalized Directional Gradients, Backward Stochastic Differential Equations and Mild Solutions of Semilinear Parabolic Equations

We study a forward-backward system of stochastic differential equations in an infinite-dimensional framework and its relationships with a semilinear parabolic differential equation on a Hilbert space, in the spirit of the approach of Pardoux-Peng. We prove that the stochastic system allows us to construct a unique solution of the parabolic equation in a suitable class of locally Lipschitz real functions. The parabolic equation is understood in a mild sense which requires the notion of a generalized directional gradient, that we introduce by a probabilistic approach and prove to exist for locally Lipschitz functions. The use of the generalized directional gradient allows us to cover various applications to option pricing problems and to optimal stochastic control problems (including control of delay equations and reaction--diffusion equations), where the lack of differentiability of the coefficients precludes differentiability of solutions to the associated parabolic equations of Black--Scholes or Hamilton-Jacobi-Bellman type.     

The optimal execution strategy of employee stock options

In this paper, we develop an optimal execution strategy for employee stock options by means of the fluid model, in which a voluntary turnover is considered. We show that the value function is the viscosity solution of the Hamilton-Jacobi-Bellman variational inequality and prove the uniqueness of the viscosity solution. Finally, we present numerical illustrative examples and numerical solutions of optimal strategies which are computed by the finite difference method.     

Existence and uniqueness for a nonlinear parabolic/Hamilton–Jacobi coupled system describing the dynamics of dislocation densities

We study a mathematical model describing the dynamics of dislocation densities in crystals. This model is expressed as a 1D system of a parabolic equation and a first order Hamilton–Jacobi equation that are coupled together. We examine an associated Dirichlet boundary value problem. We prove the existence and uniqueness of a viscosity solution among those assuming a lower-bound on their gradient for all time including the initial time. Moreover, we show the existence of a viscosity solution when we have no such restriction on the initial data. We also state a result of existence and uniqueness of entropy solution for the initial value problem of the system obtained by spatial derivation. The uniqueness of this entropy solution holds in the class of bounded-from-below solutions. In order to prove our results on the bounded domain, we use an “extension and restriction” method, and we exploit a relation between scalar conservation laws and Hamilton–Jacobi equations, mainly to get our gradient estimates.     

Optimal control problems constrained by the stochastic Navier–Stokes equations with multiplicative Lévy noise

We consider the controlled stochastic Navier–Stokes equations in a bounded multidimensional domain, where the noise term allows jumps. In order to prove existence and uniqueness of an optimal control w.r.t. a given control problem, we first need to show the existence and uniqueness of a local mild solution of the considered controlled stochastic Navier–Stokes equations. We then discuss the control problem, where the related cost functional includes stopping times dependent on controls. Based on the continuity of the cost functional, we can apply existence and uniqueness results provided in [4], which enables us to show that a unique optimal control exists.     

Limit Theorem for Controlled Backward SDEs and Homogenization of Hamilton–Jacobi–Bellman Equations

We prove a convergence theorem for a family of value functions associated with stochastic control problems whose cost functions are defined by backward stochastic differential equations. The limit function is characterized as a viscosity solution to a fully nonlinear partial differential equation of second order. The key assumption we use in our approach is shown to be a necessary and sufficient assumption for the homogenizability of the control problem. The results generalize partially homogenization problems for Hamilton–Jacobi–Bellman equations treated recently by Alvarez and Bardi by viscosity solution methods. In contrast to their approach, we use mainly probabilistic arguments, and discuss a stochastic control interpretation for the limit equation.     

On the L2 improperly posed problems

This paper considers a problem of approximation of functions proposed by Bellman [1]. The results include a representation lemma for the solutions and an algorithm for computing such solutions. A sufficient condition for the convergence of the algorithm to the optimal solutions is shown to be related to the uniqueness of solutions of a pair of functional equations. To the author's knowledge, not much is known about these functional equations and these may prove to be of interest in future research. An example is included to illustrate the algorithm.     

Ergodic BSDEs under weak dissipative assumptions

In this paper we study ergodic backward stochastic differential equations (EBSDEs) dropping the strong dissipativity assumption needed in Fuhrman et al. (2009) [12]. In other words we do not need to require the uniform exponential decay of the difference of two solutions of the underlying forward equation, which, on the contrary, is assumed to be non-degenerate.We show the existence of solutions by the use of coupling estimates for a non-degenerate forward stochastic differential equation with bounded measurable nonlinearity. Moreover we prove the uniqueness of “Markovian” solutions by exploiting the recurrence of the same class of forward equations.Applications are then given for the optimal ergodic control of stochastic partial differential equations and to the associated ergodic Hamilton-Jacobi-Bellman equations.     

Optimal portfolio and consumption selection with default risk

We investigate an optimal portfolio and consumption choice problem with a defaultable security. Under the goal of maximizing the expected discounted utility of the average past consumption, a dynamic programming principle is applied to derive a pair of second-order parabolic Hamilton-Jacobi-Bellman (HJB) equations with gradient constraints. We explore these HJB equations by a viscosity solution approach and characterize the post-default and pre-default value functions as a unique pair of constrained viscosity solutions to the HJB equations.