Stochastic homogenization of Hamilton‐Jacobi‐Bellman equations

We study the homogenization of some Hamilton‐Jacobi‐Bellman equations with a vanishing second‐order term in a stationary ergodic random medium under the hyperbolic scaling of time and space. Imposing certain convexity, growth, and regularity assumptions on the Hamiltonian, we show the locally uniform convergence of solutions of such equations to the solution of a deterministic “effective” first‐order Hamilton‐Jacobi equation. The effective Hamiltonian is obtained from the original stochastic Hamiltonian by a minimax formula. Our homogenization results have a large‐deviations interpretation for a diffusion in a random environment. © 2005 Wiley Periodicals, Inc.     

Bellman equations associated to the optimal feedback control of stochastic Navier‐Stokes equations

In this paper we study infinite‐dimensional, second‐order Hamilton‐Jacobi‐Bell‐man equations associated to the feedback synthesis of stochastic Navier‐Stokes equations forced by space‐time white noise. Uniqueness and existence of viscosity solutions are proven for these infinite‐dimensional partial differential equations. © 2005 Wiley Periodicals, Inc.     

Controllability Properties of Linear Mean-Field Stochastic Systems

We study some controllability properties for linear stochastic systems of mean-field type. First, we give necessary and sufficient criteria for exact terminal-controllability. Second, we characterize the approximate and approximate null-controllability via duality techniques. Using Riccati equations associated to linear quadratic problems in the control of mean-field systems, we provide a (conditional) viability criterion for approximate null-controllability. In the classical diffusion framework, approximate and approximate null-controllability are equivalent. This is no longer the case for mean-field systems. We provide sufficient (algebraic) invariance conditions implying approximate null-controllability. We also present a general class of systems for which our criterion is equivalent to approximate null-controllability property. We also introduce some rank conditions under which approximate and approximate null-controllability are equivalent. Several examples and counter-examples as well as a partial algorithm are provided.     

Optimal control of diffusion processes and hamilton–jacobi–bellman equations part 2 : viscosity solutions and uniqueness

We consider general optimal stochastic control problems and the associated Hamilton–Jacobi–Bellman equations. We develop a general notion of week solutions – called viscosity solutions – of the amilton–Jocobi–Bellman equations that is stable and we show that the optimal cost functions of the control problems are always solutions in that sense of the Hamilton–Jacobi–Bellman equations. We then prove general uniqueness results for viscosity solutions of the Hamilton–Jacobi–Bellman equations.     

Sprays metrizable by Finsler functions of constant flag curvature

In this paper we characterize sprays that are metrizable by Finsler functions of constant flag curvature. By solving a particular case of the Finsler metrizability problem, we provide the necessary and sufficient conditions that can be used to decide whether or not a given homogeneous system of second order ordinary differential equations represents the geodesic equations of a Finsler function of constant flag curvature. The conditions we provide are tensorial equations on the Jacobi endomorphism. We identify the class of homogeneous SODE where the Finsler metrizability is equivalent with the metrizability by a Finsler function of constant flag curvature.     

A deterministic‐control‐based approach to fully nonlinear parabolic and elliptic equations

We show that a broad class of fully nonlinear, second‐order parabolic or elliptic PDEs can be realized as the Hamilton‐Jacobi‐Bellman equations of deterministic two‐person games. More precisely: given the PDE, we identify a deterministic, discrete‐time, two‐person game whose value function converges in the continuous‐time limit to the viscosity solution of the desired equation. Our game is, roughly speaking, a deterministic analogue of the stochastic representation recently introduced by Cheridito, Soner, Touzi, and Victoir. In the parabolic setting with no u‐dependence, it amounts to a semidiscrete numerical scheme whose timestep is a min‐max. Our result is interesting, because the usual control‐based interpretations of second‐order PDEs involve stochastic rather than deterministic control. © 2009 Wiley Periodicals, Inc.     

Some results on the large-time behavior of weakly coupled systems of first-order Hamilton–Jacobi equations

Systems of Hamilton–Jacobi equations arise naturally when we study optimal control problems with pathwise deterministic trajectories with random switching. In this work, we are interested in the large-time behavior of weakly coupled systems of first-order Hamilton–Jacobi equations in the periodic setting. First results have been obtained by Camilli et al. (NoDEA Nonlinear Diff Eq Appl, 2012) and Mitake and Tran (Asymptot Anal, 2012) under quite strict conditions. Here, we use a PDE approach to extend the convergence result proved by Barles and Souganidis (SIAM J Math Anal 31(4):925–939 (electronic), 2000) in the scalar case. This result permits us to treat general cases, for instance, systems of nonconvex Hamiltonians and systems of strictly convex Hamiltonians. We also obtain some other convergence results under different assumptions. These results give a clearer view on the large-time behavior for systems of Hamilton–Jacobi equations.     

Multi-Target Control Problems

We study control problems with several targets in the case of nonlinear dynamic systems. The map associating with every initial condition the minimal time to reach successively two given targets is characterized in the framework of differential inclusions through the notion of viability kernel. This approach allows one to treat the problem without assumptions of regularity and to build numerical schemes computing the minimal time. We also study the problem where an order of visit of the targets is required. The statements are also extended to the case of p targets under state constraints. Equivalent formulations in terms of Hamilton–Jacobi equations are also provided.     

Stochastic Optimal Control for the Stochastic Heat Equation with Exponentially Growing Coefficients and with Control and Noise on a Subdomain

Abstract

We consider stochastic optimal control problems in Banach spaces, related to nonlinear controlled equations with dissipative non linearities: on the nonlinear term we do not impose any growth condition. The problems are treated via the backward stochastic differential equations approach, that allows also to solve in mild sense Hamilton Jacobi Bellman equations in Banach spaces. We apply the results to controlled stochastic heat equation, in space dimension 1, with control and noise acting on a subdomain.     

Stability of a class of stochastic distributed parameter systems with random boundary conditions

In this article we consider the question of stability of a class of stochastic systems governed by elliptic and parabolic second order partial differential equations with Neumann boundary conditions. Results on the “stability in the mean” are given in Theorems 1 and 2, and those on “almost sure stability” are presented in Theorems 3 and 4. These results are proved under the assumption that the perturbing forces are measurable stochastic processes defined on $\text{I × Ω}$. In Theorem 5 it is shown that the proofs require only minor modification to admit progressively measurable (predictable or optional) processes.     

Scale of Viability and Minimal Time of Crisis

In this paper, we introduce and study the minimal time of a crisis map which measures the minimal time spent outside a given closed domain of constraints by trajectory solutions of a differential inclusion. The interest of such a notion is basically to tackle simultaneously viability and target issues. The main mathematical result characterizes the epigraph of the crisis map in terms of a viability kernel of an augmented problem. This allows the obtaining of the numerical schemes we specify and to derive an equivalent Hamilton–Jacobi formulation. A simple economic example illustrates the results.     

Bilateral Fixed-Points and Algebraic Properties of Viability Kernels and Capture Basins of Sets

Many concepts of viability theory such as viability or invariance kernels and capture or absorption basins under discrete multivalued systems, differential inclusions and dynamical games share algebraic properties that provide simple – yet powerful – characterizations as either largest or smallest fixed points or unique minimax (or bilateral fixed-point) of adequate maps defined on pairs of subsets. Further, important algorithms such as the Saint-Pierre viability kernel algorithm for computing viability kernels under discrete system and the Cardaliaguet algorithm for characterizing discriminating kernels under dynamical games are algebraic in nature. The Matheron Theorem as well as the Galois transform find applications in the field of control and dynamical games allowing us to clarify concepts and simplify proofs.     

Stochastic Differential Games with Multiple Modes and Applications to Portfolio Optimization

Abstract

We study a zero-sum stochastic differential game with multiple modes. The state of the system is governed by “controlled switching” diffusion processes. Under certain conditions, we show that the value functions of this game are unique viscosity solutions of the appropriate Hamilton–Jacobi–Isaac' system of equations. We apply our results to the analysis of a portfolio optimization problem where the investor is playing against the market and wishes to maximize his terminal utility. We show that the maximum terminal utility functions are unique viscosity solutions of the corresponding Hamilton–Jacobi–Isaac' system of equations.     

Deep Learning for Constrained Utility Maximisation

This paper proposes two algorithms for solving stochastic control problems with deep learning, with a focus on the utility maximisation problem. The first algorithm solves Markovian problems via the Hamilton Jacobi Bellman (HJB) equation. We solve this highly nonlinear partial differential equation (PDE) with a second order backward stochastic differential equation (2BSDE) formulation. The convex structure of the problem allows us to describe a dual problem that can either verify the original primal approach or bypass some of the complexity. The second algorithm utilises the full power of the duality method to solve non-Markovian problems, which are often beyond the scope of stochastic control solvers in the existing literature. We solve an adjoint BSDE that satisfies the dual optimality conditions. We apply these algorithms to problems with power, log and non-HARA utilities in the Black-Scholes, the Heston stochastic volatility, and path dependent volatility models. Numerical experiments show highly accurate results with low computational cost, supporting our proposed algorithms.

     

Viability with Probabilistic Knowledge of Initial Condition, Application to Optimal Control

In this paper we provide an extension of the Viability and Invariance Theorems in the Wasserstein metric space of probability measures with finite quadratic moments in ? ^d for controlled systems of which the dynamic is bounded and Lipschitz. Then we characterize the viability and invariance kernels as the largest viability (resp. invariance) domains. As application of our result we consider an optimal control problem of Mayer type with lower semicontinuous cost function for the same controlled system with uncertainty on the initial state modeled by a probability measure. Following Frankowska, we prove using the epigraphical viability approach that the value function is the unique lower semicontinuous proximal episolution of a suitable Hamilton Jacobi equation.     

复杂随机系统的生存性问题研究

为了刻画复杂随机系统的理性决策,提出了复杂随机系统的生存性及不变性的概念,给出并证明了复杂随机系统的生存性定理及不变性定理.并提出了均方相依锥,生存域与不变域的概念.得到了与文献中的一致的结论.     

Nonlinear elliptic systems and mean-field games

We consider a class of quasilinear elliptic systems of PDEs consisting of N Hamilton–Jacobi–Bellman equations coupled with N divergence form equations, generalising to N > 1 populations the PDEs for stationary Mean-Field Games first proposed by Lasry and Lions. We provide a wide range of sufficient conditions for the existence of solutions to these systems: either the Hamiltonians are required to behave at most linearly for large gradients, as it occurs when the controls of the agents are bounded, or they must grow faster than linearly and not oscillate too much in the space variables, in a suitable sense. We show the connection of these systems with the classical strongly coupled systems of Hamilton–Jacobi–Bellman equations of the theory of N-person stochastic differential games studied by Bensoussan and Frehse. We also prove the existence of Nash equilibria in feedback form for some N-person games.     

Cosmically Lipschitz Set-Valued Mappings

This paper introduces a kind of sub-Lipschitz continuity for set-valued mappings based on the cosmic metric. This type of Lipschitz behavior has applications with regards to necessary optimality conditions, the Hamilton–Jacobi equation, and invariance of unbounded differential inclusions. Cosmically Lipschitz assumptions allow for broader applications than previously allowed under Lipschitz assumptions. It is also shown that a cosmically Lipschitz mapping can be characterized by the normal cones to its graph using the coderivative, and various rules are presented in order to more easily identify such a mapping.     

HJB equations in infinite dimensions with locally Lipschitz Hamiltonian and unbounded terminal condition

We study Hamilton Jacobi Bellman equations in an infinite dimensional Hilbert space, with Lipschitz coefficients, where the Hamiltonian has superquadratic growth with respect to the derivative of the value function, and the final condition is not bounded. This allows to study stochastic optimal control problems for suitable controlled state equations with unbounded control processes. The results are applied to a controlled wave equation.     

Weak KAM theorem for Hamilton‐Jacobi equations with Neumann boundary conditions on noncompact manifolds

In this paper, we consider Hamilton–Jacobi equations with homogeneous Neumann boundary condition. We establish some results on noncompact manifold with homogeneous Neumann boundary conditions in view of weak Kolmogorov‐Arnold‐Moser (KAM) theory, which is a generalization of the results obtained by Fathi under the non‐bounded condition. Copyright © 2014 John Wiley & Sons, Ltd.