Some Applications of Linear Programming Formulations in Stochastic Control

We present two applications of the linearization techniques in stochastic optimal control. In the first part, we show how the assumption of stability under concatenation for control processes can be dropped in the study of asymptotic stability domains. Generalizing Zubov??s method, the stability domain is then characterized as some level set of a semicontinuous generalized viscosity solution of the associated Hamilton?CJacobi?CBellman equation. In the second part, we extend our study to unbounded coefficients and apply the method to obtain a linear formulation for control problems whenever the state equation is a stochastic variational inequality.     

Mayer and optimal stopping stochastic control problems with discontinuous cost

We study two classes of stochastic control problems with semicontinuous cost: the Mayer problem and optimal stopping for controlled diffusions. The value functions are introduced via linear optimization problems on appropriate sets of probability measures. These sets of constraints are described deterministically with respect to the coefficient functions. Both the lower and upper semicontinuous cases are considered. The value function is shown to be a generalized viscosity solution of the associated HJB system, respectively, of some variational inequality. Dual formulations are given, as well as the relations between the primal and dual value functions. Under classical convexity assumptions, we prove the equivalence between the linearized Mayer problem and the standard weak control formulation. Counter-examples are given for the general framework.     

Computation of viability kernels: a case study of by-catch fisheries

Traditional means of studying environmental economics and management problems consist of optimal control and dynamic game models that are solved for optimal or equilibrium strategies. Notwithstanding the possibility of multiple equilibria, the models’ users—managers or planners—will usually be provided with a single optimal or equilibrium strategy no matter how reliable, or unreliable, the underlying models and their parameters are. In this paper we follow an alternative approach to policy making that is based on viability theory. It establishes “satisficing” (in the sense of Simon), or viable, policies that keep the dynamic system in a constraint set and are, generically, multiple and amenable to each manager’s own prioritisation. Moreover, they can depend on fewer parameters than the optimal or equilibrium strategies and hence be more robust. For the determination of these (viable) policies, computation of “viability kernels” is crucial. We introduce a MATLAB application, under the name of VIKAASA, which allows us to compute approximations to viability kernels. We discuss two algorithms implemented in VIKAASA. One approximates the viability kernel by the locus of state space positions for which solutions to an auxiliary cost-minimising optimal control problem can be found. The lack of any solution implies the infinite value function and indicates an evolution which leaves the constraint set in finite time, therefore defining the point from which the evolution originates as belonging to the kernel’s complement. The other algorithm accepts a point as viable if the system’s dynamics can be stabilised from this point. We comment on the pros and cons of each algorithm. We apply viability theory and the VIKAASA software to a problem of by-catch fisheries exploited by one or two fleets and provide rules concerning the proportion of fish biomass and the fishing effort that a sustainable fishery’s exploitation should follow.     

Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov’s Method

We aim at characterizing domains of attraction for controlled piecewise deterministic processes using an occupational measure formulation and Zubov??s approach. Firstly, we provide linear programming (primal and dual) formulations of discounted, infinite horizon control problems for PDMPs. These formulations involve an infinite-dimensional set of probability measures and are obtained using viscosity solutions theory. Secondly, these tools allow to construct stabilizing measures and to avoid the assumption of stability under concatenation for controls. The domain of controllability is then characterized as some level set of a convenient solution of the associated Hamilton-Jacobi integral-differential equation. The theoretical results are applied to PDMPs associated to stochastic gene networks. Explicit computations are given for Cook??s model for gene expression.     

Discontinuous differential games and control systems with supremum cost

This paper concerns with the existence of a value for a zero sum two-player differential game with supremum cost of the form C_t0,x0(u,v)=sup_τ[t0,T]h(x(τ;t₀,x_0,u,v)) under Isaacs' condition. We characterize the value function as the unique solution—in a suitable sense—to a PDE, namely the Hamilton–Jacobi–Isaacs equation. As a byproduct, we obtain a PDE characterization of the value function for control system.     

Viscosity solutions for partial differential equations with Neumann type boundary conditions and some aspects of Aubry-Mather theory

We study partial differential inequalities (PDI) of the type where NK(⋅) is the normal cone to the set K. We prove existence of a constant such that the PDI of Hamilton-Jacobi type has a unique (global) Lipschitz viscosity solution. We provide a formula to calculate this constant. Moreover, we define a subset of K such that any two solutions of the previous PDI which coincide on will coincide on K. Our paper generalizes results of the case without boundary conditions for convex Hamiltonians obtained by L.C. Evans and A. Fathi.     

Primal-Lower-Nice Property of Value Functions in Optimization and Control Problems

The paper studies value functions associated with optimization problems and with Mayer-type control problems. Using methods belonging to proximal analysis and control theory, we establish new results for the primal-lower-nice (pln) property of the value functions for these problems.     

Characterization of the optimal trajectories for the averaged dynamics associated to singularly perturbed control systems

The aim of this paper is to study singularly perturbed control systems. Firstly, we provide linearized formulation version for the calculus of the value function associated with the averaged dynamics. Secondly, we obtain necessary and sufficient conditions in order to identify the optimal trajectory of the averaged system.     

Optimality conditions for reflecting boundary control problems

We consider a control problem with reflecting boundary and obtain necessary optimality conditions in the form of the maximum Pontryagin principle. To derive these results we transform the constrained problem in an unconstrained one or we use penalization techniques of Morreau-Yosida type to approach the original problem by a sequence of optimal control problems with Lipschitz dynamics. Then nonsmooth analysis theory is used to study the convergence of the penalization in order to obtain optimality conditions.