Maximum principle for optimal control problems involving impulse controls with nonsmooth data

This paper is concerned with the stochastic maximum principle for impulse optimal control problems of forward–backward systems, where the coefficients of the forward part are Lipschitz continuous. The domain of the regular controls is not necessarily convex. We establish a Pontryagins maximum principle for this control problem by applying Ekelands variational principle to a sequence of approximated control problems with smooth coefficients of the initial problems.     

Optimal control of stochastic functional differential equations with a bounded memory

This paper treats a finite time horizon optimal control problem in which the controlled state dynamics are governed by a general system of stochastic functional differential equations with a bounded memory. An infinite dimensional Hamilton–Jacobi–Bellman (HJB) equation is derived using a Bellman-type dynamic programming principle. It is shown that the value function is the unique viscosity solution of the HJB equation.     

Taylor expansions of the value function associated with a bilinear optimal control problem

A general bilinear optimal control problem subject to an infinite-dimensional state equation is considered. Polynomial approximations of the associated value function are derived around the steady state by repeated formal differentiation of the Hamilton–Jacobi–Bellman equation. The terms of the approximations are described by multilinear forms, which can be obtained as solutions to generalized Lyapunov equations with recursively defined right-hand sides. They form the basis for defining a suboptimal feedback law. The approximation properties of this feedback law are investigated. An application to the optimal control of a Fokker–Planck equation is also provided.     

Approximation of an optimal value of a Bolza functional

In this article, we show that under reasonable assumptions every Lipschitz-continuous solution to a Hamilton–Jacobi inequality approximates with a priori known error the optimal value of a respective Bolza functional and that such approximation is stable. The solutions of Hamilton–Jacobi variational inequalities can be easily obtained by well-known numerical methods as approximate solutions of Hamilton–Jacobi equations resulting from related Bolza functionals. The main strength of this approach lies in the fact that both precise solution to the Hamilton–Jacobi PDE and the distance between that solution and its numerical approximation need not be known in order to solve the original Bolza problem.     

Stochastic Optimal Control for the Stochastic Heat Equation with Exponentially Growing Coefficients and with Control and Noise on a Subdomain

Abstract

We consider stochastic optimal control problems in Banach spaces, related to nonlinear controlled equations with dissipative non linearities: on the nonlinear term we do not impose any growth condition. The problems are treated via the backward stochastic differential equations approach, that allows also to solve in mild sense Hamilton Jacobi Bellman equations in Banach spaces. We apply the results to controlled stochastic heat equation, in space dimension 1, with control and noise acting on a subdomain.     

Optimum Constrained Portfolio Rules in a Diffusion Market

A portfolio selection model is derived for diffusions where inequality constraints are imposed on portfolio security weights. Using the method of stochastic dynamic programming Hamilton–Jacobi–Bellman (HJB) equations are obtained for the problem of maximizing the expected utility of terminal wealth over a finite time horizon. Optimal portfolio weights are given in feedback form in terms of the solution of the HJB equations and its partial derivatives. An analysis of the no‐constraining (NC) region of a portfolio is also conducted.     

A stochastic portfolio optimization model with complete memory

In this article, we consider a portfolio optimization problem of the Merton’s type with complete memory over a finite time horizon. The problem is formulated as a stochastic control problem on a finite time horizon and the state evolves according to a process governed by a stochastic process with memory. The goal is to choose investment and consumption controls such that the total expected discounted utility is maximized. Under certain conditions, we derive the explicit solutions for the associated Hamilton–Jacobi–Bellman (HJB) equations in a finite-dimensional space for exponential, logarithmic, and power utility functions. For those utility functions, verification results are established to ensure that the solutions are equal to the value functions, and the optimal controls are also derived.     

Mayer control problem with probabilistic uncertainty on initial positions

In this paper we introduce and study an optimal control problem in the Mayer's form in the space of probability measures on ${\mathbb{R}}^n$ endowed with the Wasserstein distance. Our aim is to study optimality conditions when the knowledge of the initial state and velocity is subject to some uncertainty, which are modeled by a probability measure on ${\mathbb{R}}^d$ and by a vector-valued measure on ${\mathbb{R}}^d$, respectively. We provide a characterization of the value function of such a problem as unique solution of an Hamilton–Jacobi–Bellman equation in the space of measures in a suitable viscosity sense. Some applications to a pursuit-evasion game with uncertainty in the state space is also discussed, proving the existence of a value for the game.     

Existence of an optimal control for stochastic systems governed by ito equations

We give existence theorems for stochastic control problems with a lower semicontinuous cost functional and governed by Ito equations. We prove that two formulations of the fundamental problem are equivalent, one involving nonanticipative controls and the other involving (measurable) feedback controls. We then use the concept ofconvergence in distribution to prove existence for the first problem, and hence for the second as well. While our work has certain similarities with a paper of Kushner, our techniques are different and lead to more general results.     

Explicit solutions of optimal consumption,investment and insurance problems with regime switching

We consider an investor who wants to select his optimal consumption, investment and insurance policies. Motivated by new insurance products, we allow not only the financial market but also the insurable loss to depend on the regime of the economy. The objective of the investor is to maximize his expected total discounted utility of consumption over an infinite time horizon. For the case of hyperbolic absolute risk aversion (HARA) utility functions, we obtain the first explicit solutions for simultaneous optimal consumption, investment, and insurance problems when there is regime switching. We determine that the optimal insurance contract is either no-insurance or deductible insurance, and calculate when it is optimal to buy insurance. The optimal policy depends strongly on the regime of the economy. Through an economic analysis, we calculate the advantage of buying insurance.     

Coexistence and optimal control problems for a degenerate predator-prey model

In this paper we present a predator-prey mathematical model for two biological populations which dislike crowding. The model consists of a system of two degenerate parabolic equations with nonlocal terms and drifts. We provide conditions on the system ensuring the periodic coexistence, namely the existence of two non-trivial non-negative periodic solutions representing the densities of the two populations. We assume that the predator species is harvested if its density exceeds a given threshold. A minimization problem for a cost functional associated with this process and with some other significant parameters of the model is also considered.     

Existence of optimal controls for the diffusion equation

The existence is considered of a boundary control which drives a system governed by the one-dimensional diffusion equation from the zero state to a given final state, and at the same time minimizes a given functional. The problem is first modified to one in which the minimum is sought of a functional defined on a set of Radon measures. The existence of a minimizing measure is demonstrated, and it is shown that this measure may be approximated by a piecewise constant control. Finally, conditions are given under which a minimizing measurable control exists for the unmodified problem.     

Duality theorem for the stochastic optimal control problem

We prove a duality theorem for the stochastic optimal control problem with a convex cost function and show that the minimizer satisfies a class of forward–backward stochastic differential equations. As an application, we give an approach, from the duality theorem, to h$h$-path processes for diffusion processes.     

Optimal investment strategies and risk-sharing arrangements for a hybrid pension plan

A continuous time stochastic model is used to study a hybrid pension plan, where both the contribution and benefit levels are adjusted depending on the performance of the plan, with risk sharing between different generations. The pension fund is invested in a risk-free asset and multiple risky assets. The objective is to seek an optimal investment strategy and optimal risk-sharing arrangements for plan trustees and participants so that this proposed hybrid pension system provides adequate and stable income to retirees while adjusting contributions effectively, as well as keeping its sustainability in the long run. These goals are achieved by minimizing the expected discount disutility of intermediate adjustment for both benefits and contributions and that of terminal wealth in finite time horizon. Using the stochastic optimal control approach, closed-form solutions are derived under quadratic loss function and exponential loss function. Numerical analysis is presented to illustrate the sensitivity of the optimal strategies to parameters of the financial market and how the optimal benefit changes with respect to different risk aversions. Through numerical analysis, we find that the optimal strategies do adjust the contributions and retirement benefits according to fund performance and model objectives so the intergenerational risk sharing seem effectively achieved for this collective hybrid pension plan.     

Geometrical methods for analyzing the optimal management of tipping point dynamics

Natural resources are not infinitely resilient and should not be modeled as being such. Finitely resilient resources feature tipping points and history dependence. This paper provides a didactical discussion of mathematical methods that are needed to understand the optimal management of such resources: viscosity solutions of Hamilton–Jacobi–Bellman equations, the costate equation and the associated canonical equations, exact root counting, and geometrical methods to analyze the geometry of the invariant manifolds of the canonical equations. Recommendations for Resource Managers

• Management of natural resources has to take into account the possible breakdown of resilience and induced regime shifts.
• Depending on the characteristics of the resource and on its present and future economic importance, either for all initial states the same kind of management policy is optimal, or the type of the optimal management policy depends on the initial state.
• Modeling should reflect the finiteness of the data.

     

Linear-quadratic optimal control under non-Markovian switching

We study a finite-dimensional continuous-time optimal control problem on finite horizon for a controlled diffusion driven by Brownian motion, in the linear-quadratic case. We admit stochastic coefficients, possibly depending on an underlying independent marked point process, so that our model is general enough to include controlled switching systems where the switching mechanism is not required to be Markovian. The problem is solved by means of a Riccati equation, which turned out to be a backward stochastic differential equation driven by the Brownian motion and by the random measure associated with the marked point process.     

Optimal control of diffusion processes with reflection

In this paper, we consider the problem of optimally controlling a diffusion process on a closed bounded region ofR ⁿ with reflection at the boundary. Employing methods similar to Fleming (Ref. 1), we present a constructive proof that there exists an optimal Markov control that is measurable or lower semicontinuous. We prove further that the expected cost function corresponding to the optimal control is the unique solution of the quasilinear parabolic differential equation of dynamic programming with Neumann boundary conditions and that there exists a diffusion process (in the sense of Stroock and Varadhan) corresponding to the optimal control.This work was partially supported by the National Science Foundation, Grant No. GK-18339, by the Office of Naval Research, Grant No. NR-042-264, and by the National Research Council of Canada, Grant No. A3609.The author would like to thank S. R. Pliska, J. Pisa, and N. Trudinger for helpful suggestions. He is especially grateful to Professor A. F. Veinott, Jr., for help and advice in the preparation of the doctoral dissertation, on which part of this paper is based. Finally, he wishes to thank one of the referees for the careful reading and constructive comments on an earlier version of this paper.