An inverse optimal problem in discrete-time stochastic control

In this paper, we study an inverse optimal problem in discrete-time stochastic control. We give necessary and sufficient conditions for a solution to a system of stochastic difference equations to be the solution of a certain optimal control problem. Our results extend to the stochastic case the work of Dechert. In particular, we present a stochastic version of an important principle in welfare economics.     

Optimal Trade Execution Under Stochastic Volatility and Liquidity

Abstract

We study the problem of optimally liquidating a financial position in a discrete-time model with stochastic volatility and liquidity. We consider the three cases where the objective is to minimize the expectation, an expected exponential or a mean-variance criterion of the implementation cost. In the first case, the optimal solution can be fully characterized by a forward-backward system of stochastic equations depending on conditional expectations of future liquidity. In the other two cases, we derive Bellman equations from which the optimal solutions can be obtained numerically by discretizing the control space. In all three cases, we compute optimal strategies for different simulated realizations of prices, volatility and liquidity and compare the outcomes to the ones produced by the deterministic strategies of Bertsimas and Lo (1998; Optimal control of execution costs. Journal of Financial Markets, 1, 1–50) and Almgren and Chriss (2001; Optimal execution of portfolio transactions. Journal of Risk, 3, 5–33).     

Stochastic Control for Mean-Field Stochastic Partial Differential Equations with Jumps

We study optimal control for mean-field stochastic partial differential equations (stochastic evolution equations) driven by a Brownian motion and an independent Poisson random measure, in case of partial information control. One important novelty of our problem is represented by the introduction of general mean-field operators, acting on both the controlled state process and the control process. We first formulate a sufficient and a necessary maximum principle for this type of control. We then prove the existence and uniqueness of the solution of such general forward and backward mean-field stochastic partial differential equations. We apply our results to find the explicit optimal control for an optimal harvesting problem.     

The achievable region method in the optimal control of queueing systems; formulations,bounds and policies

We survey a new approach that the author and his co-workers have developed to formulate stochastic control problems (predominantly queueing systems) asmathematical programming problems. The central idea is to characterize the region of achievable performance in a stochastic control problem, i.e., find linear or nonlinear constraints on the performance vectors that all policies satisfy. We present linear and nonlinear relaxations of the performance space for the following problems: Indexable systems (multiclass single station queues and multiarmed bandit problems), restless bandit problems, polling systems, multiclass queueing and loss networks. These relaxations lead to bounds on the performance of an optimal policy. Using information from the relaxations we construct heuristic nearly optimal policies. The theme in the paper is the thesis that better formulations lead to deeper understanding and better solution methods. Overall the proposed approach for stochastic control problems parallels efforts of the mathematical programming community in the last twenty years to develop sharper formulations (polyhedral combinatorics and more recently nonlinear relaxations) and leads to new insights ranging from a complete characterization and new algorithms for indexable systems to tight lower bounds and nearly optimal algorithms for restless bandit problems, polling systems, multiclass queueing and loss networks.     

On existence of optimal solutions for stochastic differential equations and inclusions with current velocities

For a stochastic differential inclusion given in terms of current velocities (symmetric mean derivatives) on flat n-dimensional torus, we prove the existence of optimal solution minimizing a certain cost criterion. Then this result is applied to the problem of optimal control for equations with current velocities.     

OPTIMAL EXPLOITATION OF RENEWABLE RESOURCES BY THE VISCOSITY SOLUTION METHOD

We study the stochastic optimization problem of renewable resources to maximize the expected discounted utility of exploitation. We develop the viscosity solution method to the associated Hamilton–Jacobi–Bellman equation and further show the C ²-regularity of the viscosity solution under the strict concavity of the utility function. The optimal policy is shown to exist and given in a feedback form or a stochastic version of Hotelling's rule.     

General Linear Quadratic Optimal Stochastic Control Problem Driven by a Brownian Motion and a Poisson Random Martingale Measure with Random Coefficients

In this article, we consider a linear-quadratic optimal control problem (LQ problem) for a controlled linear stochastic differential equation driven by a multidimensional Browinan motion and a Poisson random martingale measure in the general case, where the coefficients are allowed to be predictable processes or random matrices. By the duality technique, the dual characterization of the optimal control is derived by the optimality system (so-called stochastic Hamilton system), which turns out to be a linear fully coupled forward-backward stochastic differential equation with jumps. Using a decoupling technique, the connection between the stochastic Hamilton system and the associated Riccati equation is established. As a result, the state feedback representation is obtained for the optimal control. As the coefficients for the LQ problem are random, here, the associated Riccati equation is a highly nonlinear backward stochastic differential equation (BSDE) with jumps, where the generator depends on the unknown variables K, L, and H in a quadratic way (see (5.9) herein). For the case where the generator is bounded and is linearly dependent on the unknown martingale terms L and H, the existence and uniqueness of the solution for the associated Riccati equation are established by Bellman's principle of quasi-linearization.     

Indefinite stochastic linear-quadratic optimal control problems with random jumps and related stochastic Riccati equations

We discuss the stochastic linear-quadratic (LQ) optimal control problem with Poisson processes under the indefinite case. Based on the wellposedness of the LQ problem, the main idea is expressed by the definition of relax compensator that extends the stochastic Hamiltonian system and stochastic Riccati equation with Poisson processes (SREP) from the positive definite case to the indefinite case. We mainly study the existence and uniqueness of the solution for the stochastic Hamiltonian system and obtain the optimal control with open-loop form. Then, we further investigate the existence and uniqueness of the solution for SREP in some special case and obtain the optimal control in close-loop form.     

A successive approximation algorithm for stochastic control problems

We consider optimal stochastic control problems in which the state variables are governed by Itô equations. A successive approximation algorithm for optimal stochastic control is obtained. This algorithm, together with the existing numerical methods for parabolic or elliptic PDEs, provides numerical schemes for the solution of Bellman equations.     

Existence of measurable optima in stochastic nonlinear programming and control

Recently W. Heilmann proved that a certain class of stochastic linear programs possesses an optimal solution which depends on the random parameter in a measurable way, and that the optimal value is measurable. We prove a result of this type for much more general problems, including stochastic nonlinear programming and stochastic optimal control problems.     

Viscosity Solutions for a System of Integro-PDEs and Connections to Optimal Switching and Control of Jump-Diffusion Processes

We develop a viscosity solution theory for a system of nonlinear degenerate parabolic integro-partial differential equations (IPDEs) related to stochastic optimal switching and control problems or stochastic games. In the case of stochastic optimal switching and control, we prove via dynamic programming methods that the value function is a viscosity solution of the IPDEs. In our setting the value functions or the solutions of the IPDEs are not smooth, so classical verification theorems do not apply.     

The Mean-Variance Hedging of a Defaultable Option with Partial Information

Abstract

We consider the mean-variance hedging of a defaultable claim in a general stochastic volatility model. By introducing a new measure Q ⁰, we derive the martingale representation theorem with respect to the investors' filtration . We present an explicit form of the optimal-variance martingale measure by means of a stochastic Riccati equation (SRE). For a general contingent claim, we represent the optimal strategy and the optimal cost of the mean-variance hedging by means of another backward stochastic differential equation (BSDE). For the defaultable option, especially when there exists a random recovery rate we give an explicit form of the solution of the BSDE.     

OPTIMAL GRAZING MANAGEMENT RULES IN SEMI‐ARID RANGELANDS WITH UNCERTAIN RAINFALL

Abstract. We study optimal adaptive grazing management under uncertain rainfall in a discrete‐time model. As in each year actual rainfall can be observed during the short rainy season, and grazing management can be adapted accordingly for the growing season; the closed‐loop solution of the stochastic optimal control problem does not only depend on the state variable, but also on the realization of the random rainfall. This distinguishes optimal grazing management from the optimal use of most other natural resources under uncertainty, where the closed‐loop solution of the stochastic optimal control problem depends only on the state variables. Solving this unusual stochastic optimization problem allows us to critically contribute to a long‐standing controversy over how to optimally manage semi‐arid rangelands by simple rules of thumb.     

Smooth Solutions to Optimal Investment Models with Stochastic Volatilities and Portfolio Constraints

Abstract. This paper deals with an extension of Merton's optimal investment problem to a multidimensional model with stochastic volatility and portfolio constraints. The classical dynamic programming approach leads to a characterization of the value function as a viscosity solution of the highly nonlinear associated Bellman equation. A logarithmic transformation expresses the value function in terms of the solution to a semilinear parabolic equation with quadratic growth on the derivative term. Using a stochastic control representation and some approximations, we prove the existence of a smooth solution to this semilinear equation. An optimal portfolio is shown to exist, and is expressed in terms of the classical solution to this semilinear equation. This reduction is useful for studying numerical schemes for both the value function and the optimal portfolio. We illustrate our results with several examples of stochastic volatility models popular in the financial literature.     

Stochastic optimal control and algorithm of the trajectory of horizontal wells

This paper presents a nonlinear, multi-phase and stochastic dynamical system according to engineering background. We show that the stochastic dynamical system exists a unique solution for every initial state. A stochastic optimal control model is constructed and the sufficient and necessary conditions for optimality are proved via dynamic programming principle. This model can be converted into a parametric nonlinear stochastic programming by integrating the state equation. It is discussed here that the local optimal solution depends in a continuous way on the parameters. A revised Hooke–Jeeves algorithm based on this property has been developed. Computer simulation is used for this paper, and the numerical results illustrate the validity and efficiency of the algorithm.     

Variational Inequalities in Hilbert Spaces with Measures and Optimal Stopping Problems

We study the existence theory for parabolic variational inequalities in weighted L ² spaces with respect to excessive measures associated with a transition semigroup. We characterize the value function of optimal stopping problems for finite and infinite dimensional diffusions as a generalized solution of such a variational inequality. The weighted L ² setting allows us to cover some singular cases, such as optimal stopping for stochastic equations with degenerate diffusion coefficient. As an application of the theory, we consider the pricing of American-style contingent claims. Among others, we treat the cases of assets with stochastic volatility and with path-dependent payoffs.     

Smooth Solutions to Optimal Investment Models with Stochastic Volatilities and Portfolio Constraints

   Abstract. This paper deals with an extension of Merton's optimal investment problem to a multidimensional model with stochastic volatility and portfolio constraints. The classical dynamic programming approach leads to a characterization of the value function as a viscosity solution of the highly nonlinear associated Bellman equation. A logarithmic transformation expresses the value function in terms of the solution to a semilinear parabolic equation with quadratic growth on the derivative term. Using a stochastic control representation and some approximations, we prove the existence of a smooth solution to this semilinear equation. An optimal portfolio is shown to exist, and is expressed in terms of the classical solution to this semilinear equation. This reduction is useful for studying numerical schemes for both the value function and the optimal portfolio. We illustrate our results with several examples of stochastic volatility models popular in the financial literature.     

Backward Stochastic Riccati Equations and Infinite Horizon L-Q Optimal Control with Infinite Dimensional State Space and Random Coefficients

We study the Riccati equation arising in a class of quadratic optimal control problems with infinite dimensional stochastic differential state equation and infinite horizon cost functional. We allow the coefficients, both in the state equation and in the cost, to be random. In such a context backward stochastic Riccati equations are backward stochastic differential equations in the whole positive real axis that involve quadratic non-linearities and take values in a non-Hilbertian space. We prove existence of a minimal non-negative solution and, under additional assumptions, its uniqueness. We show that such a solution allows to perform the synthesis of the optimal control and investigate its attractivity properties. Finally the case where the coefficients are stationary is addressed and an example concerning a controlled wave equation in random media is proposed.     

Estimation of an optimal solution of a LP problem with unknown objective function

We consider a linear programming problem with unknown objective function. Random observations related to the unknown objective function are sequentially available. We define a stochastic algorithm, based on the simplex method, that estimates an optimal solution of the linear programming problem. It is shown that this algorithm converges with probability one to the set of optimal solutions and that its failure probability is of order inversely proportional to the sample size. We also introduce stopping criteria for the algorithm. The asymptotic normality of some suitably defined residuals is also analyzed. The proposed estimation algorithm is motivated by the stochastic approximation algorithms but it introduces a generalization of these techniques when the linear programming problem has several optimal solutions. The proposed algorithm is also close to the stochastic quasi-gradient procedures, though their usual assumptions are weakened.Mathematics Subject Classification (2000): 90C05, 62L20, 90C15Acknowledgments. I would like to thank two unknown referees for their fruitful suggestions that have helped to improve the paper.     

Necessary conditions for optimality in relaxed stochastic control problems

In this paper, we are concerned with optimal control problems where the system is driven by a stochastic differential equation of the Ito type. We study the relaxed model for which an optimal solution exists. This is an extension of the initial control problem, where admissible controls are measure valued processes. Using Ekeland's variational principle and some stability properties of the corresponding state equation and adjoint processes, we establish necessary conditions for optimality satisfied by an optimal relaxed control. This is the first version of the stochastic maximum principle that covers relaxed controls.