Stochastic Linear Quadratic Optimal Control Problems

This paper is concerned with the stochastic linear quadratic optimal control problem (LQ problem, for short) for which the coefficients are allowed to be random and the cost functional is allowed to have a negative weight on the square of the control variable. Some intrinsic relations among the LQ problem, the stochastic maximum principle, and the (linear) forward—backward stochastic differential equations are established. Some results involving Riccati equation are discussed as well. Accepted 15 May 2000. Online publication 1 December 2000     

The Value Function of Singularly Perturbed Control Systems

The limit as ɛ→ 0 of the value function of a singularly perturbed optimal control problem is characterized. Under general conditions it is shown that limit value functions exist and solve in a viscosity sense a Hamilton—Jacobi equation. The Hamiltonian of this equation is generated by an infinite horizon optimization on the fast time scale. In particular, the limit Hamiltonian and the limit Hamilton—Jacobi equation are applicable in cases where the reduction of order, namely setting ɛ = 0 , does not yield an optimal behavior. Accepted 18 November 1999     

Hamilton-Jacobi theory over time scales and applications to linear-quadratic problems

In this paper we first derive the verification theorem for nonlinear optimal control problems over time scales. That is, we show that the value function is the only solution of the Hamilton-Jacobi equation, in which the minimum is attained at an optimal feedback controller. Applications to the linear-quadratic regulator problem (LQR problem) gives a feedback optimal controller form in terms of the solution of a generalized time scale Riccati equation, and that every optimal solution of the LQR problem must take that form. A connection of the newly obtained Riccati equation with the traditional one is established. Problems with shift in the state variable are also considered. As an important tool for the latter theory we obtain a new formula for the chain rule on time scales. Finally, the corresponding LQR problem with shift in the state variable is analyzed and the results are related to previous ones.     

Deterministic and Stochastic Control of Navier—Stokes Equation with Linear, Monotone, and Hyperviscosities

This paper deals with the optimal control of space—time statistical behavior of turbulent fields. We provide a unified treatment of optimal control problems for the deterministic and stochastic Navier—Stokes equation with linear and nonlinear constitutive relations. Tonelli type ordinary controls as well as Young type chattering controls are analyzed. For the deterministic case with monotone viscosity we use the Minty—Browder technique to prove the existence of optimal controls. For the stochastic case with monotone viscosity, we combine the Minty—Browder technique with the martingale problem formulation of Stroock and Varadhan to establish existence of optimal controls. The deterministic models given in this paper also cover some simple eddy viscosity type turbulence closure models. Accepted 7 June 1999     

Autonomous Integral Functionals with Discontinuous Nonconvex Integrands: Lipschitz Regularity of Minimizers,DuBois—Reymond Necessary Conditions,and Hamilton—Jacobi Equations

This paper is devoted to the autonomous Lagrange problem of the calculus of variations with a discontinuous Lagrangian. We prove that every minimizer is Lipschitz continuous if the Lagrangian is coercive and locally bounded. The main difference with respect to the previous works in the literature is that we do not assume that the Lagrangian is convex in the velocity. We also show that, under some additional assumptions, the DuBois—Reymond necessary condition still holds in the discontinuous case. Finally, we apply these results to deduce that the value function of the Bolza problem is locally Lipschitz and satisfies (in a generalized sense) a Hamilton—Jacobi equation.     

Indefinite LQ optimal control with cross term for discrete‐time uncertain systems

Recently, there has been an increasing interest in the study on uncertain optimal control problems. In this paper, a linear quadratic (LQ) optimal control with cross term for discrete‐time uncertain systems is considered, whereas the weighting matrices in the cost function are allowed to be indefinite. Firstly, a recurrence equation for the problem is presented based on Bellman's principle of optimality in dynamic programming. Then, a necessary condition for the existence of an optimal linear state feedback control of the indefinite LQ problem is given by the recurrence equation. Moreover, a sufficient condition of well‐posedness for the indefinite LQ problem is presented by introducing a linear matrix inequality (LMI) condition. Furthermore, it is shown that the well‐posedness of the indefinite LQ problem, the solvability of the indefinite LQ problem, the LMI condition, and the solvability of the constrained difference equation are equivalent to each other. Finally, an example is presented to illustrate the results obtained.     

Second order Hamilton–Jacobi–Bellman equations with an unbounded operator

This work is devoted to the study of a class of Hamilton–Jacobi–Bellman equations associated to an optimal control problem where the state equation is a stochastic differential inclusion with a maximal monotone operator. We show that the value function minimizing a Bolza-type cost functional is a viscosity solution of the HJB equation. The proof is based on the perturbation of the initial problem by approximating the unbounded operator. Finally, by providing a comparison principle we are able to show that the solution of the equation is unique.     

Discrete-time Indefinite LQ Control with State and Control Dependent Noises

This paper deals with the discrete-time stochastic LQ problem involving state and control dependent noises, whereas the weighting matrices in the cost function are allowed to be indefinite. In this general setting, it is shown that the well-posedness and the attainability of the LQ problem are equivalent. Moreover, a generalized difference Riccati equation is introduced and it is proved that its solvability is necessary and sufficient for the existence of an optimal control which can be either of state feedback or open-loop form. Furthermore, the set of all optimal controls is identified in terms of the solution to the proposed difference Riccati equation.     

A Class of Hamilton—Jacobi Equations with Unbounded Coefficients in Hilbert Spaces

We establish existence and comparison theorems for a class of Hamilton—Jacobi equations. The class of Hamilton—Jacobi equations includes and is broader than those studied in [8] We apply the existence and uniqueness results to characterizing the value functions associated with the optimal control of systems governed by partial differential equations of parabolic type. Accepted 11 May 2001. Online publication 5 October 2001.     

Viscosity Solutions of an Infinite-Dimensional Black—Scholes—Barenblatt Equation

We study an infinite-dimensional Black—Scholes—Barenblatt equation which is a Hamilton—Jacobi—Bellman equation that is related to option pricing in the Musiela model of interest rate dynamics. We prove the existence and uniqueness of viscosity solutions of the Black—Scholes—Barenblatt equation and discuss their stochastic optimal control interpretation. We also show that in some cases the solution can be locally uniformly approximated by solutions of suitable finite-dimensional Hamilton—Jacobi—Bellman equations.     

带有交叉项的离散不定随机线性二次控制问题

研究性能指标带有交叉项的离散时间不定随机线性二次(LQ)控制问题，允许权矩阵是不定的。引入一个广义差分Riccati方程，证明了此方程的可解性是LQ问题存在最优控制的一个充分条件，并用方程的解给出了最优控制。推广了[1]的结果。     

关于带跳的反射扩散过程的随机最优控制问题

研究一类半空间上带泊松跳的反射扩散过程的随机最优控制问题。得到关于这一控制问题的非线性Nisio半群,和联系这一半群的带Neumann边界条件的哈密顿。雅可比。贝尔曼方程。讨论这一类方程的粘性解的存在唯一性等问题。证明该控制问题中的价值函数是这一方程的一个粘性解。     

An efficient parallel processing optimal control scheme for a class of nonlinear composite systems

This article presents an efficient parallel processing approach for solving the optimal control problem of nonlinear composite systems. In this approach, the original high-order coupled nonlinear two-point boundary value problem (TPBVP) derived from the Pontryagin's maximum principle is first transformed into a sequence of lower-order decoupled linear time-invariant TPBVPs. Then, an optimal control law which consists of both feedback and forward terms is achieved by using the modal series method for the derived sequence. The feedback term specified by local states of each subsystem is determined by solving a matrix Riccati differential equation. The forward term for each subsystem derived from its local information is an infinite sum of adjoint vectors. The convergence analysis and parallel processing capability of the proposed approach are also provided. To achieve an accurate feedforward-feedback suboptimal control, we apply a fast iterative algorithm with low computational effort. Finally, some comparative results are included to illustrate the effectiveness of the proposed approach.     

Smooth Solutions to Optimal Investment Models with Stochastic Volatilities and Portfolio Constraints

Abstract. This paper deals with an extension of Merton's optimal investment problem to a multidimensional model with stochastic volatility and portfolio constraints. The classical dynamic programming approach leads to a characterization of the value function as a viscosity solution of the highly nonlinear associated Bellman equation. A logarithmic transformation expresses the value function in terms of the solution to a semilinear parabolic equation with quadratic growth on the derivative term. Using a stochastic control representation and some approximations, we prove the existence of a smooth solution to this semilinear equation. An optimal portfolio is shown to exist, and is expressed in terms of the classical solution to this semilinear equation. This reduction is useful for studying numerical schemes for both the value function and the optimal portfolio. We illustrate our results with several examples of stochastic volatility models popular in the financial literature.     

Equivalence between Nonlinear H ∞ Control Problems and Existence of Viscosity Solutions of Hamilton—Jacobi—Isaacs Equations rid="*" id="*" This work was written while the author was visiting the University of California at Santa Barbara. He was partially supported by Consiglio Nazionale delle Ricerche of Italy.

In this paper we extend to completely general nonlinear systems the result stating that the suboptimal control problem is solved if and only if the corresponding Hamilton—Jacobi—Isaacs (HJI) equation has a nonnegative (super)solution. This is well known for linear systems, using the Riccati equation instead of the HJI equation. We do this using the theory of differential games and viscosity solutions. Accepted 14 February 1997     

Smooth Solutions to Optimal Investment Models with Stochastic Volatilities and Portfolio Constraints

   Abstract. This paper deals with an extension of Merton's optimal investment problem to a multidimensional model with stochastic volatility and portfolio constraints. The classical dynamic programming approach leads to a characterization of the value function as a viscosity solution of the highly nonlinear associated Bellman equation. A logarithmic transformation expresses the value function in terms of the solution to a semilinear parabolic equation with quadratic growth on the derivative term. Using a stochastic control representation and some approximations, we prove the existence of a smooth solution to this semilinear equation. An optimal portfolio is shown to exist, and is expressed in terms of the classical solution to this semilinear equation. This reduction is useful for studying numerical schemes for both the value function and the optimal portfolio. We illustrate our results with several examples of stochastic volatility models popular in the financial literature.     

Stability analysis and optimal control of stochastic singular systems

In this paper, problems of stability and optimal control for a class of stochastic singular systems are studied. Firstly, under some appropriate assumptions, some new results about mean-square admissibility are developed and the corresponding LMI sufficient condition is given. Secondly, finite-time horizon and infinite-time horizon linear quadratic (LQ) control problems for the stochastic singular system are investigated, in which the coefficients are allowed to be random in control input and quadratic criterion. Some results involving new stochastic generalized Riccati equation are discussed as well. Finally, the proposed LQ control model for stochastic singular systems provides an appropriate and effective framework to study the portfolio selection problem in light of the recent development on general stochastic LQ problems.     

Indefinite stochastic LQ control with cross term via semidefinite programming

An indefinite stochastic linear-quadratic (LQ) optimal control problem with cross term over an infinite time horizon is studied, allowing the weighting matrices to be indefinite. A systematic approach to the problem based on semidefinite programming (SDP) and related duality analysis is developed. Several implication relations among the SDP complementary duality, the existence of the solution to the generalized Riccati equation and the optimality of LQ problem are discussed. Based on these relations, a numerical procedure that provides a thorough treatment of the LQ problem via primal-dual SDP is given: it identifies a stabilizing optimal feedback control or determines the problem has no optimal solution. An example is provided to illustrate the results obtained.