On Bellman Equations in Quadratic Ergodic Control with Controller Constraints

We consider the Bellman equation related to the quadratic ergodic control problem for stochastic differential systems with controller constraints. We solve this equation rigidly in C ² -class, and give the minimal value and the optimal control. Accepted 9 January 1997     

On the Bellman Equation for Infinite Horizon Problems with Unbounded Cost Functional

We study a class of infinite horizon control problems for nonlinear systems, which includes the Linear Quadratic (LQ) problem, using the Dynamic Programming approach. Sufficient conditions for the regularity of the value function are given. The value function is compared with sub- and supersolutions of the Bellman equation and a uniqueness theorem is proved for this equation among locally Lipschitz functions bounded below. As an application it is shown that an optimal control for the LQ problem is nearly optimal for a large class of small unbounded nonlinear and nonquadratic pertubations of the same problem. Accepted 8 October 1998     

An Auxiliary Equation for the Bellman Equation in a One-Dimensional Ergodic Control

In this paper we consider the Bellman equation in a one-dimensional ergodic control. Our aim is to show the existence and the uniqueness of its solution under general assumptions. For this purpose we introduce an auxiliary equation whose solution gives the invariant measure of the diffusion corresponding to an optimal control. Using this solution, we construct a solution to the Bellman equation. Our method of using this auxiliary equation has two advantages in the one-dimensional case. First, we can solve the Bellman equation under general assumptions. Second, this auxiliary equation gives an optimal Markov control explicitly in many examples. \keywords{Bellman equation, Auxiliary equation, Ergodic control.} \amsclass{49L20, 35G20, 93E20.} Accepted 11 September 2000. Online publication 16 January 2001.     

Computational Method for Time-Optimal Switching Control

An efficient algorithm, called the time-optimal switching (TOS) algorithm, is proposed for the time-optimal switching control of nonlinear systems with a single control input. The problem is formulated in the arc times space, arc times being the durations of the arcs. A feasible switching control, or as a special case bang-bang control, is found using the STC method previously developed by the authors to get from an initial point to a target point with a given number of switchings. Then, by means of constrained optimization techniques, the cost being considered as the summation of the arc times, a minimum-time switching control solution is obtained. Example applications of the TOS algorithm involving second-order and third-order systems are presented. Comparisons are made with a well-known general optimal control software package to demonstrate the efficiency of the algorithm.     

Contingent Solutions for the Bellman Equation in Infinite Dimensions

The present paper is concerned with the study of the Hamilton–Jacobi–Bellman equation for the time optimal control problem associated with infinite-dimensional linear control systems from the point of view of continuous contingent solutions.     

Analysis of Properties of the Solutions to Parametric Time-Optimal Problems

We consider a linear time-optimal problem in which initial state values depend on a parameter and study the problem of the solution structure identification for small parameter perturbations. Properties of the time-optimal function and a point-set mapping, defined by optimal Lagrange vectors, are studied as well as the dependence of the solution on the parameter. Special attention is paid to the solution properties in irregular points.     

Pontryagin's Principle for Time-Optimal Problems

We consider time-optimal control problems for semilinear parabolic equations with pointwise state constraints and unbounded controls. A Pontryagin's principle is obtained in nonqualified form without any qualification condition. The terminal time, which is a control variable, satisfies an optimality condition, which seems to be new in the context of control problems for partial differential equations.     

具有终端不等式约束的线性二次控制问题

本文研究具有终端不等式约束的线性二次控制问题,得到了最优控制的反馈形式.所得到的反馈形式与相应的无约束问题的Riccati微分方程和一个函数矩阵的线性方程的解有关.     

Properties of the Time-Optimal Feedback Control for a Pendulum-Like System

A time-optimal control problem for a pendulum-like system is considered. The system describes the dynamics of an inertial object under the action of a bounded control force and an external force which is periodic in coordinate. The terminal set consists of points on the abscissa axis of the phase plane, and the distance between two neighboring points is equal to the period of the external force. In the general case, the solution can be obtained only numerically. An estimate is found for the amplitude of the control for which the time-optimal feedback control has the simplest structure: the number of switchings is not greater than one for any initial conditions. For the estimated interval of the control constraints, we analyze the feedback control pattern.     

Deterministic Global Optimization in Nonlinear Optimal Control Problems

The accurate solution of optimal control problems is crucial in many areas of engineering and applied science. For systems which are described by a nonlinear set of differential-algebraic equations, these problems have been shown to often contain multiple local minima. Methods exist which attempt to determine the global solution of these formulations. These algorithms are stochastic in nature and can still get trapped in local minima. There is currently no deterministic method which can solve, to global optimality, the nonlinear optimal control problem. In this paper a deterministic global optimization approach based on a branch and bound framework is introduced to address the nonlinear optimal control problem to global optimality. Only mild conditions on the differentiability of the dynamic system are required. The implementa-tion of the approach is discussed and computational studies are presented for four control problems which exhibit multiple local minima.     

Approximate Solutions to the Time-Invariant Hamilton–Jacobi–Bellman Equation

In this paper, we develop a new method to approximate the solution to the Hamilton–Jacobi–Bellman (HJB) equation which arises in optimal control when the plant is modeled by nonlinear dynamics. The approximation is comprised of two steps. First, successive approximation is used to reduce the HJB equation to a sequence of linear partial differential equations. These equations are then approximated via the Galerkin spectral method. The resulting algorithm has several important advantages over previously reported methods. Namely, the resulting control is in feedback form and its associated region of attraction is well defined. In addition, all computations are performed off-line and the control can be made arbitrarily close to optimal. Accordingly, this paper presents a new tool for designing nonlinear control systems that adhere to a prescribed integral performance criterion.     

A Linear PDE Approach to the Bellman Equation of Ergodic Control with Periodic Structure

   Abstract. In this paper we give a new proof of the existence result of Bensoussan [1, Theorem II-6.1] for the Bellman equation of ergodic control with periodic structure. This Bellman equation is a nonlinear PDE, and he constructed its solution by using the solution of a nonlinear PDE. On the contrary, our key idea is to solve two linear PDEs. Hence, we propose a linear PDE approach to this Bellman equation.     

A Linear PDE Approach to the Bellman Equation of Ergodic Control with Periodic Structure

Abstract. In this paper we give a new proof of the existence result of Bensoussan [1, Theorem II-6.1] for the Bellman equation of ergodic control with periodic structure. This Bellman equation is a nonlinear PDE, and he constructed its solution by using the solution of a nonlinear PDE. On the contrary, our key idea is to solve two linear PDEs. Hence, we propose a linear PDE approach to this Bellman equation.     

Existence and Structure of Optimal Solutions of Infinite-Dimensional Control Problems

In this work we analyze the structure of optimal solutions for a class of infinite-dimensional control systems. We are concerned with the existence of an overtaking optimal trajectory over an infinite horizon. The existence result that we obtain extends the result of Carlson, Haurie, and Jabrane to a situation where the trajectories are not necessarily bounded. Also, we show that an optimal trajectory defined on an interval [0,τ] is contained in a small neighborhood of the optimal steady-state in the weak topology for all t ∈ [0,τ] \backslash E , where E \subset [0,τ] is a measurable set such that the Lebesgue measure of E does not exceed a constant which depends only on the neighborhood of the optimal steady-state and does not depend on τ . Accepted 26 July 2000. Online publication 13 November 2000.     

On the Bellman equation of the overtaking criterion: Addendum

In a previous note, we discussed the properties of solutions to the Bellman equation of the Gale overtaking criterion. The purpose of this note is to show that the dynamic programming approach may also be used for the Brock criterion.We are indebted to an anonymous referee who contributed to improve this paper.     

The Rate of Convergence of Finite-Difference Approximations for Bellman Equations with Lipschitz Coefficients

We consider parabolic Bellman equations with Lipschitz coefficients. Error bounds of order h^1/2 for certain types of finite-difference schemes are obtained.