A Carathéodory-Hamilton-Jacobi theory for infinite-horizon optimal control problems

In this paper, we extend Carathéodory's concept of equivalent variational problems to infinite-horizon optimal control problems. In such a setting, the usual concept of a minimum need not exist, and we therefore consider a weaker definition of optimality, known as catching up optimality. The extension presented here leads us to a Hamilton-Jacobi theory for infinite-horizon optimal control problems that closely parallels the classical work of Carathéodory as well as providing sufficient conditions for optimality. Finally, we show that the results given here subsume several previously known results as a special case.This research forms part of the author's doctoral dissertation, written at the University of Delaware, Newark, Delaware, under the supervision of Professor T. S. Angell.     

Optimal feedback control for a linear-quadratic control problem with a state inequality constraint

In this paper, we consider the linear-quadratic control problem (LQCP) for systems defined by evolution operators with an inequality constraint on the state. It is shown that, under suitable assumptions, the optimal control exists, is unique, and has a closedloop structure. The synthesis of the feedback control requires one to solve two Riccati integral equations.     

A perturbation method for optimizing matrix stability

We present the first practical perturbation method for optimizing matrix stability using spectral abscissa minimization. Using perturbation theory for a matrix with simple eigenvalues and coupling this with linear programming, we successively reduce the spectral abscissa of a matrix until it reaches a local minimum. Optimality conditions for a local minimizer of the spectral abscissa are provided and proved for both the affine matrix problem and the output feedback control problem. Experiments show that this novel perturbation method is efficient, especially for a matrix with the majority of whose eigenvalues are already located in the left half of the complex plane. Moreover, unlike most available methods, the method does not require the introduction of Lyapunov variables. The method is illustrated for a small size matrix from an affine matrix problem and is then applied to large matrices actually arising from more sophisticated control problems used in the design of the Boeing 767 jet and a nuclear powered turbo-generator.     

Direct training method for a continuous-time nonlinear optimal feedback controller

The solutions of most nonlinear optimal control problems are given in the form of open-loop optimal control which is computed from a given fixed initial condition. Optimal feedback control can in principle be obtained by solving the corresponding Hamilton-Jacobi-Bellman dynamic programming equation, though in general this is a difficult task. We propose a practical and effective alternative for constructing an approximate optimal feedback controller in the form of a feedforward neural network, and we justify this choice by several reasons. The controller is capable of approximately minimizing an arbitrary performance index for a nonlinear dynamical system for initial conditions arising from a nontrivial bounded subset of the state space. A direct training algorithm is proposed and several illustrative examples are given.This research was carried out with the support of a grant from the Australian Research Council.We thank the anonymous reviewers for their helpful comments.     

Lower closure and existence theorems for optimal control problems with infinite horizon

Lower closure theorems are proved for optimal control problems governed by ordinary differential equations for which the interval of definition may be unbounded. One theorem assumes that Cesari's property (Q) holds. Two theorems are proved which do not require property (Q), but assume either a generalized Lipschitz condition or a bound on the controls in an appropriateL _p-space. An example shows that these hypotheses can hold without property (Q) holding.     

Turnpike properties for a class of piecewise deterministic systems arising in manufacturing flow control

This paper deals with a general class of piecewise deterministic control systems that encompasses FMS flow control models. One uses the Markov renewal decision process formalism to characterize optimal policies via a discrete event dynamic programming approach. A family of control problems with a random stopping time is associated with these optimality conditions. These problems can be reformulated as infinite horizon deterministic control problems. It is then shown how the so-calledturnpike property should hold for these deterministic control problems under classical convexity assumptions. These turnpikes have the same generic properties as the attractors obtained via a problem specific approach in FMS flow control models and production planning and are calledhedging points in this literature.This research has been supported by NSERC-Canada, Grants No. A4952 by FCAR-Québec, Grant No. 88EQ3528, Actions Structurantes, MESS-Québec, Grant No. 6.1/7.4(28), and FNRS-Switzerland.     

Existence of solutions for a class of nonconcave infinite horizon optimal control problems

In this article, we establish the existence of optimal solutions for a large class of nonconvex infinite horizon discrete-time optimal control problems. This class contains optimal control problems arising in economic dynamics which describe a model with nonconcave utility functions representing the preferences of the planner.     

Stochastic optimal linear feedback control systems using available measurements

The stochastic optimal control of linear systems with time-varying and partially observable parameters is synthesized under noisy measurements and a quadratic performance criterion. The structure of the regulator is given, and the optimal solution is reduced to a two-point boundary-value problem. Comments on the numerical solution by appropriate integration schemes is included.     

Conditions for the discovery of solution horizons

We present necessary and sufficient conditions for discrete infinite horizon optimization problems with unique solutions to be solvable. These problems can be equivalently viewed as the task of finding a shortest path in an infinite directed network. We provide general forward algorithms with stopping rules for their solution. The key condition required is that of weak reachability, which roughly requires that for any sequence of nodes or states, it must be possible from optimal states to reach states close in cost to states along this sequence. Moreover the costs to reach these states must converge to zero. Applications are considered in optimal search, undiscounted Markov decision processes, and deterministic infinite horizon optimization.This work was supported in part by NSF Grant ECS-8700836 to The University of Michigan.     

On the regulator problem for linear degenerate control systems

In this paper, we study the finite-time regulator problem for linear, autonomous, degenerate systems, i.e., systems described by the differential equation with det(K)=0. Two investigations of the regulator problem are presented, which depend on the quadratic cost associated with the differential equation.This work was performed under the auspices of the National Research Council of Italy, Gruppo Nazionale per l'Analisi Funzionale e le Sue Applicazioni, Rome, Italy.     

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.     

Minimax Control for a Class of Linear Systems Subject to Disturbances<Superscript>1</Superscript>

A class of linear dynamical control systems subject to uncertain but bounded disturbances is considered. The bounds imposed on the disturbances depend on the control magnitude and grow with the control. This situation occurs if the disturbances are due to the inaccuracy of the control implementation and often takes place in engineering applications such as transportation, aerospace, and robotic systems. Under certain assumptions, the minimax control problem is formulated and solved. Explicit expressions for the optimal control (both open-loop and feedback) are obtained that provide the minimax to the given performance index for arbitrary but bounded disturbances. Examples are given. ¹This work was supported by the Russian Foundation for Basic Research (Project 05-01-00647) and the Grant for Russian Scientific Schools (NSh 1627.2003.1).     

A receding horizon control approach for integrated vector management of Aedes aegypti using chemical and biological control: A mono and a multiobjective approach

This paper addresses an integrated vector management (IVM) approach for combating Aedes aegypti, the transmission vector of dengue, zika, and chikungunya diseases, some of the most important viral epidemics worldwide. In order to tackle this problem, a receding horizon control (RHC) strategy is adopted, considering a mono-objective and a multiobjective version of the optimal control model of combating the mosquito using chemical and biological control. RHC is essentially a suboptimal scheme of classical optimal control strategies considering discrete-time approximations. The integrated vector control actions used in this work consist in applying insecticides and inserting sterile males produced by irradiation in the population of mosquitoes. The cost function is defined in terms of social and economic costs, in order to quantify the effectiveness of the proposed epidemiological control throughout a time window of 4 months. Numerical simulations show that the obtained results are better than those from the optimal control strategies found in literature. Furthermore, through the application of the multiobjetive approach, varying the scenarios in the mono-objective formulation is no longer necessary and a set of optimal strategies can be obtained at once. Finally, in order to help health authorities in the choice of the best solution of the Pareto-optimal set to be implemented in practice, a cost-effectiveness analysis is performed and a strategy representing the most cost-effective control policy is obtained.     

Existence theorems for lagrange control problems with unbounded time domain

Existence theorems are proved for usual Lagrange control systems, in which the time domain is unbounded. As usual in Lagrange problems, the cost functional is an improper integral, the state equation is a system of ordinary differential equations, with assigned boundary conditions, and constraints may be imposed on the values of the state and control variables. It is shown that the boundary conditions at infinity require a particular analysis. Problems of this form can be found in econometrics (e.g., infinite-horizon economic models) and operations research (e.g., search problems).The author wishes to thank Professor L. Cesari for his many helpful comments and assistance in the preparation of this paper. This work was sponsored by the United States Air Force under Grants Nos. AF-AFOSR-69-1767-A and AFOSR-69-1662.     

Nonlinear feedback controllers and compensators: a state-dependent Riccati equation approach

State-dependent Riccati equation (SDRE) techniques are rapidly emerging as general design and synthesis methods of nonlinear feedback controllers and estimators for a broad class of nonlinear regulator problems. In essence, the SDRE approach involves mimicking standard linear quadratic regulator (LQR) formulation for linear systems. In particular, the technique consists of using direct parameterization to bring the nonlinear system to a linear structure having state-dependent coefficient matrices. Theoretical advances have been made regarding the nonlinear regulator problem and the asymptotic stability properties of the system with full state feedback. However, there have not been any attempts at the theory regarding the asymptotic convergence of the estimator and the compensated system. This paper addresses these two issues as well as discussing numerical methods for approximating the solution to the SDRE. The Taylor series numerical methods works only for a certain class of systems, namely with constant control coefficient matrices, and only in small regions. The interpolation numerical method can be applied globally to a much larger class of systems. Examples will be provided to illustrate the effectiveness and potential of the SDRE technique for the design of nonlinear compensator-based feedback controllers.     

Decision and forecast horizons,agreeable plans,and the maximum principle for infinite horizon control problems

This paper establishes a link between the concepts of optimality used in economic theory for infinite horizon planning models, and the concepts of decision and forecast horizons used in several areas of Management Science. It is shown that decision and forecast horizons induce an alternate definition of optimality which is stronger than the concept of ‘agreeable plan’ proposed by Hammond. All concepts of optimality share a common property, namely a Principle of Optimality. In an optimal control framework this implies that the maximum principle will be a necessary condition for optimality according to any of these definitions.     

Existence and structure of solutions for a class of optimal control systems with discounting arising in economic dynamics

We study the existence and a turnpike property of solutions of a discrete-time control system with discounting and with a compact metric space of states. In our recent work for this discrete-time optimal control system without discounting we establish the turnpike property and show that it is stable under perturbations of an objective function. In the present paper we show that this turnpike property together with its stability also hold for the system with discounting.     

Optimal and feedback strategies for managing multicohort populations

A Beverton and Holt type linear cohort dynamics model is integrated and combined with a nonlinear stock-recruitment relationship to obtain a discrete-time multicohort harvesting model. Assuming that each age class is individually controllable, it is shown, subject to certain assumptions, that the optimal harvesting strategy is to drive the population to the maximum sustainable yield solution in one time step. In most fisheries, this controllability assumption is not met and harvesting is agewise nonselective. In this case, it may be preferable to implement a harvesting policy based on suboptimal constant effort or stock level feedback strategies, rather than implement a more complicated optimal policy. This question is addressed through numerical studies on the management of an anchovy fishery.Dedicated to G. LeitmannThe author would like to thank M. Mangel, W. Reed, P. Sullivan, and G. Swartzman for commenting on a draft of this paper.     

Modified modulus‐based matrix splitting iteration methods for linear complementarity problems

For solving the large sparse linear complementarity problems, we establish modified modulus‐based matrix splitting iteration methods and present the convergence analysis when the system matrices are H₊‐matrices. The optima of parameters involved under some scopes are also analyzed. Numerical results show that in computing efficiency, our new methods are superior to classical modulus‐based matrix splitting iteration methods under suitable conditions. Copyright © 2015 John Wiley & Sons, Ltd.