Discontinuous Dynamical Systems in Optimal Control Theory

We develop foundations of the theory of discontinuous Hamiltonian systems appearing in the problems of optimal control. We consider analogs of the classical Poisson and Liouville theorems for discontinuous Hamiltonian systems. We study the local geometry of discontinuous dynamical systems and describe singularities in general position and the behavior of integral trajectories near an elliptical submanifold (sliding mode).     

Flood search under the California Split rule

We consider flood search on a line and show that no algorithm can achieve an average-case competitive ratio of less than 4 when compared to the optimal off-line algorithm. We also demonstrate that the optimal scanning sequences are described by simple recursive relationships that yield surprisingly complex behavior related to Hamiltonian chaos.     

Hölder Continuity of Adjoint States and Optimal Controls for State Constrained Problems

We investigate Hölder regularity of adjoint states and optimal controls for a Bolza problem under state constraints. We start by considering any optimal solution satisfying the constrained maximum principle in its normal form and we show that whenever the associated Hamiltonian function is smooth enough and has some monotonicity properties in the directions normal to the constraints, then both the adjoint state and optimal trajectory enjoy Hölder type regularity. More precisely, we prove that if the state constraints are smooth, then the adjoint state and the derivative of the optimal trajectory are Hölder continuous, while they have the two sided lower Hölder continuity property for less regular constraints. Finally, we provide sufficient conditions for Hölder type regularity of optimal controls.     

Necessary conditions for functional differential inclusions

We consider an optimal control problem in which the dynamic equation and cost function depend on the recent past of the trajectory. The regularity assumed in the basic data is Lipschitz continuity with respect to the sup norm. It is shown that, for a given optimal solution, an adjoint arc of bounded variation exists that satisfies an associated Hamiltonian inclusion. From this result, known smooth versions of the Pontryagin maximum principle for hereditary problems can be easily derived. Problems with Euclidean endpoint constraints are also considered.     

A semi-analytical direct optimal control solution for strongly excited and dissipative Hamiltonian systems

A semi-analytical direct optimal control solution for strongly excited and dissipative Hamiltonian systems is proposed based on the extended Hamiltonian principle, the Hamilton-Jacobi-Bellman (HJB) equation and its variational integral equation, and the finite time element approximation. The differential extended Hamiltonian equations for structural vibration systems are replaced by the variational integral equation, which can preserve intrinsic system structure. The optimal control law dependent on the value function is determined by the HJB equation so as to satisfy the overall optimality principle. The partial differential equation for the value function is converted into the integral equation with variational weighting. Then the successive solution of optimal control with system state is designed. The two variational integral equations are applied to sequential time elements and transformed into the algebraic equations by using the finite time element approximation. The direct optimal control on each time element is obtained respectively by solving the algebraic equations, which is unconstrained by the system state observed. The proposed control algorithm is applicable to linear and nonlinear systems with the quadratic performance index, and takes into account the effects of external excitations measured on control. Numerical examples are given to illustrate the optimal control effectiveness.     

Optimality Conditions and the Hamiltonian for a Distributed Optimal Control Problem on Controlled Domain

The paper investigates an optimal control problem for a distributed system arising in the economics of endogenous growth. The problem involves a specific coupled family of controlled ODEs parameterized by a parameter (representing the heterogeneity) running over a domain that may dynamically depend on the control and on the state of the system. Existence of an optimal control is obtained and continuity of any optimal control with respect to the parameter of heterogeneity is proved. The latter allows to substantially strengthen previously obtained necessary optimality conditions and to obtain a Pontryagin’s type maximum principle. The necessary optimality conditions obtained here have a Hamiltonian representation, and stationarity of the Hamiltonian along any optimal trajectory is proved in the case of time-independent data.     

A time transformation technique for Switched Systems Optimal Control Problems with fixed sequence of phases

A switched systems optimal control problem regarding a quadcopter trajectory optimization is considered. Introducing a particular time transformation, the problem is reformulated as an optimal control problem, whose solution can be found by the SQP algorithm. A numerical example concludes the paper. (© 2014 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

On the topological structure of singular attractors of nonlinear systems of differential equations

We study the topological structure of singular (in the sense of the Feigenbaum-Sharkovskii-Magnitskii theory) attractors of nonlinear dissipative systems of differential equations. We show that any such attractor is a stable nonperiodic trajectory lying on a two-dimensional infinitely folded heteroclinic separatrix manifold generated by the unstable two-dimensional invariant manifold of the original singular cycle as the bifurcation parameter of the system varies. The results obtained for two-dimensional nonautonomous and three-dimensional autonomous dissipative systems are generalized to autonomous multi- and infinite-dimensional dissipative systems as well as to conservative (in particular, Hamiltonian) systems.     

Classical characteristics of the bellman equation in constructions of grid optimal synthesis

We consider optimal control problems with fixed final time and terminal-integral cost functional, and address the question of constructing a grid optimal synthesis (a universal feedback) on the basis of classical characteristics of the Bellman equation. To construct an optimal synthesis, we propose a numerical algorithm that relies on the necessary optimality conditions (the Pontryagin maximum principle) and sufficient conditions in the Hamiltonian form. We obtain estimates for the efficiency of the numerical method. The method is illustrated by an example of the numerical solution of a nonlinear optimal control problem.     

Solutions to Constrained Optimal Control Problems with Linear Systems

This paper is devoted to present solutions to constrained finite-horizon optimal control problems with linear systems, and the cost functional of the problem is in a general form. According to the Pontryagin’s maximum principle, the extremal control of such problem is a function of the costate trajectory, but an implicit function. We here develop the canonical backward differential flows method and then give the extremal control explicitly with the costate trajectory by canonical backward differential flows. Moreover, there exists an optimal control if and only if there exists a unique extremal control. We give the proof of the existence of the optimal solution for this optimal control problem with Green functions.     

Stabilizing the Hamiltonian system for constructing optimal trajectories

An infinite-horizon optimal control problem based on an economic growth model is studied. The goal in the problem is to optimize the mechanisms of investment in basic production assets in order to increase the growth rate of the consumption level. The main output variable-the gross domestic product (GDP)-depends on three production factors: capital stock, human capital, and useful work. The first two factors are endogenous variables of the model, and the useful work is an exogenous factor. The dependence of the GDP on the production factors is described by the Cobb-Douglas power-type production function. The economic system under consideration is assumed to be closed, so the GDP is distributed between consumption and investment in the capital stock and human capital. The optimal control problem consists in determining optimal investment strategies that maximize the integral discounted relative consumption index on an infinite time interval. A solution to the problem is constructed on the basis of the Pontryagin maximum principle adapted to infinite-horizon problems. We examine the questions of existence and uniqueness of a solution, verify necessary and sufficient optimality conditions, and perform a qualitative analysis of Hamiltonian systems on the basis of which we propose an algorithm for constructing optimal trajectories. This algorithm uses information on solutions obtained by means of a nonlinear regulator. Finally, we estimate the accuracy of the algorithm with respect to the integral cost functional of the control process.     

Impulsive optimal control model for the trajectory of horizontal wells

This paper presents an impulsive optimal control model for solving the optimal designing problem of the trajectory of horizontal wells. We take fully into account the effect of unknown disturbances in drilling. The optimal control problem can be converted into a nonlinear parametric optimization by integrating the state equation. We discuss here that the locally optimal solution depends in a continuous way on the parameters (disturbances) and utilize this property to propose a revised Hooke–Jeeves algorithm. The uniform design technique is incorporated into the revised Hooke–Jeeves algorithm to handle the multimodal objective function. The numerical simulation is in accordance with theoretical results. The numerical results illustrate the validity of the model and efficiency of the algorithm.     

Numerical Continuation of Hamiltonian Relative Periodic Orbits

The bifurcation theory and numerics of periodic orbits of general dynamical systems is well developed, and in recent years, there has been rapid progress in the development of a bifurcation theory for dynamical systems with structure, such as symmetry or symplecticity. But as yet, there are few results on the numerical computation of those bifurcations. The methods we present in this paper are a first step toward a systematic numerical analysis of generic bifurcations of Hamiltonian symmetric periodic orbits and relative periodic orbits (RPOs). First, we show how to numerically exploit spatio-temporal symmetries of Hamiltonian periodic orbits. Then we describe a general method for the numerical computation of RPOs persisting from periodic orbits in a symmetry breaking bifurcation. Finally, we present an algorithm for the numerical continuation of non-degenerate Hamiltonian relative periodic orbits with regular drift-momentum pair. Our path following algorithm is based on a multiple shooting algorithm for the numerical computation of periodic orbits via an adaptive Poincaré section and a tangential continuation method with implicit reparametrization. We apply our methods to continue the famous figure eight choreography of the three-body system. We find a relative period doubling bifurcation of the planar rotating eight family and compute the rotating choreographies bifurcating from it.      

The Pontryagin derivative in optimal control

A basic feature of Pontryagin’s maximum principle is its native Hamiltonian format, inherent in the principle regardless of any regularity conditions imposed on the optimal problem under consideration. It canonically assigns to the problem a family of Hamiltonian systems, indexed with the control parameter, and complements the family with the maximum condition, which makes it possible to solve the initial value problem for the system by “dynamically” eliminating the parameter as we proceed along the trajectory, thus providing extremals of the problem. Much has been said about the maximum condition since its discovery in 1956, and all achievements in the field were mainly credited to it, whereas the Hamiltonian format of the maximum principle has always been taken for granted and never been discussed seriously. Meanwhile, the very possibility of formulating the maximum principle is intimately connected with its native Hamiltonian format and with the parametrization of the problem with the control parameter. Both these starting steps were made by L.S. Pontryagin in 1955 from scratch, in fact, out of nothing, and eventually led to the discovery of the maximum principle. Since the present volume is dedicated to the centenary of the birth of Lev Semenovich Pontryagin, I decided to return to this now semi-historical topic and give a short exposition of the Hamiltonian format of the maximum principle.     

On Periodic Solutions of One Class of Systems of Differential Equations

We study the problem of the existence of periodic solutions of two-dimensional linear inhomogeneous periodic systems of differential equations for which the corresponding homogeneous system is Hamiltonian. We propose a new numerical-analytic algorithm for the investigation of the problem of the existence of periodic solutions of two-dimensional nonlinear differential systems with Hamiltonian linear part and their construction. The results obtained are generalized to systems of higher orders. __________ Translated from Ukrains'kyi Matematychnyi Zhurnal, Vol. 57, No. 4, pp. 483–495, April, 2005.     

On dispersive stability of Hamiltonian systems on lattices

We study the long-time dynamics of oscillations in lattices of infinitely many particles interacting via certain non-linear potentials. The aim is to proof dispersive stability of such Hamiltonian systems analogously to results known for PDEs. To do so we first recapitulate the dynamics of linear Hamiltonian systems on an infinite chain and give optimal decay rates based on the dispersion relation. Based on this we proof that if the non-linearity is weak enough, the non-linear system shows a similar behaviour like its linearization. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

The Symplectic Study of Motions in a Perturbed Van-Der-Pol Dynamical System

We study a weakly perturbed van-der-Pol dynamical system and the structure of its trajectory behavior via the modern symplectic theory. Based on a Samoilenko–Prykarpatsky method of studying integral submanifolds of weakly perturbed completely integrable Hamiltonian systems, we prove the regularity of deformations of the Lagrangian asymptotic submanifolds in a vicinity of the hyperbolic periodic orbit.     

WELFARE MEASUREMENT OF TECHNOLOGICAL AND ENVIRONMENTAL EXTERNALITIES IN THE RAMSEY GROWTH MODEL

This paper deals with the measurement of economic welfare within the framework of the Ramsey growth model, when there are anticipated technological and/or environmental changes. It is shown that, under such circumstances, the Hamiltonian along an optimal trajectory underestimates the maximum sustainable utility, meaning that the welfare measure suggested in several recent studies would be incorrect in this case. We also derive the appropriate welfare measure in the presence of technological and environmental changes. The second part of the study concerns the welfare implications of unanticipated technological change.     

The Infinite-Dimensional Optimal Filtering Problem with Mobile and Stationary Sensor Networks

In this article, we introduce a framework to address filtering and smoothing with mobile sensor networks for distributed parameter systems. The main problem is formulated as the minimization of a functional involving the trace of the solution of a Riccati integral equation with constraints given by the trajectory of the sensor network. We prove existence and develop approximation of the solution to the Riccati equation in certain trace-class spaces. We also consider the corresponding optimization problem. Finally, we employ a Galerkin approximation scheme and implement a descent algorithm to compute optimal trajectories of the sensor network. Numerical examples are given for both stationary and moving sensor networks.     

Geometry of neighborhoods of singular trajectories in problems with multidimensional control

It is shown that the order of a singular trajectory in problems with multidimensional control is described by a flag of linear subspaces in the control space. In terms of this flag, we construct necessary conditions for the junction of a nonsingular trajectory with a singular one in affine control systems. We also give examples of multidimensional problems in which the optimal control has the form of an irrational winding of a torus that is passed in finite time.