Multidimensional Dual Dynamic Programming

In this paper, we develop a dual approach to the dynamic programming for the optimal control problem in a multidimensional case. The idea of our method consists in defining, instead of the value function, a new function which satisfies a dual first-order partial differential equation of dynamic programming. We then prove a suitable verification theorem and introduce the concept of a dual feedback control. The sufficient optimality conditions thus obtained are analogous to their one-dimensional counterparts.     

Solving Internal Habit Formation Models Through Dynamic Programming in Infinite Dimension

In this paper, we study an economic model, where internal habits play a role. Their formation is described by a more general functional form than is usually assumed in the literature, because a finite memory effect is allowed. Indeed, the problem becomes the optimal control of a standard ordinary differential equation, with the past of the control entering both the objective function and an inequality constraint. Therefore, the problem is intrinsically infinite dimensional. To solve this model, we apply the dynamic programming approach and we find an explicit solution for the associated Hamilton–Jacobi–Bellman equation, which lets us write the optimal strategies in feedback form. Therefore, we contribute to the existing literature in two ways. Firstly, we fully develop the dynamic programming approach to a type of problem not studied in previous contributions. Secondly, we use this result to unveil the global dynamics of an economy characterized by generic internal habits.     

A dual dynamic programming for minimax optimal control problems governed by parabolic equation

In the article, minimax optimal control problems governed by parabolic equations are considered. We apply a new dual dynamic programming approach to derive sufficient optimality conditions for such problems. The idea is to move all the notions from a state space to a dual space and to obtain a new verification theorem providing the conditions, which should be satisfied by a solution of the dual partial differential equation of dynamic programming. We also give sufficient optimality conditions for the existence of an optimal dual feedback control and some approximation of the problem considered, which seems to be very useful from a practical point of view.     

A reduced-order extrapolated approach to solution coefficient vectors in the Crank-Nicolson finite element method for the uniform transmission line equation

This work is concerned mainly with developing and testing the reduced-order extrapolated approach to the unknown coefficient vectors in the Crank-Nicolson finite element (CNFE) solutions for the uniform transmission line equation. For this objective, the CNFE functional and matrix models and the existence, stability, and errors of the CNFE solutions of the uniform transmission line equation are first derived. Then a reduced-order extrapolated CNFE (ROECNFE) matrix model is established by means of a proper orthogonal decomposition technique, and the existence, stability, and error estimates of the ROECNFE solutions are demonstrated by matrix analysis, leading to an elegant theoretical development. Especially, our work shows that the basis functions and accuracy of the ROECNFE matrix model are the same as those of the CNFE matrix model. Finally, some numerical tests are illustrated to computationally experimentally confirm the validity and sharpness of the ROECNFE method.     

Generalised discounting in dynamic programming with unbounded returns

In this note we study a deterministic dynamic programming model with generalised discounting. We use a modified weighted norm approach and an approximation technique to a study of the Bellman equation for unbounded return functions. Furthermore, we apply this theory to economic growth models.     

Reduced-order-based feedback control of the Kuramoto–Sivashinsky equation

In this paper, we consider the Kuramoto–Sivashinsky equation (KSE), which describes the long-wave motions of a thin film over a vertical plane. Solution procedures for the KSE often yield a large or infinite-dimensional nonlinear system. We first discuss two reduced-order methods, the approximate inertial manifold and the proper orthogonal decomposition, and show that these methods can be used to obtain a reduced-order system that can accurately describe the dynamics of the KSE. Moreover, from this resulting reduced-order system, the feedback controller can readily be designed and synthesized. For our control techniques, we use the linear and nonlinear quadratic regulator methods, which are the first- and second-order approximated solutions of the Hamilton–Jacobi–Bellman equation, respectively. Numerical simulations comparing the performance of the reduced-order-based linear and nonlinear controllers are presented.     

Approximation of optimal feedback control: a dynamic programming approach

We consider the general continuous time finite-dimensional deterministic system under a finite horizon cost functional. Our aim is to calculate approximate solutions to the optimal feedback control. First we apply the dynamic programming principle to obtain the evolutive Hamilton–Jacobi–Bellman (HJB) equation satisfied by the value function of the optimal control problem. We then propose two schemes to solve the equation numerically. One is in terms of the time difference approximation and the other the time-space approximation. For each scheme, we prove that (a) the algorithm is convergent, that is, the solution of the discrete scheme converges to the viscosity solution of the HJB equation, and (b) the optimal control of the discrete system determined by the corresponding dynamic programming is a minimizing sequence of the optimal feedback control of the continuous counterpart. An example is presented for the time-space algorithm; the results illustrate that the scheme is effective.     

Adaptive POD basis computation for parametrized nonlinear systems using optimal snapshot location

The construction of reduced-order models for parametrized partial differential systems using proper orthogonal decomposition (POD) is based on the information of the so-called snapshots. These provide the spatial distribution of the nonlinear system at discrete parameter and/or time instances. In this work a strategy is used, where the POD reduced-order model is improved by choosing additional snapshot locations in an optimal way; see Kunisch and Volkwein (ESAIM: M2AN, 44:509–529, 2010). These optimal snapshot locations influences the POD basis functions and therefore the POD reduced-order model. This strategy is used to build up a POD basis on a parameter set in an adaptive way. The approach is illustrated by the construction of the POD reduced-order model for the complex-valued Helmholtz equation.     

A simple adaptive control for full and reduced-order synchronization of uncertain time-varying chaotic systems

In this paper, a simple adaptive feedback control is proposed for full and reduced-order synchronization of time-varying and strictly uncertain chaotic systems. Our method uses only one feedback gain with parameter adaptation law and converges very fast even in the presence of noise. For full synchronization, a drive-response system consisting of two second-order identical parametrically excited oscillators achieve global synchronization; while for reduced-order synchronization, the dynamical evolution of a second-order parametrically driven oscillator is synchronized with the projection of a third-order time-varying chaotic system. The effectiveness of our approach is demonstrated using numerical simulations.     

A mathematical model for the three-dimentional ocean sound propagation

A model is developed mathematically to represent sound propagation in a three-dimensional ocean. The complete development is based on characteristics of the physical environment, mathematical theory, and computational accuracy.While the two-dimentional underwater acoustic wave propagation problem is not yet solved completely for range-dependent environments,three-dimentional environmental effects, such as fronts and eddies, often cannot be neglected. To predict underwater sound propagation, one usually deals with the solution of the Helmholtz (reduced wave) equation. This elliptical equation, along with a set of boundary conditions including a wall condition at the maximum range, forms a well-posed problem, which is pure boundary-value problem. An existing approach to economically solve this three-dimensional range-dependent problem is by means of a two-dimensional parabolic partial differential equation. This parabolic approximation approach, within the limitation of mathematical and acoustical approximations, offers efficient solutions to a class of long-range propagation problems. The parabolic wave equation is much easier to solve than the elliptic equation; one major saving is the removal of the wall boundary condition at the maximum range. The application of the two-dimensional parabolic wave equation to a number of realistic problems has been successful.We discuss the extension of the parabolic equation approach to three-dimensional problems. This paper begins with general considerations of the three-dimensional elliptic wave equation and shows how to transform this equation into parabolic equations which are easier to solve. The development of this paper focuses on wide angle three-dimensional underwater acoustic propagation and accommodates as a special case prevoius developments by other authors. In the course of our development, the physical properties, mathematical validity, and computational accuracy are the primary factors considered. We describe how parabolic wave equations are derived and how wide angle propagation is taken into consideration. Then, a discussion of the limitations and the advantages of the parabolic equation approximation is highlighted. These provide the background for the mathematical formulation of three-dimensional underwater acoustic wave propagation models.Modelling the mathematical solution to three-dimensional underwater acoustic wave propagation involves difficulties both in describing the theoretical acoustics and in performing the large scale computations. We have used the mathematical and physical properties of the problem to simplify considerably. Simplications allow us to introduce a three-dimensional mathematical model for underwater acoustic propagation predictions. Our wide angle three-dimensional parabolic equation model is theoretically justifiable and computationally accurate. This model offers a variety of capabilities to handle a class of long-range propagation problems under acoustical environments with three-dimensional variations.     

On the value function for optimal control problems with infinite horizon

We consider optimal control problems of infinite horizon type, whose control laws are given by L¹_loc‐functions and whose objective function has the meaning of a discounted utility. Our main objective is the verification of the fact that the value function is a viscosity solution of the Hamilton‐Jacobi‐Bellman (HJB) equation in this framework. The usual final condition for the HJB‐equation in the finite horizon case (V (T, x) = 0 or V (T, x) = g(x)) has to be substituted by a decay condition at the infinity. Following the dynamic programming approach, we obtain Bellman's optimality principle and the dynamic programming equation (see (3)). We also prove a regularity result (local Lipschitz continuity) for the value function.     

Stabilization of linear systems with saturation: a Sylvester equation approach

A new methodology of the partial eigenstructure assignmentby state feedback via Sylvester equation is used to obtain asolution of the constrained regulator problem for linear continuous-timeand discrete-time systems by using the reduced-order technique.     

Balanced model reduction of partially observed Langevin equations: an averaging principle

We study balanced model reduction of partially observed stochastic differential equations of Langevin type. Upon balancing, the Langevin equation turns into a singularly perturbed system of equations with slow and fast degrees of freedom. We prove that in the limit of vanishing small Hankel singular values (i.e. for infinite scale separation between fast and slow variables), its solution converges to the solution of a reduced-order Langevin equation. The approach is illustrated with several numerical examples, and we discuss the relation to model reduction of deterministic control systems having an underlying Hamiltonian structure.     

Sufficient optimality conditions for multidimensional optimal control problems with mixed constraints governed by an elliptic PDE

In this article, the idea of a dual dynamic programming is applied to the optimal control problems with multiple integrals governed by a semi-linear elliptic PDE and mixed state-control constraints. The main result called a verification theorem provides the new sufficient conditions for optimality in terms of a solution to the dual equation of a multidimensional dynamic programming. The optimality conditions are also obtained by using the concept of an optimal dual feedback control. Besides seeking the exact minimizers of problems considered some kind of an approximation is given and the sufficient conditions for an approximated optimal pair are derived.     

Control of the Burgers Equation by a Reduced-Order Approach Using Proper Orthogonal Decomposition

Proper orthogonal decomposition (POD) is a method to derive reduced-order models for dynamical systems. In this paper, POD is utilized to solve open-loop and closed-loop optimal control problems for the Burgers equation. The relative simplicity of the equation allows comparison of POD-based algorithms with numerical results obtained from finite-element discretization of the optimality system. For closed-loop control, suboptimal state feedback strategies are presented.     

The galerkin method and feedback control of linear distributed parameter systems

In the development of feedback control theory for distributed parameter systems (DPS), i.e., systems described by partial differential equations, it is important to maintain the finite dimensionality of the controller even though the DPS is infinite dimensional. Since this dimension is directly related to the available on-line computer capacity, it must be finite (and not very large) in order to make the controller implementable from an engineering standpoint. In previous work, it has been our intention to investigate what can be accomplished by finite-dimensional control of infinite-dimensional systems; in particular, we have concentrated on controller design and closed-loop stability. The starting point for all of this is some means for producing a finite-dimensional approximation—a reduced-order model—of the actual DPS. When the “modes” of the DPS are known, the natural candidate for model reduction is projection onto the modal subspace spanned by a finite number of critical modes. Unfortunately, in real engineering systems, these modes are never known exactly and some other reasonable approximation must be used. In this paper, the model reduction is based on the well-known Galerkin procedure. We generate the Galerkin reduced-order model and develop a finite-dimensional controller from it; then we analyze the stability of this controller in closed loop with the actual DPS. Our results indicate conditions under which model reduction based on consistent Galerkin approximations will lead to stable finite-dimensional control.     

Multitime Dynamic Programming for Curvilinear Integral Actions

This paper justifies dynamic programming PDEs for optimal control problems with performance criteria involving curvilinear integrals. The main novel feature, relative to the known theory, is that the multitime dynamic programming PDEs are now connected to the multitime maximum principle. For the first time, an interesting and useful connection between the multitime maximum principle and the multitime dynamic programming is given, characterizing the optimal control by means of a PDE system that may be viewed as a multitime feedback law.     

Reduced-order controllers for unstable multivariable linear systems that can be balanced

This paper considers the problem of H_∞-stabilization of unstable multivariable linear systems. The major features of the approach are: (1) a reduced-order model is obtained using low-frequency balancing, the approximant will have the same number of unstable poles as the original system, (2) the controller design is accomplished by dynamic output feedback, and (3) sufficient conditions in the form of two algebraic Riccati equations and an upper bound explicitly characterize a H_∞-controller of lower dimensions. At the end, an illustrative example is given to show the simplicity of the procedure.     

Reduced-order LQG control of a Timoshenko beam model

We present a computational approach for the construction of reduced-order controllers for the Timoshenko beam model. By means of a space discretization of the Timoshenko equations, we obtain a large-scale, finite-dimensional dynamical system, for which we compute an LQG controller for closed-loop stabilization. The solutions of the algebraic Riccati equations characterizing the LQG controller are then used to construct a balancing transformation which allows the dimensional reduction of the large-scale dynamic compensator. We present numerical tests assessing the stability and performance of the approach.