A time-domain decomposition iterative method for the solution of distributed linear quadratic optimal control problems

We study a class of time-domain decomposition-based methods for the numerical solution of large-scale linear quadratic optimal control problems. Our methods are based on a multiple shooting reformulation of the linear quadratic optimal control problem as a discrete-time optimal control (DTOC) problem. The optimality conditions for this DTOC problem lead to a linear block tridiagonal system. The diagonal blocks are invertible and are related to the original linear quadratic optimal control problem restricted to smaller time-subintervals. This motivates the application of block Gauss–Seidel (GS)-type methods for the solution of the block tridiagonal systems. Numerical experiments show that the spectral radii of the block GS iteration matrices are larger than one for typical applications, but that the eigenvalues of the iteration matrices decay to zero fast. Hence, while the GS method is not expected to convergence for typical applications, it can be effective as a preconditioner for Krylov-subspace methods. This is confirmed by our numerical tests.A byproduct of this research is the insight that certain instantaneous control techniques can be viewed as the application of one step of the forward block GS method applied to the DTOC optimality system.     

The achievable region method in the optimal control of queueing systems; formulations,bounds and policies

We survey a new approach that the author and his co-workers have developed to formulate stochastic control problems (predominantly queueing systems) asmathematical programming problems. The central idea is to characterize the region of achievable performance in a stochastic control problem, i.e., find linear or nonlinear constraints on the performance vectors that all policies satisfy. We present linear and nonlinear relaxations of the performance space for the following problems: Indexable systems (multiclass single station queues and multiarmed bandit problems), restless bandit problems, polling systems, multiclass queueing and loss networks. These relaxations lead to bounds on the performance of an optimal policy. Using information from the relaxations we construct heuristic nearly optimal policies. The theme in the paper is the thesis that better formulations lead to deeper understanding and better solution methods. Overall the proposed approach for stochastic control problems parallels efforts of the mathematical programming community in the last twenty years to develop sharper formulations (polyhedral combinatorics and more recently nonlinear relaxations) and leads to new insights ranging from a complete characterization and new algorithms for indexable systems to tight lower bounds and nearly optimal algorithms for restless bandit problems, polling systems, multiclass queueing and loss networks.     

Numerical solution of a minimax ergodic optimal control problem

In this work we consider an L^∞ minimax ergodic optimal control problem with cumulative cost. We approximate the cost function as a limit of evolutions problems. We present the associated Hamilton-Jacobi-Bellman equation and we prove that it has a unique solution in the viscosity sense. As this HJB equation is consistent with a numerical procedure, we use this discretization to obtain a procedure for the primitive problem. For the numerical solution of the ergodic version we need a perturbation of the instantaneous cost function. We give an appropriate selection of the discretization and penalization parameters to obtain discrete solutions that converge to the optimal cost. We present numerical results. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Direct methods with maximal lower bound for mixed-integer optimal control problems

Many practical optimal control problems include discrete decisions. These may be either time-independent parameters or time-dependent control functions as gears or valves that can only take discrete values at any given time. While great progress has been achieved in the solution of optimization problems involving integer variables, in particular mixed-integer linear programs, as well as in continuous optimal control problems, the combination of the two is yet an open field of research. We consider the question of lower bounds that can be obtained by a relaxation of the integer requirements. For general nonlinear mixed-integer programs such lower bounds typically suffer from a huge integer gap. We convexify (with respect to binary controls) and relax the original problem and prove that the optimal solution of this continuous control problem yields the best lower bound for the nonlinear integer problem. Building on this theoretical result we present a novel algorithm to solve mixed-integer optimal control problems, with a focus on discrete-valued control functions. Our algorithm is based on the direct multiple shooting method, an adaptive refinement of the underlying control discretization grid and tailored heuristic integer methods. Its applicability is shown by a challenging application, the energy optimal control of a subway train with discrete gears and velocity limits.      

Optimization Over the Efficient Set of Multi-objective Convex Optimal Control Problems

We consider multi-objective convex optimal control problems. First we state a relationship between the (weakly or properly) efficient set of the multi-objective problem and the solution of the problem scalarized via a convex combination of objectives through a vector of parameters (or weights). Then we establish that (i) the solution of the scalarized (parametric) problem for any given parameter vector is unique and (weakly or properly) efficient and (ii) for each solution in the (weakly or properly) efficient set, there exists at least one corresponding parameter vector for the scalarized problem yielding the same solution. Therefore the set of all parametric solutions (obtained by solving the scalarized problem) is equal to the efficient set. Next we consider an additional objective over the efficient set. Based on the main result, the new objective can instead be considered over the (parametric) solution set of the scalarized problem. For the purpose of constructing numerical methods, we point to existing solution differentiability results for parametric optimal control problems. We propose numerical methods and give an example application to illustrate our approach.     

Regularization methods for optimization problems with probabilistic constraints

We analyze nonlinear stochastic optimization problems with probabilistic constraints on nonlinear inequalities with random right hand sides. We develop two numerical methods with regularization for their numerical solution. The methods are based on first order optimality conditions and successive inner approximations of the feasible set by progressive generation of p-efficient points. The algorithms yield an optimal solution for problems involving α-concave probability distributions. For arbitrary distributions, the algorithms solve the convex hull problem and provide upper and lower bounds for the optimal value and nearly optimal solutions. The methods are compared numerically to two cutting plane methods.     

Telecommunication Link Restoration Planning with Multiple Facility Types

To ensure uninterrupted service, telecommunication networks contain excess (spare) capacity for rerouting (restoring) traffic in the event of a link failure. We study the NP-hard capacity planning problem of economically installing spare capacity on a network to permit link restoration of steady-state traffic. We present a planning model that incorporates multiple facility types, and develop optimization-based heuristic solution methods based on solving a linear programming relaxation and minimum cost network flow subproblems. We establish bounds on the performance of the algorithms, and discuss problem instances that nearly achieve these worst-case bounds. In tests on three real-world problems and numerous randomly-generated problems containing up to 50 nodes and 150 edges, the heuristics provide good solutions (often within 0.5% of optimality) to problems with single facility type, in equivalent or less time than methods from the literature. For multi-facility problems, the gap between our heuristic solution values and the linear programming bounds are larger. However, for small graphs, we show that the optimal linear programming value does not provide a tight bound on the optimal integer value, and our heuristic solutions are closer to optimality than implied by the gaps.     

A technique for speeding up the solution of the Lagrangean dual

We propose techniques for the solution of the LP relaxation and the Lagrangean dual in combinatorial optimization and nonlinear programming problems. Our techniques find the optimal solution value and the optimal dual multipliers of the LP relaxation and the Lagrangean dual in polynomial time using as a subroutine either the Ellipsoid algorithm or the recent algorithm of Vaidya. Moreover, in problems of a certain structure our techniques find not only the optimal solution value, but the solution as well. Our techniques lead to significant improvements in the theoretical running time compared with previously known methods (interior point methods, Ellipsoid algorithm, Vaidya's algorithm). We use our method to the solution of the LP relaxation and the Langrangean dual of several classical combinatorial problems, like the traveling salesman problem, the vehicle routing problem, the Steiner tree problem, thek-connected problem, multicommodity flows, network design problems, network flow problems with side constraints, facility location problems,K-polymatroid intersection, multiple item capacitated lot sizing problem, and stochastic programming. In all these problems our techniques significantly improve the theoretical running time and yield the fastest way to solve them.     

A fast heuristic method for minimizing traffic congestion on reconfigurable ring topologies

We describe the development of fast heuristics and methodologies for congestion minimization problems in directional wireless networks, and we compare their performance with optimal solutions. The focus is on the network layer topology control problem (NLTCP) defined by selecting an optimal ring topology as well as the flows on it. Solutions to NLTCP need to be computed in near realtime due to changing weather and other transient conditions and which generally preclude traditional optimization strategies. Using a mixed-integer linear programming formulation, we present both new constraints for this problem and fast heuristics to solve it. The new constraints are used to increase the lower bound from the linear programming relaxation and hence speed up the solution of the optimization problem by branch and bound. The upper and lower bounds for the optimal objective function to the mixed integer problem then serve to evaluate new node-swapping heuristics which we also present. Through a series of tests on different sized networks with different traffic demands, we show that our new heuristics achieve within about 0.5% of the optimal value within seconds.     

A spatial domain decomposition method for parabolic optimal control problems

We present a non-overlapping spatial domain decomposition method for the solution of linear–quadratic parabolic optimal control problems. The spatial domain is decomposed into non-overlapping subdomains. The original parabolic optimal control problem is decomposed into smaller problems posed on space–time cylinder subdomains with auxiliary state and adjoint variables imposed as Dirichlet boundary conditions on the space–time interface boundary. The subdomain problems are coupled through Robin transmission conditions. This leads to a Schur complement equation in which the unknowns are the auxiliary state adjoint variables on the space-time interface boundary. The Schur complement operator is the sum of space–time subdomain Schur complement operators. The application of these subdomain Schur complement operators is equivalent to the solution of an subdomain parabolic optimal control problem. The subdomain Schur complement operators are shown to be invertible and the application of their inverses is equivalent to the solution of a related subdomain parabolic optimal control problem. We introduce a new family of Neumann–Neumann type preconditioners for the Schur complement system including several different coarse grid corrections. We compare the numerical performance of our preconditioners with an alternative approach recently introduced by Benamou.     

Distributed optimal control of the viscous Burgers equation via a Legendre pseudo‐spectral approach

This paper presents a computational technique based on the pseudo‐spectral method for the solution of distributed optimal control problem for the viscous Burgers equation. By using pseudo‐spectral method, the problem is converted to a classical optimal control problem governed by a system of ordinary differential equations, which can be solved by well‐developed direct or indirect methods. For solving the resulting optimal control problem, we present an indirect method by deriving and numerically solving the first‐order optimality conditions. Numerical tests involving both unconstrained and constrained control problems are considered. Copyright © 2015 John Wiley & Sons, Ltd.     

Analysis and Computational Methods of Dirichlet Boundary Optimal Control Problems for 2D Boussinesq Equations

We study Dirichlet boundary optimal control problems for 2D Boussinesq equations. The existence of the solution of the optimization problem is proved and an optimality system of partial differential equations is derived from which optimal controls and states may be determined. Then, we present some computational methods to get the solution of the optimality system. The iterative algorithms are given explicitly. We also prove the convergence of the gradient algorithm.     

Solving some optimal control problems using the barrier penalty function method

In this paper we present a new approach to solve a two-level optimization problem arising from an approximation by means of the finite element method of optimal control problems governed by unilateral boundary-value problems. The problem considered is to find a minimum of a functional with respect to the control variablesu. The minimized functional depends on control variables and state variablesx. The latter are the optimal solution of an auxiliary quadratic programming problem, whose parameters depend onu.Our main idea is to replace this QP problem by its dual and then apply the barrier penalty method to this dual QP problem or to the primal one if it is in an appropriate form. As a result we obtain a problem approximating the original one. Its good property is the differentiable dependence of state variables with respect to the control variables. Furthermore, we propose a method for finding an approximate solution of a penalized lower-level problem if the optimal solution of the original QP problem is known. We apply the result obtained to some optimal shape design problems governed by the Dirichlet-Signorini boundary-value problem.This research was supported by the Academy of Finland and the Systems Research Institute of the Polish Academy of Sciences.     

A generalized structured doubling algorithm for the numerical solution of linear quadratic optimal control problems

We propose a generalization of the structured doubling algorithm to compute invariant subspaces of structured matrix pencils that arise in the context of solving linear quadratic optimal control problems. The new algorithm is designed to attain better accuracy when the classical Riccati equation approach for the solution of the optimal control problem is not well suited because the stable and unstable invariant subspaces are not well separated (because of eigenvalues near or on the imaginary axis) or in the case when the Riccati solution does not exist at all. We analyze the convergence of the method and compare the new method with the classical structured doubling algorithm as well as some structured QR methods. Copyright © 2012 John Wiley & Sons, Ltd.     

Optimal control policies for resource allocation in an activity network

In the paper a class of project-scheduling problems concerning the allocation of continuously divisible resources is considered. It is assumed that performing speeds of activities are continuous functions of the resource amount, and that the initial and terminal states of activities are known. For such mathematical models of project activities the problem of time-optimal resource allocation under instantaneous and integral constrains on a resource, and the problem of cost-optimal resource allocation with fixed project duration are formulated and a general solution concept is proposed. Necessary and sufficient conditions for the existence of a solution in particular cases are derived and properties of optimal schedules are given. The control policies for resource allocation are constructed for the example of the cost-optimal problem.     

Numerical methods for solving terminal optimal control problems

Numerical methods for solving optimal control problems with equality constraints at the right end of the trajectory are discussed. Algorithms for optimal control search are proposed that are based on the multimethod technique for finding an approximate solution of prescribed accuracy that satisfies terminal conditions. High accuracy is achieved by applying a second-order method analogous to Newton’s method or Bellman’s quasilinearization method. In the solution of problems with direct control constraints, the variation of the control is computed using a finite-dimensional approximation of an auxiliary problem, which is solved by applying linear programming methods.     

Investment in electricity networks with transmission switching

We consider the application of Dantzig-Wolfe decomposition to stochastic integer programming problems arising in the capacity planning of electricity transmission networks that have some switchable transmission elements. The decomposition enables a column-generation algorithm to be applied, which allows the solution of large problem instances. The methodology is illustrated by its application to a problem of determining the optimal investment in switching equipment and transmission capacity for an existing network. Computational tests on IEEE test networks with 73 nodes and 118 nodes confirm the efficiency of the approach.     

Optimal discrete-valued control computation

In this paper, we consider an optimal control problem in which the control takes values from a discrete set and the state and control are subject to continuous inequality constraints. By introducing auxiliary controls and applying a time-scaling transformation, we transform this optimal control problem into an equivalent problem subject to additional linear and quadratic constraints. The feasible region defined by these additional constraints is disconnected, and thus standard optimization methods struggle to handle these constraints. We introduce a novel exact penalty function to penalize constraint violations, and then append this penalty function to the objective. This leads to an approximate optimal control problem that can be solved using standard software packages such as MISER. Convergence results show that when the penalty parameter is sufficiently large, any local solution of the approximate problem is also a local solution of the original problem. We conclude the paper with some numerical results for two difficult train control problems.     

Optimizing bandwidth allocation in elastic optical networks with application to scheduling

We study a problem of optimal bandwidth allocation in the elastic optical networks technology, where usable frequency intervals are of variable width. In this setting, each lightpath has a lower and upper bound on the width of its frequency interval, as well as an associated profit, and we seek a bandwidth assignment that maximizes the total profit. This problem is known to be NP-complete. We strengthen this result by showing that, in fact, the problem is inapproximable within any constant ratio even on a path network. We further derive NP-hardness results and present approximation algorithms for several special cases of the path and ring networks, which are of practical interest. Finally, while in general our problem is hard to approximate, we show that an optimal solution can be obtained by allowing resource augmentation. Some of our results resolve open problems posed by Shalom et al. (2013) [28]. Our study has applications also in real-time scheduling.     

On the optimal control of the Schlögl-model

Optimal control problems for a class of 1D semilinear parabolic equations with cubic nonlinearity are considered. This class is also known as the Schlögl model. Main emphasis is laid on the control of traveling wave fronts that appear as typical solutions to the state equation. The well-posedness of the optimal control problem and the regularity of its solution are proved. First-order necessary optimality conditions are established by standard adjoint calculus. The state equation is solved by the implicit Euler method in time and a finite element technique with respect to the spatial variable. Moreover, model reduction by Proper Orthogonal Decomposition is applied and compared with the numerical solution of the full problem. To solve the optimal control problems numerically, the performance of different versions of the nonlinear conjugate gradient method is studied. Various numerical examples demonstrate the capacities and limits of optimal control methods.