Limit of the solutions for the finite horizon problems as the optimal solution to the infinite horizon optimization problems

In this paper, we consider how to construct the optimal solutions for the undiscounted discrete time infinite horizon optimization problems. We present the conditions under which the limit of the solutions for the finite horizon problems is optimal among all attainable paths for the infinite horizon problem under two modified overtaking criteria, as well as the conditions under which it is the unique optimum under the sum-of-utilities criterion. The results are applied to a parametric example of a simple one-sector growth model to examine the impacts of discounting on the optimal path.     

Finite horizon approximations of infinite horizon linear programs

This paper describes the class of infinite horizon linear programs that have finite optimal values. A sequence of finite horizon (T period) problems is shown to approximate the infinite horizon problems in the following sense: the optimal values of theT period problems converge monotonically to the optimal value of the infinite problem and the limit of any convergent subsequence of initialT period optimal decisions is an optimal decision for the infinite horizon problem.     

Convex infinite horizon programs

We establish conditions under which a sequence of finite horizon convex programs monotonically increases in value to the value of the infinite program; a subsequence of optimal solutions converges to the optimal solution of the infinite problem. If the conditions we impose fail, then (roughtly) the optimal value of the infinite horizon problem is an improper convex function. Under more restrictive conditions we establish the necessary and sufficient conditions for optimality. This constructive procedure gives us a way to solve the infinite (long range) problem by solving a finite (short range) problem. It appears to work well in practice.     

Semi-Infinite Programming Approach to Continuously-Constrained Linear-Quadratic Optimal Control Problems

Consider the class of linear-quadratic (LQ) optimal control problems with continuous linear state constraints, that is, constraints imposed on every instant of the time horizon. This class of problems is known to be difficult to solve numerically. In this paper, a computational method based on a semi-infinite programming approach is given. The LQ optimal control problem is formulated as a positive-quadratic infinite programming problem. This can be done by considering the control as the decision variable, while taking the state as a function of the control. After parametrizing the decision variable, an approximate quadratic semi-infinite programming problem is obtained. It is shown that, as we refine the parametrization, the solution sequence of the approximate problems converges to the solution of the infinite programming problem (hence, to the solution of the original optimal control problem). Numerically, the semi-infinite programming problems obtained above can be solved efficiently using an algorithm based on a dual parametrization method.     

Duality in infinite dimensional linear programming

We consider the class of linear programs with infinitely many variables and constraints having the property that every constraint contains at most finitely many variables while every variable appears in at most finitely many constraints. Examples include production planning and equipment replacement over an infinite horizon. We form the natural dual linear programming problem and prove strong duality under a transversality condition that dual prices are asymptotically zero. That is, we show, under this transversality condition, that optimal solutions are attained in both primal and dual problems and their optimal values are equal. The transversality condition, and hence strong duality, is established for an infinite horizon production planning problem.This material is based on work supported by the National Science Foundation under Grant No. ECS-8700836.     

Quadratic Control for Stochastic Systems Defined by Evolution Operators and Square Integrable Martingales

The infinite dimensional version of the linear quadratic cost control problem is studied by Curtain and Pritchard [2], Gibson [5] by using Riccati integral equations, instead of differential equations. In the present paper the corresponding stochastic case over a finite horizon is considered. The stochastic perturbations are given by Hilbert valued square integrable martingales and it is shown that the deterministic optimal feedback control is also optimal in the stochastic case. Sufficient conditions are given for the convergence of approximate solutions of optimal control problems.     

Gradient-flow approach for computing a nonlinear-quadratic optimal-output feedback gain matrix

In this paper, an approach is proposed for solving a nonlinear-quadratic optimal regulator problem with linear static state feedback and infinite planning horizon. For such a problem, approximate problems are introduced and considered, which are obtained by combining a finite-horizon problem with an infinite-horizon linear problem in a certain way. A gradient-flow based algorithm is derived for these approximate problems. It is shown that an optimal solution to the original problem can be found as the limit of a sequence of solutions to the approximate problems. Several important properties are obtained. For illustration, two numerical examples are presented.This project was partially supported by a research grant from the Australian Research Council.     

Existence of solutions for a class of nonconcave infinite horizon optimal control problems

In this article, we establish the existence of optimal solutions for a large class of nonconvex infinite horizon discrete-time optimal control problems. This class contains optimal control problems arising in economic dynamics which describe a model with nonconcave utility functions representing the preferences of the planner.     

Analytic solutions for infinite horizon stochastic optimal control problems via finite horizon approximation: A practical guide

We show how infinite horizon stochastic optimal control problems can be solved via studying their finite horizon approximations. This often leads to analytical solutions for the infinite horizon problem by studying phase diagrams, even in cases where the complexity of the finite horizon case does not permit analytic solutions. Our approach can be applied to many problems in dynamic economics.     

Degeneracy in infinite horizon optimization

We consider sequential decision problems over an infinite horizon. The forecast or solution horizon approach to solving such problems requires that the optimal initial decision be unique. We show that multiple optimal initial decisions can exist in general and refer to their existence as degeneracy. We then present a conceptual cost perturbation algorithm for resolving degeneracy and identifying a forecast horizon. We also present a general near-optimal forecast horizon.This material is based on work supported by the National Science Foundation under Grants ECS-8409682 and ECS-8700836.     

Mean-field stochastic control with elephant memory in finite and infinite time horizon

ABSTRACT

Our purpose of this paper is to study stochastic control problems for systems driven by mean-field stochastic differential equations with elephant memory, in the sense that the system (like the elephants) never forgets its history. We study both the finite horizon case and the infinite time horizon case.

• In the finite horizon case, results about existence and uniqueness of solutions of such a system are given. Moreover, we prove sufficient as well as necessary stochastic maximum principles for the optimal control of such systems. We apply our results to solve a mean-field linear quadratic control problem.

• For infinite horizon, we derive sufficient and necessary maximum principles.

  As an illustration, we solve an optimal consumption problem from a cash flow modelled by an elephant memory mean-field system.

     

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.     

Duality and uniqueness of convex solutions to stationary Hamilton-Jacobi equations

Value functions for convex optimal control problems on infinite time intervals are studied in the framework of duality. Hamilton-Jacobi characterizations and the conjugacy of primal and dual value functions are of main interest. Close ties between the uniqueness of convex solutions to a Hamilton-Jacobi equation, the uniqueness of such solutions to a dual Hamilton-Jacobi equation, and the conjugacy of primal and dual value functions are displayed. Simultaneous approximation of primal and dual infinite horizon problems with a pair of dual problems on finite horizon, for which the value functions are conjugate, leads to sufficient conditions on the conjugacy of the infinite time horizon value functions. Consequently, uniqueness results for the Hamilton-Jacobi equation are established. Little regularity is assumed on the cost functions in the control problems, correspondingly, the Hamiltonians need not display any strict convexity and may have several saddle points.

     

Algorithms for solving discrete optimal control problems with infinite time horizon and determining minimal mean cost cycles in a directed graph as decision support tool

Time-discrete systems with a finite set of states are considered. Discrete optimal control problems with infinite time horizon for such systems are formulated. We introduce a certain graph-theoretic structure to model the transitions of the dynamical system. Algorithms for finding the optimal stationary control parameters are presented. Furthermore, we determine the optimal mean cost cycles. This approach can be used as a decision support strategy within such a class of problems; especially so-called multilayered decision problems which occur within environmental emission trading procedures can be modelled by such an approach.     

A stochastic programming approach for planning horizons of infinite horizon capacity planning problems

Planning horizon is a key issue in production planning. Different from previous approaches based on Markov Decision Processes, we study the planning horizon of capacity planning problems within the framework of stochastic programming. We first consider an infinite horizon stochastic capacity planning model involving a single resource, linear cost structure, and discrete distributions for general stochastic cost and demand data (non-Markovian and non-stationary). We give sufficient conditions for the existence of an optimal solution. Furthermore, we study the monotonicity property of the finite horizon approximation of the original problem. We show that, the optimal objective value and solution of the finite horizon approximation problem will converge to the optimal objective value and solution of the infinite horizon problem, when the time horizon goes to infinity. These convergence results, together with the integrality of decision variables, imply the existence of a planning horizon. We also develop a useful formula to calculate an upper bound on the planning horizon. Then by decomposition, we show the existence of a planning horizon for a class of very general stochastic capacity planning problems, which have complicated decision structure.     

Conditions for the discovery of solution horizons

We present necessary and sufficient conditions for discrete infinite horizon optimization problems with unique solutions to be solvable. These problems can be equivalently viewed as the task of finding a shortest path in an infinite directed network. We provide general forward algorithms with stopping rules for their solution. The key condition required is that of weak reachability, which roughly requires that for any sequence of nodes or states, it must be possible from optimal states to reach states close in cost to states along this sequence. Moreover the costs to reach these states must converge to zero. Applications are considered in optimal search, undiscounted Markov decision processes, and deterministic infinite horizon optimization.This work was supported in part by NSF Grant ECS-8700836 to The University of Michigan.     

Optimal solution approximation for infinite positive-definite quadratic programming

We consider a general doubly-infinite, positive-definite, quadratic programming problem. We show that the sequence of unique optimal solutions to the natural finite-dimensional subproblems strongly converges to the unique optimal solution. This offers the opportunity to arbitrarily well approximate the infinite-dimensional optimal solution by numerically solving a sufficiently large finite-dimensional version of the problem. We then apply our results to a general time-varying, infinite-horizon, positive-definite, LQ control problem.This work was supported in part by the National Science Foundation under Grants ECS-8700836, DDM-9202849, and DDM-9214894.     

Approximate stable solutions in infinite tree problems

In this article infinite minimum cost spanning tree situations and related TU-games are studied. Since an optimal tree for these situations does not always exist, it is impossible in such situations to implement core solutions based on a particular tree. Therefore, we study core-like solutions and deal with three cases concerning the total cost of connection: when it is zero, when it is finite but larger than zero, and when it is infinite.     

Risk-sensitive portfolio optimization on infinite time horizon

We consider a continuous time portfolio optimization problems on an infinite time horizon for a factor model, recently treated by Bielecki and Pliska ["Risk-sensitive dynamic asset management", Appl. Math. Optim. , 39 (1990) 337-360], where the mean returns of individual securities or asset categories are explicitly affected by economic factors. The factors are assumed to be Gaussian processes. We see new features in constructing optimal strategies for risk-sensitive criteria of the portfolio optimization on an infinite time horizon, which are obtained from the solutions of matrix Riccati equations.     

Solutions for a class of optimal control problems with time delay,part 1

The optimal control of a system whose state is governed by a nonlinear autonomous Volterra integrodifferential equation with unbounded time interval is considered. Specifically, it is assumed that the delay occurs only in the state variable. We are concerned with the existence of an overtaking optimal trajectory over an infinite horizon. The existence result that we obtain extends the result of Carlson (Ref. 1) to a situation where the trajectories are not necessary bounded. Also, we study the structure of approximate solutions for the problem on a finite interval.The author thanks A. Leizarowitz for fruitful discussions.