Approximate solutions to infinite dimensional LQ problems over infinite time horizon

This paper is addressed to develop an approximate method to solve a class of infinite dimensional LQ optimal regulator problems over infinite time horizon. Our algorithm is based on a construction of approximate solutions which solve some finite dimensional LQ optimal regulator problems over finite time horizon, and it is shown that these approximate solutions converge strongly to the desired solution in the double limit sense.     

Analytic solutions for infinite horizon stochastic optimal control problems via finite horizon approximation: A practical guide

We show how infinite horizon stochastic optimal control problems can be solved via studying their finite horizon approximations. This often leads to analytical solutions for the infinite horizon problem by studying phase diagrams, even in cases where the complexity of the finite horizon case does not permit analytic solutions. Our approach can be applied to many problems in dynamic economics.     

Convex infinite horizon programs

We establish conditions under which a sequence of finite horizon convex programs monotonically increases in value to the value of the infinite program; a subsequence of optimal solutions converges to the optimal solution of the infinite problem. If the conditions we impose fail, then (roughtly) the optimal value of the infinite horizon problem is an improper convex function. Under more restrictive conditions we establish the necessary and sufficient conditions for optimality. This constructive procedure gives us a way to solve the infinite (long range) problem by solving a finite (short range) problem. It appears to work well in practice.     

Existence of solutions for a class of nonconcave infinite horizon optimal control problems

In this article, we establish the existence of optimal solutions for a large class of nonconvex infinite horizon discrete-time optimal control problems. This class contains optimal control problems arising in economic dynamics which describe a model with nonconcave utility functions representing the preferences of the planner.     

A stochastic programming approach for planning horizons of infinite horizon capacity planning problems

Planning horizon is a key issue in production planning. Different from previous approaches based on Markov Decision Processes, we study the planning horizon of capacity planning problems within the framework of stochastic programming. We first consider an infinite horizon stochastic capacity planning model involving a single resource, linear cost structure, and discrete distributions for general stochastic cost and demand data (non-Markovian and non-stationary). We give sufficient conditions for the existence of an optimal solution. Furthermore, we study the monotonicity property of the finite horizon approximation of the original problem. We show that, the optimal objective value and solution of the finite horizon approximation problem will converge to the optimal objective value and solution of the infinite horizon problem, when the time horizon goes to infinity. These convergence results, together with the integrality of decision variables, imply the existence of a planning horizon. We also develop a useful formula to calculate an upper bound on the planning horizon. Then by decomposition, we show the existence of a planning horizon for a class of very general stochastic capacity planning problems, which have complicated decision structure.     

General necessary conditions for infinite horizon fractional variational problems

We consider infinite horizon fractional variational problems, where the fractional derivative is defined in the sense of Caputo. Necessary optimality conditions for higher-order variational problems and optimal control problems are obtained. Transversality conditions are obtained in the case state functions are free at the initial time.     

Infinite horizon multiobjective optimal control problems in the discrete time case

This article studies multiobjective optimal control problems in the discrete time framework and in the infinite horizon case. The functions appearing in the problems satisfy smoothness conditions. This article generalizes to the multiobjective case results obtained for single-objective optimal control problems in that framework. The dynamics are governed by difference equations or difference inequations. Necessary conditions of Pareto optimality are presented, namely Pontryagin maximum principles in the weak form and in the strong form. Sufficient conditions are also provided. Other notions of Pareto optimality are defined when the infinite series do not necessarily converge.     

Finite horizon approximations of infinite horizon linear programs

This paper describes the class of infinite horizon linear programs that have finite optimal values. A sequence of finite horizon (T period) problems is shown to approximate the infinite horizon problems in the following sense: the optimal values of theT period problems converge monotonically to the optimal value of the infinite problem and the limit of any convergent subsequence of initialT period optimal decisions is an optimal decision for the infinite horizon problem.     

Existence of efficient solutions in infinite horizon optimization under continuous and discrete controls

For time-varying deterministic infinite horizon control problems, we provide conditions for the existence of efficient solutions, i.e., solutions which are optimal to each of the states through which they pass. A sufficient condition is that the mappings from controls to states be open. Applications to production planning are considered.     

Degeneracy in infinite horizon optimization

We consider sequential decision problems over an infinite horizon. The forecast or solution horizon approach to solving such problems requires that the optimal initial decision be unique. We show that multiple optimal initial decisions can exist in general and refer to their existence as degeneracy. We then present a conceptual cost perturbation algorithm for resolving degeneracy and identifying a forecast horizon. We also present a general near-optimal forecast horizon.This material is based on work supported by the National Science Foundation under Grants ECS-8409682 and ECS-8700836.     

Risk-sensitive portfolio optimization on infinite time horizon

We consider a continuous time portfolio optimization problems on an infinite time horizon for a factor model, recently treated by Bielecki and Pliska ["Risk-sensitive dynamic asset management", Appl. Math. Optim. , 39 (1990) 337-360], where the mean returns of individual securities or asset categories are explicitly affected by economic factors. The factors are assumed to be Gaussian processes. We see new features in constructing optimal strategies for risk-sensitive criteria of the portfolio optimization on an infinite time horizon, which are obtained from the solutions of matrix Riccati equations.     

Pontryagin Type Maximum Principle for budget-constrained infinite horizon optimal control problems with linear dynamics

In this paper a class of infinite horizon optimal control problems with an isoperimetrical constraint, also interpreted as a budget constraint, is considered. Herein a linear both in the state and in the control dynamic is allowed. The problem setting includes a weighted Sobolev space as the state space. For this class of problems, we establish the necessary optimality conditions in form of a Pontryagin Type Maximum Principle including a transversality condition. The proved theoretical result is applied to a linear–quadratic regulator problem.     

On the rate of convergence of infinite horizon discounted optimal value functions

In this paper we investigate the rate of convergence of the optimal value function of an infinite horizon discounted optimal control problem as the discount rate tends to zero. Using the Integration Theorem for Laplace transformations we provide conditions on averaged functionals along suitable trajectories yielding quadratic pointwise convergence. From this we derive under appropriate controllability conditions criteria for linear uniform convergence of the value functions on control sets. Applications of these results are given and an example is discussed in which both linear and slower rates of convergence occur depending on the cost functional.     

Necessity of limiting co-state arcs in Bolza-type infinite horizon problem

We investigate necessary conditions of optimality for the Bolza-type infinite horizon problem with free right end. The optimality is understood in the sense of weakly uniformly overtaking optimal control. No previous knowledge in the asymptotic behaviour of trajectories or adjoint variables is necessary. Following Seierstad’s idea, we obtain the necessary boundary condition at infinity in the form of a transversality condition for the maximum principle. Those transversality conditions may be expressed in the integral form through an Aseev–Kryazhimskii-type formulae for co-state arcs. The connection between these formulae and limiting gradients of pay-off function at infinity is identified; several conditions under which it is possible to explicitly specify the co-state arc through those Aseev–Kryazhimskii-type formulae are found. For infinite horizon problem of Bolza type, an example is given to clarify the use of the Aseev–Kryazhimskii formula as an explicit expression of the co-state arc.     

Risk-sensitive stopping problems for continuous-time Markov chains

In this paper we consider stopping problems for continuous-time Markov chains under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. More precisely our aim is to maximize the certainty equivalent of the stopping reward minus cost over the time horizon. We derive optimality equations for the value functions and prove the existence of optimal stopping times. The exponential utility is treated as a special case. In contrast to risk-neutral stopping problems it may be optimal to stop between jumps of the Markov chain. We briefly discuss the influence of the risk sensitivity on the optimal stopping time and consider a special house selling problem as an example.     

Conditions for the discovery of solution horizons

We present necessary and sufficient conditions for discrete infinite horizon optimization problems with unique solutions to be solvable. These problems can be equivalently viewed as the task of finding a shortest path in an infinite directed network. We provide general forward algorithms with stopping rules for their solution. The key condition required is that of weak reachability, which roughly requires that for any sequence of nodes or states, it must be possible from optimal states to reach states close in cost to states along this sequence. Moreover the costs to reach these states must converge to zero. Applications are considered in optimal search, undiscounted Markov decision processes, and deterministic infinite horizon optimization.This work was supported in part by NSF Grant ECS-8700836 to The University of Michigan.     

Well-posed infinite horizon variational problems on a compact manifold

We give an effective sufficient condition for a variational problem with infinite horizon on a compact Riemannian manifold M to admit a smooth optimal synthesis, i.e., a smooth dynamical system on M whose positive semi-trajectories are solutions to the problem. To realize the synthesis, we construct an invariant Lagrangian submanifold (well-projected to M) of the flow of extremals in the cotangent bundle T*M. The construction uses the curvature of the flow in the cotangent bundle and some ideas of hyperbolic dynamics.     

Mean-field stochastic control with elephant memory in finite and infinite time horizon

ABSTRACT

Our purpose of this paper is to study stochastic control problems for systems driven by mean-field stochastic differential equations with elephant memory, in the sense that the system (like the elephants) never forgets its history. We study both the finite horizon case and the infinite time horizon case.

• In the finite horizon case, results about existence and uniqueness of solutions of such a system are given. Moreover, we prove sufficient as well as necessary stochastic maximum principles for the optimal control of such systems. We apply our results to solve a mean-field linear quadratic control problem.

• For infinite horizon, we derive sufficient and necessary maximum principles.

  As an illustration, we solve an optimal consumption problem from a cash flow modelled by an elephant memory mean-field system.

     

On the value function for optimal control problems with infinite horizon

We consider optimal control problems of infinite horizon type, whose control laws are given by L¹_loc‐functions and whose objective function has the meaning of a discounted utility. Our main objective is the verification of the fact that the value function is a viscosity solution of the Hamilton‐Jacobi‐Bellman (HJB) equation in this framework. The usual final condition for the HJB‐equation in the finite horizon case (V (T, x) = 0 or V (T, x) = g(x)) has to be substituted by a decay condition at the infinity. Following the dynamic programming approach, we obtain Bellman's optimality principle and the dynamic programming equation (see (3)). We also prove a regularity result (local Lipschitz continuity) for the value function.     

Algorithms for solving discrete optimal control problems with infinite time horizon and determining minimal mean cost cycles in a directed graph as decision support tool

Time-discrete systems with a finite set of states are considered. Discrete optimal control problems with infinite time horizon for such systems are formulated. We introduce a certain graph-theoretic structure to model the transitions of the dynamical system. Algorithms for finding the optimal stationary control parameters are presented. Furthermore, we determine the optimal mean cost cycles. This approach can be used as a decision support strategy within such a class of problems; especially so-called multilayered decision problems which occur within environmental emission trading procedures can be modelled by such an approach.