Approximate solutions to infinite dimensional LQ problems over infinite time horizon

This paper is addressed to develop an approximate method to solve a class of infinite dimensional LQ optimal regulator problems over infinite time horizon. Our algorithm is based on a construction of approximate solutions which solve some finite dimensional LQ optimal regulator problems over finite time horizon, and it is shown that these approximate solutions converge strongly to the desired solution in the double limit sense.     

Finite horizon approximations of infinite horizon linear programs

This paper describes the class of infinite horizon linear programs that have finite optimal values. A sequence of finite horizon (T period) problems is shown to approximate the infinite horizon problems in the following sense: the optimal values of theT period problems converge monotonically to the optimal value of the infinite problem and the limit of any convergent subsequence of initialT period optimal decisions is an optimal decision for the infinite horizon problem.     

Limit of the solutions for the finite horizon problems as the optimal solution to the infinite horizon optimization problems

In this paper, we consider how to construct the optimal solutions for the undiscounted discrete time infinite horizon optimization problems. We present the conditions under which the limit of the solutions for the finite horizon problems is optimal among all attainable paths for the infinite horizon problem under two modified overtaking criteria, as well as the conditions under which it is the unique optimum under the sum-of-utilities criterion. The results are applied to a parametric example of a simple one-sector growth model to examine the impacts of discounting on the optimal path.     

Convex infinite horizon programs

We establish conditions under which a sequence of finite horizon convex programs monotonically increases in value to the value of the infinite program; a subsequence of optimal solutions converges to the optimal solution of the infinite problem. If the conditions we impose fail, then (roughtly) the optimal value of the infinite horizon problem is an improper convex function. Under more restrictive conditions we establish the necessary and sufficient conditions for optimality. This constructive procedure gives us a way to solve the infinite (long range) problem by solving a finite (short range) problem. It appears to work well in practice.     

Decision/forecast horizon for a single facility dynamic location/relocation problem

Procedures to solve finite horizon dynamic location/relocation problems have been reported in the literature by many authors. This paper provides several decision/forecast horizon results for a single facility dynamic location/relocation problem; these results are helpful in finding optimal initial decisions for the infinite horizon problem by using information only for a finite horizon.     

Stationary solutions of SPDEs and infinite horizon BDSDEs

In this paper we study the existence of stationary solutions for stochastic partial differential equations. We establish a new connection between valued solutions of backward doubly stochastic differential equations (BDSDEs) on infinite horizon and the stationary solutions of the SPDEs. Moreover, we prove the existence and uniqueness of the solutions of BDSDEs on both finite and infinite horizons, so obtain the solutions of initial value problems and the stationary solutions (independent of any initial value) of SPDEs. The connection of the weak solutions of SPDEs and BDSDEs has independent interests in the areas of both SPDEs and BSDEs.     

Mean-field stochastic control with elephant memory in finite and infinite time horizon

ABSTRACT

Our purpose of this paper is to study stochastic control problems for systems driven by mean-field stochastic differential equations with elephant memory, in the sense that the system (like the elephants) never forgets its history. We study both the finite horizon case and the infinite time horizon case.

• In the finite horizon case, results about existence and uniqueness of solutions of such a system are given. Moreover, we prove sufficient as well as necessary stochastic maximum principles for the optimal control of such systems. We apply our results to solve a mean-field linear quadratic control problem.

• For infinite horizon, we derive sufficient and necessary maximum principles.

  As an illustration, we solve an optimal consumption problem from a cash flow modelled by an elephant memory mean-field system.

     

Risk-sensitive portfolio optimization on infinite time horizon

We consider a continuous time portfolio optimization problems on an infinite time horizon for a factor model, recently treated by Bielecki and Pliska ["Risk-sensitive dynamic asset management", Appl. Math. Optim. , 39 (1990) 337-360], where the mean returns of individual securities or asset categories are explicitly affected by economic factors. The factors are assumed to be Gaussian processes. We see new features in constructing optimal strategies for risk-sensitive criteria of the portfolio optimization on an infinite time horizon, which are obtained from the solutions of matrix Riccati equations.     

Degeneracy in infinite horizon optimization

We consider sequential decision problems over an infinite horizon. The forecast or solution horizon approach to solving such problems requires that the optimal initial decision be unique. We show that multiple optimal initial decisions can exist in general and refer to their existence as degeneracy. We then present a conceptual cost perturbation algorithm for resolving degeneracy and identifying a forecast horizon. We also present a general near-optimal forecast horizon.This material is based on work supported by the National Science Foundation under Grants ECS-8409682 and ECS-8700836.     

A stochastic programming approach for planning horizons of infinite horizon capacity planning problems

Planning horizon is a key issue in production planning. Different from previous approaches based on Markov Decision Processes, we study the planning horizon of capacity planning problems within the framework of stochastic programming. We first consider an infinite horizon stochastic capacity planning model involving a single resource, linear cost structure, and discrete distributions for general stochastic cost and demand data (non-Markovian and non-stationary). We give sufficient conditions for the existence of an optimal solution. Furthermore, we study the monotonicity property of the finite horizon approximation of the original problem. We show that, the optimal objective value and solution of the finite horizon approximation problem will converge to the optimal objective value and solution of the infinite horizon problem, when the time horizon goes to infinity. These convergence results, together with the integrality of decision variables, imply the existence of a planning horizon. We also develop a useful formula to calculate an upper bound on the planning horizon. Then by decomposition, we show the existence of a planning horizon for a class of very general stochastic capacity planning problems, which have complicated decision structure.     

On the single item fill rate for a finite horizon

We investigate expressions for expected item fill rate in a periodic inventory system. The typical treatment of fill rate found in many operations management texts assumes infinite horizon, independent and stationary demand. For the case when the horizon is finite, we show that the expected value of the actual fill rate is greater than the value given by the infinite horizon expression. The implication of our results is that an inventory manager in a finite horizon situation who uses the infinite horizon expression to set stocking levels will achieve a higher than desired expected fill rate at greater than necessary inventory expense.     

Existence of efficient solutions in infinite horizon optimization under continuous and discrete controls

For time-varying deterministic infinite horizon control problems, we provide conditions for the existence of efficient solutions, i.e., solutions which are optimal to each of the states through which they pass. A sufficient condition is that the mappings from controls to states be open. Applications to production planning are considered.     

Stationary solutions of SPDEs and infinite horizon BDSDEs with non-Lipschitz coefficients

We prove a general theorem that the -valued solution of an infinite horizon backward doubly stochastic differential equation, if exists, gives the stationary solution of the corresponding stochastic partial differential equation. We prove the existence and uniqueness of the -valued solutions for backward doubly stochastic differential equations on finite and infinite horizon with linear growth without assuming Lipschitz conditions, but under the monotonicity condition. Therefore the solution of finite horizon problem gives the solution of the initial value problem of the corresponding stochastic partial differential equations, and the solution of the infinite horizon problem gives the stationary solution of the SPDEs according to our general result.     

Existence of solutions for a class of nonconcave infinite horizon optimal control problems

In this article, we establish the existence of optimal solutions for a large class of nonconvex infinite horizon discrete-time optimal control problems. This class contains optimal control problems arising in economic dynamics which describe a model with nonconcave utility functions representing the preferences of the planner.     

Algorithms for solving discrete optimal control problems with infinite time horizon and determining minimal mean cost cycles in a directed graph as decision support tool

Time-discrete systems with a finite set of states are considered. Discrete optimal control problems with infinite time horizon for such systems are formulated. We introduce a certain graph-theoretic structure to model the transitions of the dynamical system. Algorithms for finding the optimal stationary control parameters are presented. Furthermore, we determine the optimal mean cost cycles. This approach can be used as a decision support strategy within such a class of problems; especially so-called multilayered decision problems which occur within environmental emission trading procedures can be modelled by such an approach.     

Games of stopping with infinite horizon

We consider two-person zero-sum games of stopping: two players sequentially observe a stochastic process with infinite time horizon. Player I selects a stopping time and player II picks the distribution of the process. The pay-off is given by the expected value of the stopped process. Results of Irle (1990) on existence of value and equivalence of randomization for such games with finite time horizon, where the set of strategies for player II is dominated in the measure-theoretical sense, are extended to the infinite time case. Furthermore we treat such games when the set of strategies for player II is not dominated. A counterexample shows that even in the finite time case such games may not have a value. Then a sufficient condition for the existence of value is given which applies to prophet-type games.     

Infinite horizon optimal policy for an inventory system with two types of product sharing common hardware platforms

We consider a periodic review inventory system and present its optimal policy in the infinite horizon setting. The optimal inventory policy that maximizes the infinite horizon expected discounted profit for the model is analytically obtained by relating to the finite horizon setting using results from variational analysis. Results are provided that elucidate the operations of the inventory system in the long run.     

Duality and uniqueness of convex solutions to stationary Hamilton-Jacobi equations

Value functions for convex optimal control problems on infinite time intervals are studied in the framework of duality. Hamilton-Jacobi characterizations and the conjugacy of primal and dual value functions are of main interest. Close ties between the uniqueness of convex solutions to a Hamilton-Jacobi equation, the uniqueness of such solutions to a dual Hamilton-Jacobi equation, and the conjugacy of primal and dual value functions are displayed. Simultaneous approximation of primal and dual infinite horizon problems with a pair of dual problems on finite horizon, for which the value functions are conjugate, leads to sufficient conditions on the conjugacy of the infinite time horizon value functions. Consequently, uniqueness results for the Hamilton-Jacobi equation are established. Little regularity is assumed on the cost functions in the control problems, correspondingly, the Hamiltonians need not display any strict convexity and may have several saddle points.

     

On the value function for optimal control problems with infinite horizon

We consider optimal control problems of infinite horizon type, whose control laws are given by L¹_loc‐functions and whose objective function has the meaning of a discounted utility. Our main objective is the verification of the fact that the value function is a viscosity solution of the Hamilton‐Jacobi‐Bellman (HJB) equation in this framework. The usual final condition for the HJB‐equation in the finite horizon case (V (T, x) = 0 or V (T, x) = g(x)) has to be substituted by a decay condition at the infinity. Following the dynamic programming approach, we obtain Bellman's optimality principle and the dynamic programming equation (see (3)). We also prove a regularity result (local Lipschitz continuity) for the value function.     

Network Simulation Method for the evaluation of perturbed supply chains on a finite horizon

Due to lead times and other delays in a chain, the Net Present Value (NPV) can be easily estimated if Laplace transforms in MRP models are employed. This leads to the estimation of NPV on an infinite horizon. However, for the simultaneous perturbations of several parameters in a supply chain and activities running on the finite horizon, NPV could be overestimated. Therefore, we suggest the parallel use of the Network Simulation Method (NSM) with the MRP theory to reduce these overestimations. This paper aims to present the NSM to evaluate supply chains on a finite horizon when stochastic behaviour of time delays and other perturbations of parameters are also essential, which is typical for food and drug supply chains. The circuit simulator NGSPICE, which was previously used by certain authors in thermodynamics, also evaluates the financial consequences of simultaneous perturbations in a finite chain. This approach holds better for the stochastic processes of simultaneous perturbations, compared to our results achieved using MRP theory without these corrections. As presented in the numerical example, the shorter the horizon and lower the interest rate, the more important it is to use the correction factors obtained from the NGSPICE simulator.