Limit of the solutions for the finite horizon problems as the optimal solution to the infinite horizon optimization problems

In this paper, we consider how to construct the optimal solutions for the undiscounted discrete time infinite horizon optimization problems. We present the conditions under which the limit of the solutions for the finite horizon problems is optimal among all attainable paths for the infinite horizon problem under two modified overtaking criteria, as well as the conditions under which it is the unique optimum under the sum-of-utilities criterion. The results are applied to a parametric example of a simple one-sector growth model to examine the impacts of discounting on the optimal path.     

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.     

Approximate solutions to infinite dimensional LQ problems over infinite time horizon

This paper is addressed to develop an approximate method to solve a class of infinite dimensional LQ optimal regulator problems over infinite time horizon. Our algorithm is based on a construction of approximate solutions which solve some finite dimensional LQ optimal regulator problems over finite time horizon, and it is shown that these approximate solutions converge strongly to the desired solution in the double limit sense.     

Algorithms for solving discrete optimal control problems with infinite time horizon and determining minimal mean cost cycles in a directed graph as decision support tool

Time-discrete systems with a finite set of states are considered. Discrete optimal control problems with infinite time horizon for such systems are formulated. We introduce a certain graph-theoretic structure to model the transitions of the dynamical system. Algorithms for finding the optimal stationary control parameters are presented. Furthermore, we determine the optimal mean cost cycles. This approach can be used as a decision support strategy within such a class of problems; especially so-called multilayered decision problems which occur within environmental emission trading procedures can be modelled by such an approach.     

Degeneracy in infinite horizon optimization

We consider sequential decision problems over an infinite horizon. The forecast or solution horizon approach to solving such problems requires that the optimal initial decision be unique. We show that multiple optimal initial decisions can exist in general and refer to their existence as degeneracy. We then present a conceptual cost perturbation algorithm for resolving degeneracy and identifying a forecast horizon. We also present a general near-optimal forecast horizon.This material is based on work supported by the National Science Foundation under Grants ECS-8409682 and ECS-8700836.     

Duality in infinite dimensional linear programming

We consider the class of linear programs with infinitely many variables and constraints having the property that every constraint contains at most finitely many variables while every variable appears in at most finitely many constraints. Examples include production planning and equipment replacement over an infinite horizon. We form the natural dual linear programming problem and prove strong duality under a transversality condition that dual prices are asymptotically zero. That is, we show, under this transversality condition, that optimal solutions are attained in both primal and dual problems and their optimal values are equal. The transversality condition, and hence strong duality, is established for an infinite horizon production planning problem.This material is based on work supported by the National Science Foundation under Grant No. ECS-8700836.     

Approximating infinite horizon stochastic optimal control in discrete time with constraints

Traditional approaches to solving stochastic optimal control problems involve dynamic programming, and solving certain optimality equations. When recast as stochastic programming problems, structural aspects such as convexity are retained, and numerical solution procedures based on decomposition and duality may be exploited. This paper explores a class of stationary, infinite-horizon stochastic optimization problems with discounted cost criterion. Constraints on both states and controls are permitted, and modeled in the objective function by allowing it to take infinite values. Approximating techniques are developed using variational analysis, and intuitive lower bounds are obtained via averaging the future. These bounds could be used in a finite-time horizon stochastic programming setting to find solutions numerically. Research supported in part by a grant of the National Science Foundation. AMS Classification 46N10, 49N15, 65K10, 90C15, 90C46     

A stochastic programming approach for planning horizons of infinite horizon capacity planning problems

Planning horizon is a key issue in production planning. Different from previous approaches based on Markov Decision Processes, we study the planning horizon of capacity planning problems within the framework of stochastic programming. We first consider an infinite horizon stochastic capacity planning model involving a single resource, linear cost structure, and discrete distributions for general stochastic cost and demand data (non-Markovian and non-stationary). We give sufficient conditions for the existence of an optimal solution. Furthermore, we study the monotonicity property of the finite horizon approximation of the original problem. We show that, the optimal objective value and solution of the finite horizon approximation problem will converge to the optimal objective value and solution of the infinite horizon problem, when the time horizon goes to infinity. These convergence results, together with the integrality of decision variables, imply the existence of a planning horizon. We also develop a useful formula to calculate an upper bound on the planning horizon. Then by decomposition, we show the existence of a planning horizon for a class of very general stochastic capacity planning problems, which have complicated decision structure.     

CIR利率模型中基于对数效用的投资组合最优化问题

本文研究了CIR 利率模型中基于对数效用的最优长期投资问题和无限时间域上的最优折算消费问题. 通过求解相关的动态规划方程, 获得了这两个最优化问题的最优策略及值函数的明确表现形式.     

Finite horizon approximations of infinite horizon linear programs

This paper describes the class of infinite horizon linear programs that have finite optimal values. A sequence of finite horizon (T period) problems is shown to approximate the infinite horizon problems in the following sense: the optimal values of theT period problems converge monotonically to the optimal value of the infinite problem and the limit of any convergent subsequence of initialT period optimal decisions is an optimal decision for the infinite horizon problem.     

Existence of solutions for a class of nonconcave infinite horizon optimal control problems

In this article, we establish the existence of optimal solutions for a large class of nonconvex infinite horizon discrete-time optimal control problems. This class contains optimal control problems arising in economic dynamics which describe a model with nonconcave utility functions representing the preferences of the planner.     

Gradient-flow approach for computing a nonlinear-quadratic optimal-output feedback gain matrix

In this paper, an approach is proposed for solving a nonlinear-quadratic optimal regulator problem with linear static state feedback and infinite planning horizon. For such a problem, approximate problems are introduced and considered, which are obtained by combining a finite-horizon problem with an infinite-horizon linear problem in a certain way. A gradient-flow based algorithm is derived for these approximate problems. It is shown that an optimal solution to the original problem can be found as the limit of a sequence of solutions to the approximate problems. Several important properties are obtained. For illustration, two numerical examples are presented.This project was partially supported by a research grant from the Australian Research Council.     

Asset-selling problem with an uncertain deadline,quitting offer,and search skipping option

This paper presents a discrete-time sequential stochastic asset-selling problem with an infinite planning horizon, where the process of selling the asset may reach a deadline at any point in time with a probability. It is assumed that a quitting offer is available at every point in time and search skipping is permitted. Thus, decisions must be made as to whether or not to accept the quitting offer, to accept an appearing buyer’s offer, and to conduct a search for a buyer. The main purpose of this paper is to clarify the properties of the optimal decision rules in relation to the model’s parameters.     

Risk-sensitive stopping problems for continuous-time Markov chains

In this paper we consider stopping problems for continuous-time Markov chains under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. More precisely our aim is to maximize the certainty equivalent of the stopping reward minus cost over the time horizon. We derive optimality equations for the value functions and prove the existence of optimal stopping times. The exponential utility is treated as a special case. In contrast to risk-neutral stopping problems it may be optimal to stop between jumps of the Markov chain. We briefly discuss the influence of the risk sensitivity on the optimal stopping time and consider a special house selling problem as an example.     

Application of doubly reflected BSDEs to an impulse control problem

We establish existence results for adapted solutions of infinite horizon backward stochastic differential equations with two reflected barriers. We also apply these results to get the existence of an optimal impulse control strategy for the infinite horizon impulse control problem. The properties of the Snell envelope reduce our problem to the existence of a pair of continuous processes.     

Risk-sensitive portfolio optimization on infinite time horizon

We consider a continuous time portfolio optimization problems on an infinite time horizon for a factor model, recently treated by Bielecki and Pliska ["Risk-sensitive dynamic asset management", Appl. Math. Optim. , 39 (1990) 337-360], where the mean returns of individual securities or asset categories are explicitly affected by economic factors. The factors are assumed to be Gaussian processes. We see new features in constructing optimal strategies for risk-sensitive criteria of the portfolio optimization on an infinite time horizon, which are obtained from the solutions of matrix Riccati equations.     

Note on an alternative solution of Kamien-Schwartz's limit pricing problem

The present authors found Kamien-Schwartz's (Ref. 1) theory of limit pricing with infinite planning horizon interesting mainly because of a certain disproportion between the rather simplelooking result, viz., constancy of the optimal price strategy, and its comparatively difficult proof, viz., use of Pontryagin's Maximum Principle (PMP) which in the infinite horizon case is encumbered by the additional difficulty of not leading to a uniquely solvable system of necessary conditions. Quite in consonance with Arrow's opinion referred to in Takayama (Ref. 2, p. 446) that sometimes the infinite horizon formulation is more convenient and revealing than the finite one, it turned out that in this special case the original optimization problem could be replaced by a much simpler one and, therefore, the use of PMP avoided.     

Asymptotic properties of constrained Markov Decision Processes

We present in this paper several asymptotic properties of constrained Markov Decision Processes (MDPs) with a countable state space. We treat both the discounted and the expected average cost, with unbounded cost. We are interested in (1) the convergence of finite horizon MDPs to the infinite horizon MDP, (2) convergence of MDPs with a truncated state space to the problem with infinite state space, (3) convergence of MDPs as the discount factor goes to a limit. In all these cases we establish the convergence of optimal values and policies. Moreover, based on the optimal policy for the limiting problem, we construct policies which are almost optimal for the other (approximating) problems. Based on the convergence of MDPs with a truncated state space to the problem with infinite state space, we show that an optimal stationary policy exists such that the number of randomisations it uses is less or equal to the number of constraints plus one. We finally apply the results to a dynamic scheduling problem.This work was partially supported by the Chateaubriand fellowship from the French embassy in Israel and by the European Grant BRA-QMIPS of CEC DG XIII     

Infinite horizon multiobjective optimal control problems in the discrete time case

This article studies multiobjective optimal control problems in the discrete time framework and in the infinite horizon case. The functions appearing in the problems satisfy smoothness conditions. This article generalizes to the multiobjective case results obtained for single-objective optimal control problems in that framework. The dynamics are governed by difference equations or difference inequations. Necessary conditions of Pareto optimality are presented, namely Pontryagin maximum principles in the weak form and in the strong form. Sufficient conditions are also provided. Other notions of Pareto optimality are defined when the infinite series do not necessarily converge.     

Infinite horizon optimal policy for an inventory system with two types of product sharing common hardware platforms

We consider a periodic review inventory system and present its optimal policy in the infinite horizon setting. The optimal inventory policy that maximizes the infinite horizon expected discounted profit for the model is analytically obtained by relating to the finite horizon setting using results from variational analysis. Results are provided that elucidate the operations of the inventory system in the long run.