On the existence of sporadically catching-up optimal solutions for infinite-horizon optimal control problems

In this paper, we extend the existence theory of Brock and Haurie concerning the existence of sporadically catching-up optimal solutions for autonomous, infinite-horizon optimal control problems. This notion of optimality is one of a hierarchy of types of optimality that have appeared in the literature to deal with optimal control problems whose cost functionals, described by an improper integral, either diverge or are unbounded below. Our results rely on the now classical convexity and seminormality hypotheses due to Cesari and are weaker than those assumed in the work of Brock and Haurie. An example is presented where our results are applicable, but those of the above-mentioned authors do not.This research forms part of the author's doctoral dissertation, written at the University of Delaware, Newark, Delaware, under the supervision of Professor T. S. Angell.     

A Carathéodory-Hamilton-Jacobi theory for infinite-horizon optimal control problems

In this paper, we extend Carathéodory's concept of equivalent variational problems to infinite-horizon optimal control problems. In such a setting, the usual concept of a minimum need not exist, and we therefore consider a weaker definition of optimality, known as catching up optimality. The extension presented here leads us to a Hamilton-Jacobi theory for infinite-horizon optimal control problems that closely parallels the classical work of Carathéodory as well as providing sufficient conditions for optimality. Finally, we show that the results given here subsume several previously known results as a special case.This research forms part of the author's doctoral dissertation, written at the University of Delaware, Newark, Delaware, under the supervision of Professor T. S. Angell.     

Infinite-horizon optimal controls for problems governed by a volterra integral equation with state-and-control-dependent discount factor

In this work, we concern ourselves with the existence of optimal solutions to optimal control problems defined on an unbounded time interval with states governed by a nonlinear Volterra integral equation. These results extend both the work of Baum and others in infinite-horizon control of ordinary differential equations as well as the work of Angell concerning integral equations. In addition, we incorporate into the objective functional (described by an improper integral) a discount factor which reflects a hereditary dependence on both state and control. In this manner, we are able to generalize the recent results of Becker, Boyd, and Sung in which they establish an existence theorem in the calculus of variations with objective functionals of the so-called recursive type. Our results are obtained through the use of appropriate lower-closure theorems and compactness conditions. Examples are presented in which the applicability of our results is demonstrated.This research was supported by the National Science Foundation, Grant No. DMS-87-00706.     

Existence of finitely optimal solutions for infinite-horizon optimal control problems

In this paper, we investigate the existence of finitely optimal solutions for the Lagrange problem of optimal control defined on [0, ) under weaker convexity and seminormality hypotheses than those of previous authors. The notion of finite optimality has been introduced into the literature as the weakest of a hierarchy of types of optimality that have been defined to permit the study of Lagrange problems, arising in mathematical economics, whose cost functions either diverge or are not bounded below. Our method of proof requires us to analyze the continuous dependence of finite-interval Lagrange problems with respect to a prescribed terminal condition. Once this is done, we show that a finitely optimal solution can be obtained as the limit of a sequence of solutions to a sequence of corresponding finite-horizon optimal control problems. Our results utilize the convexity and seminormality hypotheses which are now classical in the existence theory of optimal control.This research forms part of the author's doctoral dissertation written at the University of Delaware, Newark, Delaware under the supervision of Professor Thomas S. Angell.     

On the Existence of a Classical Optimal Solution and of an Almost Strongly Optimal Solution for an Infinite-Horizon Control Problem

We consider an infinite-horizon optimal control problem with the cost functional described either by an integral over an unbounded interval (a Lebesgue integral) or by a limit of integrals (an improper Lebesgue integral). We prove some theorems on the existence of solutions to such problems. The proofs are based on appropriate lower closure theorems and some extensions of Olech’s theorem on the lower semicontinuity of an integral functional; these extensions cover the cases of functionals described by an integral over an unbounded interval and by a limit of integrals.     

Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes

ABSTRACT

The main goal of this paper is to study the infinite-horizon long run average continuous-time optimal control problem of piecewise deterministic Markov processes (PDMPs) with the control acting continuously on the jump intensity λ and on the transition measure Q of the process. We provide conditions for the existence of a solution to an integro-differential optimality inequality, the so called Hamilton-Jacobi-Bellman (HJB) equation, and for the existence of a deterministic stationary optimal policy. These results are obtained by using the so-called vanishing discount approach, under some continuity and compactness assumptions on the parameters of the problem, as well as some non-explosive conditions for the process.     

Maximum principle for infinite-horizon optimal control problems under weak regularity assumptions

The paper deals with first order necessary optimality conditions for a class of infinite-horizon optimal control problems that arise in economic applications. Neither convergence of the integral utility functional nor local boundedness of the optimal control is assumed. Using the classical needle variations technique we develop a normal form version of the Pontryagin maximum principle with an explicitly specified adjoint variable under weak regularity assumptions. The result generalizes some previous results in this direction. An illustrative economical example is presented.     

On necessary optimality conditions for infinite-horizon economic growth problems with locally unbounded instantaneous utility function

We consider a class of infinite-horizon optimal control problems that arise in studying models of optimal dynamic allocation of economic resources. In a typical problem of that kind the initial state is fixed, no constraints are imposed on the behavior of the admissible trajectories at infinity, and the objective functional is given by a discounted improper integral. Earlier, for such problems, S.M. Aseev and A.V. Kryazhimskiy in 2004–2007 and jointly with the author in 2012 developed a method of finite-horizon approximations and obtained variants of the Pontryagin maximum principle that guarantee normality of the problem and contain an explicit formula for the adjoint variable. In the present paper those results are extended to a more general situation where the instantaneous utility function need not be locally bounded from below. As an important illustrative example, we carry out a rigorous mathematical investigation of the transitional dynamics in the neoclassical model of optimal economic growth.     

Zero-sum stochastic games with average payoffs: New optimality conditions

In this paper we study zero-sum stochastic games. The optimality criterion is the long-run expected average criterion, and the payoff function may have neither upper nor lower bounds. We give a new set of conditions for the existence of a value and a pair of optimal stationary strategies. Our conditions are slightly weaker than those in the previous literature, and some new sufficient conditions for the existence of a pair of optimal stationary strategies are imposed on the primitive data of the model. Our results are illustrated with a queueing system, for which our conditions are satisfied but some of the conditions in some previous literatures fail to hold.     

Conditions for characterizing the structure of optimal strategies in infinite-horizon dynamic programs

The study of infinite-horizon nonstationary dynamic programs using the operator approach is continued. The point of view here differs slightly from that taken by others, in that Denardo's local income function is not used as a starting point. Infinite-horizon values are defined as limits of finite-horizon values, as the horizons get long. Two important conditions of an earlier paper are weakened, yet the optimality equations, the optimality criterion, and the existence of optimal structured strategies are still obtained.     

Pontryagin principles for bounded discrete-time processes

Abstract

We establish necessary conditions and sufficient conditions of optimality in the form of Pontryagin principles for infinite-horizon discrete-time optimal control problems governed by a difference inequation.     

Stabilizing the Hamiltonian system for constructing optimal trajectories

An infinite-horizon optimal control problem based on an economic growth model is studied. The goal in the problem is to optimize the mechanisms of investment in basic production assets in order to increase the growth rate of the consumption level. The main output variable-the gross domestic product (GDP)-depends on three production factors: capital stock, human capital, and useful work. The first two factors are endogenous variables of the model, and the useful work is an exogenous factor. The dependence of the GDP on the production factors is described by the Cobb-Douglas power-type production function. The economic system under consideration is assumed to be closed, so the GDP is distributed between consumption and investment in the capital stock and human capital. The optimal control problem consists in determining optimal investment strategies that maximize the integral discounted relative consumption index on an infinite time interval. A solution to the problem is constructed on the basis of the Pontryagin maximum principle adapted to infinite-horizon problems. We examine the questions of existence and uniqueness of a solution, verify necessary and sufficient optimality conditions, and perform a qualitative analysis of Hamiltonian systems on the basis of which we propose an algorithm for constructing optimal trajectories. This algorithm uses information on solutions obtained by means of a nonlinear regulator. Finally, we estimate the accuracy of the algorithm with respect to the integral cost functional of the control process.     

Markov Decision Processes with Variance Minimization: A New Condition and Approach

Abstract

This article deals with the limiting average variance criterion for discrete-time Markov decision processes in Borel spaces. The costs may have neither upper nor lower bounds. We propose another set of conditions under which we prove the existence of a variance minimal policy in the class of average expected cost optimal stationary policies. Our conditions are weaker than those in the previous literature. Moreover, some sufficient conditions for the existence of a variance minimal policy are imposed on the primitive data of the model. In particular, the stochastic monotonicity condition in this paper has been first used to study the limiting average variance criterion. Also, the optimality inequality approach provided here is different from the “optimality equation approach” widely used in the previous literature. Finally, we use a controlled queueing system to illustrate our results.     

Banach空间中非线性脉冲Volterra型积分方程组的可解性

在较弱的条件下，利用锥理论和单调迭代方法首先建立了Banach空间中一类非线性算子方程组最小最大解的存在性定理；然后作为应用，利用一个新的比较结果，得到了Banach空间中非线性脉冲Volterra型积分方程组的整体解，改进了最近的许多结果．     

Constant Sustainable Consumption Rate in Optimal Growth with Exhaustible Resources

The problem of optimal growth with an exhaustible resource deposit under R. M. Solow's criterion of maximum sustainable consumption rate, previously formulated as a minimum-resource-extraction problem, is shown to be a Mayer-type optimal-control problem. The exact solution of the relevant firstorder necessary conditions for optimality is derived for a Cobb-Douglas production function, whether or not the constant unit resource extraction cost vanishes. The related problem of maximizing the terminal capital stock over an unspecified finite planning period is investigated for the development of more efficient numerical schemes for the solution of multigrade-resource deposit problems. The results for this finite-horizon planning problem are also important from a theoretical viewpoint, since they elucidate the economic content of the optimal growth paths for infinite-horizon problems.     

Necessary conditions of Pareto optimality for multiobjective optimal control problems under constraints

This paper studies multiobjective optimal control problems in presence of constraints in the discrete time framework. Both the finite- and infinite-horizon settings are considered. The paper provides necessary conditions of Pareto optimality under lighter smoothness assumptions compared to the previously obtained results. These conditions are given in the form of weak and strong Pontryagin principles which generalize the existing ones. To obtain some of these results, we provide new multiplier rules for multiobjective static optimization problems and new Pontryagin principles for the finite horizon multiobjective optimal control problems.     

On the Construction of Optimal Monotone Cubic Spline Interpolations

In this paper we derive necessary optimality conditions for an interpolating spline function which minimizes the Holladay approximation of the energy functional and which stays monotone if the given interpolation data are monotone. To this end optimal control theory for state-restricted optimal control problems is applied. The necessary conditions yield a complete characterization of the optimal spline. In the case of two or three interpolation knots, which we call thelocalcase, the optimality conditions are treated analytically. They reduce to polynomial equations which can very easily be solved numerically. These results are used for the construction of a numerical algorithm for the optimal monotone spline in the general (global) case via Newton's method. Here, the local optimal spline serves as a favourable initial estimation for the additional grid points of the optimal spline. Some numerical examples are presented which are constructed by FORTRAN and MATLAB programs.     

Transversality condition and optimization of higher order ordinary differential inclusions

In the present paper, a Bolza problem of optimal control theory with a fixed time interval given by convex and nonconvex second-order differential inclusions (P_H) is studied. Our main goal is to derive sufficient optimality conditions for Cauchy problem of sth-order differential inclusions. The sufficient conditions including distinctive transversality condition are proved incorporating the Euler–Lagrange and Hamiltonian type inclusions. The basic concepts involved in obtaining optimality conditions are the locally adjoint mappings. Furthermore, the application of these results is demonstrated by solving the problems with third-order differential inclusions.     

New Average Optimality Conditions for Semi-Markov Decision Processes in Borel Spaces

This paper deals with semi-Markov decision processes under the average expected criterion. The state and action spaces are Borel spaces, and the cost/reward function is allowed to be unbounded from above and from below. We give another set of conditions, under which the existence of an optimal (deterministic) stationary policy is proven by a new technique of two average optimality inequalities. Our conditions are slightly weaker than those in the existing literature, and some new sufficient conditions for the verifications of our assumptions are imposed on the primitive data of the model. Finally, we illustrate our results with three examples.