Limit of the solutions for the finite horizon problems as the optimal solution to the infinite horizon optimization problems

In this paper, we consider how to construct the optimal solutions for the undiscounted discrete time infinite horizon optimization problems. We present the conditions under which the limit of the solutions for the finite horizon problems is optimal among all attainable paths for the infinite horizon problem under two modified overtaking criteria, as well as the conditions under which it is the unique optimum under the sum-of-utilities criterion. The results are applied to a parametric example of a simple one-sector growth model to examine the impacts of discounting on the optimal path.     

A stochastic target formulation for optimal switching problems in finite horizon

We consider a general optimal switching problem for a controlled diffusion and show that its value coincides with the value of a well-suited stochastic target problem associated to a diffusion with jumps. The proof consists in showing that the Hamilton–Jacobi–Bellman equations of both problems are the same and in proving a comparison principle for this equation. This provides a new family of lower bounds for the optimal switching problem, which can be computed by Monte-Carlo methods. This result has also a nice economical interpretation in terms of a firm's valuation.     

Existence of solutions for a class of nonconcave infinite horizon optimal control problems

In this article, we establish the existence of optimal solutions for a large class of nonconvex infinite horizon discrete-time optimal control problems. This class contains optimal control problems arising in economic dynamics which describe a model with nonconcave utility functions representing the preferences of the planner.     

Impulsive optimal control with finite or infinite time horizon

We consider a dynamical system subjected to feedback optimal control in such a way that the evolution of the state exhibits both sudden jumps and continuous changes. Previously obtained necessary conditions (Ref. 1) for such impulsive optimal feedback controls are generalized to admit the case of infinite time horizon; this generalization permits application to a wider class of problems. The results are illustrated by application to a version of the innkeeper's problem.Dedicated to G. Leitmann     

On the value function for optimal control problems with infinite horizon

We consider optimal control problems of infinite horizon type, whose control laws are given by L¹_loc‐functions and whose objective function has the meaning of a discounted utility. Our main objective is the verification of the fact that the value function is a viscosity solution of the Hamilton‐Jacobi‐Bellman (HJB) equation in this framework. The usual final condition for the HJB‐equation in the finite horizon case (V (T, x) = 0 or V (T, x) = g(x)) has to be substituted by a decay condition at the infinity. Following the dynamic programming approach, we obtain Bellman's optimality principle and the dynamic programming equation (see (3)). We also prove a regularity result (local Lipschitz continuity) for the value function.     

Approximating infinite horizon stochastic optimal control in discrete time with constraints

Traditional approaches to solving stochastic optimal control problems involve dynamic programming, and solving certain optimality equations. When recast as stochastic programming problems, structural aspects such as convexity are retained, and numerical solution procedures based on decomposition and duality may be exploited. This paper explores a class of stationary, infinite-horizon stochastic optimization problems with discounted cost criterion. Constraints on both states and controls are permitted, and modeled in the objective function by allowing it to take infinite values. Approximating techniques are developed using variational analysis, and intuitive lower bounds are obtained via averaging the future. These bounds could be used in a finite-time horizon stochastic programming setting to find solutions numerically. Research supported in part by a grant of the National Science Foundation. AMS Classification 46N10, 49N15, 65K10, 90C15, 90C46     

Necessary conditions for stochastic optimal control problems in infinite dimensions

The purpose of this paper is to establish the first and second order necessary conditions for stochastic optimal controls in infinite dimensions. The control system is governed by a stochastic evolution equation, in which both drift and diffusion terms may contain the control variable and the set of controls is allowed to be nonconvex. Only one adjoint equation is introduced to derive the first order necessary optimality condition either by means of the classical variational analysis approach or, under an additional assumption, by using differential calculus of set-valued maps. More importantly, in order to avoid the essential difficulty with the well-posedness of higher order adjoint equations, using again the classical variational analysis approach, only the first and the second order adjoint equations are needed to formulate the second order necessary optimality condition, in which the solutions to the second order adjoint equation are understood in the sense of the relaxed transposition.     

Lower closure and existence theorems for optimal control problems with infinite horizon

Lower closure theorems are proved for optimal control problems governed by ordinary differential equations for which the interval of definition may be unbounded. One theorem assumes that Cesari's property (Q) holds. Two theorems are proved which do not require property (Q), but assume either a generalized Lipschitz condition or a bound on the controls in an appropriateL _p-space. An example shows that these hypotheses can hold without property (Q) holding.     

Mean-field stochastic control with elephant memory in finite and infinite time horizon

ABSTRACT

Our purpose of this paper is to study stochastic control problems for systems driven by mean-field stochastic differential equations with elephant memory, in the sense that the system (like the elephants) never forgets its history. We study both the finite horizon case and the infinite time horizon case.

• In the finite horizon case, results about existence and uniqueness of solutions of such a system are given. Moreover, we prove sufficient as well as necessary stochastic maximum principles for the optimal control of such systems. We apply our results to solve a mean-field linear quadratic control problem.

• For infinite horizon, we derive sufficient and necessary maximum principles.

  As an illustration, we solve an optimal consumption problem from a cash flow modelled by an elephant memory mean-field system.

     

A PDE approach to regularity of solutions to finite horizon optimal switching problems

We study optimal 2-switching and n-switching problems and the corresponding system of variational inequalities. We obtain results on the existence of viscosity solutions for the 2-switching problem for various setups when the cost of switching is non-deterministic. For the n-switching problem we obtain regularity results for the solutions of the variational inequalities. The solutions are C^1,1-regular away for the free boundaries of the action sets.     

A PDE approach to regularity of solutions to finite horizon optimal switching problems

We study optimal 2-switching and $n$-switching problems and the corresponding system of variational inequalities. We obtain results on the existence of viscosity solutions for the 2-switching problem for various setups when the cost of switching is non-deterministic. For the $n$-switching problem we obtain regularity results for the solutions of the variational inequalities. The solutions are $C^{1,1}$-regular away for the free boundaries of the action sets.     

Pontryagin Type Maximum Principle for budget-constrained infinite horizon optimal control problems with linear dynamics

In this paper a class of infinite horizon optimal control problems with an isoperimetrical constraint, also interpreted as a budget constraint, is considered. Herein a linear both in the state and in the control dynamic is allowed. The problem setting includes a weighted Sobolev space as the state space. For this class of problems, we establish the necessary optimality conditions in form of a Pontryagin Type Maximum Principle including a transversality condition. The proved theoretical result is applied to a linear–quadratic regulator problem.     

Classical and singular stochastic control for the optimal dividend policy when there is regime switching

Motivated by economic and empirical arguments, we consider a company whose cash surplus is affected by macroeconomic conditions. Specifically, we model the cash surplus as a Brownian motion with drift and volatility modulated by an observable continuous-time Markov chain that represents the regime of the economy. The objective of the management is to select the dividend policy that maximizes the expected total discounted dividend payments to be received by the shareholders. We study two different cases: bounded dividend rates and unbounded dividend rates. These cases generate, respectively, problems of classical stochastic control with regime switching and singular stochastic control with regime switching. We solve these problems, and obtain the first analytical solutions for the optimal dividend policy in the presence of business cycles. We prove that the optimal dividend policy depends strongly on macroeconomic conditions.     

A successive approximation algorithm for stochastic control problems

We consider optimal stochastic control problems in which the state variables are governed by Itô equations. A successive approximation algorithm for optimal stochastic control is obtained. This algorithm, together with the existing numerical methods for parabolic or elliptic PDEs, provides numerical schemes for the solution of Bellman equations.     

A stochastic programming approach for planning horizons of infinite horizon capacity planning problems

Planning horizon is a key issue in production planning. Different from previous approaches based on Markov Decision Processes, we study the planning horizon of capacity planning problems within the framework of stochastic programming. We first consider an infinite horizon stochastic capacity planning model involving a single resource, linear cost structure, and discrete distributions for general stochastic cost and demand data (non-Markovian and non-stationary). We give sufficient conditions for the existence of an optimal solution. Furthermore, we study the monotonicity property of the finite horizon approximation of the original problem. We show that, the optimal objective value and solution of the finite horizon approximation problem will converge to the optimal objective value and solution of the infinite horizon problem, when the time horizon goes to infinity. These convergence results, together with the integrality of decision variables, imply the existence of a planning horizon. We also develop a useful formula to calculate an upper bound on the planning horizon. Then by decomposition, we show the existence of a planning horizon for a class of very general stochastic capacity planning problems, which have complicated decision structure.     

A discontinuous interpolated finite volume approximation of semilinear elliptic optimal control problems

In this article, we describe a discontinuous finite volume method with interpolated coefficients for the numerical approximation of the distributed optimal control problem governed by a class of semilinear elliptic equations with control constraints. The proposed distributed control problem involves three unknown variable: control, state and costate. For the approximation of control, we have adopted three different methodologies: variational discretization, piecewise constant and piecewise linear discretization, while the approximation of state and costate variables is based on discontinuous piecewise linear polynomials. As the resulted scheme is non‐symmetric, optimize‐then‐discretize approach is used to approximate the control problem. Optimal a priori error estimates in suitable natural norms for state, costate and control variables are derived. Moreover, numerical experiments are presented to support the derived theoretical results. © 2017 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 33: 2090–2113, 2017     

Robust error estimates for the finite element approximation of elliptic optimal control problems

In this paper, we consider the finite element approximation of an elliptic optimal control problem. Based on an assumption on the adjoint state of the continuous problem with a small parameter, which represents a regularization of the bang-bang type control problem, we derive robust a priori error estimates for optimal control and state and a posteriori error estimate is also presented. Numerical experiments confirm our theoretical results.     

A posteriori error estimates for mixed finite element solutions of convex optimal control problems

In this paper, we present an a posteriori error analysis for mixed finite element approximation of convex optimal control problems. We derive a posteriori error estimates for the coupled state and control approximations under some assumptions which hold in many applications. Such estimates can be used to construct reliable adaptive mixed finite elements for the control problems.