Monotone optimal control for a class of Markov decision processes

This paper provides a unified framework to study monotone optimal control for a class of Markov decision processes through D-multimodularity. We demonstrate that each system in this class can be classified as either a substitution-type or a complement-type system according to the possible transition set, which can be used as a classification mechanism that integrates a variety of models in the literature. We develop a generic proof of the structural properties of both types of system. In particular, we show that D-multimodularity is a generally sufficient condition for monotone optimal control of different types of system in this class. With this unified theory, there is no need to pursue each problem ad hoc and the structural properties of this class of MDPs follow with ease.     

Robust optimal investment and reinsurance problem for a general insurance company under Heston model

In this paper, we study a robust optimal investment and reinsurance problem for a general insurance company which contains an insurer and a reinsurer. Assume that the claim process described by a Brownian motion with drift, the insurer can purchase proportional reinsurance from the reinsurer. Both the insurer and the reinsurer can invest in a financial market consisting of one risk-free asset and one risky asset whose price process is described by the Heston model. Besides, the general insurance company’s manager will search for a robust optimal investment and reinsurance strategy, since the general insurance company faces model uncertainty and its manager is ambiguity-averse in our assumption. The optimal decision is to maximize the minimal expected exponential utility of the weighted sum of the insurer’s and the reinsurer’s surplus processes. By using techniques of stochastic control theory, we give sufficient conditions under which the closed-form expressions for the robust optimal investment and reinsurance strategies and the corresponding value function are obtained.     

Mean-risk analysis of risk aversion and wealth effects on optimal portfolios with multiple investment opportunities

In this paper, we first define risk in an axiomatic way and a class of utility functions suitable for the so-called mean-risk analysis. Then, we show that, in a portfolio selection problem with multiple risky investments, an investor who is more risk averse in the Arrow-Pratt sense prefers less risk, in the sense of this paper, with less mean return, and an investor who displays increasing (decreasing) relative risk aversion becomes more conservative (aggressive) as the initial capital increases. The risk aversion effect for diversification on optimal portfolios is also discussed.     

Decentralized optimal control of dynamical systems under uncertainty

The problem of optimal control of a group of interconnected dynamical objects under uncertainty is considered. The cases are examined in which the centralized control of the group of objects is impossible due to delay in the channel for information exchange between the group members. Optimal self-control algorithms in real time for each dynamical object are proposed. Various types of a priori and current information about the behavior of the group members and about uncertainties in the system are examined. The proposed methods supplement the earlier developed optimal control methods for an individual dynamical system and the methods of decentralized optimal control of deterministic objects. The results are illustrated with examples.     

The consequences of irreversibility on optimal intertemporal emission policies under uncertainty

This paper investigates how irreversibility affects optimal intertemporal emission policies when negative stock externalities exist. In particular it discusses the effect of irreversible emission, i.e., it concerns the physical issue whether it is possible to recollect pollutants that have been emitted or not. We depict our analysis with the greenhouse effect as a topical example and model the uncertainty with respect to the future evolution of the world’s temperature (i.e., the uncertain factor that determines the costs) as Itô-process with the drift provided by current carbon-dioxide emissions. We show analytically that irreversibility affects the optimal emission policy only if the future impact of today’s emissions is uncertain. Under uncertainty, irreversibility leads to a conservationist policy such that emissions are reduced at any level of environmental concentration of the pollutant. The level where stopping emissions is optimal decreases in the presence of irreversibility. Furthermore, the expected duration of fossil fuel use is derived. A numerical example which is calibrated to roughly reflect the global CO₂ problem illustrates the analytical findings.     

An optimal control of a periodically nonstationary generalized automaton under fuzzy conditions

The general problem of successively making optimal decisions on controlling a periodically nonstationary generalized automaton with fuzzy objective and fuzzy constraints on the control is solved. An example of a solution is given.     

The optimal policy for insurance company under consideration of internal competition and the time value of ruin

This paper considers the dividend optimization problem for an insurance company under the consideration of internal competition between different units inside the company. The objective is to find a reinsurance policy and a dividend payment scheme so as to maximize the expected discounted value of the dividend payment, and the expected present value of an amount which the insurer earns until the time of ruin. By solving the corresponding constrained Hamilton-Jacobi-Bellman （HJB） equation, we obtain the value function and the optimal reinsurance policy and dividend payment.     

Optimal control of the insurance company with proportional reinsurance policy under solvency constraints

This paper considers the optimal control problem of the insurance company with proportional reinsurance policy under solvency constraints. The management of the company controls the reinsurance rate and dividends payout processes to maximize the expected present value of the dividend until the time of bankruptcy. This is a mixed singular-regular control problem. However, the optimal dividend payout barrier may be too low to be acceptable. The company may be prohibited to pay dividend according to external reasons because this low dividend payout barrier will result in bankruptcy soon. Therefore, some constraints on the insurance company’s dividend policy will be imposed. One reasonable and normal constraint is that if b is the minimum dividend barrier, then the bankrupt probability should not be larger than some predetermined ε within the time horizon T. This paper is to work out the optimal control policy of the insurance company under the solvency constraints.     

Robust optimal feedback for terminal linear-quadratic control problems under disturbances

We consider a linear dynamic system in the presence of an unknown but bounded perturbation and study how to control the system in order to get into a prescribed neighborhood of a zero at a given final moment. The quality of a control is estimated by the quadratic functional. We define optimal guaranteed program controls as controls that are allowed to be corrected at one intermediate time moment. We show that an infinite dimensional problem of constructing such controls is equivalent to a special bilevel problem of mathematical programming which can be solved explicitely. An easy implementable algorithm for solving the bilevel optimization problem is derived. Based on this algorithm we propose an algorithm of constructing a guaranteed feedback control with one correction moment. We describe the rules of computing feedback which can be implemented in real time mode. The results of illustrative tests are given.     

On a student-related optimal control problem

In a recent paper (Ref. 1), Cheng and Teo discussed some further extensions of a student-related optimal control problem which was originally proposed by Raggettet al. (Ref. 2) and later on modified by Parlar (Ref. 3). In this paper, we treat further extensions of the problem.This paper is a modified and improved version of Ref. 4. It is based, in part, on research sponsored by NSF.     

Maintenance optimal control three-machine replacement model under technological breakthrough expectations

This paper addresses a finite-horizon profit maximization three-machine replacement problem. More precisely, a model is formulated allowing for preventive maintenance to slow down machine quality and profit reduction caused by obsolescence, to determine the timing of replacing an existing machine by another available machine with improved technology. This decision is considered under uncertainty regarding the introduction time of a machine with a not-yetachieved technology. Given an exponential probability distribution function of the introduction time, the optimality of a bang-bang nonincreasing preventive maintenance control is shown.Moreover, subproblems maximizing the expected discounted profit are analyzed. Closed-form solutions are provided to compare machines of different technologies and to derive an analytical sensitivity analysis concerned with many issues related to the problem. The results are not necessarily intuitive and simple. For example, different relationships between the planning horizon and the preventive maintenance switching time are presented for the three-machine problem versus the single-machine problem.The focus of this paper is on the formulation and the analytical analysis of the problem rather than on its computational aspects.     

On an optimal shape design problem to control a thermoelastic deformation under a prescribed thermal treatment

A shape optimization problem concerned with thermal deformation of elastic bodies is considered. In this article, measure theory approach in function space is derived, resulting in an effective algorithm for the discretized optimization problem. First the problem is expressed as an optimal control problem governed by variational forms on a fixed domain. Then by using an embedding method, the class of admissible shapes is replaced by a class of positive Borel measures. The optimization problem in measure space is then approximated by a linear programming problem. The optimal measure representing optimal shape is approximated by the solution of this finite-dimensional linear programming problem. Numerical examples are also given.     

Indefinite stochastic optimal LQR control with cross term under IQ constraints

A stochastic optimal LQR control problem under some integral quadratic (IQ) constraints is studied, with cross terms in both the cost and the constraint functionals, allowing all the control weighting matrices being indefinite. Sufficient conditions for the well-posedness of this problem are given. When these conditions are satisfied, the optimal control is explicitly derived via dual theory.     

Maximum principle for infinite-horizon optimal control problems under weak regularity assumptions

The paper deals with first order necessary optimality conditions for a class of infinite-horizon optimal control problems that arise in economic applications. Neither convergence of the integral utility functional nor local boundedness of the optimal control is assumed. Using the classical needle variations technique we develop a normal form version of the Pontryagin maximum principle with an explicitly specified adjoint variable under weak regularity assumptions. The result generalizes some previous results in this direction. An illustrative economical example is presented.     

A posteriori error estimates for an optimal control problem of laser surface hardening of steel

The main focus of this article is on the development of a posteriori error estimates for an optimal control problem of laser surface hardening of steel, governed by a dynamical system consisting of a semi-linear parabolic equation and an ordinary differential equation. A posteriori error estimators are developed for the variables representing temperature, formation of austenite, and laser energy using residual method when a continuous piecewise linear discretization has been used for the finite element approximation of space variables and a discontinuous Galerkin method has been used for time and control discretizations. The error indicators are used in the implementation and numerical results are obtained.