Monotone optimal control for a class of Markov decision processes

This paper provides a unified framework to study monotone optimal control for a class of Markov decision processes through D-multimodularity. We demonstrate that each system in this class can be classified as either a substitution-type or a complement-type system according to the possible transition set, which can be used as a classification mechanism that integrates a variety of models in the literature. We develop a generic proof of the structural properties of both types of system. In particular, we show that D-multimodularity is a generally sufficient condition for monotone optimal control of different types of system in this class. With this unified theory, there is no need to pursue each problem ad hoc and the structural properties of this class of MDPs follow with ease.     

Robust optimal investment and reinsurance problem for a general insurance company under Heston model

In this paper, we study a robust optimal investment and reinsurance problem for a general insurance company which contains an insurer and a reinsurer. Assume that the claim process described by a Brownian motion with drift, the insurer can purchase proportional reinsurance from the reinsurer. Both the insurer and the reinsurer can invest in a financial market consisting of one risk-free asset and one risky asset whose price process is described by the Heston model. Besides, the general insurance company’s manager will search for a robust optimal investment and reinsurance strategy, since the general insurance company faces model uncertainty and its manager is ambiguity-averse in our assumption. The optimal decision is to maximize the minimal expected exponential utility of the weighted sum of the insurer’s and the reinsurer’s surplus processes. By using techniques of stochastic control theory, we give sufficient conditions under which the closed-form expressions for the robust optimal investment and reinsurance strategies and the corresponding value function are obtained.     

The optimal strategy for insurance company under the influence of terminal value

This paper considers a model of an insurance company which is allowed to invest a risky asset and to purchase proportional reinsurance. The objective is to find the policy which maximizes the expected total discounted dividend pay-out until the time of bankruptcy and the terminal value of the company under liquidity constraint. We find the solution of this problem via solving the problem with zero terminal value. We also analyze the influence of terminal value on the optimal policy.     

Linear-quadratic optimal control under non-Markovian switching

We study a finite-dimensional continuous-time optimal control problem on finite horizon for a controlled diffusion driven by Brownian motion, in the linear-quadratic case. We admit stochastic coefficients, possibly depending on an underlying independent marked point process, so that our model is general enough to include controlled switching systems where the switching mechanism is not required to be Markovian. The problem is solved by means of a Riccati equation, which turned out to be a backward stochastic differential equation driven by the Brownian motion and by the random measure associated with the marked point process.     

Optimal control of a big financial company with debt liability under bankrupt probability constraints

This paper considers an optimal control of a big financial company with debt liability under bankrupt probability constraints. The company, which faces constant liability payments and has choices to choose various production/business policies from an available set of control policies with different expected profits and risks, controls the business policy and dividend payout process to maximize the expected present value of the dividends until the time of bankruptcy. However, if the dividend payout barrier is too low to be acceptable, it may result in the company’s bankruptcy soon. In order to protect the shareholders’ profits, the managements of the company impose a reasonable and normal constraint on their dividend strategy, that is, the bankrupt probability associated with the optimal dividend payout barrier should be smaller than a given risk level within a fixed time horizon. This paper aims at working out the optimal control policy as well as optimal return function for the company under bankrupt probability constraint by stochastic analysis, partial differential equation and variational inequality approach. Moreover, we establish a riskbased capital standard to ensure the capital requirement can cover the total given risk by numerical analysis, and give reasonable economic interpretation for the results.     

The consequences of irreversibility on optimal intertemporal emission policies under uncertainty

This paper investigates how irreversibility affects optimal intertemporal emission policies when negative stock externalities exist. In particular it discusses the effect of irreversible emission, i.e., it concerns the physical issue whether it is possible to recollect pollutants that have been emitted or not. We depict our analysis with the greenhouse effect as a topical example and model the uncertainty with respect to the future evolution of the world’s temperature (i.e., the uncertain factor that determines the costs) as Itô-process with the drift provided by current carbon-dioxide emissions. We show analytically that irreversibility affects the optimal emission policy only if the future impact of today’s emissions is uncertain. Under uncertainty, irreversibility leads to a conservationist policy such that emissions are reduced at any level of environmental concentration of the pollutant. The level where stopping emissions is optimal decreases in the presence of irreversibility. Furthermore, the expected duration of fossil fuel use is derived. A numerical example which is calibrated to roughly reflect the global CO₂ problem illustrates the analytical findings.     

Decentralized optimal control of dynamical systems under uncertainty

The problem of optimal control of a group of interconnected dynamical objects under uncertainty is considered. The cases are examined in which the centralized control of the group of objects is impossible due to delay in the channel for information exchange between the group members. Optimal self-control algorithms in real time for each dynamical object are proposed. Various types of a priori and current information about the behavior of the group members and about uncertainties in the system are examined. The proposed methods supplement the earlier developed optimal control methods for an individual dynamical system and the methods of decentralized optimal control of deterministic objects. The results are illustrated with examples.     

Mean-risk analysis of risk aversion and wealth effects on optimal portfolios with multiple investment opportunities

In this paper, we first define risk in an axiomatic way and a class of utility functions suitable for the so-called mean-risk analysis. Then, we show that, in a portfolio selection problem with multiple risky investments, an investor who is more risk averse in the Arrow-Pratt sense prefers less risk, in the sense of this paper, with less mean return, and an investor who displays increasing (decreasing) relative risk aversion becomes more conservative (aggressive) as the initial capital increases. The risk aversion effect for diversification on optimal portfolios is also discussed.     

An optimal control of a periodically nonstationary generalized automaton under fuzzy conditions

The general problem of successively making optimal decisions on controlling a periodically nonstationary generalized automaton with fuzzy objective and fuzzy constraints on the control is solved. An example of a solution is given.     

Mixed-integer optimal control under minimum dwell time constraints

Mathematical Programming - Tailored Mixed-Integer Optimal Control policies for real-world applications usually have to avoid very short successive changes of the active integer control. Minimum...     

Optimal control of the insurance company with proportional reinsurance policy under solvency constraints

This paper considers the optimal control problem of the insurance company with proportional reinsurance policy under solvency constraints. The management of the company controls the reinsurance rate and dividends payout processes to maximize the expected present value of the dividend until the time of bankruptcy. This is a mixed singular-regular control problem. However, the optimal dividend payout barrier may be too low to be acceptable. The company may be prohibited to pay dividend according to external reasons because this low dividend payout barrier will result in bankruptcy soon. Therefore, some constraints on the insurance company’s dividend policy will be imposed. One reasonable and normal constraint is that if b is the minimum dividend barrier, then the bankrupt probability should not be larger than some predetermined ε within the time horizon T. This paper is to work out the optimal control policy of the insurance company under the solvency constraints.     

The optimal policy for insurance company under consideration of internal competition and the time value of ruin

This paper considers the dividend optimization problem for an insurance company under the consideration of internal competition between different units inside the company. The objective is to find a reinsurance policy and a dividend payment scheme so as to maximize the expected discounted value of the dividend payment, and the expected present value of an amount which the insurer earns until the time of ruin. By solving the corresponding constrained Hamilton-Jacobi-Bellman （HJB） equation, we obtain the value function and the optimal reinsurance policy and dividend payment.     

Optimal control of the insurance company with proportional reinsurance policy under solvency constraints

This paper considers the optimal control problem of the insurance company with proportional reinsurance policy under solvency constraints. The management of the company controls the reinsurance rate and dividends payout processes to maximize the expected present value of the dividend until the time of bankruptcy. This is a mixed singular-regular control problem. However, the optimal dividend payout barrier may be too low to be acceptable. The company may be prohibited to pay dividend according to external reasons because this low dividend payout barrier will result in bankruptcy soon. Therefore, some constraints on the insurance company’s dividend policy will be imposed. One reasonable and normal constraint is that if $b$ is the minimum dividend barrier, then the bankrupt probability should not be larger than some predetermined $\epsilon$ within the time horizon $T$. This paper is to work out the optimal control policy of the insurance company under the solvency constraints.     

On a student-related optimal control problem

In a recent paper (Ref. 1), Cheng and Teo discussed some further extensions of a student-related optimal control problem which was originally proposed by Raggettet al. (Ref. 2) and later on modified by Parlar (Ref. 3). In this paper, we treat further extensions of the problem.This paper is a modified and improved version of Ref. 4. It is based, in part, on research sponsored by NSF.     

Maintenance optimal control three-machine replacement model under technological breakthrough expectations

This paper addresses a finite-horizon profit maximization three-machine replacement problem. More precisely, a model is formulated allowing for preventive maintenance to slow down machine quality and profit reduction caused by obsolescence, to determine the timing of replacing an existing machine by another available machine with improved technology. This decision is considered under uncertainty regarding the introduction time of a machine with a not-yetachieved technology. Given an exponential probability distribution function of the introduction time, the optimality of a bang-bang nonincreasing preventive maintenance control is shown.Moreover, subproblems maximizing the expected discounted profit are analyzed. Closed-form solutions are provided to compare machines of different technologies and to derive an analytical sensitivity analysis concerned with many issues related to the problem. The results are not necessarily intuitive and simple. For example, different relationships between the planning horizon and the preventive maintenance switching time are presented for the three-machine problem versus the single-machine problem.The focus of this paper is on the formulation and the analytical analysis of the problem rather than on its computational aspects.     

Robust optimal feedback for terminal linear-quadratic control problems under disturbances

We consider a linear dynamic system in the presence of an unknown but bounded perturbation and study how to control the system in order to get into a prescribed neighborhood of a zero at a given final moment. The quality of a control is estimated by the quadratic functional. We define optimal guaranteed program controls as controls that are allowed to be corrected at one intermediate time moment. We show that an infinite dimensional problem of constructing such controls is equivalent to a special bilevel problem of mathematical programming which can be solved explicitely. An easy implementable algorithm for solving the bilevel optimization problem is derived. Based on this algorithm we propose an algorithm of constructing a guaranteed feedback control with one correction moment. We describe the rules of computing feedback which can be implemented in real time mode. The results of illustrative tests are given.