Optimal impulse control for a multidimensional cash management system with generalized cost functions

We consider the optimal control of a multidimensional cash management system where the cash balances fluctuate as a homogeneous diffusion process in Rⁿ${\mathbb{R}}^n$. We formulate the model as an impulse control problem on an unbounded domain with unbounded cost functions. Under general assumptions we characterize the value function as a weak solution of a quasi-variational inequality in a weighted Sobolev space and we show the existence of an optimal policy. Moreover we prove the local uniform convergence of a finite element scheme to compute numerically the value function and the optimal cost. We compute the solution of the model in two-dimensions with linear and distance cost functions, showing what are the shapes of the optimal policies in these two simple cases. Finally our third numerical experiment computes the solution in the realistic case of the cash concentration of two bank accounts made by a centralized treasury.     

Viscosity solution and impulse control of the diffusion model with reinsurance and fixed transaction costs

We consider an optimal impulse control problem on reinsurance, dividend and reinvestment of an insurance company. To close reality, we add fixed and proportional transaction costs to this problem. The value of the company is associated with expected present value of net dividends pay out minus the net reinvestment capitals until ruin time. We focus on non-cheap proportional reinsurance. We prove that the value function is a unique solution to associated Hamilton–Jacobi–Bellman equation, and establish the regularity property of the viscosity solution under a weak assumption. We solve the non-uniformly elliptic equation associated with the impulse control problem. Finally, we derive the value function and the optimal strategy of the control problem.     

An impulse control of a geometric Brownian motion with quadratic costs

We examine an optimal impulse control problem of a stochastic system whose state follows a geometric Brownian motion. We suppose that, when an agent intervenes in the system, it requires costs consisting of a quadratic form of the system state. Besides the intervention costs, running costs are continuously incurred to the system, and they are also of a quadratic form. Our objective is to find an optimal impulse control of minimizing the expected total discounted sum of the intervention costs and running costs incurred over the infinite time horizon. In order to solve this problem, we formulate it as a stochastic impulse control problem, which is approached via quasi-variational inequalities (QVI). Under a suitable set of sufficient conditions on the given problem parameters, we prove the existence of an optimal impulse control such that, whenever the system state reaches a certain level, the agent intervenes in the system. Consequently it instantaneously reduces to another level.     

Optimal securitization of credit portfolios via impulse control

We study the optimal loan securitization policy of a commercial bank which is mainly engaged in lending activities. For this we propose a stylized dynamic model which contains the main features affecting the securitization decision. In line with reality we assume that there are non-negligible fixed and variable transaction costs associated with each securitization. The fixed transaction costs lead to a formulation of the optimization problem in an impulse control framework. We prove viscosity solution existence and uniqueness for the quasi-variational inequality associated with this impulse control problem. Iterated optimal stopping is used to find a numerical solution of this PDE, and numerical examples are discussed.     

Optimal Portfolio Selection Under Concave Price Impact

In this paper we study an optimal portfolio selection problem under instantaneous price impact. Based on some empirical analysis in the literature, we model such impact as a concave function of the trading size when the trading size is small. The price impact can be thought of as either a liquidity cost or a transaction cost, but the concavity nature of the cost leads to some fundamental difference from those in the existing literature. We show that the problem can be reduced to an impulse control problem, but without fixed cost, and that the value function is a viscosity solution to a special type of Quasi-Variational Inequality (QVI). We also prove directly (without using the solution to the QVI) that the optimal strategy exists and more importantly, despite the absence of a fixed cost, it is still in a “piecewise constant” form, reflecting a more practical perspective.     

On scheduling a multiclass queue with abandonments under general delay costs

We consider a multiclass queueing system with abandonments and general delay costs. A system manager makes dynamic scheduling decisions to minimize long-run average delay and abandonment costs. We consider the three types of delay cost: (i) linear, (ii) convex, and (iii) convex–concave, where the last one corresponds to settings where customers may have a particular deadline in mind but once that deadline passes there is increasingly little difference in the added delay. The dynamic control problem for the queueing system is not tractable analytically. Therefore, we consider the system in the conventional heavy traffic regime and study the approximating Brownian control problem (BCP). We observe that the approximating BCP does not admit a pathwise solution due to abandonments. In particular, the celebrated cμ rule and its extension, the generalized cμ rule, which is asymptotically optimal under convex delay costs with no abandonments, are not optimal in this case. Consequently, we solve the associated Bellman equation, which yields a dynamic index policy (derived from the value function) as the optimal control for the approximating BCP. Interpreting that control in the context of the original queueing system, we propose practical policies for each of the three cases considered and demonstrate their effectiveness through a simulation study.     

Numerical solution of a minimax ergodic optimal control problem

In this work we consider an L^∞ minimax ergodic optimal control problem with cumulative cost. We approximate the cost function as a limit of evolutions problems. We present the associated Hamilton-Jacobi-Bellman equation and we prove that it has a unique solution in the viscosity sense. As this HJB equation is consistent with a numerical procedure, we use this discretization to obtain a procedure for the primitive problem. For the numerical solution of the ergodic version we need a perturbation of the instantaneous cost function. We give an appropriate selection of the discretization and penalization parameters to obtain discrete solutions that converge to the optimal cost. We present numerical results. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

A MIXED CONTROL PROBLEM OF THE MANAGEMENT OF NATURAL RESOURCES

In this paper, we study the optimal solutions of a model of natural resource management which allows for both impulse and continuous harvesting policies. This type of model is known in the literature as mixed optimal control problem. In the resource management context, each type of control represents a different harvesting technology, which has a different cost. In particular, we want to know when the following conjecture made by Clark is an optimal solution to this mixed optimal control problem: if the harvesting capacity is unlimited, it is optimal to jump immediately to the steady state of the continuous time problem and then to stay there. We show that under a particular relationship between the continuous and the impulse profit function, the conjecture made by Clark is true. In other cases, however, it is either better to use only continuous control variables or to jump to resource levels which are smaller than the steady state and then let the resource grow back to the steady state. These results emphasize the importance of the cost functions in the modeling of natural resource management.     

Un modéle d'lnspections optimales

We consider the following model: we inspect the motion of a Markov process with which an “evolution cost” is associated. We inspect the process at times T ₁…, T _n ,…. If when we inspect, its value is in a given set A, it continues its evolution, otherwise we kill it. At each inspection we associate an "inspection cost" and a "killing cost". The problem consists of finding a sequence of optimal inspections. After the modelization we construct the value function by an iterative procedure as in impulse control theory, by using the theory of analytic functions and theorems of section. Thanks to the criteria of optimality we get a sequence of optimal inspections under very general hypotheses.     

Impulsive Control of Portfolios

In this paper a general model of a market with asset prices and economical factors of Markovian structure is considered. The problem is to find optimal portfolio strategies maximizing a discounted infinite horizon reward functional consisting of an integral term measuring the quality of the portfolio at each moment and a discrete term measuring the reward from consumption. There are general transaction costs which, in particular, cover fixed plus proportional costs. It is shown, under general conditions, that there exists an optimal impulse strategy and the value function is a solution to the Bellman equation which corresponds to suitable quasi-variational inequalities.     

Admission control for a multi-server queue with abandonment

In a M/M/N+M queue, when there are many customers waiting, it may be preferable to reject a new arrival rather than risk that arrival later abandoning without receiving service. On the other hand, rejecting new arrivals increases the percentage of time servers are idle, which also may not be desirable. We address these trade-offs by considering an admission control problem for a M/M/N+M queue when there are costs associated with customer abandonment, server idleness, and turning away customers. First, we formulate the relevant Markov decision process (MDP), show that the optimal policy is of threshold form, and provide a simple and efficient iterative algorithm that does not presuppose a bounded state space to compute the minimum infinite horizon expected average cost and associated threshold level. Under certain conditions we can guarantee that the algorithm provides an exact optimal solution when it stops; otherwise, the algorithm stops when a provided bound on the optimality gap is reached. Next, we solve the approximating diffusion control problem (DCP) that arises in the Halfin–Whitt many-server limit regime. This allows us to establish that the parameter space has a sharp division. Specifically, there is an optimal solution with a finite threshold level when the cost of an abandonment exceeds the cost of rejecting a customer; otherwise, there is an optimal solution that exercises no control. This analysis also yields a convenient analytic expression for the infinite horizon expected average cost as a function of the threshold level. Finally, we propose a policy for the original system that is based on the DCP solution, and show that this policy is asymptotically optimal. Our extensive numerical study shows that the control that arises from solving the DCP achieves a very similar cost to the control that arises from solving the MDP, even when the number of servers is small.     

Viability with Probabilistic Knowledge of Initial Condition, Application to Optimal Control

In this paper we provide an extension of the Viability and Invariance Theorems in the Wasserstein metric space of probability measures with finite quadratic moments in ? ^d for controlled systems of which the dynamic is bounded and Lipschitz. Then we characterize the viability and invariance kernels as the largest viability (resp. invariance) domains. As application of our result we consider an optimal control problem of Mayer type with lower semicontinuous cost function for the same controlled system with uncertainty on the initial state modeled by a probability measure. Following Frankowska, we prove using the epigraphical viability approach that the value function is the unique lower semicontinuous proximal episolution of a suitable Hamilton Jacobi equation.     

有限时区非线性系统的最优切换控制

This paper proposes a optimal control problem for a general nonlinear systems with finitely many admissible control settings and with costs assigned to switching of controls. With dynamic programming and viscosity solution theory we show that the switching lower-value function is a viscosity solution of the appropriate systems of quasi-variational inequalities(the appropriate generalization of the Hamilton-Jacobi equation in this context) and that the minimal such switching-storage function is equal to the continuous switching lower-value for the game. With the lower value function a optimal switching control is designed for minimizing the cost of running the systems.     

Impulse and continuous control of piecewise deterministic Markov processes

In this paper we consider the problem of impulse and continuous control on the jump rate and post jump location parameters of piecewise-deterministic Markov processes (PDP's). In a companion paper we studied the optimal stopping with continuous control problem of PDP's assuming only absolutely continuity along trajectories hypothesis on the final cost function. In this paper we apply these results to obtain optimality equations for the impulse and continuous control problem of PDP's in terms of a set of quasi-variational inequalities as well as on the first jump time operator of the process. No continuity or differential assumptions on the whole state space, neither stability assumptions on the parameters of the problem are required. It is shown that if the post intervention operator satisfies some locally lipschitz continuity along trajectories properties then so will the value function of the impulse and continuous control problem.     

Optimal proportional reinsurance model with transaction costs

Based on the mixed control strategy (regular control and impulse dividend control strategy), we formulate a proportional reinsurance model with transaction costs. For getting the maximal return function and associated mixed control strategy, using Itô calculus and classical mixed control theory, we derive the quasi-variational inequality solution to this optimal problem. Furthermore, we obtain its closed forms under some assumptions.     

Optimal Dividend Payments in a Dual Risk Model with both Fixed and Proportional Costs

In this paper, we study the optimal dividend problem in a dual risk model, which might be appropriate for companies that have fixed expenses and occasional profits. Assuming that dividend payments are subject to both proportional and fixed transaction costs, our object is to maximize the expected present value of dividend payments until ruin, which is defined as the first time the company's surplus becomes negative. This optimization problem is formulated as a stochastic impulse control problem. By solving the corresponding quasi-variational inequality (QVI), we obtain the analytical solutions of the value function and its corresponding optimal dividend strategy when jump sizes are exponentially distributed.     

A leavable bounded-velocity stochastic control problem

This paper studies bounded-velocity control of a Brownian motion when discretionary stopping, or ‘leaving’, is allowed. The goal is to choose a control law and a stopping time in order to minimize the expected sum of a running and a termination cost, when both costs increase as a function of distance from the origin. There are two versions of this problem: the fully observed case, in which the control multiplies a known gain, and the partially observed case, in which the gain is random and unknown. Without the extra feature of stopping, the fully observed problem originates with Beneš (Stochastic Process. Appl. 2 (1974) 127–140), who showed that the optimal control takes the ‘bang–bang’ form of pushing with maximum velocity toward the origin. We show here that this same control is optimal in the case of discretionary stopping; in the case of power-law costs, we solve the variational equation for the value function and explicitly determine the optimal stopping policy.We also discuss qualitative features of the solution for more general cost structures. When no discretionary stopping is allowed, the partially observed case has been solved by Beneš et al. (Stochastics Monographs, Vol. 5, Gordon & Breach, New York and London, pp. 121–156) and Karatzas and Ocone (Stochastic Anal. Appl. 11 (1993) 569–605). When stopping is allowed, we obtain lower bounds on the optimal stopping region using stopping regions of related, fully observed problems.     

An efficient approach for solving the lot-sizing problem with time-varying storage capacities

We address the dynamic lot size problem assuming time-varying storage capacities. The planning horizon is divided into T periods and stockouts are not allowed. Moreover, for each period, we consider a setup cost, a holding unit cost and a production/ordering unit cost, which can vary through the planning horizon. Although this model can be solved using O(T³) algorithms already introduced in the specialized literature, we show that under this cost structure an optimal solution can be obtained in O(T log T) time. In addition, we show that when production/ordering unit costs are assumed to be constant (i.e., the Wagner–Whitin case), there exists an optimal plan satisfying the Zero Inventory Ordering (ZIO) property.     

方差保费准则下的最优脉冲控制

本文研究保险公司在有再保险控制下的最优脉冲分红问题. 对保险公司的理赔损失, 假定有两家再保险公司参与分保, 且保险公司与两家再保险公司采取不同参数下的方差保费准则. 进一步, 假定保险公司有股东红利分配, 且每次分红有固定交易费和比例税收, 即脉冲分红. 在扩散逼近模型下, 本文应用随机动态规划方法研究破产前的最大期望折现分红, 给出值函数的解析表达式, 进而获得最优再保险策略和分红策略的具体形式.