Optimal stopping of switching diffusions with state dependent switching rates

This paper is concerned with a continuous-time and infinite-horizon optimal stopping problem in switching diffusion models. In contrast to the assumption commonly made in the literature that the regime-switching is modeled by an independent Markov chain, we consider in this paper the case of state-dependent regime-switching. The Hamilton–Jacobi–Bellman (HJB) equation associated with the optimal stopping problem is given by a system of coupled variational inequalities. By means of the dynamic programming (DP) principle, we prove that the value function is the unique viscosity solution of the HJB system. As an interesting application in mathematical finance, we examine the problem of pricing perpetual American put options with state-dependent regime-switching. A numerical procedure is developed based on the DP approach and an efficient discrete tree approximation of the continuous asset price process. Numerical results are reported.     

Optimal switching problems with an infinite set of modes: An approach by randomization and constrained backward SDEs

We address a general optimal switching problem over finite horizon for a stochastic system described by a differential equation driven by Brownian motion. The main novelty is the fact that we allow for infinitely many modes (or regimes, i.e. the possible values of the piecewise-constant control process). We allow all the given coefficients in the model to be path-dependent, that is, their value at any time depends on the past trajectory of the controlled system. The main aim is to introduce a suitable (scalar) backward stochastic differential equation (BSDE), with a constraint on the martingale part, that allows to give a probabilistic representation of the value function of the given problem. This is achieved by randomization of control, i.e. by introducing an auxiliary optimization problem which has the same value as the starting optimal switching problem and for which the desired BSDE representation is obtained. In comparison with the existing literature we do not rely on a system of reflected BSDE nor can we use the associated Hamilton–Jacobi–Bellman equation in our non-Markovian framework.     

Near-optimal controls of differential systems with switching and random jumps subject to fast switching and wideband noise perturbation

This work develops near-optimal controls for systems given by differential equations with wideband noise and random switching. The random switching is modeled by a continuous-time, time-inhomogeneous Markov chain. Under broad conditions, it is shown that there is an associated limit problem, which is a switching jump diffusion. Using near-optimal controls of the limit system, we then build controls for the original systems. It is shown that such constructed controls are nearly optimal.     

有限时区上的最优转换和停止问题

讨论了有限时区上的最优转换和停止问题,它是一类同时具备脉冲控制和最优停止特征的最优控制问题.问题的最优值以及最优转换和停止决策可以由具有混合障碍的多维反射倒向随机微分方程的解来刻画.接着考虑了形式更一般的反射倒向随机微分方程并证明了方程解的存在唯一性.     

有限时区上的最优转换和停止问题

讨论了有限时区上的最优转换和停止问题,它是一类同时具备脉冲控制和最优停止特征的最优控制问题.问题的最优值以及最优转换和停止决策可以由具有混合障碍的多维反射倒向随机微分方程的解来刻画.接着考虑了形式更一般的反射倒向随机微分方程并证明了方程解的存在唯一性.     

有限时区非线性系统的最优切换控制

This paper proposes a optimal control problem for a general nonlinear systems with finitely many admissible control settings and with costs assigned to switching of controls. With dynamic programming and viscosity solution theory we show that the switching lower-value function is a viscosity solution of the appropriate systems of quasi-variational inequalities(the appropriate generalization of the Hamilton-Jacobi equation in this context) and that the minimal such switching-storage function is equal to the continuous switching lower-value for the game. With the lower value function a optimal switching control is designed for minimizing the cost of running the systems.     

Dividend optimization for jump–diffusion model with solvency constraints

Belhaj (2010) established that a barrier strategy is optimal for the dividend problem under jump–diffusion model. However, if the optimal dividend barrier level is set too low, then the bankruptcy probability may be too high to be acceptable. This paper aims to address this issue by taking the solvency constrain into consideration. Precisely, we consider a dividend payment problem with solvency constraint under a jump–diffusion model. Using stochastic control and PIDE, we derive the optimal dividend strategy of the problem.     

Optimal mode-switching for hybrid systems with varying initial states

This paper concerns a particular aspect of the optimal control problem for switched systems that change modes whenever the state intersects certain switching surfaces. These surfaces are assumed to be parameterized by a finite dimensional switching parameter, and the optimization problem we consider is that of minimizing a given cost-functional with respect to the switching parameter under the assumption that the initial state of the system is not a priori known. We approach this problem from two different vantage points by first minimizing the worst possible cost over the given set of initial states using results from min–max optimization. The second approach is based on a sensitivity analysis in which variational arguments give the derivative of the switching parameters with respect to the initial conditions.     

Stochastic Maximum Principle for Forward-Backward Regime Switching Jump Diffusion Systems and Applications to Finance

The authors prove a sufficient stochastic maximum principle for the optimal control of a forward-backward Markov regime switching jump diffusion system and show its connection to dynamic programming principle. The result is applied to a cash flow valuation problem with terminal wealth constraint in a financial market. An explicit optimal strategy is obtained in this example.     

Optimal buying at the global minimum in a regime switching model

This paper addresses the problem of buying an asset at its expected globally minimal price, to that end, we model it as an optimal stopping problem with regime switching driven by a continuous-time Markov chain. We characterize the optimal stopping time by optimizing the value functions and writing them as solutions of a system of integral equations. Finally we develop a stochastic recursive algorithm for numerical implementation.     

Linear-quadratic optimal control under non-Markovian switching

We study a finite-dimensional continuous-time optimal control problem on finite horizon for a controlled diffusion driven by Brownian motion, in the linear-quadratic case. We admit stochastic coefficients, possibly depending on an underlying independent marked point process, so that our model is general enough to include controlled switching systems where the switching mechanism is not required to be Markovian. The problem is solved by means of a Riccati equation, which turned out to be a backward stochastic differential equation driven by the Brownian motion and by the random measure associated with the marked point process.     

Randomized filtering and Bellman equation in Wasserstein space for partial observation control problem

We study a stochastic optimal control problem for a partially observed diffusion. By using the control randomization method in Bandini et al. (2018), we prove a corresponding randomized dynamic programming principle (DPP) for the value function, which is obtained from a flow property of an associated filter process. This DPP is the key step towards our main result: a characterization of the value function of the partial observation control problem as the unique viscosity solution to the corresponding dynamic programming Hamilton–Jacobi–Bellman (HJB) equation. The latter is formulated as a new, fully non linear partial differential equation on the Wasserstein space of probability measures. An important feature of our approach is that it does not require any non-degeneracy condition on the diffusion coefficient, and no condition is imposed to guarantee existence of a density for the filter process solution to the controlled Zakai equation. Finally, we give an explicit solution to our HJB equation in the case of a partially observed non Gaussian linear–quadratic model.     

Near-optimal mean–variance controls under two-time-scale formulations and applications

Although the mean–variance control was initially formulated for financial portfolio management problems in which one wants to maximize the expected return and control the risk, our motivations stem from highway vehicle platoon controls that aim to maximize highway utility while ensuring zero accident. This paper develops near-optimal mean–variance controls of switching diffusion systems. To reduce the computational complexity, with motivations from earlier work on singularly perturbed Markovian systems [Sethi and Zhang, Hierarchical Decision Making in Stochastic Manufacturing Systems, Birkhäuser, Boston, MA, 1994; Yin and Zhang, Continuous-Time Markov Chains and Applications: A Singular Pertubation Approach, Springer-Verlag, New York, 1998 and Yin et al., Ann. Appl. Probab. 10 (2000), pp. 549–572], we use a two-time-scale formulation to treat the underlying system, which is represented by the use of a small parameter. As the small parameter goes to 0, we obtain a limit problem. Using the limit problem as a guide, we construct controls for the original problem, and show that the control so constructed is nearly optimal.     

On the regularity of American options with regime-switching uncertainty

We study the regularity of the stochastic representation of the solution of a class of initial–boundary value problems related to a regime-switching diffusion. This representation is related to the value function of a finite-horizon optimal stopping problem such as the price of an American-style option in finance. We show continuity and smoothness of the value function using coupling and time-change techniques. As an application, we find the minimal payoff scenario for the holder of an American-style option in the presence of regime-switching uncertainty under the assumption that the transition rates are known to lie within level-dependent compact sets.     

Optimal control problems for the reaction–diffusion–convection equation with variable coefficients

The solvability of optimal control problems is proved on both weak and strong solutions of a boundary value problem for the nonlinear reaction–diffusion–convection equation with variable coefficients. In the second case, the requirements for smoothness of the multiplicative control are reduced. The study of extremal problems is based on the proof of the solvability of the corresponding boundary value problems and on the qualitative analysis of their solutions properties. The large data existence results for weak solutions, the maximum principle as well as the local existence and uniqueness of a strong solution are established. Moreover, an optimal feedback control problem is considered. Using methods of the theory of topological degree for set-valued perturbations (with aspheric image sets) of generalized monotone operators, we obtain sufficient conditions for the solvability of this problem in the class of weak solutions.     

Stochastic Differential Games with Multiple Modes and Applications to Portfolio Optimization

Abstract

We study a zero-sum stochastic differential game with multiple modes. The state of the system is governed by “controlled switching” diffusion processes. Under certain conditions, we show that the value functions of this game are unique viscosity solutions of the appropriate Hamilton–Jacobi–Isaac' system of equations. We apply our results to the analysis of a portfolio optimization problem where the investor is playing against the market and wishes to maximize his terminal utility. We show that the maximum terminal utility functions are unique viscosity solutions of the corresponding Hamilton–Jacobi–Isaac' system of equations.     

A sequential computational approach to optimal control problems for differential-algebraic systems based on efficient implicit Runge–Kutta integration

Efficient and reliable integrators are indispensable for the design of sequential solvers for optimal control problems involving continuous dynamics, especially for real-time applications. In this paper, optimal control problems for systems represented by index-1 differential-algebraic equations are investigated. On the basis of a time-scaling transformation, the control is parameterized as a piecewise constant function with variable heights and switching time instants. Compared with control parameterization with fixed time grids, the flexibility of adjusting switching time instants increases the chance of finding the optimal solution. Furthermore, error constraints are introduced in the optimization procedure such that the optimal control obtained has a guarantee of integration accuracy. For the derived approximate nonlinear programming problem, a function evaluation and forward sensitivity propagation algorithm is proposed with an embedded implicit Runge–Kutta integrator, which executes one Newton iteration in the limit by employing a predictor-corrector strategy. This algorithm is combined with a nonlinear programming solver Ipopt to construct the optimal control solver. Numerical experiments for the solution of the optimal control problem for a Delta robot demonstrate that the computational speed of this solver is increased by a factor of 0.5–2 when compared with the same solver without the predictor-corrector strategy, and increased by a factor of 20–40 when compared with solver embedding IDAS, the Implicit Differential-Algebraic solver with Sensitivity capabilities developed by Lawrence Livermore National Laboratory. Meanwhile, the accuracy loss compared with the one using IDAS is small and admissible.     

Optimizing acyclic traffic signal switching sequences through an Extended Linear Complementarity Problem formulation

In this paper we first show how the Extended Linear Complementarity Problem, which is a mathematical programming problem, can be used to design optimal switching schemes for a class of switched systems with linear dynamics subject to saturation. More specifically, we consider the determination of the optimal switching time instants (the switching sequences are acyclic, but the phase sequence is pre-fixed). Although this method yields globally optimal switching time sequences, it is not feasible in practice due to its computational complexity. Therefore, we also discuss some approximations that lead to suboptimal switching time sequences that can be computed very efficiently and for which the value of the objective function is close to the global optimum. Finally we use these results to design optimal switching time sequences for a traffic signal controlled intersection so as to minimize criteria such as average queue length, worst case queue length, average waiting time, and so on.     

Optimal control laws for the model of information diffusion in a social group

The article investigates two models of information diffusion in a social group. The dynamics of the process is described by a one-dimensional controlled Riccati differential equation. Our two models differ from the original model of K. V. Izmodenova and A. P. Mikhailov in the choice of the functional being optimized. Two different choices of the optimand functional are considered. The optimal control problems are solved by the Pontryagin maximum principle. It is shown that the optimal control program is a relay function of time with at most one switching point. Conditions on the problem parameters are proposed that are easy to check and guarantee the existence of an optimal-control switching point. The theoretical analysis leads to a one-dimensional convex minimization problem to find the optimal-control switching point. The article also describes an alternative approach to the construction of the optimal solution, which does not resort to the maximum principle and instead utilizes a special representation of the optimand functional and works with reachability sets that are independent of the functional. For the two models considered in this article optimal feedback controls are derived from the programmed optimal controls.     

Optimal investment and premium control in a nonlinear diffusion model

This paper considers the optimal investment and premium control problem in a diffusion approximation to a non-homogeneous compound Poisson process. In the nonlinear diffusion model, it is assumed that there is an unspecified monotone function describing the relationship between the safety loading of premium and the time-varying claim arrival rate. Hence, in addition to the investment control, the premium rate can be served as a control variable in the optimization problem. Specifically, the problem is investigated in two cases: (i) maximizing the expected utility of terminal wealth, and (ii) minimizing the probability of ruin respectively. In both cases, some properties of the value functions are derived, and closed-form expressions for the optimal policies and the value functions are obtained. The results show that the optimal investment policy and the optimal premium control policy are dependent on each other. Most interestingly, as an example, we show that the nonlinear diffusion model reduces to a diffusion model with a quadratic drift coefficient when the function associated with the premium rate and the claim arrival rate takes a special form. This example shows that the model of study represents a class of nonlinear stochastic control risk model.