A Fokker–Planck control framework for multidimensional stochastic processes

An efficient framework for the optimal control of probability density functions (PDFs) of multidimensional stochastic processes is presented. This framework is based on the Fokker–Planck equation that governs the time evolution of the PDF of stochastic processes and on tracking objectives of terminal configuration of the desired PDF. The corresponding optimization problems are formulated as a sequence of open-loop optimality systems in a receding-horizon control strategy. Many theoretical results concerning the forward and the optimal control problem are provided. In particular, it is shown that under appropriate assumptions the open-loop bilinear control function is unique. The resulting optimality system is discretized by the Chang–Cooper scheme that guarantees positivity of the forward solution. The effectiveness of the proposed computational framework is validated with a stochastic Lotka–Volterra model and a noised limit cycle model.     

On a Class of Optimal Control Problems with State Jumps

In this paper, we consider a class of optimal control problems in which the dynamical system involves a finite number of switching times together with a state jump at each of these switching times. The locations of these switching times and a parameter vector representing the state jumps are taken as decision variables. We show that this class of optimal control problems is equivalent to a special class of optimal parameter selection problems. Gradient formulas for the cost functional and the constraint functional are derived. On this basis, a computational algorithm is proposed. For illustration, a numerical example is included.     

Ergodic control of reflected diffusions with jumps

We discuss ergodicity properties of a controlled jumps diffusion process reflected from the boundary of a bounded domain. The control parameters act on the drift term and on a first-order-type jump density. The controlled process is generated via a Girsanov change of probability, and a long-run average criterion is optimized. An optimal stationary feedback is constructed by means of the Hamilton-Jacobi-Bellman equation.     

On stationarity conditions in an optimal control problem with a simple contact with the phase boundary

A certain class of optimal control problems with a one-dimensional phase constraint is considered. When a trajectory contacts the phase boundary on an interval, we employ a special procedure (two-stage variation) to obtain optimality conditions in the Gamkrelidze form and then in the Dubovitskii–Milyutin form, including the sign definiteness property measure density and its jumps at junction points.     

Stability analysis for networked control systems with sampling,transmission protocols and input delays

In this paper, the stability problem is investigated for networked control systems. Input delays and multiple communication imperfections containing time-varying transmission intervals and transmission protocols are considered. A unified framework based on the hybrid systems with memory is proposed to model the whole networked control system. Hybrid systems with memory are used to model hybrid systems affected by delays and permit multiple jumps at a jumping instant. The stability analysis depends on the Lyapunov–Krasovskii functional approaches for hybrid systems with memory and the proposed stability theorem does not need strict decrease of the Lyapunov–Krasovskii functional during jumps. Based on the developed stability theorems, stability conditions for networked control systems are established. An explicit formula is given to compute the maximal allowable transmission interval. In the special case that the networked control system contains linear dynamics, an explicit Lyapunov functional is constructed and stability conditions in terms of linear matrix inequalities (LMI) are proposed. Finally, an example of a chemical batch reactor is given to illustrate the effectiveness of the proposed results.     

Stochastic optimization theory of backward stochastic differential equations with jumps and viscosity solutions of Hamilton–Jacobi–Bellman equations

In this paper we study stochastic optimal control problems with jumps with the help of the theory of Backward Stochastic Differential Equations (BSDEs) with jumps. We generalize the results of Peng [S. Peng, BSDE and stochastic optimizations, in: J. Yan, S. Peng, S. Fang, L. Wu, Topics in Stochastic Analysis, Science Press, Beijing, 1997 (Chapter 2) (in Chinese)] by considering cost functionals defined by controlled BSDEs with jumps. The application of BSDE methods, in particular, the use of the notion of stochastic backward semigroups introduced by Peng in the above-mentioned work allows a straightforward proof of a dynamic programming principle for value functions associated with stochastic optimal control problems with jumps. We prove that the value functions are the viscosity solutions of the associated generalized Hamilton–Jacobi–Bellman equations with integral-differential operators. For this proof, we adapt Peng’s BSDE approach, given in the above-mentioned reference, developed in the framework of stochastic control problems driven by Brownian motion to that of stochastic control problems driven by Brownian motion and Poisson random measure.     

A note on first-passage times of continuously time-changed Brownian motion

The probability of a Brownian motion with drift to remain between two constant barriers (for some period of time) is known explicitly. In mathematical finance, this and related results are required, for example, for the pricing of single-barrier and double-barrier options in a Black-Scholes framework. One popular possibility to generalize the Black-Scholes model is to introduce a stochastic time scale. This equips the modelled returns with desirable stylized facts such as volatility clusters and jumps. For continuous time transformations, independent of the Brownian motion, we show that analytical results for the double-barrier problem can be obtained via the Laplace transform of the time change. The result is a very efficient power series representation for the resulting exit probabilities. We discuss possible specifications of the time change based on integrated intensities of shot-noise type and of basic affine process type.     

The perturbed compound Poisson risk model with two-sided jumps

In this paper, we consider a perturbed compound Poisson risk model with two-sided jumps. The downward jumps represent the claims following an arbitrary distribution, while the upward jumps are also allowed to represent the random gains. Assuming that the density function of the upward jumps has a rational Laplace transform, the Laplace transforms and defective renewal equations for the discounted penalty functions are derived, and the asymptotic estimate for the probability of ruin is also studied for heavy-tailed downward jumps. Finally, some explicit expressions for the discounted penalty functions, as well as numerical examples, are given.     

Impulsive optimal control with finite or infinite time horizon

We consider a dynamical system subjected to feedback optimal control in such a way that the evolution of the state exhibits both sudden jumps and continuous changes. Previously obtained necessary conditions (Ref. 1) for such impulsive optimal feedback controls are generalized to admit the case of infinite time horizon; this generalization permits application to a wider class of problems. The results are illustrated by application to a version of the innkeeper's problem.Dedicated to G. Leitmann     

Reflected BSDEs with nonpositive jumps,and controller-and-stopper games

We study a class of reflected backward stochastic differential equations with nonpositive jumps and upper barrier. Existence and uniqueness of a minimal solution are proved by a double penalization approach under regularity assumptions on the obstacle. In a suitable regime switching diffusion framework, we show the connection between our class of BSDEs and fully nonlinear variational inequalities. Our BSDE representation provides in particular a Feynman–Kac type formula for PDEs associated to general zero-sum stochastic differential controller-and-stopper games, where control affects both drift and diffusion term, and the diffusion coefficient can be degenerate. Moreover, we state a dual game formula of this BSDE minimal solution involving equivalent change of probability measures, and discount processes. This gives in particular a new representation for zero-sum stochastic differential controller-and-stopper games.     

Variance-constrained dissipative observer-based control for a class of nonlinear stochastic systems with degraded measurements

This paper is concerned with the variance-constrained dissipative control problem for a class of stochastic nonlinear systems with multiple degraded measurements, where the degraded probability for each sensor is governed by an individual random variable satisfying a certain probabilistic distribution over a given interval. The purpose of the problem is to design an observer-based controller such that, for all possible degraded measurements, the closed-loop system is exponentially mean-square stable and strictly dissipative, while the individual steady-state variance is not more than the pre-specified upper bound constraints. A general framework is established so that the required exponential mean-square stability, dissipativity as well as the variance constraints can be easily enforced. A sufficient condition is given for the solvability of the addressed multiobjective control problem, and the desired observer and controller gains are characterized in terms of the solution to a convex optimization problem that can be easily solved by using the semi-definite programming method. Finally, a numerical example is presented to show the effectiveness and applicability of the proposed algorithm.     

A unifying formulation of the Fokker–Planck–Kolmogorov equation for general stochastic hybrid systems

A general formulation of the Fokker–Planck–Kolmogorov (FPK) equation for stochastic hybrid systems is presented, within the framework of Generalized Stochastic Hybrid Systems (GSHSs). The FPK equation describes the time evolution of the probability law of the hybrid state. Our derivation is based on the concept of mean jump intensity, which is related to both the usual stochastic intensity (in the case of spontaneous jumps) and the notion of probability current (in the case of forced jumps). This work unifies all previously known instances of the FPK equation for stochastic hybrid systems, and provides GSHS practitioners with a tool to derive the correct evolution equation for the probability law of the state in any given example.     

Reciprocal Class of Jump Processes

Processes having the same bridges as a given reference Markov process constitute its reciprocal class. In this paper we study the reciprocal class of compound Poisson processes whose jumps belong to a finite set \(\mathcal {A}\subset \mathbb {R}^{d}\). We propose a characterization of the reciprocal class as the unique set of probability measures on which a family of time and space transformations induces the same density, expressed in terms of the reciprocal invariants. The geometry of \(\mathcal {A}\) plays a crucial role in the design of the transformations, and we use tools from discrete geometry to obtain an optimal characterization. We deduce explicit conditions for two Markov jump processes to belong to the same class. Finally, we provide a natural interpretation of the invariants as short-time asymptotics for the probability that the reference process makes a cycle around its current state.     

First passage times of reflected Ornstein–Uhlenbeck processes with two-sided jumps

In this paper we consider the first passage problem for reflected jump-type Ornstein–Uhlenbeck processes with two-reflecting barriers. We calculate the explicit joint Laplace transform of the first passage time and the corresponding undershoot when the jumps follow a two-sided mixed exponential law. The method of contour integrals proposed by Jacobsen and Jensen (in Stoch. Process. Appl. 117: 1330–1356, 2007) is applied to obtain the explicit joint Laplace transform. Finally, a comparison concerning Laplace transforms between the reflected case and non-reflected case is presented by taking smooth-pasting conditions at reflecting barriers into account.     

Stability analysis and optimal control of stochastic singular systems

In this paper, problems of stability and optimal control for a class of stochastic singular systems are studied. Firstly, under some appropriate assumptions, some new results about mean-square admissibility are developed and the corresponding LMI sufficient condition is given. Secondly, finite-time horizon and infinite-time horizon linear quadratic (LQ) control problems for the stochastic singular system are investigated, in which the coefficients are allowed to be random in control input and quadratic criterion. Some results involving new stochastic generalized Riccati equation are discussed as well. Finally, the proposed LQ control model for stochastic singular systems provides an appropriate and effective framework to study the portfolio selection problem in light of the recent development on general stochastic LQ problems.     

A note on optimal investment–consumption–insurance in a Lévy market

In Shen and Wei (2014) an optimal investment, consumption and life insurance purchase problem for a wage earner with Brownian information has been investigated. This paper discusses the same problem but extend their results to a geometric Itô–Lévy jump process. Our modelling framework is very general as it allows random parameters which are unbounded and involves some jumps. It also covers parameters which are both Markovian and non-Markovian functionals. Unlike in Shen and Wei (2014) who considered a diffusion framework, ours solves the problem using a novel approach, which combines the Hamilton–Jacobi–Bellman (HJB) and a backward stochastic differential equation (BSDE) in a Lévy market setup. We illustrate our results by two examples.     

Probability density function of leaky integrate-and-fire model with Lévy noise and its numerical approximation

We investigate the numerical analysis of leaky integrate-and-fire model with Lévy noise. We consider a neuronal model in which probability density function of a neuron in some potential at any time is modeled by a transport equation. Lévy noise is included due to jumps by excitatory and inhibitory impulses. Due to these jumps the resulting equation is a transport equation containing two integral in right-hand side (jumps). We design, implement, and analyze numerical methods of finite volume type. Some numerical examples are also included.     

Invariant Measure for Diffusions with Jumps

Our purpose is to study an ergodic linear equation associated to diffusion processes with jumps in the whole space. This integro-differential equation plays a fundamental role in ergodic control problems of second order Markov processes. The key result is to prove the existence and uniqueness of an invariant density function for a jump diffusion, whose lower order coefficients are only Borel measurable. Based on this invariant probability, existence and uniqueness (up to an additive constant) of solutions to the ergodic linear equation are established. Accepted 24 February 1998     

A game-theoretic analysis of information security investment for multiple firms in a network

The application of Internet of Things promotes the cooperation among firms, and it also introduces some information security issues. Due to the vulnerability of the communication network, firms need to invest in information security technologies to protect their confidential information. In this paper, considering the multiple-step propagation of a security breach in a fully connected network, an information security investment game among n firms is investigated. We make meticulous theoretic and experimental analyses on both the Nash equilibrium solution and the optimal solution. The results show that a larger network size (n) or a larger one-step propagation probability (q) has a negative effect on the Nash equilibrium investment. The optimal investment does not necessarily increase in n or q, and its variation trend depends on the concrete conditions. A compensation mechanism is proposed to encourage firms to coordinate their strategies and invest a higher amount equal to the optimal investment when they make decisions individually. At last, our model is extended by considering another direct breach probability function and another network structure, respectively. We find that a higher connection density of the network will result in a greater expected cost for each firm.     

关于带跳的反射扩散过程的随机最优控制问题

研究一类半空间上带泊松跳的反射扩散过程的随机最优控制问题。得到关于这一控制问题的非线性Nisio半群,和联系这一半群的带Neumann边界条件的哈密顿。雅可比。贝尔曼方程。讨论这一类方程的粘性解的存在唯一性等问题。证明该控制问题中的价值函数是这一方程的一个粘性解。