Poisson’s equation for discrete-time single-birth processes

We consider Poisson’s equation for discrete-time single-birth processes, and we derive its solutions by solving a linear system of infinitely many equations. We apply the solution of Poisson’s equation to obtain the asymptotic variance. The results are further applied to birth–death processes and the scalar-valued GI/M/1-type Markov chains.     

State estimation for partially observed Markov chains

Recursive equations are derived for the conditional distribution of the state of a Markov chain, given observations of a function of the state. Mainly continuous time chains are considered. The equations for the conditional distribution are given in matrix form and in differential equation form. The conditional distribution itself forms a Markov process. Special cases considered are doubly stochastic Poisson processes with a Markovian intensity, Markov chains with a random time, and Markovian approximations of semi-Markov processes. Further the results are used to compute the Radon-Nikodym derivative for two probability measures for a Markov chain, when a function of the state is observed.     

离散时间Markov链几何非常返与代数非常返的判别准则

本文研究可数状态空间离散时间Markov链的几何非常返和代数非常返,利用某状态末离时的矩条件和某方程解的存在性,给出两种非常返性的判别准则.进一步,我们将所得结果应用于研究Geom/G/1排队模型的随机稳定性.     

On the uniqueness of solutions to the Poisson equations for average cost Markov chains with unbounded cost functions

We consider the Poisson equations for denumerable Markov chains with unbounded cost functions. Solutions to the Poisson equations exist in the Banach space of bounded real-valued functions with respect to a weighted supremum norm such that the Markov chain is geometrically ergodic. Under minor additional assumptions the solution is also unique. We give a novel probabilistic proof of this fact using relations between ergodicity and recurrence. The expressions involved in the Poisson equations have many solutions in general. However, the solution that has a finite norm with respect to the weighted supremum norm is the unique solution to the Poisson equations. We illustrate how to determine this solution by considering three queueing examples: a multi-server queue, two independent single server queues, and a priority queue with dependence between the queues.     

The Limit Behaviour of Imprecise Continuous-Time Markov Chains

We study the limit behaviour of a nonlinear differential equation whose solution is a superadditive generalisation of a stochastic matrix, prove convergence, and provide necessary and sufficient conditions for ergodicity. In the linear case, the solution of our differential equation is equal to the matrix exponential of an intensity matrix and can then be interpreted as the transition operator of a homogeneous continuous-time Markov chain. Similarly, in the generalised nonlinear case that we consider, the solution can be interpreted as the lower transition operator of a specific set of non-homogeneous continuous-time Markov chains, called an imprecise continuous-time Markov chain. In this context, our convergence result shows that for a fixed initial state, an imprecise continuous-time Markov chain always converges to a limiting distribution, and our ergodicity result provides a necessary and sufficient condition for this limiting distribution to be independent of the initial state.     

Optimal Consumption and Portfolio with Ambiguity to Markovian Switching

This paper considers the problem of maximizing expected utility from consumption and terminal wealth under model uncertainty for a general semimartingale market, where the agent with an initial capital and a random endowment can invest. To find a solution to the investment problem we use the martingale method. We first prove that under appropriate assumptions a unique solution to the investment problem exists. Then we deduce that the value functions of primal problem and dual problem are convex conjugate functions. Furthermore we consider a diffusion-jump-model where the coefficients depend on the state of a Markov chain and the investor is ambiguity to the intensity of the underlying Poisson process. Finally, for an agent with the logarithmic utility function, we use the stochastic control method to derive the Hamilton-Jacobi-Bellmann (HJB) equation. And the solution to this HJB equation can be determined numerically. We also show how thereby the optimal investment strategy can be computed.     

带跳的分数倒向重随机微分方程及相应的随机积分偏微分方程

本文首次把Poisson随机测度引入分数倒向重随机微分方程,基于可料的Girsanov变换证明由Brown运动、Poisson随机测度和Hurst参数在(1/2,1)范围内的分数Brown运动共同驱动的半线性倒向重随机微分方程解的存在唯一性.在此基础上,本文定义一类半线性随机积分偏微分方程的随机黏性解,并证明该黏性解由带跳分数倒向重随机微分方程的解唯一地给出,对经典的黏性解理论作出有益的补充.     

Continuous-time limit of repeated interactions for a system in a confining potential

We study the continuous-time limit of a class of Markov chains coming from the evolution of classical open systems undergoing repeated interactions. This repeated interaction model has been initially developed for dissipative quantum systems in Attal and Pautrat (2006) and was recently set up for the first time in Deschamps (2012) for classical dynamics. It was particularly shown in the latter that this scheme furnishes a new kind of Markovian evolutions based on Hamilton’s equations of motion. The system is also proved to evolve in the continuous-time limit with a stochastic differential equation. We here extend the convergence of the evolution of the system to more general dynamics, that is, to more general Hamiltonians and probability measures in the definition of the model. We also present a natural way to directly renormalize the initial Hamiltonian in order to obtain the relevant process in a study of the continuous-time limit. Then, even if Hamilton’s equations have no explicit solution in general, we obtain some bounds on the dynamics allowing us to prove the convergence in law of the Markov chain on the system to the solution of a stochastic differential equation, via the infinitesimal generators.     

On categorical time series models with covariates

We study the problem of stationarity and ergodicity for autoregressive multinomial logistic time series models which possibly include a latent process and are defined by a GARCH-type recursive equation. We improve considerably upon the existing conditions about stationarity and ergodicity of those models. Proofs are based on theory developed for chains with complete connections. A useful coupling technique is employed for studying ergodicity of infinite order finite-state stochastic processes which generalize finite-state Markov chains. Furthermore, for the case of finite order Markov chains, we discuss ergodicity properties of a model which includes strongly exogenous but not necessarily bounded covariates.     

Markov chain-based degree distributions of evolving networks

In this paper, we study a class of stochastic processes, called evolving network Markov chains, in evolving networks. Our approach is to transform the degree distribution problem of an evolving network to a corresponding problem of evolving network Markov chains. We investigate the evolving network Markov chains, thereby obtaining some exact formulas as well as a precise criterion for determining whether the steady degree distribution of the evolving network is a power-law or not. With this new method, we finally obtain a rigorous, exact and unified solution of the steady degree distribution of the evolving network.     

Nonlinear reserving in life insurance: Aggregation and mean-field approximation

We suggest a unified approach to claims reserving for life insurance policies with reserve-dependent payments driven by multi-state Markov chains. The associated prospective reserve is formulated as a recursive utility function using the framework of backward stochastic differential equations (BSDE). We show that the prospective reserve satisfies a nonlinear Thiele equation for Markovian BSDEs when the driver is a deterministic function of the reserve and the underlying Markov chain. Aggregation of prospective reserves for large and homogeneous insurance portfolios is considered through mean-field approximations. We show that the corresponding prospective reserve satisfies a BSDE of mean-field type and derive the associated nonlinear Thiele equation.     

Zero-sum risk-sensitive stochastic games for continuous time Markov chains

We study infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled continuous time Markov chains on a countable state space. For the discounted-cost game, we prove the existence of value and saddle-point equilibrium in the class of Markov strategies under nominal conditions. For the ergodic-cost game, we prove the existence of values and saddle point equilibrium by studying the corresponding Hamilton-Jacobi-Isaacs equation under a certain Lyapunov condition.     

Indefinite stochastic linear-quadratic optimal control problems with random jumps and related stochastic Riccati equations

We discuss the stochastic linear-quadratic (LQ) optimal control problem with Poisson processes under the indefinite case. Based on the wellposedness of the LQ problem, the main idea is expressed by the definition of relax compensator that extends the stochastic Hamiltonian system and stochastic Riccati equation with Poisson processes (SREP) from the positive definite case to the indefinite case. We mainly study the existence and uniqueness of the solution for the stochastic Hamiltonian system and obtain the optimal control with open-loop form. Then, we further investigate the existence and uniqueness of the solution for SREP in some special case and obtain the optimal control in close-loop form.     

Networks of infinite-server queues with nonstationary Poisson input

In this paper we focus on networks of infinite-server queues with nonhomogeneous Poisson arrival processes. We start by introducing a more general Poisson-arrival-location model (PALM) in which arrivals move independently through a general state space according to a location stochastic process after arriving according to a nonhomogeneous Poisson process. The usual open network of infinite-server queues, which is also known as a linear population process or a linear stochastic compartmental model, arises in the special case of a finite state space. The mathematical foundation is a Poisson-random-measure representation, which can be obtained by stochastic integration. It implies a time-dependent product-form result: For appropriate initial conditions, the queue lengths (numbers of customers in disjoint subsets of the state space) at any time are independent Poisson random variables. Even though there is no dependence among the queue lengths at each time, there is important dependence among the queue lengths at different times. We show that the joint distribution is multivariate Poisson, and calculate the covariances. A unified framework for constructing stochastic processes of interest is provided by stochastically integrating various functionals of the location process with respect to the Poisson arrival process. We use this approach to study the flows in the queueing network; e.g., we show that the aggregate arrival and departure processes at a given queue (to and from other queues as well as outside the network) are generalized Poisson processes (without necessarily having a rate or unit jumps) if and only if no customer can visit that queue more than once. We also characterize the aggregate arrival and departure processes when customers can visit the queues more frequently. In addition to obtaining structural results, we use the stochastic integrals to obtain explicit expressions for time-dependent means and covariances. We do this in two ways. First, we decompose the entire network into a superposition of independent networks with fixed deterministic routes. Second, we make Markov assumptions, initially for the evolution of the routes and finally for the entire location process. For Markov routing among the queues, the aggregate arrival rates are obtained as the solution to a system of input equations, which have a unique solution under appropriate qualifications, but not in general. Linear ordinary differential equations characterize the time-dependent means and covariances in the totally Markovian case.     

Existence and exponential stability for neutral stochastic fractional differential equations with impulses driven by Poisson jumps

The paper is mainly concerned with a class of neutral stochastic fractional integro-differential equation with Poisson jumps. First, the existence and uniqueness for mild solution of an impulsive stochastic system driven by Poisson jumps is established by using the Banach fixed point theorem and resolvent operator. The exponential stability in the pth moment for mild solution to neutral stochastic fractional integro-differential equations with Poisson jump is obtained by establishing an integral inequality.     

Stochastic Cahn-Hilliard equations driven by Poisson random measures

We study a stochastic Cahn-Hilliard equation driven by a Poisson random measure with Neumann boundary conditions. The global weak solution is established for the equation. Moreover, the existence of a Lyapunov function for the equation and an invariant measure associated with the transition semigroup are proved.     

Statistical inference for misspecified ergodic Lévy driven stochastic differential equation models

We consider the estimation problem of misspecified ergodic Lévy driven stochastic differential equation models based on high-frequency samples. We utilize a widely applicable and tractable Gaussian quasi-likelihood approach which focuses on mean and variance structure. It is shown that the Gaussian quasi-likelihood estimators of the drift and scale parameters still satisfy polynomial type probability estimates and asymptotic normality at the same rate as the correctly specified case. In their derivation process, the theory of extended Poisson equation for time-homogeneous Feller Markov processes plays an important role. Our result confirms the reliability of the Gaussian quasi-likelihood approach for SDE models.     

马氏环境中可数马氏链的Poisson极限率

研究了马氏环境中的可数马氏链,主要证明了过程于小柱集上的回返次数是渐近地服从Poisson分布。为此,引入熵函数h,首先给出了马氏环境中马氏链的Shannon-Mc Millan-Breiman定理,还给出了一个非马氏过程Posson逼近的例子。当环境过程退化为一常数序列时,便得到可数马氏链的Poisson极限定理。这是有限马氏链Pitskel相应结果的拓广。     

Qualitative behaviour of stochastic delay equations with a bounded memory

A dichotomy is proved concerning recurrence properties of the solution of certain stochastic delay equations. If the solution process is recurrent, there exists an invariant measure π on the state space C which is unique (up to a multiplicative constant) and the tail-field is trivial. If π happens to be a probability measure, then for every initial condition, the distribution of the process converges to it as t→∞. We will formulate a sufficient condition for the existence of an invariant probability measure (ipm) in icrnia of Lyapunov junctionals and give two examples, one Heing the stochastic-delay version of the famous logistic equation of population growth. Finally we study approximations of delay equations by Markov chains.     

The maximum principle for a jump-diffusion mean-field model and its application to the mean–variance problem

This paper establishes a necessary and sufficient stochastic maximum principle for a mean-field model with randomness described by Brownian motions and Poisson jumps. We also prove the existence and uniqueness of the solution to a jump-diffusion mean-field backward stochastic differential equation. A new version of the sufficient stochastic maximum principle, which only requires the terminal cost is convex in an expected sense, is applied to solve a bicriteria mean–variance portfolio selection problem.