Lévy-Driven Carma Processes

Properties and examples of continuous-time ARMA (CARMA) processes driven by Lévy processes are examined. By allowing Lévy processes to replace Brownian motion in the definition of a Gaussian CARMA process, we obtain a much richer class of possibly heavy-tailed continuous-time stationary processes with many potential applications in finance, where such heavy tails are frequently observed in practice. If the Lévy process has finite second moments, the correlation structure of the CARMA process is the same as that of a corresponding Gaussian CARMA process. In this paper we make use of the properties of general Lévy processes to investigate CARMA processes driven by Lévy processes {W(t)} without the restriction to finite second moments. We assume only that W (1) has finite r-th absolute moment for some strictly positive r. The processes so obtained include CARMA processes with marginal symmetric stable distributions.     

New sufficient conditions for average optimality in continuous-time Markov decision processes

This paper is devoted to studying continuous-time Markov decision processes with general state and action spaces, under the long-run expected average reward criterion. The transition rates of the underlying continuous-time Markov processes are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. We provide new sufficient conditions for the existence of average optimal policies. Moreover, such sufficient conditions are imposed on the controlled process’ primitive data and thus they are directly verifiable. Finally, we apply our results to two new examples.     

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

Abstract

In this article, we study continuous-time Markov decision processes in Polish spaces. The optimality criterion to be maximized is the expected discounted criterion. The transition rates may be unbounded, and the reward rates may have neither upper nor lower bounds. We provide conditions on the controlled system's primitive data under which we prove that the transition functions of possibly non-homogeneous continuous-time Markov processes are regular by using Feller's construction approach to such transition functions. Then, under continuity and compactness conditions we prove the existence of optimal stationary policies by using the technique of extended infinitesimal operators associated with the transition functions of possibly non-homogeneous continuous-time Markov processes, and also provide a recursive way to compute (or at least to approximate) the optimal reward values. The conditions provided in this paper are different from those used in the previous literature, and they are illustrated with an example.     

Diffusion Approximations for Queues with Markovian Bases

Consider a base family of state-dependent queues whose queue-length process can be formulated by a continuous-time Markov process. In this paper, we develop a piecewise-constant diffusion model for an enlarged family of queues, each of whose members has arrival and service distributions generalized from those of the associated queue in the base. The enlarged family covers many standard queueing systems with finite waiting spaces, finite sources and so on. We provide a unifying explicit expression for the steady-state distribution, which is consistent with the exact result when the arrival and service distributions are those of the base. The model is an extension as well as a refinement of the M/M/s-consistent diffusion model for the GI/G/s queue developed by Kimura [13] where the base was a birth-and-death process. As a typical base, we still focus on birth-and-death processes, but we also consider a class of continuous-time Markov processes with lower-triangular infinitesimal generators.     

Strong Differential Subordination and Sharp Inequalities for Orthogonal Processes

We introduce a strong differential α-subordination for continuous-time processes, which generalizes this notion from the discrete-time setting, due to Burkholder and Choi. Then we determine the best constants in the L ^p estimates for a nonnegative submartingale and its strong α-subordinate under an additional assumption on the orthogonality of these two processes.     

Bias optimality for multichain continuous-time Markov decision processes

This paper deals with the bias optimality of multichain models for finite continuous-time Markov decision processes. Based on new performance difference formulas developed here, we prove the convergence of a so-called bias-optimal policy iteration algorithm, which can be used to obtain bias-optimal policies in a finite number of iterations.     

Average sample-path optimality for continuous-time Markov decision processes in Polish spaces

In this paper we study the average sample-path cost(ASPC) problem for continuous-time Markov decision processes in Polish spaces.To the best of our knowledge,this paper is a first attempt to study the ASPC criterion on continuous-time MDPs with Polish state and action spaces.The corresponding transition rates are allowed to be unbounded,and the cost rates may have neither upper nor lower bounds.Under some mild hypotheses,we prove the existence of ε(ε≥ 0)-ASPC optimal stationary policies based on two differe...     

On the exponential process associated with a CARMA-type process

We study the correlation decay and the expected maximal increments of the exponential processes determined by continuous-time autoregressive moving average (CARMA)-type processes of order (p, q). We consider two background driving processes, namely fractional Brownian motions and Lévy processes with exponential moments. The results presented in this paper are significant extensions of those very recent works on the Ornstein–Uhlenbeck-type case (p = 1, q = 0), and we develop more refined techniques to meet the general (p, q). In the concluding section, we discuss the perspective role of exponential CARMA-type processes in stochastic modelling of the burst phenomena in telecommunications and the leverage effect in financial econometrics.     

Conservative markov processes on a topological space

A Markov operator preservingC(X) is known to induce a decomposition of the locally compact spaceX to conservative and dissipative parts. Two notions of ergodicity are defined and the existence of subprocesses is studied. A sufficient condition for the existence of a conservative subprocess is given, and then the process is assumed to be conservative. When it has no subprocesses, sufficient conditions for the existence of a σ-finite invariant measure are given, and are extended to continuous-time processes. When the invariant measure is unique, ratio limit theorems are proved for the discrete and continuous time processes. Examples show that some combinations of conservative processes are not necessarily conservative. This paper is a part of the authors’s Ph.D. thesis prepared at the Hebrew University under the direction of Professor S. R. Foguel, to whom the author is grateful for his helpful advice and kind encouragement.     

The central limit theorem for stationary Markov processes with normal generator—with applications to hypergroups

We extend the central limit theorem for additive functionals of a stationary, ergodic Markov chain with normal transition operator due to Gordin and Lif?ic, 1981 [A remark about a Markov process with normal transition operator, In: Third Vilnius Conference on Probability and Statistics 1, pp. 147–48] to continuous-time Markov processes with normal generators. As examples, we discuss random walks on compact commutative hypergroups as well as certain random walks on non-commutative, compact groups.     

High-level dependence in time series models

We present several notions of high-level dependence for stochastic processes, which have appeared in the literature. We calculate such measures for discrete and continuous-time models, where we concentrate on time series with heavy-tailed marginals, where extremes are likely to occur in clusters. Such models include linear models and solutions to random recurrence equations; in particular, discrete and continuous-time moving average and (G)ARCH processes. To illustrate our results we present a small simulation study.     

Hyperfinite construction of G-expectation

The hyperfinite G-expectation is a nonstandard discrete analogue of G-expectation (in the sense of Robinsonian nonstandard analysis). A lifting of a continuous-time G-expectation operator is defined as a hyperfinite G-expectation which is infinitely close, in the sense of nonstandard topology, to the continuous-time G-expectation. We develop the basic theory for hyperfinite G-expectations and prove an existence theorem for liftings of (continuous-time) G-expectation. For the proof of the lifting theorem, we use a new discretization theorem for the G-expectation (also established in this paper, based on the work of Dolinsky et al. [Weak approximation of G-expectations, Stoch. Process. Appl. 122(2) (2012), pp. 664–675]).     

Structure of the Particle Population for a Branching Random Walk with a Critical Reproduction Law

We consider a continuous-time symmetric branching random walk on the d-dimensional lattice, d ≥?1, and assume that at the initial moment there is one particle at every lattice point. Moreover, we assume that the underlying random walk has a finite variance of jumps and the reproduction law is described by a continuous-time Markov branching process (a continuous-time analog of a Bienamye-Galton-Watson process) at every lattice point. We study the structure of the particle subpopulation generated by the initial particle situated at a lattice point x. We replay why vanishing of the majority of subpopulations does not affect the convergence to the steady state and leads to clusterization for lattice dimensions d =?1 and d =?2.

     

Strong Law of Large Numbers and Central Limit Theorems for Functionals of Inhomogeneous Semi-Markov Processes

Limit theorems for functionals of classical (homogeneous) Markov renewal and semi-Markov processes have been known for a long time, since the pioneering work of Pyke Schaufele (Limit theorems for Markov renewal processes, Ann. Math. Statist., 35(4):1746–1764, 1964). Since then, these processes, as well as their time-inhomogeneous generalizations, have found many applications, for example, in finance and insurance. Unfortunately, no limit theorems have been obtained for functionals of inhomogeneous Markov renewal and semi-Markov processes as of today, to the best of the authors’ knowledge. In this article, we provide strong law of large numbers and central limit theorem results for such processes. In particular, we make an important connection of our results with the theory of ergodicity of inhomogeneous Markov chains. Finally, we provide an application to risk processes used in insurance by considering a inhomogeneous semi-Markov version of the well-known continuous-time Markov chain model, widely used in the literature.     

Regeneration and renovation in queues

Regenerative events for different queueing models are considered. The aim of this paper is to construct these events for continuous-time processes if they are given for the corresponding discrete-time model. The construction uses so-called renovative events revealing the property of the state at timen of the discrete-time model to be independent (in an algebraic sense) of the states referring to epochs not later thann −L (whereL is some constant) given that there are some restrictions on the “governing sequence”. Different types of multi-server and multi-phase queues are considered.     

Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes

ABSTRACT

The main goal of this paper is to study the infinite-horizon long run average continuous-time optimal control problem of piecewise deterministic Markov processes (PDMPs) with the control acting continuously on the jump intensity λ and on the transition measure Q of the process. We provide conditions for the existence of a solution to an integro-differential optimality inequality, the so called Hamilton-Jacobi-Bellman (HJB) equation, and for the existence of a deterministic stationary optimal policy. These results are obtained by using the so-called vanishing discount approach, under some continuity and compactness assumptions on the parameters of the problem, as well as some non-explosive conditions for the process.     

盈余过程的马氏性与索赔到达间隔分布为离散型的连续时间风险模型

本文研究广泛的一类连续时间风险模型盈余过程的马氏性,得到了盈余过程成为马氏过程的充分必要条件．首次建立了索赔到达间隔为离散型分布的连续时间风险模型．并对两个基本特例得到了破产概率的准确表达式．     

Equivalence of two notions of continuity for stationary continuous-time information sources

The notion of continuity for continuous-time information sources which was introduced by Pinsker has found numerous applications in information theory. Continuity in probability is an important concept in the theory of continuous-time stochastic processes. It is shown that these two forms of continuity are equivalent for stationary processes whose state space is a separable metric space.     

Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach

This paper deals with a continuous-time Markov decision process in Borel state and action spaces and with unbounded transition rates. Under history-dependent policies, the controlled process may not be Markov. The main contribution is that for such non-Markov processes we establish the Dynkin formula, which plays important roles in establishing optimality results for continuous-time Markov decision processes. We further illustrate this by showing, for a discounted continuous-time Markov decision process, the existence of a deterministic stationary optimal policy (out of the class of history-dependent policies) and characterizing the value function through the Bellman equation.     

Multiple Wiener-itô integral expansions for level-crossing-count functionals

Summary This paper applies the stochastic calculus of multiple Wiener-Itô integral expansions to express the number of crossings of the mean level by a stationary (discrete- or continuous-time) Gaussian process within a fixed time interval [0,T]. The resulting expansions involve a class of hypergeometric functions, for which recursion and differential relations and some asymptotic properties are derived. The representation obtained for level-crossing counts is applied to prove a central limit theorem of Cuzick (1976) for level crossings in continuous time, using a general central limit theorem of Chambers and Slud (1989a) for processes expressed via multiple Wiener-Itô integral expansions in terms of a stationary Gaussian process. Analogous results are given also for discrete-time processes. This approach proves that the limiting variance is strictly positive, without additional assumptions needed by Cuzick.Research supported by Office of Naval Research contracts N00014-86-K-0007 and N00014-89-J-1051