On Markov Chain Approximations to Semilinear Partial Differential Equations Driven by Poisson Measure Noise

We consider the stochastic model of water pollution, which mathematically can be written with a stochastic partial differential equation driven by Poisson measure noise. We use a stochastic particle Markov chain method to produce an implementable approximate solution. Our main result is the annealed law of large numbers establishing convergence in probability of our Markov chains to the solution of the stochastic reaction-diffusion equation while considering the Poisson source as a random medium for the Markov chains.     

Switching diffusion logistic models involving singularly perturbed Markov chains: Weak convergence and stochastic permanence

Focusing on stochastic dynamics involve continuous states as well as discrete events, this article investigates stochastic logistic model with regime switching modulated by a singular Markov chain involving a small parameter. This Markov chain undergoes weak and strong interactions, where the small parameter is used to reflect rapid rate of regime switching among each state class. Two-time-scale formulation is used to reduce the complexity. We obtain weak convergence of the underlying system so that the limit has much simpler structure. Then we utilize the structure of limit system as a bridge, to invest stochastic permanence of original system driving by a singular Markov chain with a large number of states. Sufficient conditions for stochastic permanence are obtained. A couple of examples and numerical simulations are given to illustrate our results.     

Credit risk model with contagious default dependencies affected by macro-economic condition

We consider a credit risk model with two industrial sectors, where defaults of corporations would be influenced by two factors. The first factor represents the macro economic condition which would affect the default intensities of the two industrial sectors differently. The second factor reflects the influences of the past defaults of corporations against other active corporations, where such influences would affect the two industrial sectors differently. A two-layer Markov chain model is developed, where the macro economic condition is described as a birth-death process, while another Markov chain represents the stochastic characteristics of defaults with default intensities dependent on the state of the birth-death process and the number of defaults in two sectors. Although the state space of the two-layer Markov chain is huge, the fundamental absorbing process with a reasonable state space size could capture the first passage time structure of the two-layer Markov chain, thereby enabling one to evaluate the joint probability of the number of defaults in two sectors via the uniformization procedure of Keilson. This in turn enables one to value a variety of derivatives defined on the underlying credit portfolios. In this paper, we focus on a financial product called CDO, and a related option.     

A dynamic model for social networks†

A continuous‐time binary‐matrix‐valued Markov chain is used to model the process by which social structure effects individual behavior. The model is developed in the context of sociometric networks of interpersonal affect. By viewing the network as a time‐dependent stochastic process it is possible to construct transition intensity equations for the probability that choices between group members will change. These equations can contain parameters for structural effects. Empirical estimates of the parameters can be interpreted as measures of structural tendencies. Some elementary processes are described and the application of the model to cross‐sectional data is explained in terms of the steady state solution to the process.     

State estimation for partially observed Markov chains

Recursive equations are derived for the conditional distribution of the state of a Markov chain, given observations of a function of the state. Mainly continuous time chains are considered. The equations for the conditional distribution are given in matrix form and in differential equation form. The conditional distribution itself forms a Markov process. Special cases considered are doubly stochastic Poisson processes with a Markovian intensity, Markov chains with a random time, and Markovian approximations of semi-Markov processes. Further the results are used to compute the Radon-Nikodym derivative for two probability measures for a Markov chain, when a function of the state is observed.     

Optimal liquidation under partial information with price impact

We study the optimal liquidation problem in a market model where the bid price follows a geometric pure jump process whose local characteristics are driven by an unobservable finite-state Markov chain and by the liquidation rate. This model is consistent with stylized facts of high frequency data such as the discrete nature of tick data and the clustering in the order flow. We include both temporary and permanent effects into our analysis. We use stochastic filtering to reduce the optimal liquidation problem to an equivalent optimization problem under complete information. This leads to a stochastic control problem for piecewise deterministic Markov processes (PDMPs). We carry out a detailed mathematical analysis of this problem. In particular, we derive the optimality equation for the value function, we characterize the value function as continuous viscosity solution of the associated dynamic programming equation, and we prove a novel comparison result. The paper concludes with numerical results illustrating the impact of partial information and price impact on the value function and on the optimal liquidation rate.     

Existence and stability for Fokker–Planck equations with log-concave reference measure

We study Markov processes associated with stochastic differential equations, whose non-linearities are gradients of convex functionals. We prove a general result of existence of such Markov processes and a priori estimates on the transition probabilities. The main result is the following stability property: if the associated invariant measures converge weakly, then the Markov processes converge in law. The proofs are based on the interpretation of a Fokker–Planck equation as the steepest descent flow of the relative entropy functional in the space of probability measures, endowed with the Wasserstein distance.     

On long-term arbitrage opportunities in Markovian models of financial markets

A discrete-time infinite horizon stock market model is considered where the logarithm of the price is assumed to be a Markov chain arising from the time-discretization of a stochastic differential equation. Conditions are given which ensure that there exist investment strategies producing an exponential growth of wealth with a probability converging to 1. The rate of this convergence is studied using large deviation techniques.     

Repair strategies in an uncertain environment: Markov decision process approach

This paper deals with repair strategies that maximize the time until a catastrophic event, that is, when there is a vital need for equipment, and the equipment fails to function. We examine the case where the need for the equipment varies over time according to a Markov chain. This means that the environment can be in different states, each with their own probability of the initiating event occurring. We model the form of the optimal policy for repair under this uncertain environment by Markov decision processes.     

On performance comparison of MR/GI/1 queues

In this paper we are interested in the effect that dependencies in the arrival process to a queue have on queueing properties such as mean queue length and mean waiting time. We start with a review of the well known relations used to compare random variables and random vectors, e.g., stochastic orderings, stochastic increasing convexity, and strong stochastic increasing concavity. These relations and others are used to compare interarrival times in Markov renewal processes first in the case where the interarrival time distributions depend only on the current state in the underlying Markov chain and then in the general case where these interarrivai times depend on both the current state and the next state in that chain. These results are used to study a problem previously considered by Patuwo et al. [14].Then, in order to keep the marginal distributions of the interarrivai times constant, we build a particular transition matrix for the underlying Markov chain depending on a single parameter,p. This Markov renewal process is used in the Patuwo et al. [14] problem so as to investigate the behavior of the mean queue length and mean waiting time on a correlation measure depending only onp. As constructed, the interarrival time distributions do not depend onp so that the effects we find depend only on correlation in the arrival process.As a result of this latter construction, we find that the mean queue length is always larger in the case where correlations are non-zero than they are in the more usual case of renewal arrivals (i.e., where the correlations are zero). The implications of our results are clear.     

The ergodic theory of Markov chains in random environments

Summary A general formulation of the stochastic model for a Markov chain in a random environment is given, including an analysis of the dependence relations between the environmental process and the controlled Markov chain, in particular the problem of feedback. Assuming stationary environments, the ergodic theory of Markov processes is applied to give conditions for the existence of finite invariant measure (equilibrium distributions) and to obtain ergodic theorems, which provide results on convergence of products of random stochastic matrices. Coupling theory is used to obtain results on direct convergence of these products and the structure of the tail -field. State properties including classification and communication properties are discussed.     

Optimal harvesting for a stochastic regime-switching logistic diffusion system with jumps

The optimization problem of fishing for a stochastic logistic model is studied in this paper. Besides a standard geometric Brownian motion, another two driving processes are taken into account: a stationary Poisson point process and a continuous-time finite-state Markov chain. The classical harvesting problem for this model is a big difficult puzzle since the corresponding Fokker–Planck equations with three types of noise are very difficult to solve. Our main goal of this paper is to work out the optimization problem with respect to stationary probability density. One of the main contributions is to provide a new equivalent approach to overcome this problem. More precisely, an ergodic method is used to show the almost surely equivalency between the time averaging yield and sustainable yield. Results show that the optimal strategy changes with environment. An interesting thing is that the optimal strategy for each state is equivalent to the global optimality.     

Default Times in a Continuous Time Markov Chain Economy

Abstract

A continuous time financial market is considered where randomness is modelled by a finite state Markov chain. Using the chain, a stochastic discount factor is defined. The probability distributions of default times are shown to be given by solutions of a system of coupled partial differential equations.     

Bayesian estimation of stochastic volatility models based on OU processes with marginal Gamma law

This paper discusses practical Bayesian estimation of stochastic volatility models based on OU processes with marginal Gamma laws. Estimation is based on a parameterization which is derived from the Rosiński representation, and has the advantage of being a non-centered parameterization. The parameterization is based on a marked point process, living on the positive real line, with uniformly distributed marks. We define a Markov chain Monte Carlo (MCMC) scheme which enables multiple updates of the latent point process, and generalizes single updating algorithm used earlier. At each MCMC draw more than one point is added or deleted from the latent point process. This is particularly useful for high intensity processes. Furthermore, the article deals with superposition models, where it discuss how the identifiability problem inherent in the superposition model may be avoided by the use of a Markov prior. Finally, applications to simulated data as well as exchange rate data are discussed.     

Convergence,boundedness, and ergodicity of regime-switching diusion processes with infinite memory

We study a class of diffusion processes, which are determined by solutions X(t) to stochastic functional differential equation with infinite memory and random switching represented by Markov chain Λ(t): Under suitable conditions, we investigate convergence and boundedness of both the solutions X(t) and the functional solutions X_t: We show that two solutions (resp., functional solutions) from different initial data living in the same initial switching regime will be close with high probability as time variable tends to infinity, and that the solutions (resp., functional solutions) are uniformly bounded in the mean square sense. Moreover, we prove existence and uniqueness of the invariant probability measure of two-component Markov-Feller process (X_t,Λ(t)); and establish exponential bounds on the rate of convergence to the invariant probability measure under Wasserstein distance. Finally, we provide a concrete example to illustrate our main results.     

An analytic valuation method for multivariate contingent claims with regime-switching volatilities

In this paper, we provide an analytic valuation method for European-type contingent claims written on multiple assets in a stochastic market environment. We employ a two-state Markov regime-switching volatility in order to reflect stochastically changing market conditions. The method is developed by exploiting the probability density of the occupation time for which the underlying asset processes are in a certain regime during a time period. In order to show its usefulness, we derive analytic valuation formulas for quanto options and exchange options with two underlying assets, as examples.     

Reliability of manufacturing equipment in complex environments

We present two stochastic failure models for the reliability evaluation of manufacturing equipment that degrades due to its complex operating environment. The first model examines the case when the environment is a temporally nonhomogeneous continuous-time Markov chain, and the second assumes the environment is a temporally homogeneous semi-Markov process on a finite space. Derived are transform expressions for the lifetime distributions. A few examples are provided to illustrate the main results.     

On stochastic processes in a multilevel control bulk queueing system

This article analyzes some stochastic processes that arise in a bulk single server queue with continuously operating server, state dependent compound Poisson input flow and general state dependent service process. The authors treat the queueing process as a semi–regenerative process and obtain the invariant probability measure and the transient distribution for the embedded Markov chain. The stationary probability measure for the queueing process with continuous time parameter is found by using semi-regenerative techniques. The authors also study the input and output processes and establish ergodic theorems for some functionals of these processes. The results are obtained in terms of the invariant probability measure for the embedded process and the stationary measure for the queueing process with continuous time parameter     

马氏利率下相关险种的离散风险建模和破产概率

在随机利率服从有限齐次Markov链下,建立相关险种离散风险模型,采用递推方法得到了有限时间破产概率的递推等式和最终破产概率的积分等式;给出了有限时间破产概率和最终破产概率的上界,导出了破产时刻余额分布的计算等式.