On Non-Uniqueness of Probability Solutions to the Two-Dimensional Stationary Fokker–Planck–Kolmogorov Equation

The problem of uniqueness of probability solutions to the two-dimensional stationary Fokker–Planck–Kolmogorov equation is considered. Under broad conditions, it is proved that the existence of two different probability solutions implies the existence of an infinite set of linearly independent probability solutions.     

Asymptotic behavior of solutions: An application to stochastic NLP

In this article we study the consistency of optimal and stationary (KKT) points of a stochastic non-linear optimization problem involving expectation functionals, when the underlying probability distribution associated with the random variable is weakly approximated by a sequence of random probability measures. The optimization model includes constraints with expectation functionals those are not captured in direct application of the previous results on optimality conditions exist in the literature. We first study the consistency of stationary points of a general NLP problem with convex and locally Lipschitz data and then apply those results to the stochastic NLP problem and stochastic minimax problem. Moreover, we derive an exponential bound for such approximations using a large deviation principle.

     

Stationary lower probabilities and unstable averages

Summary This paper explores the possibilities for probability-like models of stationary nondeterministic phenomena that possess divergent but bounded time averages. A random sequence described by a stationary probability measure must have almost surely convergent time averages whenever it has almost surely bounded time averages. Hence, no measure can provide the mathematical model we desire. In turning to lower probability based models we first explore the relationships between divergence, stationarity, and monotone continuity and those between monotone continuity and unicity of extensions. We then construct several examples of stationary lower probabilities for sequences of uniformly bounded random variables such that divergence of time averages occurs with lower probability one. We conclude with some remarks on the problem of estimating lower probability models on the basis of cylinder set observations.     

Maximal mean/standard deviation ratio in an undiscounted MDP

A stationary policy in an MDP (Markov decision process) induces a stationary probability distribution of the reward from each initial state. The problem analyzed here is maximization of the mean/standard deviation ratio of the stationary distribution. In the unichain case, a solution is obtained via parametric analysis of a linear program having the same number of variables and one more constraint than the formulation for gain-rate optimization. The same linear program suffices in the multichain case if the initial state is an element of choice. The easier problem of maximizing the mean/variance ratio is mentioned at the end of the paper.     

Maximizing the probability of attaining a target prior to extinction

We present a dynamic programming-based solution to the problem of maximizing the probability of attaining a target set before hitting a cemetery set for a discrete-time Markov control process. Under mild hypotheses we establish that there exists a deterministic stationary policy that achieves the maximum value of this probability. We demonstrate how the maximization of this probability can be computed through the maximization of an expected total reward until the first hitting time to either the target or the cemetery set. Martingale characterizations of thrifty, equalizing, and optimal policies in the context of our problem are also established.     

Algorithms for improving efficiency of discrete Markov chains

Sampling from an intractable probability distribution is a common and important problem in scientific computing. A popular approach to solve this problem is to construct a Markov chain which converges to the desired probability distribution, and run this Markov chain to obtain an approximate sample. In this paper, we provide two methods to improve the performance of a given discrete reversible Markov chain. These methods require the knowledge of the stationary distribution only up to a normalizing constant. Each of these methods produces a reversible Markov chain which has the same stationary distribution as the original chain, and dominates the original chain in the ordering introduced by Peskun [11]. We illustrate these methods on two Markov chains, one connected to hidden Markov models and one connected to card shuffling. We also prove a result which shows that the Metropolis-Hastings algorithm preserves the Peskun ordering for Markov transition matrices.     

整值随机变量序列发生次数与条件概率和的等价性

在没有独立性、平稳性和相依性假设的条件下,利用分析方法讨论了整值随机变量序列取某个数值的次数与随机变量序列取该数值的条件概率和的等价性问题,建立并证明了若干强极限定理.     

Optimal threshold probability and expectation in semi-Markov decision processes

We consider undiscounted semi-Markov decision process with a target set and our main concern is a problem minimizing threshold probability. We formulate the problem as an infinite horizon case with a recurrent class. We show that an optimal value function is a unique solution to an optimality equation and there exists a stationary optimal policy. Also several value iteration methods and a policy improvement method are given in our model. Furthermore, we investigate a relationship between threshold probabilities and expectations for total rewards.     

Optimal harvesting for a stochastic regime-switching logistic diffusion system with jumps

The optimization problem of fishing for a stochastic logistic model is studied in this paper. Besides a standard geometric Brownian motion, another two driving processes are taken into account: a stationary Poisson point process and a continuous-time finite-state Markov chain. The classical harvesting problem for this model is a big difficult puzzle since the corresponding Fokker–Planck equations with three types of noise are very difficult to solve. Our main goal of this paper is to work out the optimization problem with respect to stationary probability density. One of the main contributions is to provide a new equivalent approach to overcome this problem. More precisely, an ergodic method is used to show the almost surely equivalency between the time averaging yield and sustainable yield. Results show that the optimal strategy changes with environment. An interesting thing is that the optimal strategy for each state is equivalent to the global optimality.     

Markov Chains with finite convergence time

We study the properties of finite ergodic Markov Chains whose transition probability matrix P is singular. The results establish bounds on the convergence time of P^m to a matrix where all the rows are equal to the stationary distribution of P. The results suggest a simple rule for identifying the singular matrices which do not have a finite convergence time. We next study finite convergence to the stationary distribution independent of the initial distribution. The results establish the connection between the convergence time and the order of the minimal polynomial of the transition probability matrix. A queuing problem and a maintenance Markovian decision problem which possess the property of rapid convergence are presented.     

Gaussian moving averages,semimartingales and option pricing

We provide a characterization of the Gaussian processes with stationary increments that can be represented as a moving average with respect to a two-sided Brownian motion. For such a process we give a necessary and sufficient condition to be a semimartingale with respect to the filtration generated by the two-sided Brownian motion. Furthermore, we show that this condition implies that the process is either of finite variation or a multiple of a Brownian motion with respect to an equivalent probability measure. As an application we discuss the problem of option pricing in financial models driven by Gaussian moving averages with stationary increments. In particular, we derive option prices in a regularized fractional version of the Black–Scholes model.     

The conjugate gradient method for computing all the extremal stationary probability vectors of a stochastic matrix

Summary The conjugate gradient method is developed for computing stationary probability vectors of a large sparse stochastic matrixP, which often arises in the analysis of queueing system. When unit vectors are chosen as the initial vectors, the iterative method generates all the extremal probability vectors of the convex set formed by all the stationary probability vectors ofP, which are expressed in terms of the Moore-Penrose inverse of the matrix (P−I). A numerical method is given also for classifying the states of the Markov chain defined byP. One particular advantage of this method is to handle a very large scale problem without resorting to any special form ofP. The Institute of Statistical Mathematics     

中国A股股票相邻两期β系数稳定性的Chow检验

本文用Chow检验方法研究了中国A股股票相邻两期的β系数是否稳定的问题。主要的发现有:1.对于个股而言,80%以上股票的β系数在上半年和下半年是稳定的。在扩展检验时期至相邻两年后,股票相邻两期的β系数稳定的概率有所降低,但是仍然高于60%;2.股票组合β系数稳定性的概率超过70%;3.股票组合的β系数在相邻两期稳定的概率与个股并无显著差异,并且,组合中的股票数量与组合的β系数在相邻两期是否稳定的概率并无显著的相关关系。     

Convergence of a transition probability tensor of a higher‐order Markov chain to the stationary probability vector

In this paper, first we introduce a new tensor product for a transition probability tensor originating from a higher‐order Markov chain. Subsequently, some properties of the new tensor product are explained, and its relationship with the stationary probability vector is studied. Also, similarity between results obtained by this new product and the first‐order case is shown. Furthermore, we prove the convergence of a transition probability tensor to the stationary probability vector. Finally, we show how to achieve a stationary probability vector with some numerical examples and make some comparison between the proposed method and another existing method for obtaining stationary probability vectors. Copyright © 2016 John Wiley & Sons, Ltd.     

THE STATIONARY DIFFUSION PROCESSES GENERATED BY SECOND ORDER DIFFERENTIAL OPERATIONS AND THEIR PROBABILITY CURRENTS

In this paper,we are concerned with the stationary Markov processes generated by second order differential operators under the local boundary conditions, It is proved that all those processes have constnt probability currents, known as circulations of the processes, and hence the processes are called single circulation processes. The invariant measures and the circulation values of those processes are calculated in all cases of boundary classification. It is shown that thr circulation value is an elementary characteristic of irreversible stationary Markov processes and that all the reversible Markov processes in the same problem are just the special ones of the single circulation processes whose circulation values are equal to zero and whose ergodic limits in the sense of weak convergence are not trivial.     

Approximating stationary points of stochastic optimization problems in Banach space

In this paper, we present a uniform strong law of large numbers for random set-valued mappings in separable Banach space and apply it to analyze the sample average approximation of Clarke stationary points of a nonsmooth one stage stochastic minimization problem in separable Banach space. Moreover, under Hausdorff continuity, we show that with probability approaching one exponentially fast with the increase of sample size, the sample average of a convex compact set-valued mapping converges to its expected value uniformly. The result is used to establish exponential convergence of stationary sequence under some metric regularity conditions.     

Queue Length Distribution in a FIFO Single-Server Queue with Multiple Arrival Streams Having Different Service Time Distributions

This paper considers the queue length distribution in a class of FIFO single-server queues with (possibly correlated) multiple arrival streams, where the service time distribution of customers may be different for different streams. It is widely recognized that the queue length distribution in a FIFO queue with multiple non-Poissonian arrival streams having different service time distributions is very hard to analyze, since we have to keep track of the complete order of customers in the queue to describe the queue length dynamics. In this paper, we provide an alternative way to solve the problem for a class of such queues, where arrival streams are governed by a finite-state Markov chain. We characterize the joint probability generating function of the stationary queue length distribution, by considering the joint distribution of the number of customers arriving from each stream during the stationary attained waiting time. Further we provide recursion formulas to compute the stationary joint queue length distribution and the stationary distribution representing from which stream each customer in the queue arrived.     

Stationary tail probabilities in exponential server tandems with renewal arrivals

The problem considered is that of estimating the tail stationary probability for two exponential server queues in series fed by renewal arrivals. We compute the tail of the marginal queue length distribution at the second queue. The marginal at the first queue is known by the classical result for the GI/M/1 queue. The approach involves deriving necessary and sufficient conditions on the paths of the arrival and virtual service processes in order to get a large queue size at the second queue. We then use large deviations estimates of the probabilities of these paths, and solve a constrained convex optimization problem to find the most likely path leading to a large queue size. We find that the stationary queue length distribution at the second queue has an exponentially decaying tail, and obtain the exact rate of decay.Research supported in part by NSF grant NCR 88-57731 and the AT & T Foundation.     

Nonlinear autoregressive models with heavy-tailed innovation

In this paper, we discuss the relationship between the stationary marginal tail probability and the innovation's tail probability of nonlinear autoregressive models. We show that under certain conditions that ensure the stationarity and ergodicity, one dimension stationary marginal distribution has the heavy-tailed probability property with the same index as that of the innovation's tail probability.