First passage times for Markov renewal processes and applications

This paper proposes a uniformly convergent algorithm for the joint transform of the first passage time and the first passage number of steps for general Markov renewal processes with any initial state probability vector. The uniformly convergent algorithm with arbitrarily prescribed error can be efficiently applied to compute busy periods, busy cycles, waiting times, sojourn times, and relevant indices of various generic queueing systems and queueing networks. This paper also conducts a numerical experiment to implement the proposed algorithm.     

First passage Markov decision processes with constraints and varying discount factors

This paper focuses on the constrained optimality problem (COP) of first passage discrete-time Markov decision processes (DTMDPs) in denumerable state and compact Borel action spaces with multi-constraints, state-dependent discount factors, and possibly unbounded costs. By means of the properties of a so-called occupation measure of a policy, we show that the constrained optimality problem is equivalent to an (infinite-dimensional) linear programming on the set of occupation measures with some constraints, and thus prove the existence of an optimal policy under suitable conditions. Furthermore, using the equivalence between the constrained optimality problem and the linear programming, we obtain an exact form of an optimal policy for the case of finite states and actions. Finally, as an example, a controlled queueing system is given to illustrate our results.     

Bayesian nonparametric estimation of first passage distributions in semi‐Markov processes

Bayesian nonparametric (BNP) models provide a flexible tool in modeling many processes. One area that has not yet utilized BNP estimation is semi‐Markov processes (SMPs). SMPs require a significant amount of computation; this, coupled with the computation requirements for BNP models, has hampered any applications of SMPs using BNP estimation. This paper presents a modeling and computational approach for BNP estimation in semi‐Markov models, which includes a simulation study and an application of asthma patients' first passage from one state of control to another.     

Optimal risk probability for first passage models in semi-Markov decision processes

This paper studies the risk minimization problem in semi-Markov decision processes with denumerable states. The criterion to be optimized is the risk probability (or risk function) that a first passage time to some target set doesn't exceed a threshold value. We first characterize such risk functions and the corresponding optimal value function, and prove that the optimal value function satisfies the optimality equation by using a successive approximation technique. Then, we present some properties of optimal policies, and further give conditions for the existence of optimal policies. In addition, a value iteration algorithm and a policy improvement method for obtaining respectively the optimal value function and optimal policies are developed. Finally, two examples are given to illustrate the value iteration procedure and essential characterization of the risk function.     

Deriving the HIV incubation period distribution for AIDS using immunological markers and devising infection treatment strategies

Survey data suggest that it is impossible for HIV infecteds to develop AIDs if the values of their CD4+ T-cell densities are above a critical threshold. An infected whose CD4+ T-cell density falls below 200 cells per microliter is now automatically regarded as having AIDS by the CDC. Using the CD4+ T-cell density as a surrogate marker of disease progression, a model that is consistent with the data is developed and applied to the homosexual/bisexual and IVDU risk groups. Assuming that the critical CD4+ T-cell density for these risk groups are identical, it is found that their progression towards AIDS during the incubation period is identical, suggesting that the dynamics of the HIV infection may be independent of risk group. The different incubation period distributions obtained from this modelling for these two risk groups is shown to be entirely due to their different normal seronegative CD4+ T-cell density distributions. Using IFN-γ as a surrogate marker is shown to give similar results.The impact of the HIV infection on the immune system is reviewed, and immunological infection models are developed. The data suggest to this author that Homo sapiens have generally lost the ability to generate T-cells and B-cells with the specificity necessary to neutralize HIV as they evolved from the primates. It is plausible that a legacy of primate immunity to HIV still remains in the 10% of Homo sapiens who show no immune system deterioration in the first 10 years of the HIV infection. New HIV infection treatment strategies based on this model are devised and discussed.     

Numerical computation of mean passage times and absorption probabilities in Markov and Semi-Markov models

This paper discusses an efficient method to compute mean passage times and absorption probabilities in Markov and Semi-Markov models. It uses the state reduction approach introduced by Winfried Grassmann for the computation of the stationary distribution of a Markov model. The method is numerically stable and has a simple probabilistic interpretation. It is especially stressed, that the natural frame for the state reduction method is rather Semi-Markov theory than Markov theory.

---

Zusammenfassung Es wird ein wirkungsvolles Rechenverfahren zur Bestimmung von mittleren Zeiten bis zur Absorption und von Absorptions-Wahrscheinlichkeiten in Markoff- und Semi-Markoff-Modellen dargestellt. Die Methode beruht auf dem Zustands-Reduktions-Ansatz, der von Grassmann für die Berechnung stationärer Verteilungen von Markoff-Ketten eingeführt wurde. Das Verfahren ist numerisch stabil und hat eine einfache wahrscheinlichkeitstheoretische Interpretation. Es wird hervorgehoben, da\ der natürliche Rahmen der Methode eher die Semi-Markoff-Theorie als die Markoff-Theorie ist.

     

Sufficiency and completeness for classes of probability distributions generated by a single probability distribution

Let (Ω,A, P₀) denote a probability space andA some sub-σ-algebra ofA. Moreover,P _{P 0} stands for the set consisting of all probability distributionsP _{P 0} of the typeP _{P 0}(A) = ∫E _{P 0} (P _{A |B)dP,A ∈A, whereP is a probability distribution onA satisfyingP |B ≪P 0} |B. It is shown thatB is sufficient and (even totally) complete forP _{P 0}. This results is illustrated by examples. Finally, it is shown thatP _{P 0} ∈P _{P 0} is an extremal point ofP _{P 0} if and only ifP |B is {0, 1}-valued. Dedicated to the memory of Professor K. Behnen     

First passage time to detection in stochastic population dynamical models for HIV-1

The detection of HIV-1 levels in human hosts is cast as a first exit time problem for a multidimensional diffusion process. We consider a four-component model for early HIV-1 dynamics including uninfected CD4+ T-cells, latently infected cells, actively infected cells, and HIV-1 virions. An analytical framework is presented for the distribution of the time at which a given virion level is attained. A one-dimensional diffusion approximation for a branching process leads to an estimate for the distribution of the virion density and an expression for the mean detection time for any given detection threshold.     

First hitting time models for the generalized inverse Gaussian distribution

Any generalized inverse Gaussian distribution with a non-positive power parameter is shown to be the distribution of the first hitting time of level 0 for each of a variety of time-homogeneous diffusions on the interval [0, ∞). The infinite divisibility of the generalized inverse Gaussian distributions is a simple consequence of this and an elementary convolution formula for these distributions.     

On the probability distribution of total quality under sublot inspection and process restoration

A lot of N items is produced on a randomly degrading facility. The lot is split into inspection sublots for detection of faulty items, and the process is returned to as-new condition after each sublot. For exponentially distributed failure time the mean, variance and probability distribution of the total number of good items are considered. Stochastic optimality properties are developed for the equal-sublots policy in both the exponential and general failure time cases.     

The lack-of-preference law and the corresponding distributions in frequency probability theory

We obtain two new distributions in frequency probability theory and demonstrate them, in particular, in the example of frequency dictionaries. One of these distributions gives a logarithmic correction to the Zipf-Mandelbrot law, and the other describes the “tails” of the distribution.     

Option pricing under a normal mixture distribution derived from the Markov tree model

We examine a Markov tree (MT) model for option pricing in which the dynamics of the underlying asset are modeled by a non-IID process. We show that the discrete probability mass function of log returns generated by the tree is closely approximated by a continuous mixture of two normal distributions. Using this normal mixture distribution and risk-neutral pricing, we derive a closed-form expression for European call option prices. We also suggest a regression tree-based method for estimating three volatility parameters σ, σ⁺, and σ⁻ required to apply the MT model. We apply the MT model to price call options on 89 non-dividend paying stocks from the S&P 500 index. For each stock symbol on a given day, we use the same parameters to price options across all strikes and expires. Comparing against the Black–Scholes model, we find that the MT model’s prices are closer to market prices.     

On the relationship between markov strategies in multistep search games and probability distributions

A certain statement of a matrix antagonistic game, commonly encountered in discrete problems of search of objects, is considered. The statement enables one to find strategies that reduce a multistep search problem to a sequence of related one-step matrix games. Bibliography: 5 titles. Translated fromObchyslyuval'na ta Prykladna Matematyka, No. 80, 1996, pp. 119–126.     

Quantization of probability distributions and gradient flows in space dimension 2

In this paper we study a perturbative approach to the problem of quantization of probability distributions in the plane. Motivated by the fact that, as the number of points tends to infinity, hexagonal lattices are asymptotically optimal from an energetic point of view [10], [12], [15], we consider configurations that are small perturbations of the hexagonal lattice and we show that: (1) in the limit as the number of points tends to infinity, the hexagonal lattice is a strict minimizer of the energy; (2) the gradient flow of the limiting functional allows us to evolve any perturbed configuration to the optimal one exponentially fast. In particular, our analysis provides a new mathematical justification of the asymptotic optimality of the hexagonal lattice among its nearby configurations.     

Nonextensive distribution and factorization of the joint probability

The problem of factorization of Tsallis' nonextensive probability distribution is discussed in a general way. It is shown that the correlation energy between the correlated subsystems in the canonical composite system cannot be neglected even in the thermodynamic limit. In consequence, the factorization approximation should be employed carefully according to different systems. It is also shown that the zeroth law of thermodynamics can be established without factorization approximation in the framework of the incomplete statistical mechanics.     

Busemann functions and equilibrium measures in last passage percolation models

The interplay between two-dimensional percolation growth models and one-dimensional particle processes has been a fruitful source of interesting mathematical phenomena. In this paper we develop a connection between the construction of Busemann functions in the Hammersley last-passage percolation model with i.i.d. random weights, and the existence, ergodicity and uniqueness of equilibrium (or time-invariant) measures for the related (multi-class) interacting fluid system. As we shall see, in the classical Hammersley model, where each point has weight one, this approach brings a new and rather geometrical solution of the longest increasing subsequence problem, as well as a central limit theorem for the Busemann function.