Numerical computation of mean passage times and absorption probabilities in Markov and Semi-Markov models

This paper discusses an efficient method to compute mean passage times and absorption probabilities in Markov and Semi-Markov models. It uses the state reduction approach introduced by Winfried Grassmann for the computation of the stationary distribution of a Markov model. The method is numerically stable and has a simple probabilistic interpretation. It is especially stressed, that the natural frame for the state reduction method is rather Semi-Markov theory than Markov theory.

---

Zusammenfassung Es wird ein wirkungsvolles Rechenverfahren zur Bestimmung von mittleren Zeiten bis zur Absorption und von Absorptions-Wahrscheinlichkeiten in Markoff- und Semi-Markoff-Modellen dargestellt. Die Methode beruht auf dem Zustands-Reduktions-Ansatz, der von Grassmann für die Berechnung stationärer Verteilungen von Markoff-Ketten eingeführt wurde. Das Verfahren ist numerisch stabil und hat eine einfache wahrscheinlichkeitstheoretische Interpretation. Es wird hervorgehoben, da\ der natürliche Rahmen der Methode eher die Semi-Markoff-Theorie als die Markoff-Theorie ist.

     

First passage time problems in time-dependent fields

This paper discusses the simplest first passage time problems for random walks and diffusion processes on a line segment. When a diffusing particle moves in a time-varying field, use of the adjoint equation does not lead to any simplification in the calculation of moments of the first passage time as is the case for diffusion in a time-invariant field. We show that for a discrete random walk in the presence of a sinusoidally varying field there is a resonant frequency * for which the mean residence time on the line segment is a minimum. It is shown that for a random walk on a line segment of lengthL the mean residence time goes likeL ² for largeL when *, but when =* the dependence is proportional toL. The results of our simulation are numerical, but can be regarded as exact. Qualitatively similar results are shown to hold for diffusion processes by a perturbation expansion in powers of a dimensionless velocity. These results are extended to higher values of this parameter by a numerical solution of the forward equation.     

A first passage time problem for random walk occupancy

Two recent studies of diffusion and flow properties of polymers in a melt have suggested the problem of finding the average time form Brownian particles to leave a sphere for the first time, given that exited particles can also reenter the sphere. We prove that the asymptotic density (asm) for the time to first emptiness of the sphere for zero-mean Brownian motion is a delta function, characterized by the exit timea(m/lnm)^2/D ,a being a constant andD being the dimension. The presence of a field leaves the delta-function form for the density, but changes the time dependence toa lnm, with only the constanta depending on the dimension. Simulations of the process suggest that the value ofm needed for the validity of the asymptotic result is orders of magnitude greater than 1000.     

Weak Variation of Gaussian Processes

Let X(t) (tR) be a real-valued centered Gaussian process with stationary increments. We assume that there exist positive constants ₀, C ₁, and c ₂ such that for any tR and hR with |h|₀ and for any 0r<min{|t|, ₀} where is regularly varying at zero of order (0 < < 1). Let be an inverse function of near zero such that (s)=(s) log log(1/s) is increasing near zero. We obtain exact estimates for the weak -variation of X(t) on [0,a].     

Micelle-polymer complexes as studied by the ESR spin probe technique

Complexes between sodium dodecyl sulfate micelles and the water-soluble polymers poly(N-vinylpyrrolidone), poly(ethylene oxide) and a copolymer of poly (vinyl alcohol) and poly(vinyl acetate) have been studied in aqueous solution by the electron spin resonance (ESR) technique using di-tert-butyl nitroxide as a spin probe. The effective rotational correlation times reveal lowering of the critical micelle concentration and decreased headgroup packing in the micelle upon interaction with the polymers.     

Bounds on the cover time

Consider a particle that moves on a connected, undirected graphG withn vertices. At each step the particle goes from the current vertex to one of its neighbors, chosen uniformly at random. Tocover time is the first time when the particle has visited all the vertices in the graph starting from a given vertex. In this paper, we present upper and lower bounds that relate the expected cover time for a graph to the eigenvalues of the Markov chain that describes the random walk above. An interesting consequence is that regular expander graphs have expected cover time (n logn).This research was done while this author was a postdoctoral fellow in the Department of Computer Science, Princeton University, and it was supported in part by DNR grant N00014-87-K-0467.     

Parameter-dependent renewal theorems with applications

Let ₁, ₂, ... be a sequence of i.i.d. random variables with positive mean and finite variance and letr(b), b0, be real numbers tending to 0 asb . Definings _n=₁+...+_n andS _n=S_n(b)=s_n+r(b)n, the stopping time =(b)=inf {n>/1:S_n >b} whereb=b(b) , will be considered with special regard to the excess over the boundaryR _b=s+r(b)–b. It turns out that the limiting distribution ofR _b is the same as in the caser(b)0 for allb. Proving this, Blackwell's renewal theorem and its integral version have to be established first in the above stated situation. Finally, an expansion ofE to vanishing terms asb will be provided and applied to some examples arising in economics.

---

Zusammenfassung Seien ₁, ₂, ... unabhängige identisch verteilte Zufallsgrößen mit positivem Erwartungswert und endlicher Varianz sowier(b), b0, reelle Zahlen mitr(b)0 für b. Sei ferners ₁, s₂, ... der zugehörige Summenprozeß,S _n= S_n(b)=s_n+r(b)n fürn1 und =(b)=inf {n1: S_n>b, wobeib=b(b) fürb . Es wird gezeigt, daß die asymptotische Verteilung des ExzessesR _b=s +r(b) –b mit der im Fallr(·)0 übereinstimmt. Dazu werden sowohl das Blackwellsche Erneuerungstheorem als auch seine Integralversion in der vorher beschriebenen parameterabhängigen Situation geeignet formuliert und bewiesen. Als Folgerung ergibt sich dann eine asymptotische Entwicklung vonE(b) fürb bis zu Termen o(1). Anh- and einiger Beispiele aus dem ökonomischen Bereich wird schließlich noch aufgezeigt, wo Approximationen fürE(b) von Interesse sein können.

     

Comprehensive two-dimensional gas chromatography with time-of-flight mass spectrometry detection: analysis of amino acid and organic acid trimethylsilyl derivatives, with application to the analysis of metabolites in rye grass samples

First, standard mixtures of trimethylsilyl (TMS) derivatives of amino acid and organic acid are analyzed by comprehensive two-dimensional (2D) gas chromatography (GC) coupled to time-of-flight mass spectrometry (GC × GC/TOFMS) in order to illustrate important issues regarding application of the technique. Specifically of interest is the extent to which the peak capacity of the 2D separation space has been utilized and the procedure by which the derivative standards are identified in the 2D separations using the mass spectral information. The resulting 2D separation is found to make extensive use of the GC × GC separation space provided by the complementary stationary phases employed.Second, in order to demonstrate GC × GC/TOFMS on two real sample types, trimethylsilyl metabolite derivatives were analyzed from extracts of common lawn grass samples (i.e., perennial rye grass), as a means to provide insight into both the pre and post harvest physiology. Various chemical components in the two rye grass extract samples were found to either emerge or disappear in relation to the trauma response. For example, a significant difference in the peak for the TMS derivative of malic acid was found. The successful analysis of various components was readily facilitated by the 2D separation, while a 1D separation would have produced too much peak overlap, thus impeding the analysis. The importance of using a GC × GC separation approach for the analysis of complex samples, such as metabolite extracts, is therefore demonstrated. The real-time analysis capability of GC × GC/TOFMS for multidimensional metabolite analysis makes this technique well suited to the high-throughput analysis of metabolomic samples, especially compared to slower, stopped-flow type separation approaches.     

Cyclic Bernoulli polling

We introduce, analyse and optimize the class of Bernoulli random polling systems. The server movescyclically among N channels (queues), butChange-over times between stations are composed ofwalking times required to move from one channel to another andswitch-in times that are incurredonly when the server actually enters a station to render service. The server uses aBernoulli random mechanism to decide whether to serve a queue or not: upon arrival to channeli, it switches in with probabilityp _i , or moves on to the next queue (w.p. 1 —p _i ) without serving any customer (e.g. packet or job). The Cyclic Bernoulli Polling (CBP) scheme is independent of the service regime in any particular station, and may be applied to any service discipline. In this paper we analyse three different service disciplines under the CBP scheme: Gated, Partially Exhaustive and Fully Exhaustive. For each regime we derive expressions for (i) the generating functions and moments of the number of customers (jobs) at the various queues at polling instants, (ii) the expected number of jobs that an arbitrary departing job leaves behind it, and (iii) the LST and expectation of the waiting time of a cutomer at any given queue. The fact that these measures of performance can be explicitly obtained under the CBP is an advantage over all parameterized cyclic polling schemes (such as the k-limited discipline) that have been studied in the literature, and for which explicit measures of performance are hard to obtain. The choice of thep _i 's in the CBP allows for fine tuning and optimization of performance measures, as well as prioritization between stations (this being achieved at a low computational cost). For this purpose, we develop a Pseudo-conservation law for amixed system comprised of channels from all three service disciplines, and define a Mathematical Program to find the optimal values of the probabilities {p _i } _i ^N =1 so as to minimize the expected amount of unfinished work in the system. Any CBP scheme for which the optimalp _i 's are not all equal to one, yields asmaller amount of the expected unfinished work in the system than that in the standard cyclic polling procedure with equivalent parameters. We conclude by showing that even in the case of a single queue, it is not always true thatp ₁=1 is the best strategy, and derive conditions under which it is optimal to havep ₁ < 1.Supported by a Grant from the France-Israel Scientific Cooperation (in Computer Science and Engineering) between the French Ministry of Research and Technology and the Israeli Ministry of Science and Technology, Grant Number 3321190.     

Can colored noise improve stochastic resonance?

The phenomenon of stochastic resonance is studied in the presence of colored noise. Several sources of colored noise are introduced and the consequences for the asymptotic time-periodic probability and the (phase-averaged) power spectrum are discussed. Based on space-time symmetry considerations, selection rules for the occurrence of-spikes in the power spectrum are derived. The effect of colored noise on the amplification of small periodic signals is studied in terms of effective, time-periodic Fokker-Planck equations: In overdamped systems driven by colored noise, we find that SR is suppressed with increasing noise color. In contrast, for colored noise induced by inertia (as well as for asymmetric dichotomic noise), one obtains an enhancement of SR. This latter result is obtained by studying the Kramers equation perturbed by a small periodic force.