On some algorithms for limiting average Markov decision processes

We consider limiting average Markov decision processes (MDP) with finite state and action spaces. We propose some algorithms to determine optimal strategies for deterministic and general MDPs. These algorithms are based on graph theory and the construction of levels in some aggregated MDP.     

Some Markov processes with Brownian exit distributions

Summary LetD be a bounded domain inR ^d with regular boundary. LetX=(X_t, P^x) be a standard Markov process inD with continuous paths up to its lifetime. IfX satisfies some weak conditions, then it is possible to add a non-local part to its generator, and construct the corresponding standard Markov process inD with Brownian exit distributions fromD.This work was done while the author was an Alexander von Humboldt fellow at the Universität des Saarlandes in Saarbrücken, Germany     

Markov processes with identical last exit distributions

Summary LetX andY be two transient locally Hunt Markov processes. IfX andY enjoy the same last exit distributions from compact sets, thenY is equivalent to a time change ofX by the inverse of a strictly increasing continuous additive functional. This result can also be interpreted (with natural auxiliary hypotheses) as a statement in potential theory involving equilibrium measures.Research supported in part by NSF grant MCS-8002659 and an CNRS Fellowship while the author was visiting I.M.S.S., Universite de Grenoble II     

A χ2 goodness-of-fit test for Markov renewal processesgoodness-of-fit test for Markov renewal processes

A Markov Renewal Process (M.R.P.) is a process similar to a Markov chain, except that the time required to move from one state to another is not fixed, but is a random variable whose distribution may depend on the two states between which the transition is made. For an M.R.P. ofm (<∞) states we derive a goodness-of-fit test for a hypothetical matrix of transition probabilities. This test is similar to the test Bartlett has derived for Markov chains. We calculate the first two moments of the test statistic and modify it to fit the moments of a standard χ². Finally, we illustrate the above procedure numeerically for a particular case of a two-state M.R.P. Dwight B. Brock is mathematical statistican, Office of Statistical Methods, National Center for Health Statistics, Rockville, Maryland. A. M. Kshisagar is Associate Professor, Department of Statistics, Southern Methodist University. This research was partially supported by Office of Naval Research Contract No. N000 14-68-A-0515, and by NIH Training Grant GM-951, both with Southern Methodist University. This article is partially based on Dwight B. Brock's Ph.D. dissertation accepted by Southern Methodist University.     

On integral and local limit theorems for recursive Markov renewal processes

Translated fromProblemy Ustoichivosti Stokhasticheskikh Modelei. Trudy Seminara, 1988, pp. 100–115.     

On the curvature on G-manifolds with finitely many non-principal orbits

We investigate the curvature of invariant metrics on G-manifolds with finitely many non-principal orbits. We prove existence results for metrics of positive Ricci curvature, and discuss some families of examples to which these existence results apply. In fact, many of our examples also admit invariant metrics of non-negative sectional curvature.     

Limit theorems for recurrent semi-Markov processes and Markov renewal processes

For Harris recurrent Markov renewal processes and semi-Markov processes one obtains a central limit theorem. One also obtains Berry-Esseen type estimates for this theorem. Their proof is based on the Kolmogorov-Doeblin regenerative method.Translated from Zapiski Nauchnykh Seminarov Leningradskogo Otdeleniya Matematicheskogo Instituta im. V. A. Steklova AN SSSR, Vol. 142, pp. 86–97, 1985.     

Markov network processes with product form stationary distributions

This study concerns the equilibrium behavior of a general class of Markov network processes that includes a variety of queueing networks and networks with interacting components or populations. The focus is on determining when these processes have product form stationary distributions. The approach is to relate the marginal distributions of the process to the stationary distributions of “node transition functions” that represent the nodes in isolation operating under certain fictitious environments. The main result gives necessary and sufficient conditions on the node transition functions for the network process to have a product form stationary distribution. This result yields a procedure for checking for a product form distribution and obtaining such a distribution when it exits. An important subclass of networks are those in which the node transition rates have Poisson arrival components. In this setting, we show that the network process has a product form distribution and is “biased locally balanced” if and only if the network is “quasi-reversible” and certain traffic equations are satisfied. Another subclass of networks are those with reversible routing. We weaken the known sufficient condition for such networks to be product form. We also discuss modeling issues related to queueing networks including time reversals and reversals of the roles of arrivals and departures. The study ends by describing how the results extend to networks with multi-class transitions. This revised version was published online in June 2006 with corrections to the Cover Date.     

On groups factorized by finitely many subgroups

We prove that every group factorizable into a product of finitely many pairwise permutable central-by-finite minimax subgroups is a soluble-by-finite group.

---

Про групи, факторизовані скінченним числом шдгруп

Розвивається спектральна теорія та теорія розсіяння для одпого класу самоспряжених матричних диференціальних операторів змішаного порядку.

---

---

This work was done while the author was visiting the Ukrainian Academy of Sciences in Kiev. He is grateful to the Institute of Mathematics for its warm hospitality.     

Equivalence systems with finitely many relations

We consider a setX with a finite totally ordered setE of equivalence relations onX. We describe the automorphism group of this system, that is, the group of all those permutations ofX that leave each relation inE invariant.     

Groups with finitely many linear orders

It is proved that there exist orderable groups having exactly 6 and 14 distinct linear orders. For any natural number k, we construct examples of orderable groups on which 2(4k+3) linear orders are defined. Supported by RFFR grant No. 96-01-00088. Translated fromAlgebra i Logika, Vol. 38, No. 2, pp. 176–200, March–April, 1999.