Superposition of Spatially Interacting Aggregated Continuous Time Markov Chains

A system s{ X(t)} = {X ₁(t),X ₂(t),..., X _N(t)} of N interacting time reversible continuous time Markov chains is considered. The state space of each of the processes {X _i(t)} (i = 1, 2,...,N) is partitioned into two aggregates. Interaction between the processes {X _i(t)},{X ₂(t)},...,{X _N(t)} is introduced by allowing the transition rates of an individual process at time t to depend on the configuration of aggregates occupied by the other N - 1 processes at that time. The motivation for this work comes from ion channel modeling, where {(X}(t)} describes the gating mechanisms of N channels and the partitioning of the state space of {X _i(t)} correspond to whether the channel is conducting or not. Let S(t) denote the number of conducting channels at time t. For a time-reversible class of such processes, expressions are derived for the mean and probability density function of the sojourns of {S(t)} at its different levels when {X(t)} is in equilibrium. Particular attention is paid to the situation when the N channels are located on a circle with nearest neighbor interaction. Necessary and sufficient conditions for a general co-operative multiple channel system to be time reversible are derived.     

Existence and Regularity of a Nonhomogeneous Transition Matrix under Measurability Conditions

This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t≥0) and conservative matrix Q(t)=[q _ij (t)] of nonhomogeneous transition rates q _ij (t) and use it to construct the transition probability matrix. Here we obtain the same result except that the q _ij (t) are only required to satisfy a mild measurability condition, and Q(t) may not be conservative. Moreover, the resulting transition matrix is shown to be the minimum transition matrix, and, in addition, a necessary and sufficient condition for it to be regular is obtained. These results are crucial in some applications of nonhomogeneous continuous-time Markov processes, such as stochastic optimal control problems and stochastic games, and this was the main motivation for this work. Supported by NSFC and RFDP. The research of O. Hernández-Lerma was partially supported by CONACYT grant 45693-F.     

Why are there only three quantum noises?

In quantum stochastic calculus on the symmetric Fock space over L ²(ℝ₊), adapted processes of operators are integrated with respect to creation, annihilation and number processes. The main property which allows this integration is that the increments of integrators between s and t act only on Fock space over L ²([s, t]). In this article, we prove that there are no other process of closable operators on coherent vectors with this property. Thus the only possible integrators in quantum stochastic calculus are the creation, annihilation and number processes.     

On processes of ornstein-uhlenbeck type in hilbert space

A process fo Ornstein-Uhlenbeck type is a mild solution of the stochastic differential system in Hilbert space dXt=AX _t dt+dZ _t, where A generates a semigroup of operators and Z _tis a process with homogeneous independent increments. The explicit integral formula for the process of O-U type is given. The main purpose is to study stationary distributions for such processes. Sufficient and necessary conditions for existence and characterization are given. The difference between finite and infinite dimensional cases is illustrated by examples     

Adaptive control for discrete-time Markov processes with unbounded costs: Average criterion

The paper deals with a class of discrete-time Markov control processes with Borel state and action spaces, and possibly unbounded one-stage costs. The processes are given by recurrent equations x _{t +1}=F(x _t ,a _t ,ξ _t ), t=1,2,… with i.i.d. ℜ ^k – valued random vectors ξ _t whose density ρ is unknown. Assuming observability of ξ _t , and taking advantage of the procedure of statistical estimation of ρ used in a previous work by authors, we construct an average cost optimal adaptive policy. Received March/Revised version October 1997     

On the Infinitesimal Generators of Ornstein–Uhlenbeck Processes with Jumps in Hilbert Space

We study Hilbert space valued Ornstein–Uhlenbeck processes (Y(t), t ≥ 0) which arise as weak solutions of stochastic differential equations of the type dY = JY + CdX(t) where J generates a C ₀ semigroup in the Hilbert space H, C is a bounded operator and (X(t), t ≥ 0) is an H-valued Lévy process. The associated Markov semigroup is of generalised Mehler type. We discuss an analogue of the Feller property for this semigroup and explicitly compute the action of its generator on a suitable space of twice-differentiable functions. We also compare the properties of the semigroup and its generator with respect to the mixed topology and the topology of uniform convergence on compacta.      

Sojourn times with finite time-horizon in finite semi-markov processes

Let Y = {Y_t:t ≥ 0} be a semi-Markov process with finite state space S. Assume that Y is either irreducible and S is then partitioned into two classes A and B, or, that Y is absorbing and S is partitioned into A, B and C, where C is the set of all absorbing states of Y. Denote by T_{A, m}(t) the mth sojourn time of Y in A during [0, t]. T_{A, m}(t) is thus defined as the duration in [0, t] of the mth visit of Y to A if A is visited by Y during [0, t] at least m times; T_{A, m}(t) = 0 otherwise. We derive a recurrence relation for the vectors of double Laplace transforms g_m**(T₁,T₂) = {g_m**(T₁, T₂;S):sSC}, m = 1,2,… which are defined by with T₁, T₂, Re(T₁), Re(T₂) > 0. This result is then applied to alternating renewal processes. Symbolic Laplace transform inversion with MAPLE is used to obtain the first two moments of T_{A, m}(t). The assumed holding time distributions are exponential and Erlang respectively. This paper is a continuation of some of the author's recent work on the distribution theory of sojourn times in a subset of the finite state space of a (semi-)Markov process where the time horizon t = + ∞. The practical importance of considering a finite time horizon for semi-Markov reliability models has been discussed recently by Jack (1991), Jack and Dagpunar (1992), and Christer and Jack (1991).     

Adaptive control of stochastic systems with unknown disturbance distribution: discounted criteria

We consider a class of discrete-time stochastic control systems, with Borel state and action spaces, and possibly unbounded costs. The processes evolve according to the equation x _{t +1}=F(x _t , a _t , ξ _t ), t=0, 1, ..., where the ξ _t are i.i.d. random vectors whose common distribution is unknown. Assuming observability of {ξ _t }, we use the empirical estimator of its distribution to construct adaptive policies which are asymptotically discounted cost optimal .AMS Subject Classification (2000) 93E10, 90C40     

Counting Observations: A Note on State Estimation Sensitivity with an L 1 -Bound

Let (X _t , Y _t ) be a pure jump Markov process: the state X _t takes real values and the observation Y _t is a counting process. The two processes are allowed to have common jump times. Let ϕ(X(⋅)) be a functional of the state trajectory restricted to the time interval [0, T] . If we change the infinitesimal parameters and/ or the initial distribution, then we introduce an error in computing the conditional law of ϕ(X(⋅)) given the observation up to time T . In this paper we give an explicit L ¹ -bound for this error. Accepted 9 March 2001. Online publication 20 June 2001.     

Parameter estimation in linear filtering

Suppose on a probability space (Ω, F, P), a partially observable random process (x_t, y_t), t ≥ 0; is given where only the second component (y_t) is observed. Furthermore assume that (x_t, y_t) satisfy the following system of stochastic differential equations driven by independent Wiener processes (W₁(t)) and (W₂(t)): dx_t=−βx_tdt+dW₁(t), x₀=0, dy_t=αx_tdt+dW₂(t), y₀=0; α, β∞(a,b), a>0. We prove the local asymptotic normality of the model and obtain a large deviation inequality for the maximum likelihood estimator (m.l.e.) of the parameter θ = (α, β). This also implies the strong consistency, efficiency, asymptotic normality and the convergence of moments for the m.l.e. The method of proof can be easily extended to obtain similar results when vector valued instead of one-dimensional processes are considered and θ is a k-dimensional vector.     

On the transition from a Markov chain to a continuous time process

Starting from a real-valued Markov chain X₀,X₁,…,X_n with stationary transition probabilities, a random element {Y(t);t[0, 1]} of the function space D[0, 1] is constructed by letting Y(k/n)=X_k, k= 0,1,…,n, and assuming Y (t) constant in between. Sample tightness criteria for sequences {Y(t);t[0,1]};_n of such random elements in D[0, 1] are then given in terms of the one-step transition probabilities of the underlying Markov chains. Applications are made to Galton-Watson branching processes.     

The linear quadratic minimum-time problem for a class of discrete systems

We consider a linear discrete-time systems controlled by inputs on L ²([0, t _N ], U), where (t _i )_{1?≤?i?≤?N} is a given sequence of times. The final time t _N (or N) is considered to be free. Given an initial state x ₀ and a final one x _d , we investigate the optimal control which steers the system from x ₀ to x _d with a minimal cost J(N, u) that includes the final time and energy terms. We treat this problem for both infinte and finite dimensional state space. We use a method similar to the Hilbert Uniqueness Method. A numerical simulation is given.     

Convergence of non-ergodic dynamical systems

Summary Let {T _t} be a flow on a probability space (S,L,}) which describes the time evolution of a dynamical system with state space S, and interpret as the initial distribution of the system. Then the distribution of the system at time t is given by T _t ^–1 . Our aim is to study the asymptotic behavior of T _t ^–1 both in general and in the particular cases of random rate and almost periodic systems. The results seem to indicate that convergence or mean convergence is the normal behavior in the non-ergodic case.     

Properties of Hida processes on . 2. Prediction and interpolation problems for processes on

If E is an ordered set, we study the processes Y_t, t E, for which the vectorial spaces _t generated by all the conditional expectations E(Y_sβ _t) for s ≥ t have finite dimensions d(t) ≤ N. ( _t is some convenient filtration.) We first develop a geometrical approach in the general situation and give a “Goursat's representation” Y_t = Σf_i(t)M_i(t), where the M_i(t) are martingales. We then restrict us to the cases E = or E = ² and give representations of the processes by the mean of stochastic integrals of “Goursat's kernels.” The special case when Y_t is the solution of a differential equation is considered.     

Limit Law of the Local Time for Brox’s Diffusion

We consider Brox’s model: a one-dimensional diffusion in a Brownian potential W. We show that the normalized local time process (L(t,m _log t +x)/t, x∈R), where m _log t is the bottom of the deepest valley reached by the process before time t, behaves asymptotically like a process which only depends on W. As a consequence, we get the weak convergence of the local time to a functional of two independent three-dimensional Bessel processes and thus the limit law of the supremum of the normalized local time. These results are discussed and compared to the discrete time and space case for which the same questions have been answered recently by Gantert, Peres, and Shi (Ann. Inst. Henri Poincaré, Probab. Stat. 46(2):525–536, 2010).     

Spatiotemporal dynamics on small-world neuronal networks: The roles of two types of time-delayed coupling

We investigate temporal coherence and spatial synchronization on small-world networks consisting of noisy Terman–Wang (TW) excitable neurons in dependence on two types of time-delayed coupling: {x_j(t − τ) − x_i(t)} and {x_j(t − τ) − x_i(t − τ)}. For the former case, we show that time delay in the coupling can dramatically enhance temporal coherence and spatial synchrony of the noise-induced spike trains. In addition, if the delay time τ is tuned to nearly match the intrinsic spike period of the neuronal network, the system dynamics reaches a most ordered state, which is both periodic in time and nearly synchronized in space, demonstrating an interesting resonance phenomenon with delay. For the latter case, however, we cannot achieve a similar spatiotemporal ordered state, but the neuronal dynamics exhibits interesting synchronization transitions with time delay from zigzag fronts of excitations to dynamic clustering anti-phase synchronization (APS), and further to clustered chimera states which have spatially distributed anti-phase coherence separated by incoherence. Furthermore, we also show how these findings are influenced by the change of the noise intensity and the rewiring probability of the small-world networks. Finally, qualitative analysis is given to illustrate the numerical results.     

Large-time asymptotics for the Gt/Mt/st+GIt many-server fluid queue with abandonment

We previously introduced and analyzed the G _t /M _t /s _t +GI _t many-server fluid queue with time-varying parameters, intended as an approximation for the corresponding stochastic queueing model when there are many servers and the system experiences periods of overload. In this paper, we establish an asymptotic loss of memory (ALOM) property for that fluid model, i.e., we show that there is asymptotic independence from the initial conditions as time t evolves, under regularity conditions. We show that the difference in the performance functions dissipates over time exponentially fast, again under the regularity conditions. We apply ALOM to show that the stationary G/M/s+GI fluid queue converges to steady state and the periodic G _t /M _t /s _t +GI _t fluid queue converges to a periodic steady state as time evolves, for all finite initial conditions.     

Jones Index of a Quantum Dynamical Semigroup

In this paper we consider a semigroup of completely positive maps τ=(τ _t ,t≥0) with a faithful normal invariant state φ on a type-II ₁ factor A₀\mathcal{A}_{0} and propose an index theory. We achieve this via a Kolmogorov’s type of construction for stationary Markov processes which naturally associate a nested family of isomorphic von-Neumann algebras. In particular this construction generalises well known Jones construction associated with a sub-factor of a type-II₁ factor.