Stability of a pure random delay system with two-time-scale Markovian switching

This work examines almost sure stability of a pure random delay system whose delay time is modeled by a finite state continuous-time Markov chain with two-time scales. The Markov chain contains a fast-varying part and a slowly-changing part. Using the properties of the weighted occupation measure of the Markov chain, it is shown that the overall system?s almost-sure-asymptotic stability can be obtained by using the “averaged” delay. This feature implies that even if some longer delay times may destabilize the system individually, the system may still be stable if their impact is balanced. In other words, the Markov chain becomes a stabilizing factor. Numerical results are provided to demonstrate our results.     

Robust stability criteria for uncertain neutral type stochastic system with Takagi–Sugeno fuzzy model and Markovian jumping parameters

In this paper, the robust stability for uncertain neutral stochastic system with Takagi–Sugeno (T–S) fuzzy model and Markovian jumping parameters (MJPs) are investigated. The jumping parameters considered here are generated from a continuous-time discrete-state homogeneous Markov process, which are governed by a Markov process with discrete and finite-state space. Some novel sufficient conditions are derived to guarantee the asymptotic stability of the equilibrium point in the mean square. By utilizing the Lyapunov–Krasovskii functional, stochastic analysis theory, some free weighting matrices and linear matrix inequality (LMI) technique, the upper bound of time-varying delay is obtained by using Matlab® control toolbox. Finally, some numerical examples are given to show the effectiveness of the obtained results.     

时滞不确定性Markov切换线性随机微分系统的指数鲁棒稳定性

研究了一类时滞不确定性Markov切换随机微分系统的均方指数鲁棒随机稳定性\bd 系统中的时滞是时变的, 不确定项结构为范数有界, Markov切换是连续时间、离散状态的时齐Markov过程{\bf\!.} 利用随机Lyapunov函数方法和LMI技术, 得到了几个判定系统均方指数鲁棒随机稳定性的充分性条件\bd 一个数值例子说明了判据的有效性和可行性.     

STABILITY ANALYSIS OF THE SPLIT-STEP THETA METHOD FOR NONLINEAR REGIME-SWITCHING JUMP SYSTEMS

In this paper,we investigate the stability of the split-step theta(SST)method for a class of nonlinear regime-switching jump systems–neutral stochastic delay differential equations(NSDDEs)with Markov switching and jumps.As we know,there are few results on the stability of numerical solutions for NSDDEs with Markov switching and jumps.The purpose of this paper is to enrich conclusions in such respect.It first devotes to show that the trivial solution of the NSDDE with Markov switching and jumps is exponentially mean square stable and asymptotically mean square stable under some suitable conditions.If the drift coefficient also satisfies the linear growth condition,it then proves that the SST method applied to the NSDDE with Markov switching and jumps shares the same conclusions with the exact solution.Moreover,a numerical example is demonstrated to illustrate the obtained results.     

Queues with Service Rate Controlled by a Delayed Feedback

We consider a single queue with a Markov modulated Poisson arrival process. Its service rate is controlled by a scheduler. The scheduler receives the workload information from the queue after a delay. This queue models the buffer in an earth station in a satellite network where the scheduler resides in the satellite. We obtain the conditions for stability, rates of convergence to the stationary distribution and the finiteness of the stationary moments. Next we extend these results to the system where the scheduler schedules the service rate among several competing queues based on delayed information about the workloads in the different queues.     

Discretisation of stochastic control problems for continuous time dynamics with delay

As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretising time and space. A new feature in this context is to allow for delay in the dynamics. The existence of an optimal strategy with respect to the cost functional can be guaranteed in the class of relaxed controls. Weak convergence of the approximating extended Markov chains to the original process together with convergence of the associated optimal strategies is established.     

Stability of hybrid stochastic delay systems whose discrete components have a large state space: A two-time-scale approach

This work is concerned with stability of stochastic differential delay equations with Markovian switching, where the modulating Markov chain has a large state space and is subject to both fast and slow movements. Under simple conditions, we demonstrate that if the limit systems are pth-moment exponentially stable, then the original systems are pth-moment exponentially stable in an appropriate sense. In addition, the exponential stability is also investigated. Moreover, stability in distribution is obtained for such hybrid systems.     

Markov processes generated by linear stochastic evolution equations †

A class of Hilbert space-valued Markov processes which can be expressed as the mild solution of a linear abstract evolution equation is studied. Sufficient conditions for the generator of the Markov process to be well-defined are given and Kolmogorov's equation and an equation for the characteristic function of the process are derived. The theory is illustrated by examples of parabolic, hyperbolic and delay stochastic differential equations.     

Exponential Stability in Mean Square for a General Class of Discrete-Time Linear Stochastic Systems

Abstract

The problem of the mean square exponential stability for a class of discrete-time linear stochastic systems subject to independent random perturbations and Markovian switching is investigated. The case of the linear systems whose coefficients depend both to present state and the previous state of the Markov chain is considered. Three different definitions of the concept of exponential stability in mean square are introduced and it is shown that they are not always equivalent. One definition of the concept of mean square exponential stability is done in terms of the exponential stability of the evolution defined by a sequence of linear positive operators on an ordered Hilbert space. The other two definitions are given in terms of different types of exponential behavior of the trajectories of the considered system. In our approach the Markov chain is not prefixed. The only available information about the Markov chain is the sequence of probability transition matrices and the set of its states. In this way one obtains that if the system is affected by Markovian jumping the property of exponential stability is independent of the initial distribution of the Markov chain.

The definition expressed in terms of exponential stability of the evolution generated by a sequence of linear positive operators, allows us to characterize the mean square exponential stability based on the existence of some quadratic Lyapunov functions.

The results developed in this article may be used to derive some procedures for designing stabilizing controllers for the considered class of discrete-time linear stochastic systems in the presence of a delay in the transmission of the data.     

Robust μ-stability analysis of Markovian switching uncertain stochastic genetic regulatory networks with unbounded time-varying delays

This paper investigates the global robust stability problem of Markovian switching uncertain stochastic genetic regulatory networks with unbounded time-varying delays and norm bounded parameter uncertainties. The structure variations at discrete time instances during the process of gene regulations known as hybrid genetic regulatory networks based on Markov process is proposed. The jumping parameters considered here are generated from a continuous-time discrete-state homogeneous Markov process, which are governed by a Markov process with discrete and finite state space. The concept of global robust μ-stability in the mean square for genetic regulatory networks is given. Based on Lyapunov function, stochastic theory and Itô’s differential formula, the stability criteria are presented in the form of linear matrix inequalities (LMIs). Numerical examples are presented to demonstrate the effectiveness of the main result.     

����״̬������Markov�л����ʱ��ϵͳ�ľ���ָ���ȶ���

考察一类Markov切换时变时滞随机系统的均方指数稳定性. 利用基于Liapunov函数和线性矩阵不等式的方法, 给出了使状态反馈控制系统能克服不确定性和随机干扰, 在均方意义下达到指数稳定的充分条件. 当Markov链遍历所有模态时, 给出了一个独立于Markov链模态集的增益矩阵, 使得状态反馈控制系统均方指数稳定     

Stability of a class of networked control systems with Markovian characterization

This paper studies the stability problem for a class of networked control systems (NCSs) with the plant being a Markovian jump system. The random delays from the sensor to the controller and from the controller to the actuator are modeled as two Markov chains. The necessary and sufficient conditions for the stochastic stability are established. The state-feedback controller gain that depends on not only the delay modes but also the system mode is obtained through the iterative linear matrix inequality approach. An illustrative example is presented to demonstrate the effectiveness of the proposed method.     

Dam processes with state dependent batch sizes and intermittent production processes with state dependent rates

We consider a dam process with a general (state dependent) release rule and a pure jump input process, where the jump sizes are state dependent. We give sufficient conditions under which the process has a stationary version in the case where the jump times and sizes are governed by a marked point process which is point (Palm) stationary and ergodic. We give special attention to the Markov and Markov regenerative cases for which the main stability condition is weakened. We then study an intermittent production process with state dependent rates. We provide sufficient conditions for stability for this process and show that if these conditions are satisfied, then an interesting new relationship exists between the stationary distribution of this process and a dam process of the type we explore here.Supported in part by The Israel Science Foundation, grant no. 372/93-1.     

Observability and nonlinear filtering

This paper develops a connection between the asymptotic stability of nonlinear filters and a notion of observability. We consider a general class of hidden Markov models in continuous time with compact signal state space, and call such a model observable if no two initial measures of the signal process give rise to the same law of the observation process. We demonstrate that observability implies stability of the filter, i.e., the filtered estimates become insensitive to the initial measure at large times. For the special case where the signal is a finite-state Markov process and the observations are of the white noise type, a complete (necessary and sufficient) characterization of filter stability is obtained in terms of a slightly weaker detectability condition. In addition to observability, the role of controllability is explored. Finally, the results are partially extended to non-compact signal state spaces.     

具有修理延迟的两不同部件冷贮备系统的可靠性分析

研究了有修理延迟的两个不同部件和两个修理工组成的冷贮备系统.假定部件的工作寿命服从一般分布,故障后的延迟修理时间和修理时间均服从指数分布.利用马尔可夫更新过程、拉普拉斯变换和拉普拉斯-司梯阶变换工具,得到了系统的首次故障前时间、可用度和平均故障次数等可靠性指标.     

Stability and monotonicity for interactive Markov chains

In a Markov chain model of a social process, interest often centers on the distribution of the population by state. One question, the stability question, is whether this distribution converges to an equilibrium value. For an ordinary Markov chain (a chain with constant transition probabilities), complete answers are available. For an interactive Markov chain (a chain which allows the transition probabilities governing each individual to depend on the locations by state of the rest of the population), few stability results are available. This paper presents new results. Roughly, the main result is that an interactive Markov chain with unique equilibrium will be stable if the chain satisfies a certain monotonicity property. The property is a generalization to interactive Markov chains of the standard definition of monotonicity for ordinary Markov chains.     

Delay-range dependent stability criteria for neural networks with Markovian jumping parameters

The paper is concerned with a stability analysis problem for neural networks with Markovian jumping parameters. The jumping parameters considered here are generated from a continuous-time discrete-state homogenous Markov process, which are governed by a Markov process with discrete and finite state space. A new type of Markovian jumping matrix P_i is introduced in this paper. The discrete delays are assumed to be time-varying and belong to a given interval, which means that the lower and upper bounds of interval time-varying delays are available. Based on the new Lyapunov–Krasovskii functional, delay-interval dependent stability criteria are obtained in terms of linear matrix inequalities (LMIs). Finally, a numerical example is provided to demonstrate the lower conservatism and the effectiveness of the proposed LMI conditions.     

马尔可夫模型的市场预测方法及其应用

本文简述了马尔可夫模型的基本原理，介绍了利用马氏过程的基本特征．即其平稳性和无后效性．对随机现象进行统计预测的方法。本文选取市场为研究对象，根据产品的市场占有率的随机变化过程，建立起市场占有率的马尔可夫预测模型，并进行了实例分析。在模型的实际应用时，要注意系统状态转移的平稳性和进行系统状态划分的有效性。     

Finite dimensional Markov process approximation for stochastic time-delayed dynamical systems

This paper presents a method of finite dimensional Markov process (FDMP) approximation for stochastic dynamical systems with time delay. The FDMP method preserves the standard state space format of the system, and allows us to apply all the existing methods and theories for analysis and control of stochastic dynamical systems. The paper presents the theoretical framework for stochastic dynamical systems with time delay based on the FDMP method, including the FPK equation, backward Kolmogorov equation, and reliability formulation. A simple one-dimensional stochastic system is used to demonstrate the method and the theory. The work of this paper opens a door to various studies of stochastic dynamical systems with time delay.