关于有限马氏链相对熵密度和随机条件熵的一类极限定理

本文引进有限非齐次马链随机条件熵的概念，研究这个概念与相对熵密度的关系，并通过数列的绝对平均收敛的概念给出了有限非齐次马氏链的相对频率，相对熵密度和平均随机条件熵ａ．ｅ收敛于常数及有限非齐次马氏链熵率存在的条件。     

树上二重非齐次马氏链随机转移概率的极限性质

研究树上二重非齐次马氏链随机转移概率的调和平均极限性质,作为推论,得到了树上非齐次马氏链以及非齐次马氏链上的随机转移概率调和平均极限性质.     

关于树指标非齐次马氏链的广义熵遍历定理

主要研究了树指标非齐次马氏链的广义熵遍历定理.首先证明了树指标非齐次马氏链上的二元函数延迟平均的强极限定理.然后得到了树指标非齐次马氏链上状态出现延迟频率的强大数定律,以及树指标非齐次马氏链的广义熵遍历定理.作为推论,推广了一些已有结果.同时,证明了局部有限无穷树树指标有限状态随机过程广义熵密度的一致可积性.     

双无限随机环境下马氏链的一类强极限定理

本文的目的 是要研究双无限随机环境下马氏链的一个强极限定理.作为推论得到了非齐次马氏链的一个强大数定律.最后,得到双无限随机环境中马氏链的随机转移概率调和平均的强极限定理.     

关于一类有限非齐次马尔可夫链熵率的收敛速度

运用随机条件熵的概念和绝对平均收敛的一些性质,利用H S Chang研究齐次马氏链熵率收敛速度的方法考虑了在给定条件下的一类有限非齐次马氏链熵率的指数收敛速度.     

一类非齐次马氏链的绝对平均强遍历性

引用马氏链绝对平均强遍历的概念,首先给出齐次马氏链绝对平均强遍历与强遍历的等价性,其次通过引进另一个强遍历的非齐次马氏链,给出一个非齐次马氏链绝对平均强遍历的充分条件.     

随机环境中马氏链的状态性质

本文研究了随机环境中马氏链的状态性质, 利用乘积空间的正则本质性和不可约性得到了随机环境中马氏链弱常返的充分条件.得到了随机环境中马氏链弱常返的充分条件.     

关于渐近循环马氏链泛函的强大数定律

研究了一类非齐次马氏链———渐近循环马氏链泛函的强大数定律,首先引出了渐近循环马氏链的概念,然后给出了若干引理.利用了渐近循环马氏链关于状态序偶出现频率的强大数定理给出并证明了关于渐近循环马氏链泛函的强大数定律,所得定理作为推论可得到已有的结果.     

二重非齐次马氏链及其随机变换的若干强极限定理

利用鞅方法,给出二重非齐次马尔可夫链三元函数的几个强极限定理.作为特例,将赌博系统的随机变换概念推广到二重马氏链情形,得到二重马氏链随机选择与随机公平比的若干极限定理.     

随机环境中马氏链函数加权和的极限定理

本文研究随机环境中马氏链函数的强极限定理,得到了随机环境中马氏链函数加权和的强收敛性成立的若干充分条件.     

Switching diffusion logistic models involving singularly perturbed Markov chains: Weak convergence and stochastic permanence

Focusing on stochastic dynamics involve continuous states as well as discrete events, this article investigates stochastic logistic model with regime switching modulated by a singular Markov chain involving a small parameter. This Markov chain undergoes weak and strong interactions, where the small parameter is used to reflect rapid rate of regime switching among each state class. Two-time-scale formulation is used to reduce the complexity. We obtain weak convergence of the underlying system so that the limit has much simpler structure. Then we utilize the structure of limit system as a bridge, to invest stochastic permanence of original system driving by a singular Markov chain with a large number of states. Sufficient conditions for stochastic permanence are obtained. A couple of examples and numerical simulations are given to illustrate our results.     

Quantized stabilization of stochastic systems with multiplicative noise under Markovian switching

A problem of quantized state feedback quadratic mean-square stabilization of discrete-time stochastic processes under Markovian switching and multiplicative noise is considered. A static quantizer is used in the feedback channel and the jump Markovian switching is modeled by a discrete-time Markov chain. The control input is simultaneously applied to both the rate vector and the diffusion term. It is shown that the coarsest quantization density that permits quadratic mean-square stabilization of this system is achieved with the use of a logarithmic quantizer, and the coarsest quantization density is determined by an algebraic Riccati equation, which is also the solution to a special linear stochastic Markovian switching control system. Also, sufficient conditions for exponential mean-square stabilization of such systems are also explored. An example is given to demonstrate the obtained results.     

Hybrid Impulsive Control of Stochastic Systems with Multiplicative Noise Under Markovian Switching

A problem of state output feedback stabilization of discrete-time stochastic systems with multiplicative noise under Markovian switching is considered. Under some appropriate assumptions, the stability of this system under pure impulsive control is given. Further under hybrid impulsive control, the output feedback stabilization problem is investigated. The hybrid control action is formulated as a combination of the regular control along with an impulsive control action. The jump Markovian switching is modeled by a discrete-time Markov chain. The control input is simultaneously applied to both the stochastic and the deterministic terms. Sufficient conditions based on stochastic semi-definite programming and linear matrix inequalities (LMIs) for both stochastic stability and stabilization are obtained. Such a nonconvex problem is solved using the existing optimization algorithms and the nonconvex CVX package. The robustness of the stability and stabilization concepts against all admissible uncertainties are also investigated. The parameter uncertainties we consider here are norm bounded. Two examples are given to demonstrate the obtained results.     

Sto chastic Maximum Principle for Optimal Control of Forward-backward Sto chastic Pantograph Systems with Regime Switching

In this paper, we derive the stochastic maximum principle for optimal control problems of the forward-backward Markovian regime-switching system. The control system is described by an anticipated forward-backward stochastic pantograph equation and modulated by a continuous-time finite-state Markov chain. By virtue of classical variational approach, duality method, and convex analysis, we obtain a stochastic maximum principle for the optimal control.     

Stochastic impulse control with regime switching for the optimal dividend policy when there are business cycles,taxes and fixed costs

We consider the problem of finding the optimal dividend policy for a company whose cash reserve follows a Brownian motion with drift and volatility modulated by an observable finite-state continuous-time Markov chain. The Markov chain represents the regime of the economy. We allow fixed costs and taxes associated with the dividend payments. This optimization problem generates a stochastic impulse control problem with regime switching. We solve this problem and obtain the first analytical solutions for the optimal dividend policy when there are simultaneously fixed costs, taxes and business cycles. Our results show that the optimal dividend policy depends strongly on the regime of the economy, on fixed costs and on taxes.     

On asymptotic optimization of a class of nonlinear stochastic hybrid systems

We consider the problem of control for continuous time stochastic hybrid systems in finite time horizon. The systems considered are nonlinear: the state evolution is a nonlinear function of both the control and the state. The control parameters change at discrete times according to an underlying controlled Markov chain which has finite state and action spaces. The objective is to design a controller which would minimize an expected nonlinear cost of the state trajectory. We show using an averaging procedure, that the above minimization problem can be approximated by the solution of some deterministic optimal control problem. This paper generalizes our previous results obtained for systems whose state evolution is linear in the control.This work is supported by the Australian Research Council. All correspondence should be directed to the first author.     

Stabilization of stochastic systems under Markovian switching

The problem of state feedback stabilization of discrete-time stochastic processes under Markovian switching is considered. The jump Markovian switching is modeled by a discrete-time Markov chain, and the noise or stochastic environmental disturbance is modeled by a sequence of identically independently normally distributed random variables. Necessary and sufficient conditions based on linear matrix inequalities (LMI’s) for stochastic stability is obtained. The proposed control law for this stochastic stabilization result depends on the mode of the system as well as the environmental disturbances. The robustness results of such stability concepts against all admissible uncertainties are also investigated. An example is given to demonstrate the obtained results.     

Robust Control of Markovian Switching Stochastic Systems with Noise Dependent States and Inputs

A problem of state feedback stabilization of discrete-time stochastic processes under Markovian switching and random diffusion (noise) is considered. The jump Markovian switching is modeled by a discrete-time Markov chain. The control input is simultaneously applied to both the rate vector and the diffusion term. Sufficient conditions based on linear matrix inequalities (LMI's) for stochastic stability is obtained. The robustness results of such stability concept against all admissible uncertainties are also investigated. An example is given to demonstrate the obtained results.     

有限时区上的最优转换和停止问题

讨论了有限时区上的最优转换和停止问题,它是一类同时具备脉冲控制和最优停止特征的最优控制问题.问题的最优值以及最优转换和停止决策可以由具有混合障碍的多维反射倒向随机微分方程的解来刻画.接着考虑了形式更一般的反射倒向随机微分方程并证明了方程解的存在唯一性.     

Dynamical behavior of a hybrid switching SIS epidemic model with vaccination and Lévy jumps

In this paper, the dynamical behavior of a hybrid switching SIS epidemic model with vaccination and Lévy jumps is considered. Besides a standard geometric Brownian motion, another two driving processes are taken into account: a stationary Poisson point process and a continuous time finite-state Markov chain. Firstly, we establish sufficient conditions for persistence in the mean of the disease. Then we obtain sufficient conditions for extinction of the disease. In addition, we also establish sufficient conditions for the existence of positive recurrence of the solutions to the model by constructing a suitable stochastic Lyapunov function with regime switching.