Numerical Solution for Linear-Quadratic Control Problems of Markov Jump Linear Systems and Weak Detectability Concept

A method for solving the linear-quadratic problem of Markov jump linear systems is developed in this paper, relying on the assumption of weak detectability. The concept of weak detectability generalizes previous concepts relevant to this class of systems, and most importantly, it allows us to revisit the quadratic control problem. In the main result of the paper, we show that, for weakly detectable systems, the solution obtained with the new method converges to the solution of the coupled algebraic Riccati equation that arises in the control problem if and only if the system is mean-square stabilizable. The paper shows how the concepts and the method involved are applied by means of numerical examples and comparisons.     

Turnpike Properties for Stochastic Linear-Quadratic Optimal Control Problems*

This paper analyzes the limiting behavior of stochastic linear-quadratic optimal control problems in finite time-horizon [0, T ] as T → ∞. The so-called turnpike properties are established for such problems, under stabilizability condition which is weaker than the controllability, normally imposed in the similar problem for ordinary differential systems.In dealing with the turnpike problem, a crucial issue is to determine the corresponding static optimization problem. Intuitively mimicking the deterministic situations, it seems to be natural to include both the drift and the diffusion expressions of the state equation to be zero as constraints in the static optimization problem. However, this would lead us to a wrong direction. It is found that the correct static problem should contain the diffusion as a part of the objective function, which reveals a deep feature of the stochastic turnpike problem.     

网络化马尔可夫跳变系统的鲁棒无源控制

针对基于网络的凸多面体不确定离散时间马尔可夫跳变系统,研究其鲁棒无源控制问题.在网络诱导时滞是时变且有界的情况下,基于李雅普诺夫稳定性理论,通过构造参数依赖的随机李雅普诺夫泛函和运用广义系统变换,提出了不依赖模态的无源控制器存在的时滞依赖充分条件.所设计的鲁棒无源控制器保证了相应的闭环系统是鲁棒随机稳定且具有指定耗散率.将鲁棒无源控制器设计问题转化为一组线性矩阵不等式的可解性问题.仿真算例证明了本文方法的有效性.     

Robust Stability and Stabilizability of Markov Jump Linear Uncertain Systems with Mode-Dependent Time Delays

This paper studies the class of uncertain linear systems with time delay and Markov jump disturbance, in which the time delay is assumed to be dependent on the system mode. An LMI-based condition for this class of systems to be robustly stable is established. Sufficient conditions for the robust stabilizability under a state feedback controller are developed, and an LMI-based method to design the state feedback is proposed. Numerical examples are worked out to show the usefulness of the theoretical results.     

Controllability Properties of Linear Mean-Field Stochastic Systems

We study some controllability properties for linear stochastic systems of mean-field type. First, we give necessary and sufficient criteria for exact terminal-controllability. Second, we characterize the approximate and approximate null-controllability via duality techniques. Using Riccati equations associated to linear quadratic problems in the control of mean-field systems, we provide a (conditional) viability criterion for approximate null-controllability. In the classical diffusion framework, approximate and approximate null-controllability are equivalent. This is no longer the case for mean-field systems. We provide sufficient (algebraic) invariance conditions implying approximate null-controllability. We also present a general class of systems for which our criterion is equivalent to approximate null-controllability property. We also introduce some rank conditions under which approximate and approximate null-controllability are equivalent. Several examples and counter-examples as well as a partial algorithm are provided.     

随机线性二次最优控制:从离散到连续时间模型

在一般情形下,分析了离散时间LQ问题与连续时间情形两者之间的自然联系.首先回顾了连续时间和离散时间随机LQ问题及对应Riccati微分/差分方程的相关结论.接下来在假设Riccati微分方程有解的前提下,证明了离散化步长足够小时,Riccati差分方程有解.然后针对连续和离散时间模型,采用配对问题最优控制的反馈形式,分别构造了一个辅助反馈控制,并证明该控制可驱使对应模型的性能指标逼近于配对问题的值函数,以此得到了关于两个模型之间联系的初步结论.最后藉由前述结论以及控制问题的特性,揭晓了连续时间和离散时间模型之间的自然联系,并给出了Riccati差分方程和微分方程的解之间的误差估计.由此联系,可构造相应离散系统和LQ问题,以适当的阶估计连续时间LQ问题的解,抑或为离散时间模型构造一个近似最优控制.无论哪种思路,都旨在降低直接求解原问题的难度和复杂性.     

Necessary and Sufficient Condition for Robust Stability and Stabilizability of Continuous-Time Linear Systems with Markovian Jumps

In this paper, we investigate the quadratic stability and quadratic stabilizability of the class of continuous-time linear systems with Markovian jumps and norm-bound uncertainties in the parameters. Under some appropriate assumptions, a necessary and sufficient condition is established for mean-square quadratic stability and mean-square quadratic stabilizability of this class of systems. The quadratic guaranteed cost control problem is also addressed via a LMI optimization problem.     

Guaranteed Cost Control of a Markov Jump Linear Uncertain System Using a Time-Multiplied Cost Function

This paper addresses the guaranteed cost control problem of jump linear systems with norm-bounded uncertain parameters. A time-multiplied performance index is considered. The performance is calculated first and an LMI-based algorithm is developed to design a state feedback control law with constant gain matrices which robustly stabilizes the system in the mean-square quadratically stable sense.     

Stochastic Detectability and Mean Bounded Error Covariance of the Recursive Kalman Filter with Markov Jump Parameters

In this article, we study the error covariance of the recursive Kalman filter when the parameters of the filter are driven by a Markov chain taking values in a countably infinite set. We do not assume ergodicity nor require the existence of limiting probabilities for the Markov chain. The error covariance matrix of the filter depends on the Markov state realizations, and hence forms a stochastic process. We show in a rather direct and comprehensive manner that this error covariance process is mean bounded under the standard stochastic detectability concept. Illustrative examples are included.     

Suboptimal Regulators for Discrete-Time Jump Linear Systems with Time-Multiplied Performance Index

This paper addresses the suboptimal regulator design problem of a discrete-time jump linear system by using a time-multiplied performance index. For a given state feedback control with constant gain matrices, the time-multiplied performance index is derived and a necessary condition for such a control to minimize the time-multiplied cost is provided. A gradient-based algorithm to find a suboptimal control law is developed and some numerical results are given to show the usefulness of the theoretical results.     

Minimax Control for a Class of Linear Systems Subject to Disturbances<Superscript>1</Superscript>

A class of linear dynamical control systems subject to uncertain but bounded disturbances is considered. The bounds imposed on the disturbances depend on the control magnitude and grow with the control. This situation occurs if the disturbances are due to the inaccuracy of the control implementation and often takes place in engineering applications such as transportation, aerospace, and robotic systems. Under certain assumptions, the minimax control problem is formulated and solved. Explicit expressions for the optimal control (both open-loop and feedback) are obtained that provide the minimax to the given performance index for arbitrary but bounded disturbances. Examples are given. ¹This work was supported by the Russian Foundation for Basic Research (Project 05-01-00647) and the Grant for Russian Scientific Schools (NSh 1627.2003.1).     

Forward-backward Stochastic Differential Equations and Backward Linear Quadratic Stochastic Optimal Control Problem

In this paper, we use the solutions of forward-backward stochastic differential equations to get the optimal control for backward stochastic linear quadratic optimal control problem. And we also give the linear feedback regulator for the optimal control problem by using the solutions of a group of Riccati equations.     

On Periodic Solutions of Linear Control Systems

We prove a theorem of the alternative concerning periodic solutions of linear control systems. We show that, if the attainable sets have nonempty interior, then the main condition assumed in Theorem 4.2 of Ref. 1 (a tangential condition) can be weakened and simplified, and a stronger conclusion can be obtained.     

一类脉冲控制模型的最小费用问题

本文研究了一类脉冲控制模型的最小费用问题,在一定条件下,给出了相应的最佳费用函数具体解析式．     

线性等式约束系统广义Riccati代数方程的求解*

本文基于定常离散LQ控制问题的动力学方程、价值泛函及系统的约束方程,根据极大值原理,给出了线性等式约束系统下的广义Riccati方程,进而对上述方程进行了深入的探讨,并给出了相应的数值例题。     

线性时变系统二次最优控制问题的保辛近似求解

状态空间的最优控制体系是保守的,其近似算法应当保辛．提出了基于分段常值精细积分方法的保辛摄动近似方法,在同一框架下求解了线性时变LQ最优控制中的计算问题,即变系数矩阵Riccati方程和状态反馈方程．该算法是保辛的,具有很好的数值稳定性和精度．算例验证了算法的有效性．     

Linear Programming Approximations for Markov Control Processes in Metric Spaces

We develop a general framework to analyze the convergence of linear-programming approximations for Markov control processes in metric spaces. The approximations are based on aggregation and relaxation of constraints, as well as inner approximations of the decision variables. In particular, conditions are given under which the control problems optimal value can be approximated by a sequence of finite-dimensional linear programs.     

Robust Output Tracking Control of an Uncertain Linear System via a Modified Optimal Linear-Quadratic Method

This study investigates the robust output tracking problem for a class of uncertain linear systems. The uncertainties are assumed to be time invariant and to satisfy the matching conditions. According to the selected nominal parameters, an optimal solution with a prescribed degree of stability is determined. Then, an auxiliary input via the use of an adapting factor, connected to the nominal optimal control, is introduced to guarantee the robustness and prescribed degree of stability for the output tracking control of the uncertain linear systems. This method is very simple and effective and can reject bounded uncertainties imposed on the states. A maglev vehicle model example is given to show its effectiveness.     

HMM-based quantized dissipative control for 2-D Markov jump systems

In this paper, the dissipative quantized control problem is addressed for Markov jump two-dimensional systems based on Roesser model, in which both asynchronous phenomenon and signal quantization between system modes and controller modes are taken into consideration simultaneously. Moreover, the hidden Markov model (HMM) is adopted to tackle such an asynchronous phenomenon. The principal goal is to devise a state feedback controller, which guarantees that the established closed-loop system achieves asymptotic mean square stability as well as satisfies a prescribed extended dissipative property. Drawing support from Lyapunov function approach and inequality technique, some less conservative criteria ensuring the implementability of the desired controller are derived. Ultimately, the availability and practicability of the developed results are certified through a simulation example.     

Genetic Algorithm for the Jump Number Scheduling Problem

This paper introduces genetic algorithms for the jump number scheduling problem. Given a set of tasks subject to precedence constraints, the problem is to construct a schedule to minimize the number of jumps. We show that genetic algorithms outperform the previously known Knuth and Szwarcfiter's exhaustive search algorithm when applied to some classes of orders in which no polynomial time algorithms exist in solving the jump number problem. Values for various parameters of genetic jump number algorithms are tested and results are discussed.