Stability,stabilizability and detectability for Markov jump discrete-time linear systems with multiplicative noise in Hilbert spaces

In this article we discuss stability, stabilizability and detectability problems for Markov-jump discrete-time linear systems (MJDLSs) with multiplicative noise (MN) and countably infinite state space of the Markov chain. On the basis of a new solution representation formula, we give new deterministic characterizations of the stability and the detectability properties of MJDLSs with MN. These results are obtained using an operatorial approach and the properties of certain positive evolution operators defined on ordered Banach spaces of sequences of nuclear operators. Assuming detectability conditions and avoiding stochastic proofs, we prove that any global, nonnegative and bounded solution of the Riccati equation of control is stabilizing for the MJDLSs with MN and control. Finally, we apply our results to solve a linear quadratic optimal control problem. The theory is illustrated by an example.     

Stochastic observability and applications

In this paper the problem of stochastic observability of alinear system affected by multiplicative white noise and Markovianjumping is investigated. The definition of stochastic observabilityadopted here extends to this framework the definition of thewell known uniform observability of a deterministic time-varyinglinear system. By several examples we show that the conceptof stochastic observability introduced in this paper is lessrestrictive than those introduced in other existing works andit does not always imply stochastic detectability as would beexpected. Finally we prove that this kind of stochastic observabilityallows us to derive a Barbasin–Krasovskii type resultfor exponential stability in mean square. This provides a sufficientcondition which guarantees that any semipositive solution ofcorresponding Riccati differential equation is a stabilizingsolution.     

Optimal control of linear stochastic evolution equations in Hilbert spaces and uniform observability

In this paper we study the existence of the optimal (minimizing) control for a tracking problem, as well as a quadratic cost problem subject to linear stochastic evolution equations with unbounded coefficients in the drift. The backward differential Riccati equation (BDRE) associated with these problems (see [2], for finite dimensional stochastic equations or [21], for infinite dimensional equations with bounded coefficients) is in general different from the conventional BDRE (see [10], [18]). Under stabilizability and uniform observability conditions and assuming that the control weight-costs are uniformly positive, we establish that BDRE has a unique, uniformly positive, bounded on ℝ ₊ and stabilizing solution. Using this result we find the optimal control and the optimal cost. It is known [18] that uniform observability does not imply detectability and consequently our results are different from those obtained under detectability conditions (see [10]).      

On unified concepts of detectability and observability for continuous-time stochastic systems

This paper is concerned with detectability and observability of continuous-time stochastic linear systems. By adopting the idea used in defining these two concepts for time-varying systems and Markovian jump linear systems that have been studied in the literature, corresponding definitions for continuous-time stochastic linear systems are proposed. These two definitions are not only able to unify some recent definitions on these two concept reported in the literature, but also allow us to propose an efficient rank criterion to test observability of continuous-time stochastic linear systems. It seems that this rank criterion is quite analogous to the rank criterion for deterministic linear systems. With the help of these two concepts and the new criteria, the stochastic Lyapunov equation is revisited and some recent published work on this equation are generalized. Numerical examples are given to illustrate the effectiveness of the proposed approach.     

On asymptotic optimization of a class of nonlinear stochastic hybrid systems

We consider the problem of control for continuous time stochastic hybrid systems in finite time horizon. The systems considered are nonlinear: the state evolution is a nonlinear function of both the control and the state. The control parameters change at discrete times according to an underlying controlled Markov chain which has finite state and action spaces. The objective is to design a controller which would minimize an expected nonlinear cost of the state trajectory. We show using an averaging procedure, that the above minimization problem can be approximated by the solution of some deterministic optimal control problem. This paper generalizes our previous results obtained for systems whose state evolution is linear in the control.This work is supported by the Australian Research Council. All correspondence should be directed to the first author.     

Adaptive control of linear stochastic evolution systems

An adaptive control problem for some linear stochastic evolution systems in Hilbert spaces is formulated and solved in this paper. The solution includes showing the strong consistency of a family of least squares estimates of the unknown parameters and the convergence of the average quadratic costs with a control based on these estimates to the optimal average cost. The unknown parameters in the model appear affinely in the infinitesimal generator of the C ₀ semigroup that defines the evolution system. A recursive equation is given for a family of least squares estimates and the bounded linear operator solution of the stationary Riccati equation is shown to be a continuous function of the unknown parameters in the uniform operator topology     

Suboptimal control of systems with multiple delays

The application of Pontryagin's maximum principle to the optimization of linear systems with time delays results in a system of coupled two-point boundary-value problems involving both delay and advance terms. The exact solution of this system of TPBV problems is extremely difficult, if not impossible. In this paper, a fast-converging iterative approach is developed for obtaining the suboptimal control for nonstationary linear systems with multiple state and control delays and with quadratic cost. At each step of the proposed method, a linear nondelay system with an extra perturbing input must be optimized. The procedure can be extended for the optimization of nonlinear systems with multiple time-varying delays, provided that some of the nonlinearities satisfy the Lipschitz condition.     

LP-stabilization problem for linear stochastic control systems with multiplicative noise

For the deterministic case, a linear controlled system is alwayspth order stable as long as we use the control obtained as the solution of the so-called LQ-problem. For the stochastic case, however, a linear controlled system with multiplicative noise is not alwayspth mean stable for largep, even if we use the LQ-optimal control. Hence, it is meaningful to solve the LP-optimal control problem (i.e., linear system,pth order cost functional) for eachp. In this paper, we define the LP-optimal control problem and completely solve it for the scalar case. For the multidimensional case, we get some results, but the general solution of this problem seems to be impossible. So, we consider thepth mean stabilization problem more intensively and give a sufficient condition for the existence of apth mean stabilizing control by using the contraction mapping method in a Hilbert space. Some examples are also given.This research was conducted while the author was a visitor at the Forschungsschwerpunkt Dynamische Systeme, Universität Bremen, Bremen, West Germany. The author is grateful to Professor L. Arnold for providing interesting seminars and excellent working conditions during his stay. The financial assistance given by the Alexander von Humboldt Foundation during the author's stay is also gratefully acknowledged.     

Optimal control for infinite dimensional stochastic differential equations with infinite Markov jumps and multiplicative noise

In this paper we solve an infinite-horizon linear quadratic control problem for a class of differential equations with countably infinite Markov jumps and multiplicative noise. The global solvability of the associated differential Riccati-type equations is studied under detectability hypotheses. A nonstochastic, operatorial approach is used. Some properties of the linear stochastic systems, such as stability, stabilizability and detectability, are also discussed on the basis of a new solution representation result. A generalized Ito's formula which applies to infinite dimensional stochastic differential equations with countably infinite Markov jumps is also provided.     

Linear programming approach to the control of discrete-time periodic systems with uncertain inputs

The problem is considered of finding a control strategy for a linear discrete-time periodic system with state and control bounds in the presence of unknown disturbances that are only known to belong to a given compact set. This kind of problem arises in practice in resource distribution systems where the demand has typically a periodic behavior, but cannot be estimated a priori without an uncertainty margin. An infinite-horizon keeping problem is formulated, which consists in confining the state within its constraint set using the allowable control, whatever the allowed disturbances may be. To face this problem, the concepts of periodically invariant set and sequence are introduced. They are used to formulate a solution strategy that solves the keeping problem. For the case of polyhedral state, control, and disturbance constraints, a computationally feasible procedure is proposed. In particular, it is shown that periodically invariant sequences may be computed off-line, and then they may be used to synthesize on-line a control strategy. Finally, an optimization criterion for the control law is discussed.     

Almost-periodic solutions for Riccati equations of stochastic control

We consider an average quadratic cost criteria for affine stochastic differential equations with almost-periodic coefficients. Under stabilizability and detectability conditions we show that the Riccati equation associated with the quadratic control problem has a unique almost-periodic solution. In the periodic case the corresponding result is proved in [4].     

Optimal control for linear discrete-time systems with Markov perturbations in Hilbert spaces

Email: vio{at}utgjiu.ro Received on September 12, 2007; Accepted on December 26, 2008 In this article, we discuss a quadratic control problem forlinear discrete-time systems with Markov perturbations in Hilbertspaces, which is linked to a discrete-time Riccati equationdefined on certain infinite-dimensional ordered Banach space.We prove that under stabilizability and stochastic uniform observabilityconditions, the Riccati equation has a unique, uniformly positive,bounded on N and stabilizing solution. Based on this result,we solve the proposed optimal control problem. An example illustratesthe theory.     

Digital stochastic control of distributed-parameter systems

This paper presents a method for discrete-time control and estimation of flexible structures in the presence of actuator and sensor noise. The approach consists of complete decoupling of the modal equations and estimator dynamics based on the independent modal-space control technique and modal spatial filtering of the system output. The solution for the Kalman filter gains reduces to that of independent second-order modal estimators, thus permitting real-time digital control of distributed-parameter systems in a noisy environment. The method can be used to control and estimate any number of modes without computational restraints and is theoretically free of observation spillover. Two examples, the first using nonlinear, quantized control and the second using linear, state feedback control are presented.This work was supported by the National Science Foundation, Grant No. PFR-80-20623.     

Exponential stabilization of distributed semilinear systems by optimal control

This paper studies exponential stabilization of distributed semilinear systems. The paper (i) gives a constrained feedback control that ensure the exponential stabilizability and (ii) shows that this control is the unique solution of an appropriate minimization problem. Examples of hyperbolic partial equations are provided.     

Direct Shooting Method for the Numerical Solution of Higher-Index DAE Optimal Control Problems

A method for the numerical solution of state-constrained optimal control problems subject to higher-index differential-algebraic equation (DAE) systems is introduced. For a broad and important class of DAE systems (semiexplicit systems with algebraic variables of different index), a direct multiple shooting method is developed. The multiple shooting method is based on the discretization of the optimal control problem and its transformation into a finite-dimensional nonlinear programming problem (NLP). Special attention is turned to the mandatory calculation of consistent initial values at the multiple shooting nodes within the iterative solution process of (NLP). Two different methods are proposed. The projection method guarantees consistency within each iteration, whereas the relaxation method achieves consistency only at an optimal solution. An illustrative example completes this article.     

Asymptotic Solution of a Boundary-Value Problem for Linear Singularly-Perturbed Functional Differential Equations Arising in Optimal Control Theory

The Hamiltonian boundary-value problem, associated with a singularly-perturbed linear-quadratic optimal control problem with delay in the state variables, is considered. A formal asymptotic solution of this boundary-value problem is constructed by application of the boundary function method. The justification of this asymptotic solution is done. The asymptotic solution of the Hamiltonian boundary-value problem is constructed and justified assuming boundary-layer stabilizability and detectability.     

Variance-constrained dissipative observer-based control for a class of nonlinear stochastic systems with degraded measurements

This paper is concerned with the variance-constrained dissipative control problem for a class of stochastic nonlinear systems with multiple degraded measurements, where the degraded probability for each sensor is governed by an individual random variable satisfying a certain probabilistic distribution over a given interval. The purpose of the problem is to design an observer-based controller such that, for all possible degraded measurements, the closed-loop system is exponentially mean-square stable and strictly dissipative, while the individual steady-state variance is not more than the pre-specified upper bound constraints. A general framework is established so that the required exponential mean-square stability, dissipativity as well as the variance constraints can be easily enforced. A sufficient condition is given for the solvability of the addressed multiobjective control problem, and the desired observer and controller gains are characterized in terms of the solution to a convex optimization problem that can be easily solved by using the semi-definite programming method. Finally, a numerical example is presented to show the effectiveness and applicability of the proposed algorithm.     

Optimal control of bioprocess systems using hybrid numerical optimization algorithms

In the paper, we consider the bioprocess system optimal control problem. Generally speaking, it is very difficult to solve this problem analytically. To obtain the numerical solution, the problem is transformed into a parameter optimization problem with some variable bounds, which can be efficiently solved using any conventional optimization algorithms, e.g. the improved Broyden–Fletcher–Goldfarb–Shanno algorithm. However, in spite of the improved Broyden–Fletcher–Goldfarb–Shanno algorithm is very efficient for local search, the solution obtained is usually a local extremum for non-convex optimal control problems. In order to escape from the local extremum, we develop a novel stochastic search method. By performing a large amount of numerical experiments, we find that the novel stochastic search method is excellent in exploration, while bad in exploitation. In order to improve the exploitation, we propose a hybrid numerical optimization algorithm to solve the problem based on the novel stochastic search method and the improved Broyden–Fletcher–Goldfarb–Shanno algorithm. Convergence results indicate that any global optimal solution of the approximate problem is also a global optimal solution of the original problem. Finally, two bioprocess system optimal control problems illustrate that the hybrid numerical optimization algorithm proposed by us is low time-consuming and obtains a better cost function value than the existing approaches.     

随机混合动态系统的状态反馈控制

针对一类以有限齐次马氏链δ(k)作为切换信号的随机混合系统,首先,通过构造随机混合Lyapunov函数,得到整个随机混合系统渐近稳定的充分条件.然后,引入可调转移概率等相关概念,通过对有限齐次马氏链δ(k)及各子系统加入控制,以实现状态反馈控制.进一步,得到随机混合闭环系统渐近稳定的充分条件.     

一类不确定广义周期时变系统的鲁棒H_∞控制

本文研究了状态矩阵具不确定性广义周期时变系统的鲁棒H∞控制问题.利用线性矩阵不等式(LMI)方法,在给出不确定广义周期时变系统广义可镇定和广义二次可镇定且具有H∞性能指标概念的基础上,得到了该系统广义二次可镇定且具有H∞性能指标γ的充要条件,并给出了相应的鲁棒H∞状态反馈控制律的设计方法,推广了周期系统的鲁棒控制理论结果.最后,通过数值算例说明了设计方法的有效性.