Solutions for the Linear-Quadratic Control Problem of Markov Jump Linear Systems

The paper is concerned with recursive methods for obtaining the stabilizing solution of coupled algebraic Riccati equations arising in the linear-quadratic control of Markovian jump linear systems by solving at each iteration uncoupled algebraic Riccati equations. It is shown that the new updates carried out at each iteration represent approximations of the original control problem by control problems with receding horizon, for which some sequences of stopping times define the terminal time. Under this approach, unlike previous results, no initialization conditions are required to guarantee the convergence of the algorithms. The methods can be ordered in terms of number of iterations to reach convergence, and comparisons with existing methods in the current literature are also presented. Also, we extend and generalize current results in the literature for the existence of the mean-square stabilizing solution of coupled algebraic Riccati equations.     

On some iterations for optimal control of jump linear equations

We consider different iterative methods for computing Hermitian solutions of the coupled Riccati equations of the optimal control problem for jump linear systems. We have constructed a sequence of perturbed Lyapunov algebraic equations whose solutions define matrix sequences with special properties proved under proper initial conditions. Several numerical examples are included to illustrate the effectiveness of the considered iterations.     

On the convergence of inexact Newton methods for discrete-time algebraic Riccati equations

In this paper, we present a convergence analysis of the inexact Newton method for solving Discrete-time algebraic Riccati equations (DAREs) for large and sparse systems. The inexact Newton method requires, at each iteration, the solution of a symmetric Stein matrix equation. These linear matrix equations are solved approximatively by the alternating directions implicit (ADI) or Smith?s methods. We give some new matrix identities that will allow us to derive new theoretical convergence results for the obtained inexact Newton sequences. We show that under some necessary conditions the approximate solutions satisfy some desired properties such as the d-stability. The theoretical results developed in this paper are an extension to the discrete case of the analysis performed by Feitzinger et al. (2009) [8] for the continuous-time algebraic Riccati equations. In the last section, we give some numerical experiments.     

Systems of matrix rational differential equations arising in connection with linear stochastic systems with Markovian jumping

In this paper, a class of systems of matrix nonlinear differential equations containing as particular cases the systems of coupled Riccati differential equations arising in connection with control of some linear stochastic systems is considered.The system of differential equations considered in this paper are converted in a suitable nonlinear differential equation on a finite-dimensional Hilbert space adequately choosen.This allows us to use the positivity properties of the linear evolution operator defined by the linear differential equations of Lyapunov type.Our aim is to investigate properties of stabilizing and bounded solutions of the considered differential equations and to obtain some conditions ensuring the existence of such solutions.Conditions providing the existence of a maximal solution (minimal solution respectively) with respect to some classes of global solutions are presented. It is shown that if the coefficients of the equations are periodic functions all these special solutions (stabilizing, maximal, minimal) are periodic functions, too.Whenever possible the probabilistic arguments were avoided and so the results proved in the paper appear as results in the field of differential equations with interest in themselves.     

A New Approach to Linearly Perturbed Riccati Equations Arising in Stochastic Control

In this paper a linearly perturbed version of the well-known matrix Riccati equations which arise in certain stochastic optimal control problems is studied. Via the concepts of mean square stabilizability and mean square detectability we improve previous results on both the convergence properties of the linearly perturbed Riccati differential equation and the solutions of the linearly perturbed algebraic Riccati equation. Furthermore, our approach unifies, in some way, the study for this class of Riccati equations with the one for classical theory, by eliminating a certain inconvenient assumption used in previous works (e.g., [10] and [26]). The results are derived under relatively weaker assumptions and include, inter alia, the following: (a) An extension of Theorem 4.1 of [26] to handle systems not necessarily observable. (b) The existence of a strong solution, subject only to the mean square stabilizability assumption. (c) Conditions for the existence and uniqueness of stabilizing solutions for systems not necessarily detectable. (d) Conditions for the existence and uniqueness of mean square stabilizing solutions instead of just stabilizing. (e) Relaxing the assumptions for convergence of the solution of the linearly perturbed Riccati differential equation and deriving new convergence results for systems not necessarily observable. Accepted 30 July 1996     

Krylov subspace methods of Hessenberg based for algebraic Riccati equation

In this paper, we propose a class of special Krylov subspace methods to solve continuous algebraic Riccati equation (CARE), i.e., the Hessenberg-based methods. The presented approaches can obtain efficiently the solution of algebraic Riccati equation to some extent. The main idea is to apply Kleinman-Newton"s method to transform the process of solving algebraic Riccati equation into Lyapunov equation at every inner iteration. Further, the Hessenberg process of pivoting strategy combined with Petrov-Galerkin condition and minimal norm condition is discussed for solving the Lyapunov equation in detail, then we get two methods, namely global generalized Hessenberg (GHESS) and changing minimal residual methods based on the Hessenberg process (CMRH) for solving CARE, respectively. Numerical experiments illustrate the efficiency of the provided methods.     

Backward error,sensitivity, and refinement of computed solutions of algebraic Riccati equations

In this paper, a new backward error criterion, together with a sensitivity measure, is presented for assessing solution accuracy of nonsymmetric and symmetric algebraic Riccati equations (AREs). The usual approach to assessing reliability of computed solutions is to employ standard perturbation and sensitivity results for linear systems and to extend them further to AREs. However, such methods are not altogether appropriate since they do not take account of the underlying structure of these matrix equations. The approach considered here is to first compute the backward error of a computed solution X? that measures the amount by which data must be perturbed so that X? is the exact solution of the perturbed original system. Conventional perturbation theory is used to define structured condition numbers that fully respect the special structure of these matrix equations. The new condition number, together with the backward error of computed solutions, provides accurate estimates for the sensitivity of solutions. Optimal perturbations are then used in an iterative refinement procedure to give further more accurate approximations of actual solutions. The results are derived in their most general setting for nonsymmetric and symmetric AREs. This in turn offers a unifying framework through which it is possible to establish similar results for Sylvester equations, Lyapunov equations, linear systems, and matrix inversions.     

Temporal Difference Methods for the Maximal Solution of Discrete-Time Coupled Algebraic Riccati Equations

In this paper, we present an iterative technique for deriving the maximal solution of a set of discrete-time coupled algebraic Riccati equations, based on temporal difference methods, which are related to the optimal control of Markovian jump linear systems and have been studied extensively over the last few years. We trace a parallel with the theory of temporal difference algorithms for Markovian decision processes to develop a -policy iteration like algorithm for the maximal solution of these equations. For the special cases in which =0 and =1, we have the situation in which the algorithm reduces to the iterations of the Riccati difference equations (value iteration) and quasilinearization method (policy iteration), respectively. The advantage of the proposed method is that an appropriate choice of between 0 and 1 can speed up the convergence of the policy evaluation step of the policy iteration method by using value iteration.     

Exact slow-fast decomposition of the singularly perturbed matrix differential Riccati equation

In this paper the Hamiltonian matrix formulation of the Riccati equation is used to derive the reduced-order pure-slow and pure-fast matrix differential Riccati equations of singularly perturbed systems. These pure-slow and pure-fast matrix differential Riccati equations are obtained by decoupling the singularly perturbed matrix differential Riccati equation of dimension n₁+n₂ into the pure-slow regular matrix differential Riccati equation of dimension n₁ and the pure-fast stiff matrix differential Riccati equation of dimension n₂. A formula is derived that produces the solution of the original singularly perturbed matrix differential Riccati equation in terms of solutions of the pure-slow and pure-fast reduced-order matrix differential Riccati equations and solutions of two reduced-order initial value problems. In addition to its theoretical importance, the main result of this paper can also be used to implement optimal filtering and control schemes for singularly perturbed linear time-invariant systems independently in pure-slow and pure-fast time scales.     

Perturbations and projections of Kalman–Bucy semigroups

We analyse various perturbations and projections of Kalman–Bucy semigroups and Riccati equations. For example, covariance inflation-type perturbations and localisation methods (projections) are common in the ensemble Kalman filtering literature. In the limit of these ensemble methods, the regularised sample covariance tends toward a solution of a perturbed/projected Riccati equation. With this motivation, results are given characterising the error between the nominal and regularised Riccati flows and Kalman–Bucy filtering distributions. New projection-type models are also discussed; e.g. Bose–Mesner projections. These regularisation models are also of interest on their own, and in, e.g., differential games, control of stochastic/jump processes, and robust control.     

On inexact Newton methods based on doubling iteration scheme for non‐symmetric algebraic Riccati equations

Newton iteration method can be used to find the minimal non‐negative solution of a certain class of non‐symmetric algebraic Riccati equations. However, a serious bottleneck exists in efficiency and storage for the implementation of the Newton iteration method, which comes from the use of some direct methods in exactly solving the involved Sylvester equations. In this paper, instead of direct methods, we apply a fast doubling iteration scheme to inexactly solve the Sylvester equations. Hence, a class of inexact Newton iteration methods that uses the Newton iteration method as the outer iteration and the doubling iteration scheme as the inner iteration is obtained. The corresponding procedure is precisely described and two practical methods of monotone convergence are algorithmically presented. In addition, the convergence property of these new methods is studied and numerical results are given to show their feasibility and effectiveness for solving the non‐symmetric algebraic Riccati equations. Copyright © 2010 John Wiley & Sons, Ltd.     

Positive operator based iterative algorithms for solving Lyapunov equations for Itô stochastic systems with Markovian jumps

This paper studies the iterative solutions of Lyapunov matrix equations associated with Itô stochastic systems having Markovian jump parameters. For the discrete-time case, when the associated stochastic system is mean square stable, two iterative algorithms with one in direct form and the other one in implicit form are established. The convergence of the implicit iteration is proved by the properties of some positive operators associated with the stochastic system. For the continuous-time case, a transformation is first performed so that it is transformed into an equivalent discrete-time Lyapunov equation. Then the iterative solution can be obtained by applying the iterative algorithm developed for discrete-time Lyapunov equation. Similar to the discrete-time case, an implicit iteration is also proposed for the continuous case. For both discrete-time and continuous-time Lyapunov equations, the convergence rates of the established algorithms are analyzed and compared. Numerical examples are worked out to validate the effectiveness of the proposed algorithms.     

一类Riccati方程组对称自反解的两种迭代算法

针对源于Markov跳变线性二次控制问题中的一类对偶代数Riccati方程组,分别采用修正共轭梯度算法和正交投影算法作为非精确Newton算法的内迭代方法,建立求其对称自反解的非精确Newton-MCG算法和非精确Newton-OGP算法.两种迭代算法仅要求Riccati方程组存在对称自反解,对系数矩阵等没有附加限定.数值算例表明,两种迭代算法是有效的.     

Solvability Criterion for the Constrained Discrete Lyapunov and Riccati Equations

Conditions for the solvability of the discrete Lyapunov and the discrete Riccati equations subject to linear equality constraints are derived. These problems arise naturally in the context of output min-max robust control. It is shown that the following problems are equivalent to one another: (a) the solvability of the constrained discrete Riccati equation; and (b) the existence of a feedback gain that guarantees the solvability of the constrained discrete Lyapunov equation of the resulting closed loop. A simple criterion for the existence of a solution to both problems is presented. These problems are shown to be related to the discrete positive real property.     

Convergence properties of preconditioned Hermitian and skew-Hermitian splitting methods for non-Hermitian positive semidefinite matrices

For the non-Hermitian and positive semidefinite systems of linear equations, we derive necessary and sufficient conditions for guaranteeing the unconditional convergence of the preconditioned Hermitian and skew-Hermitian splitting iteration methods. We then apply these results to block tridiagonal linear systems in order to obtain convergence conditions for the corresponding block variants of the preconditioned Hermitian and skew-Hermitian splitting iteration methods.

     

Stein iterations for the coupled discrete-time Riccati equations

We consider a set of discrete-time coupled algebraic Riccati equations that arise in quadratic optimal control of Markovian jump linear systems. Two iterations for computing a symmetric (maximal) solution of this system are investigated. We construct sequences of the solutions of the decoupled Stein equations and show that these sequences converge to a solution of the considered system. Numerical experiments are given.     

Approximation of low rank solutions for linear quadratic control of partial differential equations

Algebraic Riccati equations (ARE) of large dimension arise when using approximations to design controllers for systems modeled by partial differential equations. We use a modified Newton method to solve the ARE that takes advantage of several special features of these problems. The modified Newton method leads to a right-hand side of rank equal to the number of inputs regardless of the weights. Thus, the resulting Lyapunov equation can be more efficiently solved. The Cholesky-ADI algorithm is used to solve the Lyapunov equation resulting at each step. The algorithm is straightforward to code. Performance is illustrated with a number of standard examples. An example on controlling the deflection of the Euler-Bernoulli beam indicates that for weakly damped problems a low rank solution to the ARE may not exist. Further analysis supports this point.     

Solving Differential Matrix Equations using Parareal

Differential matrix equations appear in many applications like optimal control of partial differential equations, balanced truncation model order reduction of linear time varying systems and many more. Here, we will focus on differential Riccati equations (DRE). Solving such matrix-valued ordinary differential equations (ODE) is a highly time consuming process. We present a Parareal based algorithm applied to Rosenbrock methods for the solution of the matrix-valued differential Riccati equations. Considering problems of moderate size, direct matrix equation solvers for the solution of the algebraic Lyapunov equations arising inside the time intgration methods are used. (© 2016 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Riccati微分方程特解新求法的研究

通过对一般Riccati方程进行初等变换,使之变为特殊的Riccati方程,然后利用公式、观察实验,或利用二阶微分方程的特解,或利用一阶微分方程组的特解等方法,求得这些Riccati方程的特解.     

Alternately linearized implicit iteration methods for the minimal nonnegative solutions of the nonsymmetric algebraic Riccati equations

For the non‐symmetric algebraic Riccati equations, we establish a class of alternately linearized implicit (ALI) iteration methods for computing its minimal non‐negative solutions by technical combination of alternate splitting and successive approximating of the algebraic Riccati operators. These methods include one iteration parameter, and suitable choices of this parameter may result in fast convergent iteration methods. Under suitable conditions, we prove the monotone convergence and estimate the asymptotic convergence factor of the ALI iteration matrix sequences. Numerical experiments show that the ALI iteration methods are feasible and effective, and can outperform the Newton iteration method and the fixed‐point iteration methods. Besides, we further generalize the known fixed‐point iterations, obtaining an extensive class of relaxed splitting iteration methods for solving the non‐symmetric algebraic Riccati equations. Copyright © 2006 John Wiley & Sons, Ltd.