Indefinite LQ optimal control with cross term for discrete‐time uncertain systems

Recently, there has been an increasing interest in the study on uncertain optimal control problems. In this paper, a linear quadratic (LQ) optimal control with cross term for discrete‐time uncertain systems is considered, whereas the weighting matrices in the cost function are allowed to be indefinite. Firstly, a recurrence equation for the problem is presented based on Bellman's principle of optimality in dynamic programming. Then, a necessary condition for the existence of an optimal linear state feedback control of the indefinite LQ problem is given by the recurrence equation. Moreover, a sufficient condition of well‐posedness for the indefinite LQ problem is presented by introducing a linear matrix inequality (LMI) condition. Furthermore, it is shown that the well‐posedness of the indefinite LQ problem, the solvability of the indefinite LQ problem, the LMI condition, and the solvability of the constrained difference equation are equivalent to each other. Finally, an example is presented to illustrate the results obtained.     

Stochastic Linear Quadratic Optimal Control Problems

This paper is concerned with the stochastic linear quadratic optimal control problem (LQ problem, for short) for which the coefficients are allowed to be random and the cost functional is allowed to have a negative weight on the square of the control variable. Some intrinsic relations among the LQ problem, the stochastic maximum principle, and the (linear) forward—backward stochastic differential equations are established. Some results involving Riccati equation are discussed as well. Accepted 15 May 2000. Online publication 1 December 2000     

STOCHASTIC LINEAR QUADRATIC OPTIMAL CONTROL PROBLEMS WITH RANDOM COEFFICIENTS

61. IntroductionLet (fi, F, P, {R}tZo) be a complete filtered probability space on which a standard onedimensional Brownian motion w(') is defined such that {R}tZo is the natural filtrationgenerated by w(.), augmented by all the p-null sets in i. We consider the following stateequationwhere T E T[0, TI, the set of all {R}tZo-stopping times taking values in [0, T], (E sigLlt (fi;IR"); A, B, C, D are matrix-valued {R}tZo-adapted bounded processes. In the above, u(.) EU[T, T]gLI(T, T…     

随机线性二次最优控制:从离散到连续时间模型

在一般情形下,分析了离散时间LQ问题与连续时间情形两者之间的自然联系.首先回顾了连续时间和离散时间随机LQ问题及对应Riccati微分/差分方程的相关结论.接下来在假设Riccati微分方程有解的前提下,证明了离散化步长足够小时,Riccati差分方程有解.然后针对连续和离散时间模型,采用配对问题最优控制的反馈形式,分别构造了一个辅助反馈控制,并证明该控制可驱使对应模型的性能指标逼近于配对问题的值函数,以此得到了关于两个模型之间联系的初步结论.最后藉由前述结论以及控制问题的特性,揭晓了连续时间和离散时间模型之间的自然联系,并给出了Riccati差分方程和微分方程的解之间的误差估计.由此联系,可构造相应离散系统和LQ问题,以适当的阶估计连续时间LQ问题的解,抑或为离散时间模型构造一个近似最优控制.无论哪种思路,都旨在降低直接求解原问题的难度和复杂性.     

Semi-Infinite Programming Approach to Continuously-Constrained Linear-Quadratic Optimal Control Problems

Consider the class of linear-quadratic (LQ) optimal control problems with continuous linear state constraints, that is, constraints imposed on every instant of the time horizon. This class of problems is known to be difficult to solve numerically. In this paper, a computational method based on a semi-infinite programming approach is given. The LQ optimal control problem is formulated as a positive-quadratic infinite programming problem. This can be done by considering the control as the decision variable, while taking the state as a function of the control. After parametrizing the decision variable, an approximate quadratic semi-infinite programming problem is obtained. It is shown that, as we refine the parametrization, the solution sequence of the approximate problems converges to the solution of the infinite programming problem (hence, to the solution of the original optimal control problem). Numerically, the semi-infinite programming problems obtained above can be solved efficiently using an algorithm based on a dual parametrization method.     

Stability analysis and optimal control of stochastic singular systems

In this paper, problems of stability and optimal control for a class of stochastic singular systems are studied. Firstly, under some appropriate assumptions, some new results about mean-square admissibility are developed and the corresponding LMI sufficient condition is given. Secondly, finite-time horizon and infinite-time horizon linear quadratic (LQ) control problems for the stochastic singular system are investigated, in which the coefficients are allowed to be random in control input and quadratic criterion. Some results involving new stochastic generalized Riccati equation are discussed as well. Finally, the proposed LQ control model for stochastic singular systems provides an appropriate and effective framework to study the portfolio selection problem in light of the recent development on general stochastic LQ problems.     

伊藤-泊松型随机微分方程的线性二次控制

本文研究伊藤-泊松型随机微分方程的线性二次控制问题,利用动态规划方法、伊藤公式等技巧,通过解HJB方程,我们得到了随机Riccati方程及另外两个微分方程,求出控制变量,解决了线性二次最优控制最优问题.     

Indefinite Stochastic Linear Quadratic Control with Markovian Jumps in Infinite Time Horizon

This paper studies a stochastic linear quadratic (LQ) control problem in the infinite time horizon with Markovian jumps in parameter values. In contrast to the deterministic case, the cost weighting matrices of the state and control are allowed to be indinifite here. When the generator matrix of the jump process – which is assumed to be a Markov chain – is known and time-invariant, the well-posedness of the indefinite stochastic LQ problem is shown to be equivalent to the solvability of a system of coupled generalized algebraic Riccati equations (CGAREs) that involves equality and inequality constraints. To analyze the CGAREs, linear matrix inequalities (LMIs) are utilized, and the equivalence between the feasibility of the LMIs and the solvability of the CGAREs is established. Finally, an LMI-based algorithm is devised to slove the CGAREs via a semidefinite programming, and numerical results are presented to illustrate the proposed algorithm.     

Elliptic partial differential equation and optimal control

The theory of optimal control and the semianalytical method of elliptic partial differential equation (PDE) in a prismatic domain are mutually simulated issues. The simulation of discrete-time linear quadratic (LQ) control with the substructural chain problem in static structural analysis is given first. From the minimum potential energy variational principle of substructural chain, the generalized variational principle with two kinds of variables and the dual equations are derived. The simulation relation is then recognized by comparing the variational principle and dual equations of the LQ control theory. The simulation between elliptic PDE in the prismatic domain and continuous-time LQ control is established in the same way, and the interval energy is naturally introduced, as in the case of substructural chain. The assembling and condensation equations can help one to derive the differential equations of the submatrices of potential energy and mixed energy. The well known Riccati equation is one of them. The interval assembling and condensation algorithm can be used to solve the Riccati equation. Some numerical examples are given to illustrate the method.     

Indefinite stochastic LQ control with cross term via semidefinite programming

An indefinite stochastic linear-quadratic (LQ) optimal control problem with cross term over an infinite time horizon is studied, allowing the weighting matrices to be indefinite. A systematic approach to the problem based on semidefinite programming (SDP) and related duality analysis is developed. Several implication relations among the SDP complementary duality, the existence of the solution to the generalized Riccati equation and the optimality of LQ problem are discussed. Based on these relations, a numerical procedure that provides a thorough treatment of the LQ problem via primal-dual SDP is given: it identifies a stabilizing optimal feedback control or determines the problem has no optimal solution. An example is provided to illustrate the results obtained.     

General Linear Quadratic Optimal Stochastic Control Problem Driven by a Brownian Motion and a Poisson Random Martingale Measure with Random Coefficients

In this article, we consider a linear-quadratic optimal control problem (LQ problem) for a controlled linear stochastic differential equation driven by a multidimensional Browinan motion and a Poisson random martingale measure in the general case, where the coefficients are allowed to be predictable processes or random matrices. By the duality technique, the dual characterization of the optimal control is derived by the optimality system (so-called stochastic Hamilton system), which turns out to be a linear fully coupled forward-backward stochastic differential equation with jumps. Using a decoupling technique, the connection between the stochastic Hamilton system and the associated Riccati equation is established. As a result, the state feedback representation is obtained for the optimal control. As the coefficients for the LQ problem are random, here, the associated Riccati equation is a highly nonlinear backward stochastic differential equation (BSDE) with jumps, where the generator depends on the unknown variables K, L, and H in a quadratic way (see (5.9) herein). For the case where the generator is bounded and is linearly dependent on the unknown martingale terms L and H, the existence and uniqueness of the solution for the associated Riccati equation are established by Bellman's principle of quasi-linearization.     

广义LQ最优控制问题的极小化序列

将经典LQ问题的评价泛函中关于控制变量的二次型推广为一类偶次多项式,证明了这类广义LQ无约束最优控制问题的一个等价扩张逼近可由一列半径递增的球约束最优控制问题加以实现.进而利用P0ntryagin极值原理建立相应的球约束最优控制问题的二次规划,并通过Canonical倒向微分流及不动点定理,求解常微分方程边值问题,得到球约束最优控制问题的最优值.随着约束球半径趋于无穷大,形成原广义LQ最优控制问题的一个极小化序列,从而得到原问题的最优值.     

一般目标信号和干扰信号下多采样率系统的最优预见控制器设计

针对目标信号和干扰信号为多项式的情形,研究了多采样率离散时间控制系统的最优预见控制问题.首先利用离散时间系统提升技术,把所研究的系统转化成单采样率的扩大系统.然后构造扩大误差系统,把问题转化为包含预见信号的最优调节问题.最后利用最优预见控制理论的结果得到系统的最优预见控制输入,其中包含积分器和预见前馈补偿.本文还对扩大误差系统的能控性和能观测性和相应的代数Riccati方程的可解性进行了讨论.     

Continuous-Time Mean-Variance Portfolio Selection: A Stochastic LQ Framework

This paper is concerned with a continuous-time mean-variance portfolio selection model that is formulated as a bicriteria optimization problem. The objective is to maximize the expected terminal return and minimize the variance of the terminal wealth. By putting weights on the two criteria one obtains a single objective stochastic control problem which is however not in the standard form due to the variance term involved. It is shown that this nonstandard problem can be ``embedded' into a class of auxiliary stochastic linear-quadratic (LQ) problems. The stochastic LQ control model proves to be an appropriate and effective framework to study the mean-variance problem in light of the recent development on general stochastic LQ problems with indefinite control weighting matrices. This gives rise to the efficient frontier in a closed form for the original portfolio selection problem. Accepted 24 November 1999     

线性等式约束系统广义Riccati代数方程的求解*

本文基于定常离散LQ控制问题的动力学方程、价值泛函及系统的约束方程,根据极大值原理,给出了线性等式约束系统下的广义Riccati方程,进而对上述方程进行了深入的探讨,并给出了相应的数值例题。     

On the existence of the stabilizing solution of generalized Riccati equations arising in zero-sum stochastic difference games: the time-varying case

In this paper, a large class of time-varying Riccati equations arising in stochastic dynamic games is considered. The problem of the existence and uniqueness of some globally defined solution, namely the bounded and stabilizing solution, is investigated. As an application of the obtained existence results, we address in a second step the problem of infinite-horizon zero-sum two players linear quadratic (LQ) dynamic game for a stochastic discrete-time dynamical system subject to both random switching of its coefficients and multiplicative noise. We show that in the solution of such an optimal control problem, a crucial role is played by the unique bounded and stabilizing solution of the considered class of generalized Riccati equations.     

On the Bellman Equation for Infinite Horizon Problems with Unbounded Cost Functional

We study a class of infinite horizon control problems for nonlinear systems, which includes the Linear Quadratic (LQ) problem, using the Dynamic Programming approach. Sufficient conditions for the regularity of the value function are given. The value function is compared with sub- and supersolutions of the Bellman equation and a uniqueness theorem is proved for this equation among locally Lipschitz functions bounded below. As an application it is shown that an optimal control for the LQ problem is nearly optimal for a large class of small unbounded nonlinear and nonquadratic pertubations of the same problem. Accepted 8 October 1998     

带有交叉项的离散不定随机线性二次控制问题

研究性能指标带有交叉项的离散时间不定随机线性二次(LQ)控制问题，允许权矩阵是不定的。引入一个广义差分Riccati方程，证明了此方程的可解性是LQ问题存在最优控制的一个充分条件，并用方程的解给出了最优控制。推广了[1]的结果。     

Indefinite stochastic linear-quadratic optimal control problems with random jumps and related stochastic Riccati equations

We discuss the stochastic linear-quadratic (LQ) optimal control problem with Poisson processes under the indefinite case. Based on the wellposedness of the LQ problem, the main idea is expressed by the definition of relax compensator that extends the stochastic Hamiltonian system and stochastic Riccati equation with Poisson processes (SREP) from the positive definite case to the indefinite case. We mainly study the existence and uniqueness of the solution for the stochastic Hamiltonian system and obtain the optimal control with open-loop form. Then, we further investigate the existence and uniqueness of the solution for SREP in some special case and obtain the optimal control in close-loop form.     

Time-Varying Linear-Quadratic Control

This paper discusses the performance of controlled linear dynamic systems that use time-varying feedforward signals and time-varying linear-quadratic (LQ) feedback gains. Such a time-varying LQ controller can bring a dynamic system to a desired final state in roughly half the time required by a time-invariant LQ controller, since it pushes at both ends, i.e., it uses significant control effort near the end of the maneuver, as well as at the beginning, to meet the specified end conditions; there is no overshoot and no settling time. This requires a more complex controller and some care with the high gains near the final time. A MATLAB³ code is listed that synthesizes and simulates zero-order-hold time-varying LQ controllers. The precision landing of a helicopter using four controls is treated as an example.