Optimal control for linear discrete-time systems with Markov perturbations in Hilbert spaces

Email: vio{at}utgjiu.ro Received on September 12, 2007; Accepted on December 26, 2008 In this article, we discuss a quadratic control problem forlinear discrete-time systems with Markov perturbations in Hilbertspaces, which is linked to a discrete-time Riccati equationdefined on certain infinite-dimensional ordered Banach space.We prove that under stabilizability and stochastic uniform observabilityconditions, the Riccati equation has a unique, uniformly positive,bounded on N and stabilizing solution. Based on this result,we solve the proposed optimal control problem. An example illustratesthe theory.     

Optimal control of stationary Markov processes

Sufficient conditions are given for the optimal control of Markov processes when the control policy is stationary and the process possesses a stationary distribution. The costs are unbounded and additive, and may or may not be discounted. Applications to Semi-Markov processes are included, and the results for random walks are related to the author's previous papers on diffusion processes.     

Synchronized stationary distribution for stochastic multi-links systems with Markov jump

This paper is concerned with the existence of a synchronized stationary distribution for stochastic multi-links systems with Markov jump (SMMJs). By employing Lyapunov method, Kirchhoff’s Matrix Tree Theorem in graph theory as well as M-matrix method, several criteria are given to guarantee the existence of a synchronized stationary distribution of SMMJs, including the Lyapunov-type theorem and a coefficients-type theorem. As a subsequent, the theoretical results are applied to a class of stochastic Markov jump oscillators with multi-links and stochastic multi-links Chua’s circuits with Markov jump, which indicates the results present widely applied prospect in various physical systems. Eventually, two examples together with numerical simulations are provided to validate the effectiveness of the theoretical results.     

Parametrized riccati equations for controlled linear differential systems with jump Markov perturbations

An input-output linear time-varying differential system with homogeneous jump Markov parameters and mean square exponential stable evolution is considered. We define a family T(t), t ≥ 0 of linear bounded input-output operators. It is proved that if sup ‖T(t)‖<γ then a parametrized by γ differential Riccati type system has a unique global bounded and stablizing solution. An application to the estimate of a stability radius is given     

Optimal feedback control for dynamic systems with state constraints: An exact penalty approach

In this paper, we consider a class of nonlinear dynamic systems with terminal state and continuous inequality constraints. Our aim is to design an optimal feedback controller that minimizes total system cost and ensures satisfaction of all constraints. We first formulate this problem as a semi-infinite optimization problem. We then show that by using a new exact penalty approach, this semi-infinite optimization problem can be converted into a sequence of nonlinear programming problems, each of which can be solved using standard gradient-based optimization methods. We conclude the paper by discussing applications of our work to glider control.     

Optimal control of fluid dynamic systems with state constraint of pointwise type

The aim of this work is to obtain the maximum principle by spike perturbation for the optimal control problem with pointwise type state constraint governed by 3-dimensional fluid dynamic systems.     

Optimal stabilizing compensator for linear systems under white noise perturbations

For time varying linear stochastic systems it is proved that the best stabilizing compensator is the one constructed by using stabilizing solutions for the standard control and estimation Riccati equations     

Optimal control for parabolic systems with competitive interactions

We consider a two-dimensional parabolic system with general competitive interactions as a two-player game with conflicting objectives and with controls on the inhomogencous (source) terms. We show the existence of an optimal solution of the game as the saddle point of a suitable objective functional.     

Nonlinear perturbations of quasi-linear control systems

Several sufficient conditions are developed for controllability of the perturbed quasi-linear system $\text{x}\text{?}\text{= A(t, x, u)x + B(t, x, u)u + f(t, x, u)}$. In particular, these reduce to conditions on the perturbation function f which guarantee that if the linear system $\text{x}\text{?}\text{= A(t)x + B(t)u}$ is controllable, then the system $\text{x}\text{?}\text{= A(t)x + B(t)u + f(t, x, u)}$ is controllable. These conditions are growth conditions in (x, u) and are obtained by solving a system of nonlinear integral equations.     

Dynamic event-triggered control for discrete-time nonlinear Markov jump systems using policy iteration-based adaptive dynamic programming

This paper investigates a dynamic event-triggered optimal control problem of discrete-time (DT) nonlinear Markov jump systems (MJSs) via exploring policy iteration (PI) adaptive dynamic programming (ADP) algorithms. The performance index function (PIF) defined in each subsystem is updated by utilizing an online PI algorithm, and the corresponding control policy is derived via solving the optimal PIF. Then, we adopt neural network (NN) techniques, including an actor network and a critic network, to estimate the iterative PIF and control policy. Moreover, the designed dynamic event-triggered mechanism (DETM) is employed to avoid wasting additional resources when the estimated iterative control policy is updated. Finally, based on the Lyapunov difference method, it is proved that the system stability and the convergence of all signals can be guaranteed under the developed control scheme. A simulation example for DT nonlinear MJSs with two system modes is presented to demonstrate the feasibility of the control design scheme.     

Output-feedback guaranteed cost control for uncertain time-delay systems with Markov jumps

This paper considers the guaranteed cost control problem ofthe jump linear system with uncertain parameters and constanttime delay. The sufficient and necessary condition for the systemto be mean square quadratically stable (MSQS) is obtained. Usingthe LMI technique, an algorithm to design an output-feedbackguaranteed controller which stabilizes the system in the MSQSsense is developed.     

Optimal sliding mode control for nonlinear systems with time-delay

This paper is concerned with the sliding mode control (SMC) for a class of nonlinear systems with time-delay. A novel optimal sliding mode is proposed by using the successive approximation approach (SAA). The stability of the nonlinear sliding mode is analyzed. The switching manifold ensures that the state trajectories of the closed-loop system converge to zero in an optimal fashion on the ideal sliding surface. Furthermore, the convergence velocity of every state trajectory on the ideal sliding surface can be adjusted through choosing the parameters of the quadratic performance index. A numerical simulation is given to show the effectiveness of the proposed design approach.     

Optimal control for nonlinear systems calculated with small computers

Using Ritz's procedure of representing the control functions of an optimal control problem by a function series with parameters to be optimized, it is shown that, from the well-known gradient procedure for dynamic problems, a simple iteration formula for the optimization of these parameters can be derived. Using an example with a technical background, the effectiveness of the program realization of this approach is demonstrated and is compared with the results of unrestricted dynamic optimization.This work was performed at the Technische Hochschule in Darmstadt, West Germany, with financial support from the DFG (Deutsche Forschungs-Gemeinschaft).     

Moments for stationary Markov chains with asymptotically zero drift

We consider a Markov chain on ℝ⁺ with asymptotically zero drift and finite second moments of jumps. We assume that the chain has invariant distribution. The paper is devoted to the existence and nonexistence of moments of invariant distribution. Our analysis is based on the technique of test functions.     

Optimal stopping with continuous control of piecewise deterministic Markov processes

In this paper we consider the problem of optimal stopping and continuous control on some local parameters of a piecewise-deterministic Markov processes (PDP's). Optimality equations are obtained in terms of a set of variational inequalities as well as on the first jump time operator of the PDP. It is shown that if the final cost function is absolutely continuous along trajectories then so is the value function of the optimal stopping problem with continuous control. These results unify and generalize previous ones in the current literature.     

Stabilization of stationary affine control systems with discrete time

We obtain new sufficient conditions for the local and global asymptotic stabilization of the zero solution of a nonlinear affine control system with discrete time and with constant coefficients by a continuous state feedback. We assume that the zero solution of the free system is Lyapunov stable. For systems with linear drift, we construct a bounded control in the problem of global asymptotic state and output stabilization. Corollaries for bilinear systems are obtained.