Dynamic programming in stochastic control of systems with delay

We consider optimal control problems for systems described by stochastic differential equations with delay (SDDE). We prove a version of Bellman's principle of optimality (the dynamic programming principle) for a general class of such problems. That the class in general means that both the dynamics and the cost depends on the past in a general way. As an application, we study systems where the value function depends on the past only through some weighted average. For such systems we obtain a Hamilton-Jacobi-Bellman partial differential equation that the value function must solve if it is smooth enough. The weak uniqueness of the SDDEs we consider is our main tool in proving the result. Notions of strong and weak uniqueness for SDDEs are introduced, and we prove that strong uniqueness implies weak uniqueness, just as for ordinary stochastic differential equations.     

The delay approximation of discrete control in higher-order systems

Physical machinery of high mathematical order operating in industry increasingly nowadays requires the replacement of early analog controllers by imbedded digital systems. Using a 7^th-order longwall shearer as the motivating case-study, the paper examines the possibility of easing the digital control system design process by substituting a continuous transport-delay for the digital sample-hold operation. Based on first- and second-order models, this is shown to produce a design procedure that is pessimistic (i.e., safe) in the main and to a practically acceptable degree. Similar response speeds are generated by the discrete- and delay-models. By simulation, the technique is shown to carry over well to the higher-order case study aforementioned. This provides confidence in the delay-for-sampler substitution, the advantages of which are that the need to generate multiple z-transform models for a range of sampling intervals and the difficulties of interpreting z-plane plots (compared to those of familiar Nyquist diagrams and root loci) are avoided.     

Optimization of control systems with a piecewise-linear output

A linear terminal problem of optimal control with a piecewise-linear terminal constraints is considered. On the base of the concept of a support the optimality criterion is proved and sufficient optimality condition in the form of the maximum principle is formulated. The support enables us to choose from the set of the Lagrange vectors a special one which in the terminology of linear programming is called the basic vector [1]. In the case of nondegeneracy of the support control the sufficient condition under question is proved to be necessary condition     

Optimal control computation for parabolic systems with boundary conditions involving time delays

In this paper, we consider a class of optimal control problems involving a second-order, linear parabolic partial differential equation with Neumann boundary conditions. The time-delayed arguments are assumed to appear in the boundary conditions. A necessary and sufficient condition for optimality is derived, and an iterative method for solving this optimal control problem is proposed. The convergence property of this iterative method is also investigated.On the basis of a finite-element Galerkin's scheme, we convert the original distributed optimal control problem into a sequence of approximate problems involving only lumped-parameter systems. A computational algorithm is then developed for each of these approximate problems. For illustration, a one-dimensional example is solved.     

Convergence of a strong variational algorithm for relaxed controls involving a distributed optimal control problem of parabolic type 1

In this paper, we consider a class of Optimal Control problems involving first boundary value problems of parabolic type. A strong variational algorithm has been obtained for solving this class of optimal control problems in a paper by the author and D. W. Reid. It was also shown that any L∞ accumulation points of control sequences generated by the algorithm satisfy a necessary condition for optimality. Since such accumulation points need not exist, it is shown in this paper that control sequences generated by the algorithm always have accumulation points in the sense of control measure, and these accumulation points satisfy a necessary condition for optimality for the corresponding relaxed control problem.     

Optimal control of parabolic problems with state constraints: a penalization method for optimality conditions

In this paper state constrained optimal control problems governed by parabolic evolution equations are studied. Our purpose is to obtain a (first-order) decoupled optimality system (that ensures the Lagrange multipliers existence). In a first step we are led to Slater-like assumptions and we are then allowed to extend the application field of the decoupled system we obtain. With a weaker assumption the existence of Lagrange multipliers (that are measures) for nonqualified problems may be established.     

Optimal control of a system governed by a parabolic equation with an infinite number of variables and time delay

A distributed control problem with delay for the parabolic operator with an infinite number of variables is considered. The performance index has an integral form. Constraints on controls are assumed. To obtain optimality conditions, the generalization of the Dubovicki-Milutin theorem given by Walczak in Ref. 1 was applied.     

Asymmetric games for convolution systems with applications to feedback control of constrained parabolic equations

The paper is devoted to the study of some classes of feedback control problems for linear parabolic equations subject to hard/pointwise constraints on both Dirichlet boundary controls and state dynamic/output functions in the presence of uncertain perturbations within given regions. The underlying problem under consideration, originally motivated by automatic control of the groundwater regime in irrigation networks, is formalized as a minimax problem of optimal control, where the control strategy is sought as a feedback law. Problems of this type are among the most important in control theory and applications — while most challenging and difficult. Based on the Maximum Principle for parabolic equations and on the time convolution structure, we reformulate the problems under consideration as certain asymmetric games, which become the main object of our study in this paper. We establish some simple conditions for the existence of winning and losing strategies for the game players, which then allow us to clarify controllability issues in the feedback control problem for such constrained parabolic systems.     

Global stabilization of switched control systems with time delay

In this paper, the stabilization problem of switched control systems with time delay is investigated for both linear and nonlinear cases. First, a new global stabilizability concept with respect to state feedback and switching law is given. Then, based on multiple Lyapunov functions and delay inequalities, the state feedback controller and the switching law are devised to make sure that the resulting closed-loop switched control systems with time delay are globally asymptotically stable and exponentially stable.     

State feedback linearization of nonlinear control systems on homogeneous time scales

The paper addresses the state feedback linearization problem for nonlinear systems, defined on homogeneous time scale. Necessary and sufficient solvability conditions are given within the algebraic framework of differential one-forms. The conditions concerning the exact dynamic state feedback linearization are equivalent to the property of differential flatness of the system. An output function which defines a right invertible system without zero-dynamics is shown to exist if and only if the basis of some space of one-forms can be transformed, via polynomial matrix operator over the field of meromorphic functions, into a system of exact one-forms. The results extend the corresponding results for the continuous-time case.     

Optimal control of impulsive hybrid systems

In this paper, we deal with optimization techniques for a class of hybrid systems that comprise continuous controllable dynamics and impulses (jumps) in the state. Using the mathematical techniques of distributional derivatives and impulse differential equations, we rewrite the original hybrid control system as a system with autonomous location transitions. For the obtained auxiliary dynamical system and the corresponding optimal control problem (OCP), we apply the Lagrange approach and derive the reduced gradient formulas. Moreover, we formulate necessary optimality conditions for the above hybrid OCPs, and discuss the newly elaborated Pontryagin-type Maximum Principle for impulsive OCPs. As in the case of the conventional OCPs, the proposed first order optimization techniques provide a basis for constructive computational algorithms.     

Stability and attractivity of periodic solutions of parabolic systems with time delays

This paper is concerned with the existence, stability, and global attractivity of time-periodic solutions for a class of coupled parabolic equations in a bounded domain. The problem under consideration includes coupled system of parabolic and ordinary differential equations, and time delays may appear in the nonlinear reaction functions. Our approach to the problem is by the method of upper and lower solutions and its associated monotone iterations. The existence of time-periodic solutions is for a class of locally Lipschitz continuous reaction functions without any quasimonotone requirement using Schauder fixed point theorem, while the stability and attractivity analysis is for quasimonotone nondecreasing and mixed quasimonotone reaction functions using the monotone iterative scheme. The results for the general system are applied to the standard parabolic equations without time delay and to the corresponding ordinary differential system. Applications are also given to three Lotka-Volterra reaction diffusion model problems, and in each problem a sufficient condition on the reaction rates is obtained to ensure the stability and global attractivity of positive periodic solutions.     

On an optimal control problem involving first order hyperbolic systems with boundary controls

In this paper, we consider an optimal control problem involving a class of first order hyperbolic systems with boundary controls. A computational algorithm which generates minimizing sequences of controls is devised and the convergence properties of the algorithm are investigated. Moreover, a necessary and sufficient condition for optimality is derived and a result on the existence of optimal controls is obtained.     

Time-optimal control of parabolic systems with boundary conditions involving time delays

The present paper is concerned with the control of certain parabolic systems whose boundary conditions involve time delays. The optimal controls are characterized in terms of an adjoint system and shown to be unique and bang-bang. These results extend to certain cases of nonlinear control and to fixed-time, minimum-norm control problems.     

Time inhomogeneous Stochastic Differential Equations involving the local time of the unknown process,and associated parabolic operators

In this paper we study time inhomogeneous versions of one-dimensional Stochastic Differential Equations (SDE) involving the Local Time of the unknown process on curves. After proving existence and uniqueness for these SDEs under mild assumptions, we explore their link with Parabolic Differential Equations (PDE) with transmission conditions. We study the regularity of solutions of such PDEs and ensure the validity of a Feynman–Kac representation formula. These results are then used to characterize the solutions of these SDEs as time inhomogeneous Markov Feller processes.     

Sliding mode control experiments of uncertain dynamical systems with time delay

This article presents the simulation and experimental studies for the nonlinear time-delayed dynamical systems with uncertainties. A rotary flexible joint made by Quanser is chosen as the model system to investigate the method for sliding mode control design. We considered the geometric nonlinearity of the flexible joint consisting of two linear springs. The system is assumed to have constant delay time and uncertain parameters with known upper and lower bounds. We also design an optimal sliding surface for the sliding mode control. Simulations and experiments are carried out to demonstrate the utility of the control method. Finally, the results from the simulations and experiments are in excellent agreement.     

Stability of singularly perturbed switched systems with time delay and impulsive effects

This paper studies the stability properties of singularly perturbed switched systems with time delay and impulsive effects. Such systems are assumed to consist of both unstable and stable subsystems. By using the multiple Lyapunov functions technique and the dwell time approach, some stability criteria are established. Our results show that impulses do contribute in order to obtain stability properties even when the system consists of only unstable subsystems. Numerical examples are presented to verify our theoretical results.     

Optimal control of parabolic equation with an infinite number of variables for non-standard functional and time delay

A distributed control problem for the parabolic operator withan infinite number of variables and time delay is considered.The performance index has an integral form. Constraints on controlsare imposed. To obtain optimality conditions for the Neumannproblem, the generalization of the Dubovitskii–Milyutintheorem given by Walczak in WALCZAK, S. Folia Mathematics, 1,187–196 and WALCZAK, S. J. Optim. Theory Appl., 42, 561–582was applied.     

Existence and stability of periodic solutions for parabolic systems with time delays

This paper is concerned with the existence and stability time-periodic solutions for a class of coupled parabolic equations with time delay, and time delays may appear in the nonlinear reaction functions. The existence of time-periodic solutions is for a class of locally Lipschitz continuous reaction functions without any quasimonotone requirement. Our approach to the problem is by the method of upper and lower solution and using Schauder fixed point theorem. Some methods for proving the stability of the periodic solution are also given. The results for the general system can be applied to the standard parabolic equations without time delay and corresponding ordinary differential system. Finally, a model arising from chemistry is used to illustrate the obtained results.