Discrete time controllable processes with bounded drift and their application in queueing systems

Recursive algorithms are considered for construction of optimal policies for some special classes of controllable stochastic processes, which are called discrete-time controllable semi-regenerative processes. By means of these processes many controllable queueing systems can be modelled. Numerical examples are given for the investigation of optimal policies in such systems.     

一类非线性系统的迭代学习控制

讨论一类非线性系统的迭代学习控制,系统的非线性动态对状态不快于多项式增长,而量测方程含有噪声.控制序列并非直接输给系统,而是先经过死区、预载及饱和等非线性函数.递推地给出了学习控制序列,并证明它的有界性及最优跟踪性.     

Approximability of nonlinear affine control systems

This paper focuses on a strong approximability property for nonlinear affine control systems. We consider control processes governed by ordinary differential equations (ODEs) and study an initial system and the associated generalized system. Our theoretical approach makes it possible to prove a strong approximability result for the above dynamical systems. The latter can be effectively applied to some classes of variable structure and hybrid control systems. In particular, this paper deals with applications of the strong approximability property obtained to the conventional sliding mode processes and to hybrid control systems with autonomous location transitions. We also take into consideration some optimal control problems for the above class of hybrid systems.     

Using a hybrid preconditioner for solving large-scale linear systems arising from interior point methods

We devise a hybrid approach for solving linear systems arising from interior point methods applied to linear programming problems. These systems are solved by preconditioned conjugate gradient method that works in two phases. During phase I it uses a kind of incomplete Cholesky preconditioner such that fill-in can be controlled in terms of available memory. As the optimal solution of the problem is approached, the linear systems becomes highly ill-conditioned and the method changes to phase II. In this phase a preconditioner based on the LU factorization is found to work better near a solution of the LP problem. The numerical experiments reveal that the iterative hybrid approach works better than Cholesky factorization on some classes of large-scale problems.     

Optimal processes in smooth-convex minimization problems

The paper elaborates a general method for studying smooth-convex conditional minimization problems that allows one to obtain necessary conditions for solutions of these problems in the case where the image of the mapping corresponding to the constraints of the problem considered can be of infinite codimension. On the basis of the elaborated method, the author proves necessary optimality conditions in the form of an analog of the Pontryagin maximum principle in various classes of quasilinear optimal control problems with mixed constraints; moreover, the author succeeds in preserving a unified approach to obtaining necessary optimality conditions for control systems without delays, as well as for systems with incommensurable delays in state coordinates and control parameters. The obtained necessary optimality conditions are of a constructive character, which allows one to construct optimal processes in practical problems (from biology, economics, social sciences, electric technology, metallurgy, etc.), in which it is necessary to take into account an interrelation between the control parameters and the state coordinates of the control object considered. The result referring to systems with aftereffect allows one to successfully study many-branch product processes, in particular, processes with constraints of the “bottle-neck” type, which were considered by R. Bellman, and also those modern problems of flight dynamics, space navigation, building, etc. in which, along with mixed constraints, it is necessary to take into account the delay effect. The author suggests a general scheme for studying optimal process with free right endpoint based on the application of the obtained necessary optimality conditions, which allows one to find optimal processes in those control systems in which no singular cases arise. The author gives an effective procedure for studying the singular case (the procedure for calculating a singular control in quasilinear systems with mixed constraints. Using the obtained necessary optimality conditions, the author constructs optimal processes in concrete control systems. __________ Translated from Sovremennaya Matematika i Ee Prilozheniya (Contemporary Mathematics and Its Applications), Vol. 42, Optimal Control, 2006.     

Two-person game with nash equilibrium in optimal control problems

A two-person game with a Nash equilibrium is formulated for optimal control problems with a free right end and a linear differential system. The game is reduced to the calculation of a fixed point of an extremal mapping, which in turn is reduced to a variational inequality with linear constraints generated by systems of linear differential controllable processes. An extra-gradient iterative method is proposed for calculating the Nash equilibrium of the dynamic game. The convergence of the method is proved.     

Décomposition de domaine pour le contrôle de systèmes gouvernés par des équations d'évolution

We present a non overlapping iterative domain decomposition method with “coupled” Robin transmission conditions. We prove its convergence on an optimal control problem for the wave equation. The linear part of the “feed-back” law associated to the local optimal control problems set on subdomains is independent of the iterative process. The method can be applied, at least formally, to the optimal control of systems governed by evolution equations.     

Choosing Optimal Parameters in Iterative Processes

We consider the problem of choosing optimal parameters in certain iterative procedures. Specifically, we are interested in finite-step processes for which it is possible to estimate the computational cost and the error relaxation in terms of the process parameters. The problem of finding the optimal process that provides the required error relaxation with a minimal total computational cost is defined and studied. To solve the problem, it is generally necessary to solve a series of mathematical programming problems with rapidly increasing dimension. We suggest two ways to avoid that difficulty. The first is to find a process that is close to the optimal process, by solving only one mathematical programming problem. The second is to define optimal processes in some special cases when this problem can be simplified. We define conditions under which processes with geometrically decreasing error are optimal or asymptotically optimal. The methods of finding parameters of such processes are also provided. We illustrate our ideas with two examples: the bilevel gradient method for unconstrained function minimization and the iterative process for solving an optimal design problem.     

Minimum-order observers for discrete-time systems

We consider the synthesis of a minimum-order state or functional observer for a linear dynamical system. The synthesis problem is solved for completely certain systems of general form and for some classes of uncertain systems. Various approaches are described, which ultimately lead to the same task: finding a minimum-dimension Hurwitz solution for a system of linear equations with a Hankel matrix. For scalar and vector linear systems, prior upper and lower bounds on the observer dimension are derived, which makes it possible to switch to an iterative procedure of finding an optimal solution. The discussion is set out for discrete-time dynamical systems.     

Two Optimal General Classes of Iterative Methods with Eighth-Order

Two new three-step classes of optimal iterative methods to approximate simple roots of nonlinear equations, satisfying the Kung-Traub’s conjecture, are designed. The development of the methods and their convergence analysis are provided joint with a generalization of both processes. In order to check the goodness of the theoretical results, some concrete methods are extracted and numerical and dynamically compared with some known methods.     

Optimal control computation for parabolic systems with boundary conditions involving time delays

In this paper, we consider a class of optimal control problems involving a second-order, linear parabolic partial differential equation with Neumann boundary conditions. The time-delayed arguments are assumed to appear in the boundary conditions. A necessary and sufficient condition for optimality is derived, and an iterative method for solving this optimal control problem is proposed. The convergence property of this iterative method is also investigated.On the basis of a finite-element Galerkin's scheme, we convert the original distributed optimal control problem into a sequence of approximate problems involving only lumped-parameter systems. A computational algorithm is then developed for each of these approximate problems. For illustration, a one-dimensional example is solved.     

Stochastic control problems with delay

We consider optimal control problems for systems described by stochastic differential equations with delay. We state conditions for certain classes of such systems under which the stochastic control problems become finite-dimensional. These conditions are illustrated with three applications. First, we solve some linear quadratic problems with delay. Then we find the optimal consumption rate in a financial market with delay. Finally, we solve explicitly a deterministic fluid problem with delay which arises from admission control in ATM communication networks.     

document

This work develops asymptotically optimal controls for discrete-time singularly perturbed Markov decision processes (MDPs) having weak and strong interactions. The focus is on finite-state-space-MDP problems. The state space of the underlying Markov chain can be decomposed into a number of recurrent classes or a number of recurrent classes and a group of transient states. Using a hierarchical control approach, continuous-time limit problems that are much simpler to handle than the original ones are derived. Based on the optimal solutions for the limit problems, nearly optimal decisions for the original problems are obtained. The asymptotic optimality of such controls is proved and the rate of convergence is provided. Infinite horizon problems are considered; both discounted costs and long-run average costs are examined.     

ANALYSIS OF A MECHANICAL SOLVER FOR LINEAR SYSTEMS OF EQUATIONS S

1. Introduction/ A new approaCh to solve systems of linear eqttations, equlvaleat to solve the ~ion of adamped harmonic oscillator, has been PrOPosed in a previous paper[11. Due to this parallelism,we call such methods Mechanical Solvers for systems of linear equations. The present study isdevoted to the analysis of these methods.Let be the linear systemwhere we assume that A is an m x m nonsingular matriX (i.e. the system has a ~ solution).We may associate to it the Newton's equation for …     

Analysis of a Mechanical Solver for Linear Systems of Equations

In this contribution we analyse some fundamental features of an iterative method to solve systems of linear equations, following the approach introduced in a previous work [1]. Such questions range from optimal parameters and initial conditions to comparison with other methods. An interesting result is that a priori we can give an estimation of the number of iterations to get a given accuracy.     

Formulation and numerical solution of finite-level quantum optimal control problems

Optimal control of finite-level quantum systems is investigated, and iterative solution schemes for the optimization of a control representing laser pulses are developed. The purpose of this external field is to channel the system's wavefunction between given states in its most efficient way. Physically motivated constraints, such as limited laser resources or population suppression of certain states, are accounted for through an appropriately chosen cost functional. First-order necessary optimality conditions and second-order sufficient optimality conditions are investigated. For solving the optimal control problems, a cascadic non-linear conjugate gradient scheme and a monotonic scheme are discussed. Results of numerical experiments with a representative finite-level quantum system demonstrate the effectiveness of the optimal control formulation and efficiency and robustness of the proposed approaches.     

On the control of queueing systems with failures and repairs

Several classes of problems can be modelled as queueing systems with failures where preventative maintenance has to be optimized: relevant examples are computer systems, databases and production lines. This paper studies a simple case of such a situation based on the M/M/1 queue where the state of the server can take two values (on, off); the transition on–off happens either when there is a failure or when one decides to maintain. The characterization and an approximation of an optimal policy are given. Extensions to more general models, including general Markov processes, are examined.     

A Unified Class of Directly Solvable Semidefinite Programming Problems

We propose a class of semidefinite programming (SDP) problems for which an optimal solution can be calculated directly, i.e., without using an iterative method. Several classes of such SDP problems have been proposed. Among them, Vanderbei and Yang (1995), Ohara (1998), and Wolkovicz (1996) are well known. We show that our class contains all of the three classes as special cases.     

Optimal controller placement in modal control of complex systems

Within the framework of modal control of large systems, a simple approach is advanced for the determination of optimal control configuration under an energy constraint, i.e., optimal locations of a limited number of controllers such that the total energy requirement for control is minimized. It is shown that the resulting design criterion is a simple function of projections of the control matrix onto components of eigenvectors associated with the affected eigenvalues. Furthermore, it is applicable to both single-input and multi-input systems. Systems possessing distinct complex eigenvalues are considered but the approach is equally applicable to other types of systems. Examples show that the minimum-energy control configuration also tends to be the most effective in terms of accomplishing control objectives.     

Decentralized iterative learning control for a class of large scale interconnected dynamical systems

The problem of decentralized iterative learning control for a class of large scale interconnected dynamical systems is considered. In this paper, it is assumed that the considered large scale dynamical systems are linear time-varying, and the interconnections between each subsystem are unknown. For such a class of uncertain large scale interconnected dynamical systems, a method is presented whereby a class of decentralized local iterative learning control schemes is constructed. It is also shown that under some given conditions, the constructed decentralized local iterative learning controllers can guarantee the asymptotic convergence of the local output error between the given desired local output and the actual local output of each subsystem through the iterative learning process. Finally, as a numerical example, the system coupled by two inverted pendulums is given to illustrate the application of the proposed decentralized iterative learning control schemes.