HIV dynamics: Analysis and robust multirate MPC-based treatment schedules

Analysis and control of human immunodeficiency virus (HIV) infection have attracted the interests of mathematicians and control engineers during the recent years. Several mathematical models exist and adequately explain the interaction of the HIV infection and the immune system up to the stage of clinical latency, as well as viral suppression and immune system recovery after treatment therapy. However, none of these models can completely exhibit all that is observed clinically and account the full course of infection. Besides model inaccuracies that HIV models suffer from, some disturbances/uncertainties from different sources may arise in the modelling. In this paper we study the basic properties of a 6-dimensional HIV model that describes the interaction of HIV with two target cells, CD4⁺ T cells and macrophages. The disturbances are modelled in the HIV model as additive bounded disturbances. Highly Active AntiRetroviral Therapy (HAART) is used. The control input is defined to be dependent on the drug dose and drug efficiency. We developed treatment schedules for HIV infected patients by using robust multirate Model Predictive Control (MPC)-based method. The MPC is constructed on the basis of the approximate discrete-time model of the nominal model. We established a set of conditions, which guarantee that the multirate MPC practically stabilizes the exact discrete-time model with disturbances. The proposed method is applied to the stabilization of the uninfected steady state of the HIV model. The results of simulations show that, after initiation of HAART with a strong dosage, the viral load drops quickly and it can be kept under a suitable level with mild dosage of HAART. Moreover, the immune system is recovered with some fluctuations due to the presence of disturbances.     

Approximate receding horizon approach for Markov decision processes: average reward case

We consider an approximation scheme for solving Markov decision processes (MDPs) with countable state space, finite action space, and bounded rewards that uses an approximate solution of a fixed finite-horizon sub-MDP of a given infinite-horizon MDP to create a stationary policy, which we call “approximate receding horizon control.” We first analyze the performance of the approximate receding horizon control for infinite-horizon average reward under an ergodicity assumption, which also generalizes the result obtained by White (J. Oper. Res. Soc. 33 (1982) 253-259). We then study two examples of the approximate receding horizon control via lower bounds to the exact solution to the sub-MDP. The first control policy is based on a finite-horizon approximation of Howard's policy improvement of a single policy and the second policy is based on a generalization of the single policy improvement for multiple policies. Along the study, we also provide a simple alternative proof on the policy improvement for countable state space. We finally discuss practical implementations of these schemes via simulation.     

Boundaries of the receding horizon control for interconnected systems

In recent years, the finite-horizon quadratic minimization problem has become popular in process control, where the horizon is constantly rolled back. In this paper, this type of control, which is also called the receding horizon control, is considered for interconnected systems. First, the receding horizon control equations are formulated; then, some stability conditions depending on the interconnection norms and the horizon lengths are presented. For -coupled systems, stability results similar to centralized systems are obtained. For interconnected systems which are not -coupled, the existence of a horizon length and a corresponding stabilizing receding horizon control are derived. Finally, the performance of a locally computed receding horizon control for time-invariant and time-varying systems with different updating intervals is examined in an example.     

Markov跳变系统滚动时域有限记忆控制

针对离散时间Markov跳变系统,提出滚动时域有限记忆控制的方法.在一段有限滤波时域上,利用系统输入与输出变量的线性组合构造一段有限控制时域上的输出反馈控制器.首先,不考虑跳变系统均方可镇定,基于最优控制的方法,获得以迭代计算形式给出的控制器,并使其在无偏条件下能优化二次型性能指标.其次,进一步考虑在成本衰减条件下确定终端加权矩阵,并以它作为边界条件计算得到最优控制律,调节系统均方稳定.为便于求解,成本衰减条件以线性矩阵不等式的形式给出.仿真实例验证了所提方法的可行性和有效性.     

Infinite-horizon minimax control with pointwise cost functional

This paper is devoted to the discussion of a minimax optimal control problem over an infinite-time horizon, where the functional to be minimized is the highest instantaneous cost that may occur under the worst combination of disturbances. The problem is formulated for a general stationary discrete-time dynamic system, and a dynamic programming algorithm is proposed for its solution. The relationship existing between the cost functional associated to a control law and the reachability properties of the resulting controlled system is discussed.     

A numerical method to approximate optimal production and maintenance plan in a flexible manufacturing system

The simultaneous planning of the production and the maintenance in a flexible manufacturing system is considered in this paper. The manufacturing system is composed of one machine that produces a single product. There is a preventive maintenance plan to reduce the failure rate of the machine. This paper is different from the previous researches in this area in two separate ways. First, the failure rate of the machine is supposed to be a function of its age. Second, we assume that the demand of the manufacturing product is time dependent and its rate depends on the level of advertisement on that product. The objective is to maximize the expected discounted total profit of the firm over an infinite time horizon. In the process of finding a solution to the problem, we first characterize an optimal control by introducing a set of Hamilton–Jacobi–Bellman partial differential equations. Then we realize that under practical assumptions, this set of equations can not be solved analytically. Thus to find a suboptimal control, we approximate the original stochastic optimal control model by a discrete-time deterministic optimal control problem. Then proposing a numerical method to solve the steady state Riccati equation, we approximate a suboptimal solution to the problem.     

Digital modeling and control of multiple time-delayed distributed power grid

Distributed power grid (DPG) control systems are so highly interconnected that the effects of local disturbances as well as transmission time delays can be amplified as they propagate through a complex network of transmission lines. These effects deteriorate control performance and could possibly destabilize the overall system. In this paper, a new approximated discretization method and digital design for DPG control systems with multiple state, input and output delays as well as a generalized bilinear transformation method are presented. Based on a procedure for the generation of impulse response data, the multiple fractional/integer time-delayed continuous-time system is transformed to a discrete-time model with multiple integer time delays. To implement the digital modeling, the singular value decomposition (SVD) of a Hankel matrix together with an energy loss level is employed to obtain an extended discrete-time state space model. Then, the extended discrete-time state space model of the DPG control system is reformulated as an integer time-delayed discrete-time system by computing its observable canonical form. The proposed method can closely approximate the step response of the original continuous time-delayed DPG control system by choosing various energy loss levels. For completeness, an optimal digital controller design for the DPG control system and a generalized bilinear transformation method with a tunable parameter are also provided, which can re-transform the integer time-delayed discrete-time model to its continuous-time model. Illustrative examples are given to demonstrate the effectiveness of the developed method.     

A receding horizon control approach for integrated vector management of Aedes aegypti using chemical and biological control: A mono and a multiobjective approach

This paper addresses an integrated vector management (IVM) approach for combating Aedes aegypti, the transmission vector of dengue, zika, and chikungunya diseases, some of the most important viral epidemics worldwide. In order to tackle this problem, a receding horizon control (RHC) strategy is adopted, considering a mono-objective and a multiobjective version of the optimal control model of combating the mosquito using chemical and biological control. RHC is essentially a suboptimal scheme of classical optimal control strategies considering discrete-time approximations. The integrated vector control actions used in this work consist in applying insecticides and inserting sterile males produced by irradiation in the population of mosquitoes. The cost function is defined in terms of social and economic costs, in order to quantify the effectiveness of the proposed epidemiological control throughout a time window of 4 months. Numerical simulations show that the obtained results are better than those from the optimal control strategies found in literature. Furthermore, through the application of the multiobjetive approach, varying the scenarios in the mono-objective formulation is no longer necessary and a set of optimal strategies can be obtained at once. Finally, in order to help health authorities in the choice of the best solution of the Pareto-optimal set to be implemented in practice, a cost-effectiveness analysis is performed and a strategy representing the most cost-effective control policy is obtained.     

Efficient robust optimization for robust control with constraints

This paper proposes an efficient computational technique for the optimal control of linear discrete-time systems subject to bounded disturbances with mixed linear constraints on the states and inputs. The problem of computing an optimal state feedback control policy, given the current state, is non-convex. A recent breakthrough has been the application of robust optimization techniques to reparameterize this problem as a convex program. While the reparameterized problem is theoretically tractable, the number of variables is quadratic in the number of stages or horizon length N and has no apparent exploitable structure, leading to computational time of per iteration of an interior-point method. We focus on the case when the disturbance set is ∞-norm bounded or the linear map of a hypercube, and the cost function involves the minimization of a quadratic cost. Here we make use of state variables to regain a sparse problem structure that is related to the structure of the original problem, that is, the policy optimization problem may be decomposed into a set of coupled finite horizon control problems. This decomposition can then be formulated as a highly structured quadratic program, solvable by primal-dual interior-point methods in which each iteration requires time. This cubic iteration time can be guaranteed using a Riccati-based block factorization technique, which is standard in discrete-time optimal control. Numerical results are presented, using a standard sparse primal-dual interior point solver, that illustrate the efficiency of this approach.     

Predictive control of nonlinear dynamic processes

Predictive control can be applied if the reference value of the process is known in advance and the deterministic disturbances can be predicted. A cost function defined in the future horizon is minimized. The control signal is calculated for a control horizon, but only the first one is applied and the procedure is repeated (receding horizon strategy). Processes with mild analytical nonlinear characteristics are considered. The possible process models are either nonparametric (linear, Hammerstein, and Volterra weighting function series) or parametric ones (generalized Hammerstein, parametric Volterra, and bilinear models). The algorithms of the optimal and suboptimal predictive control based on the nonparametric and the parametric models mentioned are derived. Several simulations present how effective these methods are. The adaptive case is dealt with as well.     

Impulsively-controlled systems and reverse dwell time: A linear programming approach

We present a receding horizon algorithm that converges to the exact solution in polynomial time for a class of optimal impulse control problems with uniformly distributed impulse instants and governed by so-called reverse dwell time conditions. The cost has two separate terms, one depending on time and the second monotonically decreasing on the state norm. The obtained results have both theoretical and practical relevance. From a theoretical perspective we prove certain geometrical properties of the discrete set of feasible solutions. From a practical standpoint, such properties reduce the computational burden and speed up the search for the optimum thus making the algorithm suitable for the on-line implementation in real-time problems. Our approach consists in approximating the optimal impulse control problem via a binary linear programming problem with a totally unimodular constraint matrix. Hence, solving the binary linear programming problem is equivalent to solving its linear relaxation. Then, given the feasible solution from the linear relaxation, we find the optimal solution via receding horizon and local search. Numerical illustrations of a queueing system are performed.     

Low-Infrastructure Real-Time Embedded Control via Variational Integrators

This paper presents a discrete time receding horizon control scheme that leverages the numerical properties of a variational integrator to facilitate real-time control generation on an embedded system. The variational integrator employed is well-suited to classical estimation and control algorithms, e.g. LQR, extended Kalman filters, and particle filters. The structure-preserving properties of this variational integrator lead to increased performance of estimation and control routines, especially in low-bandwidth applications. Several experimental examples are presented that illustrate the features of this receding horizon control scheme when leveraging the desirable numerical properties of the variational integrator. © 2016 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Suboptimal Explicit Receding Horizon Control via Approximate Multiparametric Quadratic Programming

Algorithms for solving multiparametric quadratic programming (MPQP) were recently proposed in Refs. 1–2 for computing explicit receding horizon control (RHC) laws for linear systems subject to linear constraints on input and state variables. The reason for this interest is that the solution to MPQP is a piecewise affine function of the state vector and thus it is easily implementable online. The main drawback of solving MPQP exactly is that, whenever the number of linear constraints involved in the optimization problem increases, the number of polyhedral cells in the piecewise affine partition of the parameter space may increase exponentially. In this paper, we address the problem of finding approximate solutions to MPQP, where the degree of approximation is arbitrary and allows to tradeoff between optimality and a smaller number of cells in the piecewise affine solution. We provide analytic formulas for bounding the errors on the optimal value and the optimizer, and for guaranteeing that the resulting suboptimal RHC law provides closed-loop stability and constraint fulfillment.     

On quadratic stage costs for mobile robots in model predictive control

We consider nonholonomic mobile robots. Since the system is finite time controllable, it is stabilizable by a receding horizon control scheme with purely quadratic stage costs if an infinite optimization horizon is employed. However, due to the so called short-sightedness of model predictive control, these stability properties are not preserved if the control problem is only optimized on a truncated and, thus, finite prediction horizon — even if an arbitrarily large terminal weight is added. Hence, it is necessary to either incorporate structurally different terminal costs or use non-quadratic stage costs to appropriately penalize the deviation from the desired set point. (© 2017 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Controllability and stabilizability of linear time‐varying distributed hereditary control systems

This paper is concerned with the controllability and stabilizability problem for control systems described by a time‐varying linear abstract differential equation with distributed delay in the state variables. An approximate controllability property is established, and for periodic systems, the stabilization problem is studied. Assuming that the semigroup of operators associated with the uncontrolled and non delayed equation is compact, and using the characterization of the asymptotic stability in terms of the spectrum of the monodromy operator of the uncontrolled system, it is shown that the approximate controllability property is a sufficient condition for the existence of a periodic feedback control law that stabilizes the system. The result is extended to include some systems which are asymptotically periodic. Copyright © 2014 John Wiley & Sons, Ltd.     

Application of Interior-Point Methods to Model Predictive Control

We present a structured interior-point method for the efficient solution of the optimal control problem in model predictive control. The cost of this approach is linear in the horizon length, compared with cubic growth for a naive approach. We use a discrete-time Riccati recursion to solve the linear equations efficiently at each iteration of the interior-point method, and show that this recursion is numerically stable. We demonstrate the effectiveness of the approach by applying it to three process control problems.     

Receding Horizon Robust Control for Nonlinear Systems Based on Linear Differential Inclusion of Neural Networks

In this paper, we present a new receding horizon neural robust control scheme for a class of nonlinear systems based on the linear differential inclusion (LDI) representation of neural networks. First, we propose a linear matrix inequality (LMI) condition on the terminal weighting matrix for a receding horizon neural robust control scheme. This condition guarantees the nonincreasing monotonicity of the saddle point value of the finite horizon dynamic game. We then propose a receding horizon neural robust control scheme for nonlinear systems, which ensures the infinite horizon robust performance and the internal stability of closed-loop systems. Since the proposed control scheme can effectively deal with input and state constraints in an optimization problem, it does not cause the instability problem or give the poor performance associated with the existing neural robust control schemes.     

Digital modeling and hybrid control of sampled-data uncertain system with input time delay using the law of mean

This paper utilizes an interval Pade approximate method together with interval arithmetic operation to convert a continuous-time uncertain system with input time-delay to an equivalent discrete-time interval model and transforms the robust control law of a continuous-time uncertain system with input time delay into an equivalent one of a sampled-data uncertain system with input time delay. The developed discrete-time interval model tightly encloses the exact discrete-time uncertain system with input time delay. Based on the law of mean and inclusion theory, a perturbed digital control law of input time-delay sampled-data uncertain system is newly presented, so that the states of the digitally controlled sample-data uncertain system closely match those of the originally well-designed continuous-time uncertain system.     

Receding-Horizon Control Method for the Stabilization of an Unpredictably Changing Discrete-Time Uncertain System

The paper is concerned with the applicability of the receding-horizon control method for stabilizing a discrete-time uncertain system with a time-varying linear nominal part. The specific feature of the problem is that the nominal system is supposed to be known only up to the actual time. The effectiveness of the method is illustrated by an example.     

A receding horizon D-optimization approach for model identification–oriented input design and application in combustion engines

In this paper, a receding horizon D-optimization approach for model identification–oriented input design is proposed, and a practical application is demonstrated for internal combustion engines. The proposed approach consists of a recursive parameter identification algorithm and an input signal design algorithm; where the latter provides D-optimal excitation signal for the adaptation of the parameter estimation in the following identification phase. The D-optimization algorithm is constructed with the Continuation/GMRES method, which provides an approximate solution according to the current parameter of the model. To validate effectiveness and feasibility of the proposed approach, testing results applying the proposed approach to an internal combustion engine are demonstrated and conducted on a full-scale engine test bench.