Switching control of piecewise-deterministic processes

A finite collection of piecewise-deterministic processes are controlled in order to minimize the expected value of a performance functional with continuous operating cost and discrete switching control costs. The solution of the associated dynamic programming equation is obtained by an iterative approximation using optimal stopping time problems.This research was supported in part by NSF Grant No. DMS-8508651 and by University of Tennessee Science Alliance Research Incentive Award.     

Optimal impulsive control of piecewise deterministic Markov processes

In this paper, we study the infinite-horizon expected discounted continuous-time optimal control problem for Piecewise Deterministic Markov Processes with both impulsive and gradual (also called continuous) controls. The set of admissible control strategies is supposed to be formed by policies possibly randomized and depending on the past-history of the process. We assume that the gradual control acts on the jump intensity and on the transition measure, but not on the flow. The so-called Hamilton–Jacobi–Bellman (HJB) equation associated to this optimization problem is analyzed. We provide sufficient conditions for the existence of a solution to the HJB equation and show that the solution is in fact unique and coincides with the value function of the control problem. Moreover, the existence of an optimal control strategy is proven having the property to be stationary and non-randomized.     

A useful technique for piecewise deterministic Markov decision processes

This note presents a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary PDMDP from the original one. The auxiliary PDMDP possesses certain desired properties, which may not be possessed by the original PDMDP. We apply this technique to risk-sensitive PDMDPs with total cost criteria, and comment on its connection with the uniformization technique.     

On a stochastic demand jump inventory model

This paper considers a Quasi-Variational Inequality (QVI) arising from a stochastic demand jump inventory model in a continuous review setting with a fixed ordering cost and where demand is made up of a deterministic part (which is a function of the stock level) punctuated by random jumps. Under some restrictions on the parameters, a solution to the QVI is found which corresponds to an (s,S) policy.     

On the compactness method in general ergodic impulsive control of markov processes

In this paper the impulsive control of Feller Markov processes on compact state space with long run average cost criterion is studied. Under the assumption of compactness of the resolvent operator the optimal strategies corresponding to general cost for impulses are constructed. Also the case of purely jump Markov processes is considered     

Event-triggered and self-triggered impulsive control for two-time-scale systems

This paper studies the stability problem of two-time-scale system via event-triggered impulsive control and self-triggered impulsive control. The overall system is modeled with the hybrid formalism. Two Chang transformations are introduced to completely decouple the hybrid system states into flow set and jump set. A composite impulsive controller based on slow and fast system states is proposed, under which the slow and fast subsystems are simultaneously triggered by event-triggered and self-triggered mechanism, respectively. As a result, the stability conditions are derived for the system under event-triggered and self-triggered impulsive control, respectively. Furthermore, the theoretical result of self-triggered impulsive control is applied to the consensus of the interconnected two-time-scale systems. Finally, simulation examples and comparison study show the effectiveness of the proposed control strategies.     

Event-triggered delayed impulsive control for input-to-state stability of nonlinear impulsive systems

This paper studies the input-to-state stability (ISS) and integral input-to-state stability (iISS) of nonlinear impulsive systems in the framework of event-triggered impulsive control (ETIC), where the stabilizing effect of time delays in impulses is fully considered. Some sufficient conditions which can avoid Zeno behavior and guarantee the ISS/iISS property of impulsive systems are proposed, where external inputs are considered in both the continuous dynamics and impulsive dynamics. A novel event-triggered delayed impulsive control (ETDIC) strategy which establishes a relationship among event-triggered parameters, impulse strength and time delays in impulses is presented. It is shown that time delays in impulses can contribute to the stabilization of impulsive systems in ISS/iISS sense. Finally, the effectiveness of the proposed theoretical results is illustrated by two numerical examples.     

On optimal control problems of a class of impulsive switching systems with terminal states constraints

The global optimal control problem is proposed for a special class of hybrid dynamical systems, i.e. impulsive switching systems. Then the necessary condition of the above problem, the minimum principle, is given. Ekeland’s variational principle and the matrix cost functional structure expression are utilized in the process of the proof. Based on the main result, a special linear hybrid impulsive and switching system (HISS) is illustrated and the optimal control algorithm is presented. Moreover, the cases of pure impulsive systems and pure switched systems are included in this paper.     

Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes

ABSTRACT

The main goal of this paper is to study the infinite-horizon long run average continuous-time optimal control problem of piecewise deterministic Markov processes (PDMPs) with the control acting continuously on the jump intensity λ and on the transition measure Q of the process. We provide conditions for the existence of a solution to an integro-differential optimality inequality, the so called Hamilton-Jacobi-Bellman (HJB) equation, and for the existence of a deterministic stationary optimal policy. These results are obtained by using the so-called vanishing discount approach, under some continuity and compactness assumptions on the parameters of the problem, as well as some non-explosive conditions for the process.     

Necessary and sufficient optimality conditions for control of piecewise deterministic markov processes

Piecewise deterministic Markov processes (PDPs) are continuous time homogeneous Markov processes whose trajectories are solutions of ordinary differential equations with random jumps between the different integral curves. Both continuous deterministic motion and the random jumps of the processes are controlled in order to minimize the expected value of a performance criterion involving discounted running and boundary costs. Under fairly general assumptions, we will show that there exists an optimal control, that the value function is Lipschitz continuous and that a generalized Bellman-Hamilton-Jacobi (BHJ) equation involving the Clarke generalized gradient is a necessary and sufficient optimality condition for the problem.     

Stability of piecewise deterministic Markovian metapopulation processes on networks

The purpose of this paper is to study a Markovian metapopulation model on a directed graph with edge-supported transfers and deterministic intra-nodal population dynamics. We first state tractable stability conditions for two typical frameworks motivated by applications: constant jump rates with multiplicative transfer amplitudes, and coercive jump rates with unitary transfers. More general criteria for boundedness, petiteness and ergodicity are then given.     

An implicit systems characterization of a class of impulsive linear switched control processes. Part 1: Modeling

Our paper focuses on a fundamental structural property associated with a family of linear switched control systems in the presence of impulsive dynamics. We consider dynamic processes governed by piecewise linear ODEs with controlled location transitions and describe the resulting system in a constructive form of an implicit dynamic system. The proposed algebraic-based modeling framework follows the celebrated behavioral approach (see Polderman and Willems (1998)) and makes it possible to apply to the switched dynamics some conventional techniques from the well-established time-invariant implicit systems theory. The analytic results of our paper constitute a formal theoretical extension of switched control systems methodology and can be used (as an auxiliary step) in a concrete control design procedure. In this first part, we consider modeling aspects.     

On the impulsive boundary value problems for nonlinear Hamiltonian systems

In this work, we deal with two‐point boundary value problems for nonlinear impulsive Hamiltonian systems with sub‐linear or linear growth. A theorem based on the Schauder fixed point theorem is established, which gives a result that yields existence of solutions without implications that solutions must be unique. An upper bound for the solution is also established. Examples are given to illustrate the main result. Copyright © 2016 John Wiley & Sons, Ltd.     

Stochastic control for a class of random evolution models

We construct the explicit connection existing between a solvable model of the discrete velocities non-linear Boltzmann equation and the Hamilton-Bellman-Jacobi equation associated with a simple optimal control of a piecewise deterministic process. This study extends the known relation that exists between the Burgers equation and a simple controlled diffusion problem. In both cases the resulting partial differential equations can be linearized via a logarithmic transformation and hence offer the possibility to solve physically relevant non-linear field models in full generality.     

Existence of solutions for second‐order impulsive differential inclusions

In this paper, we investigate the existence and multiplicity of solutions to the following second‐order impulsive Hamiltonian systems: where is a continuousmap form the interval [0, T] to the set of N‐order symmetric matrices. Our methods are based on critical point theory for nondifferentiable functionals. Copyright © 2014 John Wiley & Sons, Ltd.     

Finite-horizon piecewise deterministic Markov decision processes with unbounded transition rates

This paper is concerned with the problem of minimizing the expected finite-horizon cost for piecewise deterministic Markov decision processes. The transition rates may be unbounded, and the cost functions are allowed to be unbounded from above and from below. The optimality is over the general history-dependent policies, where the control is continuously acting in time. The infinitesimal approach is employed to establish the associated Hamilton-Jacobi-Bellman equation, via which the existence of optimal policies is proved. An example is provided to verify all the assumptions proposed.     

A impulsive infective transmission SI model for pest control

In this paper, we propose a model with impulsive control of epidemics for pest management. By using Floquet's theorem, small‐amplitude perturbation skills and comparison theorem, we show that there exists a globally asymptotically stable susceptible pest‐eradication periodic solution when the release amount of infective pests is larger than some critical value. However, when the amount of infective pests released is less than this critical value, the system is shown to be permanent, which implies that the trivial periodic susceptible pest‐eradication solution loses its stability. Further, the existence of a positive periodic endemic solution and other rich dynamics are also studied by numerical simulation. Therefore, we can use the amount of release of infective pests to control susceptible pests at desirable low levels. Copyright © 2007 John Wiley & Sons, Ltd.     

Homoclinic solutions for second order impulsive Hamiltonian systems with small forcing terms

In this paper, we establish some new sufficient conditions on the existence of homoclinic solution for a class of second‐order impulsive Hamiltonian systems. By using the mountain pass theorem, we demonstrate that the limit of a 2kT‐periodic approximation solution is a homoclinic solution of our problem. We also present some examples to illustrate the applications of our main results. Copyright © 2016 John Wiley & Sons, Ltd.     

Optimal control of unilateral obstacle problem with a source term

We consider an optimal control problem for the obstacle problem with an elliptic variational inequality. The obstacle function which is the control function is assumed in H². We use an approximate technique to introduce a family of problems governed by variational equations. We prove optimal solutions existence and give necessary optimality conditions. The author is grateful to Prof. M. Bergounioux for her instructive suggestions.