Large deviations of Markovian polling models with applications to admission control

In this paper we consider large deviations and admission control problems for a discrete-time Markovian polling system. The system consists of two-parallel queues and multiple heterogeneous servers. The arrival process of each queue is a superposition of mutually independent Markovian on/off processes, and the multiple servers serve independently the two queues according to the so called Bernoulli service schedule. Using the large deviations techniques, we derive upper and lower bounds of the overflow probabilities, and then we present an admission control criterion by which different Quality of Service (QoS) requirements for the two queues are guaranteed.     

Optimal control of price and production in an assemble-to-order system

We study the optimal control of an assembly system that produces one assembled-to-order final product with multiple made-to-stock components and sells it at variable price. It is shown that a threshold control on component production, product price, and product orders maximizes total discounted profit over an infinite horizon.     

On the optimal control of a two-queue polling model

This paper deals with two M/M/1 queues served by a single server with threshold switching. Our main goal is to solve the Poisson equation and, as a result, give expressions for the long-run expected average cost of holding units and switching actions of the server, and the bias vector.     

Optimal policy for minimizing risk models in Markov decision processes

We consider the minimizing risk problems in discounted Markov decisions processes with countable state space and bounded general rewards. We characterize optimal values for finite and infinite horizon cases and give two sufficient conditions for the existence of an optimal policy in an infinite horizon case. These conditions are closely connected with Lemma 3 in White (1993), which is not correct as Wu and Lin (1999) point out. We obtain a condition for the lemma to be true, under which we show that there is an optimal policy. Under another condition we show that an optimal value is a unique solution to some optimality equation and there is an optimal policy on a transient set.     

A survey of Markov decision models for control of networks of queues

We review models for the optimal control of networks of queues. Our main emphasis is on models based on Markov decision theory and the characterization of the structure of optimal control policies.This research was partially supported by the National Science Foundation under Grant No. DDM-8719825. The Government has certain rights in this material. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. The research was also partially supported by the C.I.E.S. (France), while the author was on leave at INRIA, Sophia-Antipolis, 1991–92.     

Optimal control policies for ambulance diversion

Ambulance diversion (AD) is used by emergency departments (EDs) to relieve congestion by requesting ambulances to bypass the ED and transport patients to another facility. We study optimal AD control policies using a Markov Decision Process (MDP) formulation that minimizes the average time that patients wait beyond their recommended safety time threshold. The model assumes that patients can be treated in one of two treatment areas and that the distribution of the time to start treatment at the neighboring facility is known. Assuming Poisson arrivals and exponential times for the length of stay in the ED, we show that the optimal AD policy follows a threshold structure, and explore the behavior of optimal policies under different scenarios. We analyze the value of information on the time to start treatment in the neighboring hospital, and show that optimal policies depend strongly on the congestion experienced by the other facility. Simulation is used to compare the performance of the proposed MDP model to that of simple heuristics under more realistic assumptions. Results indicate that the MDP model performs significantly better than the tested heuristics under most cases. Finally, we discuss practical issues related to the implementation of the policies prescribed by the MDP.     

Aggregation and disaggregation in Markov decision models for inventory control

In this paper the possibility is investigated of using aggregation in the action space for some Markov decision processes of inventory control type. For the standard (s, S) inventory control model the policy improvement procedure can be executed in a very efficient way, therefore, aggregation in the action space is not of much use. However, in situations where the decisions have some aftereffect and, hence, the old decision has to be incorporated in the state, it might be rewarding to aggregate actions. Some variants for aggregation and disaggregation are formulated and analyzed. Numerical evidence is presented.     

Monotonicity and stability of periodic polling models

This paper deals with the stability of periodic polling models with a mixture of service policies. Customers arrive according to independent Poisson processes. The service times and the switchover times are independent with general distributions. The necessary and sufficient condition for the stability of such polling systems is established. The proof is based on the stochastic monotonicity of the state process at the polling instants. The stability of only a subset of the queues is also analyzed and, in case of heavy traffic, the order of explosion of the queues is given. The results are valid for a model with set-up times, and also when there is a local priority rule at the queues.This work was supported in part by a Fellowship of the Netherlands Organization for Scientific Research NWO-ECOZOEK.     

Stability of periodic polling system with BMAP arrivals

This paper considers the stability of BMAP/GI/1 periodic polling models with mixed service disciplines. The server attends the N stations in a repeating sequence of stages. Customers arrive to the stations according to batch Markov arrival processes (BMAPs). The service times of the stations are general independent and identically distributed. The characterization of global stability of the system, the order of instability of stations and the necessary and sufficient condition for the stability are given. Our stability analysis is based on the investigation of the embedded Markovian chains at the polling epochs, which allows a much simpler discussion than the formerly applied approaches. This work can also be seen as a survey on stability of a quite general set of polling models, since the majority of the known results of the field is a special case of the presented ones.     

Stability and continuity of polling systems

The stability of a polling system with exhaustive service and a finite number of users, each with infinite buffers is considered. The arrival process is more general than a Poisson process and the system is not slotted. Stochastic continuity of the stationary distributions, rates of convergence and functional limit theorems for the queue length and waiting time processes have also been proved. The results extend to the gated service discipline.