Round robin scheduling of heterogeneous parallel servers in heavy traffic

We consider scheduling for heterogeneous server systems, where tasks arrive according to a Poisson process, with their processing requirements following a discrete distribution with finite support. For a system with a dispatcher and several heterogeneous servers, we propose an optimized multi-layered round robin routing policy followed by shortest remaining processing time scheduling at each server. Using a heavy traffic approximation, we show that the proposed policy performs as well as the optimal scheduling policy for a heterogeneous servers system with a single queue (no routing) in heavy traffic. Additional simulation results suggest that such policies will be effective in more general settings.     

Heavy traffic resource pooling in parallel‐server systems

We consider a queueing system with r non‐identical servers working in parallel, exogenous arrivals into m different job classes, and linear holding costs for each class. Each arrival requires a single service, which may be provided by any of several different servers in our general formulation; the service time distribution depends on both the job class being processed and the server selected. The system manager seeks to minimize holding costs by dynamically scheduling waiting jobs onto available servers. A linear program involving only first‐moment data (average arrival rates and mean service times) is used to define heavy traffic for a system of this form, and also to articulate a condition of overlapping server capabilities which leads to resource pooling in the heavy traffic limit. Assuming that the latter condition holds, we rescale time and state space in standard fashion, then identify a Brownian control problem that is the formal heavy traffic limit of our rescaled scheduling problem. Because of the assumed overlap in server capabilities, the limiting Brownian control problem is effectively one‐dimensional, and it admits a pathwise optimal solution. That is, in the limiting Brownian control problem the multiple servers of our original model merge to form a single pool of service capacity, and there exists a dynamic control policy which minimizes cumulative cost incurred up to any time t with probability one. Interpreted in our original problem context, the Brownian solution suggests the following: virtually all backlogged work should be held in one particular job class, and all servers can and should be productively employed except when the total backlog is small. It is conjectured that such ideal system behavior can be approached using a family of relatively simple scheduling policies related to the cμ rule. This revised version was published online in June 2006 with corrections to the Cover Date.     

Critical Thresholds for Dynamic Routing in Queueing Networks

This paper studies dynamic routing in a parallel server queueing network with a single Poisson arrival process and two servers with exponential processing times of different rates. Each customer must be routed at the time of arrival to one of the two queues in the network. We establish that this system operating under a threshold policy can be well approximated by a one-dimensional reflected Brownian motion when the arrival rate to the network is close to the processing capacity of the two servers. As the heavy traffic limit is approached, thresholds which grow at a logarithmic rate are critical in determining the behavior of the limiting system. We provide necessary and sufficient conditions on the growth rate of the threshold for (i) approximation of the network by a reflected Brownian motion (ii) positive recurrence of the limiting Brownian diffusion and (iii) asymptotic optimality of the threshold policy.     

Analysis of job assignment with batch arrivals among heterogeneous servers

We revisit the problem of job assignment to multiple heterogeneous servers in parallel. The system under consideration, however, has a few unique features. Specifically, repair jobs arrive to the queueing system in batches according to a Poisson process. In addition, servers are heterogeneous and the service time distributions of the individual servers are general. The objective is to optimally assign each job within a batch arrival to minimize the long-run average number of jobs in the entire system. We focus on the class of static assignment policies where jobs are routed to servers upon arrival according to pre-determined probabilities. We solve the model analytically and derive the structural properties of the optimal static assignment. We show that when the traffic is below a certain threshold, it is better to not assign any jobs to slower servers. As traffic increases (either due to an increase in job arrival rate or batch size), more slower servers will be utilized. We give an explicit formula for computing the threshold. Finally we compare and evaluate the performance of the static assignment policy to two dynamic policies, specifically the shortest expected completion policy and the shortest queue policy.     

Asymptotic optimality of the Round–Robin policy in multipath routing with resequencing

We consider a model of a multipath routing system where arriving customers are routed to a set of identical, parallel, single server queues according to balancing policies operating without state information. After completion of service, customers are required to leave the system in their order of arrival, thus incurring an additional resequencing delay. We are interested in minimizing the end-to-end delay (including time at the resequencing buffer) experienced by arriving customers. To that end we establish the optimality of the Round–Robin routing assignment in two asymptotic regimes, namely heavy and light traffic: In heavy traffic, the Round–Robin customer assignment is shown to achieve the smallest (in the increasing convex stochastic ordering) end-to-end delay amongst all routing policies operating without queue state information. In light traffic, and for the special case of Poisson arrivals, we show that Round–Robin is again an optimal (in the strong stochastic ordering) routing policy. We illustrate the stochastic comparison results by several simulation examples. The work of the first author was supported through an ARCHIMEDES grant by the Greek Ministry of Education. The work of the second author was prepared through collaborative participation in the Communications and Networks Consortium sponsored by the U.S. Army Research Laboratory under the Collaborative Technology Alliance Program, Cooperative Agreement DAAD19-01-2-0011. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government.     

A fork-join queueing model: Diffusion approximation,integral representations and asymptotics

We consider two parallel M/M/1 queues which are fed by a single Poisson arrival stream. An arrival splits into two parts, with each part joining a different queue. This is the simplest example of a fork-join model. After the individual parts receive service, they may be joined back together, though we do not consider the join part here. We study this model in the heavy traffic limit, where the service rate in either queue is only slightly larger than the arrival rate. In this limit we obtain asymptotically the joint steady-state queue length distribution. In the symmetric case, where the two servers are identical, this distribution has a very simple form. In the non-symmetric case we derive several integral representations for the distribution. We then evaluate these integrals asymptotically, which leads to simple formulas which show the basic qualitative structure of the joint distribution function.     

Heavy traffic limit for a tandem queue with identical service times

We consider a two-node tandem queueing network in which the upstream queue is M/G/1 and each job reuses its upstream service requirement when moving to the downstream queue. Both servers employ the first-in-first-out policy. We investigate the amount of work in the second queue at certain embedded arrival time points, namely when the upstream queue has just emptied. We focus on the case of infinite-variance service times and obtain a heavy traffic process limit for the embedded Markov chain.     

On the average sojourn time under M/M/1/SRPT

We study an M/M/1 queueing system under the shortest remaining processing time (SRPT) policy. We show that the average sojourn time varies as , where ρ is the system load. Thus, SRPT offers a Θ(ln(e/(1−ρ))) factor improvement over policies that ignore knowledge of job sizes while scheduling.     

Optimal order for two servers in tandem

We consider two servers (serveri, i=1, 2) in tandem for which the order of servers can be changed. Server 1 has a general service time distribution and server 2 has either its shifted or truncated distribution. This permits that the service times at the two servers are overlapping. An unlimited queue is allowed in front of the first server. For the systems having zero buffer capacity between the servers, we show that the sojourn time of every customer is stochastically minimized under any arrival process if server 2 is first. For the systems with infinite buffer capacity and a Poisson arrivals, we show that this order of servers minimizes mean customer delay when traffic is light. Several numerical examples are presented to demonstrate that this optimal order is invariant under any arrival process (the interarrival times are i.i.d. r.v.'s) and mild traffic condition.Research funded by NEC Corporation C & C Laboratory.     

Erlang arrivals joining the shorter queue

We consider a system in which customers join upon arrival the shortest of two single-server queues. The interarrival times between customers are Erlang distributed and the service times of both servers are exponentially distributed. Under these assumptions, this system gives rise to a Markov chain on a multi-layered quarter plane. For this Markov chain we derive the equilibrium distribution using the compensation approach. The expression for the equilibrium distribution matches and refines tail asymptotics obtained earlier in the literature.     

The FCFS service discipline: Stable network topologies, bounds on traffic burstiness and delay, and control by regulators

We consider the well-known First Come First Serve (FCFS) scheduling policy under bursty arrivals. This policy is commonly used in scheduling communication networks and manufacturing systems. Recently it has been shown that the FCFS policy can be unstable for some nonacyclic network topologies.We identify some network topologies under which FCFS is stable for all arrival and service rate vectors within the system's capacity. This is done by determining a sharp bound on the burstiness of traffic exiting from a tandem section of the system, in terms of the burstiness of the incoming traffic. This burstiness bound further allows us to provide a bound on the maximum number of parts in the system, and the maximum delay. It also enables us to analyze the performance of some systems controlled by the use of traffic smoothing regulators. The maximum delay can remain bounded even in the heavy traffic limit, when all stations are 100% utilized.     

Optimal preemptive scheduling on uniform machines with discounted flowtime objectives

We consider the problem of scheduling n jobs on m parallel machines. Each job has a deterministic processing time and a weight associated with it. For uniform machines we show that discounted flowtime is minimized by serving jobs preemptively in increasing order of their remaining processing times, assigning the job with the shortest remaining processing time to the fastest available machine.     

A priority rule for minimizing weighted flow time in a class of parallel machine scheduling problems

The problem of scheduling n jobs with known process times on m identical parallel machines with an objective of minimizing weighted flow time is NP-hard. However, when job weights are identical, it is well known that the problem is easily solved using the shortest processing time rule. In this paper, we show that a generalization of the shortest processing time rule minimizes weighted flow time in a class of problems where job weights are not identical.     

Optimal resource allocation for multiqueue systems with a shared server pool

We study optimal allocation of servers for a system with multiple service facilities and with a shared pool of servers. Each service facility poses a constraint on the maximum expected sojourn time of a job. A central decision maker can dynamically allocate servers to each facility, where adding more servers results in faster processing speeds but against higher utilization costs. The objective is to dynamically allocate the servers over the different facilities such that the sojourn-time constraints are met at minimal costs. This situation occurs frequently in practice, for example, in Grid systems for real-time image processing (iris scans, fingerprints). We model this problem as a Markov decision process and derive structural properties of the relative value function. These properties, which are hard to derive for multidimensional systems, give a full characterization of the optimal policy. We demonstrate the effectiveness of these policies by extensive numerical experiments.     

Stability and instability of routings through a queueing network

We consider a network of N nodes with given distances in which customers arrive at one of the nodes and request transportation to one of the other nodes. There is a finite number of servers (or transporters) with possibly different capacities and speed available. We show that there exists a critical arrival rate κ_c, such that if the arrival rate is larger than κ_c then the number of customers in the system increases to ∞ with linear speed no matter which routing strategy for the transporters is chosen (even if we allow the strategy to depend on the future arrival process). If, on the other hand, κ < _c then there exists even a fixed periodic routing strategy which keeps the system stable (in a sense to be made precise). We prefer to start with a deterministic set-up which allows very general arrival processes and look at the stochastic case only in the final section where we get more refined results by assuming that the arrival process has integrable stationary ergodic increments (which still allows batch arrivals and long-range dependence). Potential applications include the control of elevators in highrise buildings. This revised version was published online in June 2006 with corrections to the Cover Date.     

Round robin tournaments and three index assignments

Scheduling a sports league can be seen as a difficult combinatorial optimization problem. We study some variants of round robin tournaments and analyze the relationship with the planar three-index assignment problem. The complexity of scheduling a minimum cost round robin tournament is established by a reduction from the planar three-index assignment problem. Furthermore, we introduce integer programming models. We pick up a popular idea and decompose the overall problem in order to obtain two subproblems which can be solved sequentially. We show that the latter subproblem can be casted as a planar three-index assignment problem. This makes existing solution techniques for the planar three-index assignment problem amenable to sports league scheduling.     

Monotonicity properties of user equilibrium policies for parallel batch systems

We study a simple network with two parallel batch-service queues, where service at a queue commences when the batch is full and each queue is served by infinitely many servers. A stream of general arrivals observe the current state of the system on arrival and choose which queue to join to minimize their own expected transit time. We show that for each set of parameter values there exists a unique user equilibrium policy and that it possesses various monotonicity properties. User equilibrium policies for probabilistic routing are also discussed and compared with the state-dependent setting.     

Open queueing networks in discrete time-some limit theorems

A finite number of nodes, each with a single server and infinite buffers, is considered in discrete time. The service may be FIFO and the service times are constant. The external arrivals and the routing decision variables form a general stationary sequence. Stability of the system is proved under these assumptions. Extension to multiple servers at a node and general stationary distributions holds. If the external input is i.i.d. and the routing is Markovian then stochastic ordering, continuity of stationary distributions, rates of convergence, a functional CLT and a functional LIL and various other limit theorems for the queue length process are also proved. Generalizations to multiple servers at nodes, customers with priority, multiple customer classes, general service length and Markov modulated external arrival cases are discussed.     

On the Shortest Queue Version of the Erlang Loss Model

We consider two parallel M / M / N / N queues. Thus there are N servers in each queue and no waiting line(s). The network is fed by a single Poisson arrival stream of rate λ, and the 2 N servers are identical exponential servers working at rate μ. A new arrival is routed to the queue with the smaller number of occupied servers. If both have the same occupancy then the arrival is routed randomly, with the probability of joining either queue being 1/2. This model may be viewed as the shortest queue version of the classic Erlang loss model. If all 2 N servers are occupied further arrivals are turned away and lost. We let  ρ=λ/μ  and   a = N /ρ= N μ/λ  . We study this model both numerically and asymptotically. For the latter we consider heavily loaded systems (ρ→∞) with a comparably large number of servers (   N →∞  with   a = O (1))  . We obtain asymptotic approximations to the joint steady state distribution of finding m servers occupied in the first queue and n in the second. We also consider the marginal distribution of the number of occupied servers in the second queue, as well as some conditional distributions. We show that aspects of the solution are much different according as   a > 1/2, a ≈ 1/2, 1/4 < a < 1/2, a ≈ 1/4  or  0 < a < 1/4  . The asymptotic approximations are shown to be quite accurate numerically.     

A stochastic and dynamic routing policy using branching processes with state dependent immigration

A stochastic and dynamic vehicle routing problem called the Dynamic Traveling Repairman Problem (DTRP) was introduced by Bertsimas and van Ryzin. Several routing policies were analyzed in light traffic and in heavy traffic conditions. But, the good light traffic policies become very quickly unstable with increasing traffic intensity, and the good heavy traffic policies are inefficient in light traffic conditions. In this paper, a new routing policy is defined and analyzed, using results from branching processes with state dependent immigration. This policy not only performs optimally in light traffic, but also performs very well in heavy traffic. This is important to the designer of a service system because the traffic conditions may be variable and/or be unpredictable, and having to switch routing policies could prove to be costly and difficult to implement.