Optimal buffer size and dynamic rate control for a queueing system with impatient customers in heavy traffic

We address a rate control problem associated with a single server Markovian queueing system with customer abandonment in heavy traffic. The controller can choose a buffer size for the queueing system and also can dynamically control the service rate (equivalently the arrival rate) depending on the current state of the system. An infinite horizon cost minimization problem is considered here. The cost function includes a penalty for each rejected customer, a control cost related to the adjustment of the service rate and a penalty for each abandoning customer. We obtain an explicit optimal strategy for the limiting diffusion control problem (the Brownian control problem or BCP) which consists of a threshold-type optimal rejection process and a feedback-type optimal drift control. This solution is then used to construct an asymptotically optimal control policy, i.e. an optimal buffer size and an optimal service rate for the queueing system in heavy traffic. The properties of generalized regulator maps and weak convergence techniques are employed to prove the asymptotic optimality of this policy. In addition, we identify the parameter regimes where the infinite buffer size is optimal.     

Convergence of a queueing system in heavy traffic with general patience-time distributions

We analyze a sequence of single-server queueing systems with impatient customers in heavy traffic. Our state process is the offered waiting time, and the customer arrival process has a state dependent intensity. Service times and customer patient-times are independent; i.i.d. with general distributions subject to mild constraints. We establish the heavy traffic approximation for the scaled offered waiting time process and obtain a diffusion process as the heavy traffic limit. The drift coefficient of this limiting diffusion is influenced by the sequence of patience-time distributions in a non-linear fashion. We also establish an asymptotic relationship between the scaled version of offered waiting time and queue-length. As a consequence, we obtain the heavy traffic limit of the scaled queue-length. We introduce an infinite-horizon discounted cost functional whose running cost depends on the offered waiting time and server idle time processes. Under mild assumptions, we show that the expected value of this cost functional for the n-th system converges to that of the limiting diffusion process as n tends to infinity.     

Multiscale diffusion approximations for stochastic networks in heavy traffic

Stochastic networks with time varying arrival and service rates and routing structure are studied. Time variations are governed by, in addition to the state of the system, two independent finite state Markov processes X and Y. The transition times of X are significantly smaller than typical inter-arrival and processing times whereas the reverse is true for the Markov process Y. By introducing a suitable scaling parameter one can model such a system using a hierarchy of time scales. Diffusion approximations for such multiscale systems are established under a suitable heavy traffic condition. In particular, it is shown that, under certain conditions, properly normalized buffer content processes converge weakly to a reflected diffusion. The drift and diffusion coefficients of this limit model are functions of the state process, the invariant distribution of X, and a finite state Markov process which is independent of the driving Brownian motion.     

Optimal replacement for discrete additive damage models: Algorithm

A system receives shocks at successive random points of discrete time, and each shock causes a positive integer-valued random amount of damage which accumulates on the system one after another. The system is subject to failure and it fails once the total cumulative damage level first exceeds a fixed threshold. Upon failure the system must be replaced by a new and identical one and a cost is incurred. If the system is replaced before failure, a smaller cost is incurred. In previous work, under some assumptions, we specified a replacement rule which minimizes the long-run (expected) average cost per unit time and possesses control limit property. In this paper, a general algorithm for such models is developed. This research has been jointly supported by ITDC, contract No.105-82150 and the National Natural Science Foundation of China.     

Heavy traffic analysis of controlled multiplexing systems

The paper develops the mathematics of the heavy traffic approach to the control and optimal control problem for multiplexing systems, where there are many mutually independent sources which feed into a single channel via a multiplexer (or of networks composed of such subsystems). Due to the widely varying bit rates over all sources, control over admission, bandwidth, etc., is needed to assure good performance. Optimal control and heavy traffic analysis has been shown to yield systems with greatly improved performance. Indeed, the heavy traffic approach covers many cases of great current interest, and provides a useful and practical approach to problems of analysis and control arising in modern high speed telecommunications. Past works on the heavy traffic approach to the multiplexing problem concentrated on the uncontrolled system or on the use of the heavy traffic limit control problem for applications, and did not provide details of the proofs. This is done in the current paper. The basic control problem for the physical system is hard, and the heavy traffic approach provides much simplification. Owing to the presence of the control, as well as to the fact that the cost function of main interest is “ergodic”, the problem cannot be fully treated with “classical” methods of heavy traffic analysis for queueing networks. A basic result is that the optimal average costs per unit time for the physical problem converge to the optimal cost per unit time for the limit stationary process as the number of sources and the time interval goes to infinity. This convergence is both in the mean and pathwise senses. Furthermore, a “nice” nearly optimal control for the limit system provides nearly optimal values for the physical system, under heavy traffic, in both a mean and pathwise sense. This revised version was published online in June 2006 with corrections to the Cover Date.     

Optimal admission control in a queueing system with heterogeneous traffic

We consider optimal admission control of the GI/PH/1-type queueing system. The problem is then reduced to that of determining multi-threshold strategies. Some numerical examples are presented. The results have applications in the optimal input control of information flow in a computer communication network with heterogeneous traffic.     

Heavy traffic resource pooling in parallel‐server systems

We consider a queueing system with r non‐identical servers working in parallel, exogenous arrivals into m different job classes, and linear holding costs for each class. Each arrival requires a single service, which may be provided by any of several different servers in our general formulation; the service time distribution depends on both the job class being processed and the server selected. The system manager seeks to minimize holding costs by dynamically scheduling waiting jobs onto available servers. A linear program involving only first‐moment data (average arrival rates and mean service times) is used to define heavy traffic for a system of this form, and also to articulate a condition of overlapping server capabilities which leads to resource pooling in the heavy traffic limit. Assuming that the latter condition holds, we rescale time and state space in standard fashion, then identify a Brownian control problem that is the formal heavy traffic limit of our rescaled scheduling problem. Because of the assumed overlap in server capabilities, the limiting Brownian control problem is effectively one‐dimensional, and it admits a pathwise optimal solution. That is, in the limiting Brownian control problem the multiple servers of our original model merge to form a single pool of service capacity, and there exists a dynamic control policy which minimizes cumulative cost incurred up to any time t with probability one. Interpreted in our original problem context, the Brownian solution suggests the following: virtually all backlogged work should be held in one particular job class, and all servers can and should be productively employed except when the total backlog is small. It is conjectured that such ideal system behavior can be approached using a family of relatively simple scheduling policies related to the cμ rule. This revised version was published online in June 2006 with corrections to the Cover Date.     

Long time asymptotics for constrained diffusions in polyhedral domains

We study long time asymptotic properties of constrained diffusions that arise in the heavy traffic analysis of multiclass queueing networks. We first consider the classical diffusion model with constant coefficients, namely a semimartingale reflecting Brownian motion (SRBM) in a d$d$-dimensional positive orthant. Under a natural stability condition on a related deterministic dynamical system [P. Dupuis, R.J. Williams, Lyapunov functions for semimartingale reflecting brownian motions, Annals of Probability 22 (2) (1994) 680–702] showed that an SRBM is ergodic. We strengthen this result by establishing geometric ergodicity for the process. As consequences of geometric ergodicity we obtain finiteness of the moment generating function of the invariant measure in a neighborhood of zero, uniform time estimates for polynomial moments of all orders, and functional central limit results. Similar long time properties are obtained for a broad family of constrained diffusion models with state dependent coefficients under a natural condition on the drift vector field. Such models arise from heavy traffic analysis of queueing networks with state dependent arrival and service rates.     

Diffusion approximation forGI/G/1 controlled queues

A queueing model is considered in which a controller can increase the service rate. There is a holding cost represented by functionh and the service cost proportional to the increased rate with coefficientl. The objective is to minimize the total expected discounted cost.Whenh andl are small and the system operates in heavy traffic, the control problem can be approximated by a singular stochastic control problem for the Brownian motion, namely, the so-called reflected follower problem. The optimal policy in this problem is characterized by a single numberz ^* so that the optimal process is a reflected diffusion in [0,z ^*]. To obtainz ^* one needs to solve a free boundary problem for the second order ordinary differential equation. For the original problem the policy which increases to the maximum the service rate when the normalized queue-length exceedsz ^* is approximately optimal.     

On the link between infinite horizon control and quasi-stationary distributions

We study infinite horizon control of continuous-time non-linear branching processes with almost sure extinction for general (positive or negative) discount. Our main goal is to study the link between infinite horizon control of these processes and an optimization problem involving their quasi-stationary distributions and the corresponding extinction rates. More precisely, we obtain an equivalent of the value function when the discount parameter is close to the threshold where the value function becomes infinite, and we characterize the optimal Markov control in this limit. To achieve this, we present a new proof of the dynamic programming principle based upon a pseudo-Markov property for controlled jump processes. We also prove the convergence to a unique quasi-stationary distribution of non-linear branching processes controlled by a Markov control conditioned on non-extinction.     

Random relaxed controls and partially observed stochastic systems

We introduce a class of generalized controls called random relaxed controls, and show that under quite general conditions, a partially observed, controlled diffusion will have an optimal random relaxed control whose cost equals the infimum over the costs of all ordinary controls. We also show that the optimal admissible control can be approximated arbitrarily well by very simple, ordinary controls. The proofs are based on a close analysis of the standard parts of nonstandard controls.     

Numerical methods for controlled and uncontrolled multiplexing and queueing systems

We deal with a very useful numerical method for both controlled and uncontrolled queuing and multiplexing type systems. The basic idea starts with a heavy traffic approximation, but it is shown that the results are very good even when working far from the heavy traffic regime. The underlying numerical method is a version of what is known as the Markov chain approximation method. It is a powerful methodology for controlled and uncontrolled stochastic systems, which can be approximated by diffusion or reflected diffusion type systems, and has been used with success on many other problems in stochastic control. We give a complete development of the relevant details, with an emphasis on multiplexing and particular queueing systems. The approximating process is a controlled or uncontrolled Markov chain which retains certain essential features of the original problem. This problem is generally substantially simpler than the original physical problem, and there are associated convergence theorems. The non-classical associated ergodic cost problem is derived, and put into a form such that reliable and good numerical algorithms, based on multigrid type ideas, can be used. Data for both controlled and uncontrolled problems shows the value of the method.Supported by ARO contract DAAL-03-92-G-0115, AFOSR contract F49620-92-J-0081, and DARPA contract AFOSR-91-0375.Formerly at Brown University. Supported by DARPA contract AFOSR-91-0375.     

Asymptotics in the random assignment problem

Summary We show that, in the usual probabilistic model for the random assignment problem, the optimal cost tends to a limit constant in probability and in expectation. The method involves construction of an infinite limit structure, in terms of which the limit constant is defined. But we cannot improve on the known numerical bounds for the limit.Research supported by NSF Grant MCS90-01710     

Explicit characterization of the super-replication strategy in financial markets with partial transaction costs

We consider a continuous time multivariate financial market with proportional transaction costs and study the problem of finding the minimal initial capital needed to hedge, without risk, European-type contingent claims. The model is similar to the one considered in Bouchard and Touzi [B. Bouchard, N. Touzi, Explicit solution of the multivariate super-replication problem under transaction costs, The Annals of Applied Probability 10 (3) (2000) 685–708] except that some of the assets can be exchanged freely, i.e. without paying transaction costs. In this context, we generalize the result of the above paper and prove that the super-replication price is given by the cost of the cheapest hedging strategy in which the number of non-freely exchangeable assets is kept constant over time. Our proof relies on the introduction of a new auxiliary control problem whose value function can be interpreted as the super-hedging price in a model with unbounded stochastic volatility (in the directions where transaction costs are non-zero). In particular, it confirms the usual intuition that transaction costs play a similar role to stochastic volatility.     

An optimal stopping problem for fragmentation processes

In this article we consider a toy example of an optimal stopping problem driven by fragmentation processes. We show that one can work with the concept of stopping lines to formulate the notion of an optimal stopping problem and moreover, to reduce it to a classical optimal stopping problem for a generalized Ornstein–Uhlenbeck process associated with Bertoin’s tagged fragment. We go on to solve the latter using a classical verification technique thanks to the application of aspects of the modern theory of integrated exponential Lévy processes.     

Scheduling networks of queues: Heavy traffic analysis of a simple open network

We consider a queueing network with two single-server stations and two types of customers. Customers of type A require service only at station 1 and customers of type B require service first at station 1 and then at station 2. Each server has a different general service time distribution, and each customer type has a different general interarrival time distribution. The problem is to find a dynamic sequencing policy at station 1 that minimizes the long-run average expected number of customers in the system.The scheduling problem is approximated by a dynamic control problem involving Brownian motion. A reformulation of this control problem is solved, and the solution is interpreted in terms of the queueing system in order to obtain an effective sequencing policy. Also, a pathwise lower bound (for any sequencing policy) is obtained for the total number of customers in the network. We show via simulation that the relative difference between the performance of the proposed policy and the pathwise lower bound becomes small as the load on the network is increased toward the heavy traffic limit.     

Asymptotic optimality of the Round–Robin policy in multipath routing with resequencing

We consider a model of a multipath routing system where arriving customers are routed to a set of identical, parallel, single server queues according to balancing policies operating without state information. After completion of service, customers are required to leave the system in their order of arrival, thus incurring an additional resequencing delay. We are interested in minimizing the end-to-end delay (including time at the resequencing buffer) experienced by arriving customers. To that end we establish the optimality of the Round–Robin routing assignment in two asymptotic regimes, namely heavy and light traffic: In heavy traffic, the Round–Robin customer assignment is shown to achieve the smallest (in the increasing convex stochastic ordering) end-to-end delay amongst all routing policies operating without queue state information. In light traffic, and for the special case of Poisson arrivals, we show that Round–Robin is again an optimal (in the strong stochastic ordering) routing policy. We illustrate the stochastic comparison results by several simulation examples. The work of the first author was supported through an ARCHIMEDES grant by the Greek Ministry of Education. The work of the second author was prepared through collaborative participation in the Communications and Networks Consortium sponsored by the U.S. Army Research Laboratory under the Collaborative Technology Alliance Program, Cooperative Agreement DAAD19-01-2-0011. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government.