Optimal control of tandem reentrant queues

We consider optimal policies for reentrant queues in which customers may be served several times at the same station. We show that for tandem reentrant queues the last-buffer first-served (LBFS) policy stochastically maximizes the departure process. This revised version was published online in June 2006 with corrections to the Cover Date.     

Optimal threshold policies in a two-class preemptive priority queue with admission and termination control

We consider a two-class 1 preemptive priority queue in which there are two essential, on-line decisions that have to be taken. The first is the decision to either accept or reject new type-1 or type-2 jobs. The second is the decision to abort jobs, i.e., to remove any type-1 or type-2 jobs from the system. We show that there exist optimal threshold policies for these two types of, decisions.     

Scheduling in a multi-class series of queues with deterministic service times

We consider a problem of scheduling in a multi-class network of single-server queues in series, in which service times at the nodes are constant and equal. Such a model has potential application to automated manufacturing systems or packet-switched communication networks, where a message is divided into packets (or cells) of fixed lengths. The network is a series-type assembly or transfer line, with the exception that there is an additional class of jobs that requires processing only at the first node (class 0). There is a holding cost per unit time that is proportional to the total number of customers in the system. The objective is to minimize the (expected) total discounted holding cost over a finite or an infinite horizon. We show that an optimal policy gives priority to class-0 jobs at node 1 when at least one of a set ofm–1 inequalities on partial sums of the components of the state vector is satisfied. We solve the problem by two methods. The first involves formulating the problem as a (discrete-time) Markov decision process and using induction on the horizon length. The second is a sample-path approach using an interchange argument to establish optimality.The research of this author was supported by the National Science Foundation under Grant No. DDM-8719825. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.     

Admission control and pricing in a queue with batch arrivals

We investigate a problem of admission control in a queue with batch arrivals. We consider a single server with exponential service times and a compound Poisson arrival process. Each arriving batch computes its expected benefit and decides whether or not to enter the system. The controller’s problem is to set state dependent prices for arriving batches. Once prices have been set we formulate the admission control problem, derive properties of the value function, and obtain the optimal admission policy.     

A feedback fluid queue with two congestion control thresholds

Feedback fluid queues play an important role in modeling congestion control mechanisms for packet networks. In this paper we present and analyze a fluid queue with a feedback-based traffic rate adaptation scheme which uses two thresholds. The higher threshold B ₁ is used to signal the beginning of congestion while the lower threshold B ₂ signals the end of congestion. These two parameters together allow to make the trade-off between maximizing throughput performance and minimizing delay. The difference between the two thresholds helps to control the amount of feedback signals sent to the traffic source. In our model the input source can behave like either of two Markov fluid processes. The first applies as long as the upper threshold B ₁ has not been hit from below. As soon as that happens, the traffic source adapts and switches to the second process, until B ₂ (smaller than B ₁) is hit from above. We analyze the model by setting up the Kolmogorov forward equations, then solving the corresponding balance equations using a spectral expansion, and finally identifying sufficient constraints to solve for the unknowns in the solution. In particular, our analysis yields expressions for the stationary distribution of the buffer occupancy, the buffer delay distribution, and the throughput.     

Analysis of an algorithm for priority queue administration

A special kind of priority queue structure, priority trees (p-trees), and an algorithm for building such trees are investigated. The main part of the paper is devoted to a mathematical analysis of the algorithm showing that the expected number of key comparisons to insert a random element into a randomp-tree withn nodes isO((logn)²). Also some practical experiments, comparing different types of priority queue strategies, are presented.     

Optimal control asymptotics of a magnetohydrodynamic flow

An optimal multiplicative control problem is considered for a one-dimensional magnetohydrodynamic flow between parallel planes (Hartman flow). The external magnetic field is used as a control function. An optimality system is derived, and the asymptotics of an optimal control with respect to a regularization parameter and the Reynolds number are constructed.     

A sequential technique for the control of traffic intensity in Markovian queues

A sequential parameter control technique previously introduced by the author is modified in this paper so as to make it simple in practice. The detailed procedure involving two phases, a warning phase with control limits and a testing phase using an appropriate test is illustrated for a queueing system with an embedded Markov chain. Operating characteristics of the procedure are also examined.     

Optimal control of a production-inventory system with customer impatience

We consider the control of a production-inventory system with impatient customers. We show that the optimal policy can be described using two thresholds: a production base-stock level that determines when production takes place and an admission threshold that determines when orders should be accepted. We describe an algorithm for computing the performance of the system for any choice of base-stock level and admission threshold. In a numerical study, we compare the performance of the optimal policy against several other policies.     

A single server queue in a hard-real-time environment

We consider a single server first in first out queue in which each arriving task has to be completed within a certain period of time (its deadline). More precisely, each arriving task has its own deadline - a non-negative real number - and as soon as the response time of one task exceeds its deadline, the whole system in considered to have failed. (In that sense the deadline is hard.) The main practical motivation for analyzing such queues comes from the need to evaluate mathematically the reliability of computer systems working with real time constraints (space or aircraft systems for instance). We shall therefore be mainly concerned with the analytical characterization of the transient behavior of such a queue in order to determine the probability of meeting all hard deadlines during a finite period of time (the ‘mission time’). The probabilistic methods for analyzing such systems are suggested by earlier work on impatience in telecommunication systems [1,2].     

Optimal control of a queue under a quality-of-service constraint with bounded and unbounded rates

We consider a simple Markovian queue with Poisson arrivals and exponential service times for jobs. The controller chooses state-dependent service rates from an action space. The queue has a finite buffer, and when full, new jobs get rejected. The controller’s objective is to choose optimal service rates that meet a quality-of-service constraint. We solve this problem analytically and compute it numerically under two cases: When the action space is unbounded and when it is bounded.     

A two-stage tandem queue attended by a moving server with holding and switching costs

We consider a two-stage tandem queue attended by a moving server, with homogeneous Poisson arrivals and general service times. Two different holding costs for stages 1 and 2 and different switching costs from one stage to the other are considered. We show that the optimal policy in the second stage is greedy; and if the holding cost rate in the second stage is greater or equal to the rate in the first stage, then the optimal policy in the second stage is also exhaustive. Then, the optimality condition for sequential service policy in systems with zero switchover times is introduced. Considering some properties of the optimal policy, we then define a Triple-Threshold (TT) policy to approximate the optimal policy in the first stage. Finally, a model is introduced to find the optimal TT policy, and using numerical results, it is shown that the TT policy accurately approximates the optimal policy. This revised version was published online in June 2006 with corrections to the Cover Date.     

A single server queue with cyclically indexed arrival and service rates

In this paper we consider a queueing model that results from at least two apparently unrelated areas. One motivation to study a system of this type results from a test case of a computer simulation factor screening technique calledfrequency domain methodology. A second motivation comes from manufacturing, where due to cyclic scheduling of upstream machines, the arrival process to downstream machines is periodic. The model is a single server queue with FIFO service discipline and exponential interarrival and service times where the arrival and/or service rates are deterministic cyclic functions of the customer sequence number. We provide steady state results for the mean number in the system for the model with cyclic arrival and fixed service rates and for the model with fixed arrival and cyclic service rates. For the model with both cyclic arrival and service rates, upper and lower bounds are developed for the steady state mean waiting time in the system. Throughout the paper various implications and/or insights derived from the results of this study are discussed for frequency domain methodology.The authors acknowledge the financial support of the CBA/GSB Faculty Research Committee of the College of Business Administration, The University of Texas at Austin.     

Optimal control in a macroeconomic problem

The Pontryagin maximum principle is used to develop an original algorithm for finding an optimal control in a macroeconomic problem. Numerical results are presented for the optimal control and optimal trajectory of the development of a regional economic system. For an optimal control satisfying a certain constraint, an invariant of a macroeconomic system is derived.     

Optimal control of a signorini contact problem

We are interested in finding the coefficient of friction which leads us to a given displacement on the contact surface between an elastic solid body and a rigid foundation. The mathematical formulation of the problem is an optimal control problem governed by a quasivariational inequality. We obtain an approximative caracterization, by using two families of penalized and regularized problems, for a given optimal control.     

A survey of Markov decision models for control of networks of queues

We review models for the optimal control of networks of queues. Our main emphasis is on models based on Markov decision theory and the characterization of the structure of optimal control policies.This research was partially supported by the National Science Foundation under Grant No. DDM-8719825. The Government has certain rights in this material. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. The research was also partially supported by the C.I.E.S. (France), while the author was on leave at INRIA, Sophia-Antipolis, 1991–92.     

Optimal feedback control for a linear-quadratic control problem with a state inequality constraint

In this paper, we consider the linear-quadratic control problem (LQCP) for systems defined by evolution operators with an inequality constraint on the state. It is shown that, under suitable assumptions, the optimal control exists, is unique, and has a closedloop structure. The synthesis of the feedback control requires one to solve two Riccati integral equations.     

Optimal Operating Policy for an M/G/1 Exhaustive Server-Vacation Model

We consider an M/G/1 queueing system controlled by an exhaustive server–vacation policy, i.e, the server is turned off whenever the system becomes empty and it is turned on after a random time with at least a customer present in the system. In this paper, it is proved that there exists an exhaustive optimal policy which is of the form X + a(T - X)⁺, where, starting with the server off, X represents the time for the first arrival and T and a are non-negative real numbers. Using a classical average cost structure, the optimization problem is treated under the asymptotic average criterion. A structured definition of exhaustive policy is also derived.     

Optimal guaranteed control of linear systems under disturbances

This paper is devoted to the problem of the minimax control of a dynamical system with quadratic performance functional under external disturbances and geometric control constraints. The optimal guaranteed control strategy is obtained in explicit form.Translated fromMatematicheskie Zametki, Vol. 60, No. 2, pp. 198–205, August, 1996.This research was supported by the Russian Foundation for Basic Research under grant No. 95-01-000771a.