Optimal admission policies for a finite queue with bursty arrivals

We consider a service system with a single server, a finite waiting room and two classes of customers with deterministic service time. Primary jobs arrive at random and are admitted whenever there is room in the system. At the beginning of each period, secondary jobs can be admitted from an infinite pool. A revenue is earned upon admission of each job, with the primary jobs bringing a higher contribution than the secondary jobs, the objective being to maximize the total discounted revenue over an infinite horizon. We model the system as a discrete time Markov decision process and show that a monotone admission policy is optimal when the number of primary arrivals has a fixed distribution. Moreover, when the primary arrival distribution varies with time according to a finite state Markov chain, we show that the optimal policy is conditionally monotone and that the numerical computation of an optimal policy, in this case, is substantially more difficult than in the case of stationary arrivals.This research was supported in part by the National Science Foundation, under grant ECS-8803061, while the author was at the University of Arizona.     

Optimality of a Threshold Policy in theM/M/ queue with repeated vacations

Consider anM/M/1 queueing system with server vacations where the server is turned off as soon as the queue gets empty. We assume that the vacation durations form a sequence of i.i.d. random variables with exponential distribution. At the end of a vacation period, the server may either be turned on if the queue is non empty or take another vacation. The following costs are incurred: a holding cost ofh per unit of time and per customer in the system and a fixed cost of each time the server is turned on. We show that there exists a threshold policy that minimizes the long-run average cost criterion. The approach we use was first proposed in Blanc et al. (1990) and enables us to determine explicitly the optimal threshold and the optimal long-run average cost in terms of the model parameters.     

Optimal threshold policies in a two-class preemptive priority queue with admission and termination control

We consider a two-class 1 preemptive priority queue in which there are two essential, on-line decisions that have to be taken. The first is the decision to either accept or reject new type-1 or type-2 jobs. The second is the decision to abort jobs, i.e., to remove any type-1 or type-2 jobs from the system. We show that there exist optimal threshold policies for these two types of, decisions.     

Admission control and pricing in a queue with batch arrivals

We investigate a problem of admission control in a queue with batch arrivals. We consider a single server with exponential service times and a compound Poisson arrival process. Each arriving batch computes its expected benefit and decides whether or not to enter the system. The controller’s problem is to set state dependent prices for arriving batches. Once prices have been set we formulate the admission control problem, derive properties of the value function, and obtain the optimal admission policy.     

Optimal control of stochastic functional differential equations with a bounded memory

This paper treats a finite time horizon optimal control problem in which the controlled state dynamics are governed by a general system of stochastic functional differential equations with a bounded memory. An infinite dimensional Hamilton–Jacobi–Bellman (HJB) equation is derived using a Bellman-type dynamic programming principle. It is shown that the value function is the unique viscosity solution of the HJB equation.     

Stability and optimality of a multi-product production and storage system under demand uncertainty

This work develops a discrete event model for a multi-product multi-stage production and storage (P&S) problem subject to random demand. The intervention problem consists of three types of possible decisions made at the end of one stage, which depend on the observed demand (or lack of) for each item: (i) to proceed further with the production of the same product, (ii) to proceed with the production of another product or (iii) to halt the production. The intervention problem is formulated in terms of dynamic programming (DP) operators and each possible solution induces an homogeneous Markov chain that characterizes the dynamics. However, solving directly the DP problem is not a viable task in situations involving a moderately large number of products with many production stages, and the idea of the paper is to detach from strict optimality with monitored precision, and rely on stability. The notion of stochastic stability brought to bear requires a finite set of positive recurrent states and the paper derives necessary and sufficient conditions for a policy to induce such a set in the studied P&S problem. An approximate value iteration algorithm is proposed, which applies to the broader class of control problems described by homogeneous Markov chains that satisfy a structural condition pointed out in the paper. This procedure iterates in a finite subset of the state space, circumventing the computational burden of standard dynamic programming. To benchmark the approach, the proposed algorithm is applied to a simple two-product P&S system.     

Dynamic power control in a fading downlink channel subject to an energy constraint

A central controller chooses a state-dependent transmission rate for each user in a fading, downlink channel by varying transmission power over time. For each user, the state of the channel evolves over time according to an exogenous continuous-time Markov chain (CTMC), which affects the quality of transmission. The traffic for each user, arriving at the central controller, is modeled as a finite-buffer Markovian queue with adjustable service rates. That is, for each user data packets arrive to the central controller according to a Poisson process and packet size is exponentially distributed; an arriving packet is dropped if the associated buffer is full, which results in degradation of quality of service. The controller forwards (downlink) the arriving packets to the corresponding user according to an optimally chosen transmission rate from a fixed set A _i of available values for each user i, depending on the backlog in the system and the channel state of all users. The objective is to maximize quality of service subject to an upper bound on the long-run average power consumption. We show that the optimal transmission rate for each user is solely a function of his own packet queue length and channel state; the dependence among users is captured through a penalty rate. Further, we explicitly characterize the optimal transmission rate for each user. This project is partially supported by Motorola grant # 0970-350-AF24. The authors thank Phil Fleming,Randy Berry and Achal Bassamboo for helpful comments.     

Multivariate Bayesian process control for a finite production run

Most industrial products and processes are characterized by several, typically correlated measurable variables, which jointly describe the product or process quality. Various control charts such as Hotelling’s T², EWMA and CUSUM charts have been developed for multivariate quality control, where the values of the chart parameters, namely the sample size, sampling interval and the control limits are determined to satisfy given economic and/or statistical requirements. It is well known that this traditional non-Bayesian approach to a control chart design is not optimal, but very few results regarding the form of the optimal Bayesian control policy have appeared in the literature, all limited to a univariate chart design. In this paper, we consider a multivariate Bayesian process mean control problem for a finite production run under the assumption that the observations are values of independent, normally distributed vectors of random variables. The problem is formulated in the POMDP (partially observable Markov decision process) framework and the objective is to determine a control policy minimizing the total expected cost. It is proved that under standard operating and cost assumptions the control limit policy is optimal. Cost comparisons with the benchmark chi-squared chart and the MEWMA chart show that the Bayesian chart is highly cost effective, the savings are larger for smaller values of the critical Mahalanobis distance between the in-control and out-of-control process mean.     

Optimal regulation of the service rate for a queue with finite waiting space

The optimal control for the service rate of a single-server queue with limited waiting space is derived. Costs are associated with providing the service and with waiting. A comparison with the traditional steady-state result is made.     

Sensitivity of optimal prices to system parameters in a steady-state service facility

We consider the problem of maximizing the long-run average reward in a service facility with dynamic pricing. We investigate sensitivity of optimal pricing policies to the parameters of the service facility which is modelled as an M/M/s/K$M/M/s/K$ queueing system. Arrival process to the facility is Poisson with arrival rate a decreasing function of the price currently being charged by the facility. We prove structural results on the optimal pricing policies when the parameters in the facility change. Namely, we show that optimal prices decrease when the capacity of the facility or the number of servers in the facility increase. Under a reasonable assumption, we also show that optimal prices increase as the overall demand for the service provided by the facility increases or when the service rate of the facility decreases. We illustrate how these structural results simplify the required computational effort while finding the optimal policy.     

Mixed control problem under partial observation

In this paper, we consider the control problem with optimal stopping of a jump process. Using compactification methods, we obtain the existence of an optimal Markovian optimal control.     

On the control of queueing systems with failures and repairs

Several classes of problems can be modelled as queueing systems with failures where preventative maintenance has to be optimized: relevant examples are computer systems, databases and production lines. This paper studies a simple case of such a situation based on the M/M/1 queue where the state of the server can take two values (on, off); the transition on–off happens either when there is a failure or when one decides to maintain. The characterization and an approximation of an optimal policy are given. Extensions to more general models, including general Markov processes, are examined.     

Analysis of optimal and nearly optimal sequencing policies for a closed queueing network

We analyze sequencing policies designed to most effectively utilize the resources of a closed queueing network representation of a manufacturing system. A continuous time Markov decision process formulation is used to compare the performance of optimal sequencing policies and a heuristic developed by analyzing a heavy traffic approximation of the system.     

Optimal dynamic pricing and inventory control with stock deterioration and partial backordering

This paper studies the optimal dynamic pricing and inventory control policies in a periodic-review inventory system with fixed ordering cost and additive demand. The inventory may deteriorate over time and the unmet demand may be partially backlogged. We identify two sufficient conditions under which (s,S,p) policies are optimal.     

The BMAP/PH/N retrial queue with Markovian flow of breakdowns

We consider a multi-server retrial queue with the Batch Markovian Arrival Process (BMAP). The servers are identical and independent of each other. The service time distribution of a customer by a server is of the phase (PH) type. If a group of primary calls meets idle servers the primary calls occupy the corresponding number of servers. If the number of idle servers is insufficient the rest of calls go to the orbit of unlimited size and repeat their attempts to get service after exponential amount of time independently of each other. Busy servers are subject to breakdowns and repairs. The common flow of breakdowns is the MAP. An event of this flow causes a failure of any busy server with equal probability. When a server fails the repair period starts immediately. This period has PH type distribution and does not depend on the repair time of other broken-down servers and the service time of customers occupying the working servers. A customer whose service was interrupted goes to the orbit with some probability and leaves the system with the supplementary probability. We derive the ergodicity condition and calculate the stationary distribution and the main performance characteristics of the system. Illustrative numerical examples are presented.     

Optimal control of a finite dam with a sample path constraint

In this paper, we discuss the 2-stage output procedure of a finite dam under the condition that water must be released by a fixed time. From this standpoint, the reservoir model we consider is subject to a sample path constraint and has a more general cost function than the earlier contributions. We analytically derive explicit formulas for the long-run average and the expected total discounted costs for an infinite time span and numerically calculate the optimal control policy. Finally, the optimal policy is compared with one by Zuckerman [1] and the effect of the fixed release time is discussed further.     

Markov control processes with randomized discounted cost

In this paper we consider Markov Decision Processes with discounted cost and a random rate in Borel spaces. We establish the dynamic programming algorithm in finite and infinity horizon cases. We provide conditions for the existence of measurable selectors. And we show an example of consumption-investment problem. This research was partially supported by the PROMEP grant 103.5/05/40.     

An assignment problem for a parallel queueing system with two heterogeneous servers

In this paper, we consider an optimization problem for a parallel queueing system with two heterogeneous servers. Each server has its own queue and customers arrive at each queue according to independent Poisson processes. Each service time is independent and exponentially distributed. When a customer arrives at queue 1, the customers in queue 1 can be transferred to queue 2 by paying an assignment cost which is proportional to the number of moved customers. Holding cost is a function of the pair of queue lengths of the two servers. Our objective is to minimize the expected total discounted cost. We use the dynamic programming approach for this problem. Considering the pair of queue lengths as a state space, we show that the optimal policy has a switch over structure under some conditions on the holding cost.     

Optimal control of price and production in an assemble-to-order system

We study the optimal control of an assembly system that produces one assembled-to-order final product with multiple made-to-stock components and sells it at variable price. It is shown that a threshold control on component production, product price, and product orders maximizes total discounted profit over an infinite horizon.