Markov renewal queues of M/M(a,d)/l/(b,n) system-part i

This paper discusses a general bulk service queue which falls into the Markov renewal class. Applying an analysis similar to the one by Hunter (1983) for M/M1/N type of feedback queues, certain properties of discrete and continuous time queue length processe are studied here. The results and formulas are then applied to a numerical illustration.     

A queue with instantaneous tri-route decision process

This paper focuses on the study of several random processes associated with M/G1 queue with instantaneous tri-route decision process. The stationary distribution of the output process is derived. Some particular queues with feedback and without feedback are also analysed. Some operating characteristics are studied for this queue. Optimum service rate is obtained. A numerical study is carried out to test the feasibility of the queueing model.     

A queueing system with linear repeated attempts,bernoulli schedule and feedback

We analyse a single-server retrial queueing system with infinite buffer, Poisson arrivals, general distribution of service time and linear retrial policy. If an arriving customer finds the server occupied, he joins with probabilityp a retrial group (called orbit) and with complementary probabilityq a priority queue in order to be served. After the customer is served completely, he will decide either to return to the priority queue for another service with probability ϑ or to leave the system forever with probability =1−ϑ, where 0≤ϑ<1. We study the ergodicity of the embedded Markov chain, its stationary distribution function and the joint generating function of the number of customers in both groups in the steady-state regime. Moreover, we obtain the generating function of system size distribution, which generalizes the well-knownPollaczek-Khinchin formula. Also we obtain a stochastic decomposition law for our queueing system and as an application we study the asymptotic behaviour under high rate of retrials. The results agree with known special cases. Finally, we give numerical examples to illustrate the effect of the parameters on several performance characteristics.     

Sojourn times in queues with instantaneous tri-route decision process

In this paper, we study the total sojourn time in a queueing system with an instantaneous tri-route decision process. Even though the computations are more difficult, we give here the structure of the sojourn time process for the M/G/1 queue with tri-route decision process. A numerical study is carried out in this paper.     

On stochastic processes in a multilevel control bulk queueing system

This article analyzes some stochastic processes that arise in a bulk single server queue with continuously operating server, state dependent compound Poisson input flow and general state dependent service process. The authors treat the queueing process as a semi–regenerative process and obtain the invariant probability measure and the transient distribution for the embedded Markov chain. The stationary probability measure for the queueing process with continuous time parameter is found by using semi-regenerative techniques. The authors also study the input and output processes and establish ergodic theorems for some functionals of these processes. The results are obtained in terms of the invariant probability measure for the embedded process and the stationary measure for the queueing process with continuous time parameter     

Matrix analytic methods for a multi-server retrial queue with buffer

We present numerical methods for obtaining the stationary distribution of states for multi-server retrial queues with Markovian arrival process, phase type service time distribution with two states and finite buffer; and moments of the waiting time. The methods are direct extensions of the ones for the single server retrial queues earlier developed by the authors. The queue is modelled as a level dependent Markov process and the generator for the process is approximated with one which is spacially homogeneous above some levelN. The levelN is chosen such that the probability associated with the homogeneous part of the approximated system is bounded by a small tolerance and the generator is eventually truncated above that level. Solutions are obtained by efficient application of block Gaussian elimination.     

A repairable Geo X /G/1 retrial queue with Bernoulli feedback and impatient customers

Abstract This paper deals with a discrete-time batch arrival retrial queue with the server subject to starting failures.Diferent from standard batch arrival retrial queues with starting failures,we assume that each customer after service either immediately returns to the orbit for another service with probabilityθor leaves the system forever with probability 1θ(0≤θ1).On the other hand,if the server is started unsuccessfully by a customer(external or repeated),the server is sent to repair immediately and the customer either joins the orbit with probability q or leaves the system forever with probability 1 q(0≤q1).Firstly,we introduce an embedded Markov chain and obtain the necessary and sufcient condition for ergodicity of this embedded Markov chain.Secondly,we derive the steady-state joint distribution of the server state and the number of customers in the system/orbit at arbitrary time.We also derive a stochastic decomposition law.In the special case of individual arrivals,we develop recursive formulae for calculating the steady-state distribution of the orbit size.Besides,we investigate the relation between our discrete-time system and its continuous counterpart.Finally,some numerical examples show the influence of the parameters on the mean orbit size.     

On the optimal control of arrivals to a single queue with arbitrary feedback delay

We consider a problem of admission control to a single queue in discrete time. The controller has access to k step old queue lengths only, where k can be arbitrary. The problem is motivated, in particular, by recent advances in high-speed networking where information delays have become prominent. We formulate the problem in the framework of Completely Observable Controlled Markov Chains, in terms of a multi-dimensional state variable. Exploiting the structure of the problem, we show that under appropriate conditions, the multi-dimensional Dynamic Programming Equation (DPE) can be reduced to a unidimensional one. We then provide simple computable upper and lower bounds to the optimal value function corresponding to the reduced unidimensional DPE. These upper and lower bounds, along with a certain relationship among the parameters of the problem, enable us to deduce partially the structural features of the optimal policy. Our approach enables us to recover simply, in part, the recent results of Altman and Stidham, who have shown that a multiple-threshold-type policy is optimal for this problem. Further, under the same relationship among the parameters of the problem, we provide easily computable upper bounds to the multiple thresholds and show the existence of simple relationships among these upper bounds. These relationships allow us to gain very useful insights into the nature of the optimal policy. In particular, the insights obtained are of great importance for the problem of actually computing an optimal policy because they reduce the search space enormously. This revised version was published online in June 2006 with corrections to the Cover Date.     

Decay rate for a PH/M/2 queue with shortest queue discipline

In this paper, we consider a PH/M/2 queue in which each server has its own queue and arriving customers join the shortest queue. For this model, it has been conjectured that the decay rate of the tail probabilities for the shortest queue length in the steady state is equal to the square of the decay rate for the queue length in the corresponding PH/M/2 model with a single queue. We prove this fact in the sense that the tail probabilities are asymptotically geometric when the difference of the queue sizes and the arrival phase are fixed. Our proof is based on the matrix analytic approach pioneered by Neuts and recent results on the decay rates. AMS subject classifications: 60K25 · 60K20 · 60F10 · 90B22     

A modified Gauss-Seidel-algorithm with exclusion of suboptimal actions for A class of semi-Markovian decision problems

In this paper it is shown that a modified Gauss-Seidel-Algorithm with exclusion of suboptimai actions can be used for approximative solving a continuously discounted Semi-Markovian decision problem. Moreover it is proved that the overrelaxation factor of the algorithm introduced in [5] can be improved. If the sojourn times are constant and the overrelaxation factor is independent of states and actions one obtains results of Reetz [6,7] for (discrete) discounted Markovian decision problems. Finally, an example illustrates this method including exclusion of suboptimai actions     

A complete and simple solution for a discrete-time multi-server queue with bulk arrivals and deterministic service times

A complete distribution for the system content of a discrete-time multi-server queue with an infinite buffer is presented, where each customer arriving in a group requires a deterministic service time that could be greater than one slot. In addition, when the service time equals one slot, a complete distribution for the delay is also presented.     

Existence,uniqueness and ergodicity of Markov branching processes with immigration and instantaneous resurrection

We consider a modified Markov branching process incorporating with both state-independent immigration and instantaneous resurrection.The existence criterion of the process is firstly considered.We prove that if the sum of the resurrection rates is finite,then there does not exist any process.An existence criterion is then established when the sum of the resurrection rates is infinite.Some equivalent criteria,possessing the advantage of being easily checked,are obtained for the latter case.The uniqueness criterion for such process is also investigated.We prove that although there exist infinitely many of them,there always exists a unique honest process for a given q-matrix.This unique honest process is then constructed.The ergodicity property of this honest process is analysed in detail.We prove that this honest process is always ergodic and the explicit expression for the equilibrium distribution is established.     

Stationary tail asymptotics of a tandem queue with feedback

Motivated by applications in manufacturing systems and computer networks, in this paper, we consider a tandem queue with feedback. In this model, the i.i.d. interarrival times and the i.i.d. service times are both exponential and independent. Upon completion of a service at the second station, the customer either leaves the system with probability p or goes back, together with all customers currently waiting in the second queue, to the first queue with probability 1−p. For any fixed number of customers in one queue (either queue 1 or queue 2), using newly developed methods we study properties of the exactly geometric tail asymptotics as the number of customers in the other queue increases to infinity. We hope that this work can serve as a demonstration of how to deal with a block generating function of GI/M/1 type, and an illustration of how the boundary behaviour can affect the tail decay rate.     

Equilibrium customer strategies in a single server Markovian queue with setup times

We consider a single server Markovian queue with setup times. Whenever this system becomes empty, the server is turned off. Whenever a customer arrives to an empty system, the server begins an exponential setup time to start service again. We assume that arriving customers decide whether to enter the system or balk based on a natural reward-cost structure, which incorporates their desire for service as well as their unwillingness to wait. We examine customer behavior under various levels of information regarding the system state. Specifically, before making the decision, a customer may or may not know the state of the server and/or the number of present customers. We derive equilibrium strategies for the customers under the various levels of information and analyze the stationary behavior of the system under these strategies. We also illustrate further effects of the information level on the equilibrium behavior via numerical experiments.      

Performance and reliability analysis of a repairable discrete‐time Geo/G/1 queue with Bernoulli feedback and randomized policy

This paper is concerned with a discrete‐time G e o /G /1 repairable queueing system with Bernoulli feedback and randomized ‐policy. The service station may be subject to failures randomly during serving customers and therefore is sent for repair immediately. The ‐policy means that when the number of customers in the system reaches a given threshold value N , the deactivated server is turned on with probability p or is still left off with probability 1?p . Applying the law of total probability decomposition, the renewal theory and the probability generating function technique, we investigate the queueing performance measures and reliability indices simultaneously in our work. Both the transient queue length distribution and the recursive expressions of the steady‐state queue length distribution at various epochs are explicitly derived. Meanwhile, the stochastic decomposition property is presented for the proposed model. Various reliability indices, including the transient and the steady‐state unavailability of the service station, the expected number of the service station breakdowns during the time interval and the equilibrium failure frequency of the service station are also discussed. Finally, an operating cost function is formulated, and the direct search method is employed to numerically find the optimum value of N for minimizing the system cost. Copyright © 2017 John Wiley & Sons, Ltd.     

Steady state analysis of an M/G/1 queue with linear retrial policy and two phase service under Bernoulli vacation schedule

We consider a single server queueing system with two phases of heterogeneous service and Bernoulli vacation schedule which operate under the so called linear retrial policy. This model extends both the classical M/G/1 retrial queue with linear retrial policy as well as the M/G/1 queue with two phases of service and Bernoulli vacation model. We carry out an extensive analysis of the model.     

Numerical solution of a long-term average control problem for singular stochastic processes

This paper analyzes numerically a long-term average stochastic control problem involving a controlled diffusion on a bounded region. The solution technique takes advantage of an infinite-dimensional linear programming formulation for the problem which relates the stationary measures to the generators of the diffusion. The restriction of the diffusion to an interval is accomplished through reflection at one end point and a jump operator acting singularly in time at the other end point. Different approximations of the linear program are obtained using finite differences for the differential operators (a Markov chain approximation to the diffusion) and using a finite element method to approximate the stationary density. The numerical results are compared with each other and with dynamic programming. This research has been supported in part by the U.S. National Security Agency under Grant Agreement Number H98230-05-1-0062. The United States Government is authorized to reproduce and distribute reprints notwithstanding any copyright notation herein.     

The onset of dominance in balls‐in‐bins processes with feedback

Consider a balls‐in‐bins process in which each new ball goes into a given bin with probability proportional to f(n), where n is the number of balls currently in the bin and f is a fixed positive function. It is known that these so‐called balls‐in‐bins processes with feedback have a monopolistic regime: if f(x) = x^p for p > 1, then there is a finite time after which one of the bins will receive all incoming balls. Our goal in this article is to quantify the onset of monopoly. We show that the initial number of balls is large and bin 1 starts with a fraction α > 1/2 of the balls, then with very high probability its share of the total number of balls never decreases significantly below α. Thus a bin that obtains more than half of the balls at a “large time” will most likely preserve its position of leadership. However, the probability that the winning bin has a non‐negligible advantage after n balls are in the system is ～const. × n^1‐p, and the number of balls in the losing bin has a power‐law tail. Similar results also hold for more general functions f. © 2009 Wiley Periodicals, Inc. Random Struct. Alg., 2009     

Equivalence of Lyapunov stability criteria in a class of Markov decision processes

We are concerned with Markov decision processes with countable state space and discrete-time parameter. The main structural restriction on the model is the following: under the action of any stationary policy the state space is acommunicating class. In this context, we prove the equivalence of ten stability/ergodicity conditions on the transition law of the model, which imply the existence of average optimal stationary policies for an arbitrary continuous and bounded reward function; these conditions include the Lyapunov function condition (LFC) introduced by A. Hordijk. As a consequence of our results, the LFC is proved to be equivalent to the following: under the action of any stationary policy the corresponding Markov chain has a unique invariant distribution which depends continuously on the stationary policy being used. A weak form of the latter condition was used by one of the authors to establish the existence of optimal stationary policies using an approach based on renewal theory.This research was supported in part by the Third World Academy of Sciences (TWAS) under Grant TWAS RG MP 898-152.