Markovian network processes: Congestion-dependent routing and processing

A Markovian network process describes the movement of discrete units among a set of nodes that process the units. There is considerable knowledge of such networks, often called queueing networks, in which the nodes operate independently and the routes of the units are independent. The focus of this study, in contrast, is on networks with dependent nodes and routings. Examples of dependencies are parallel processing across several nodes, blocking of transitions because of capacity constraints on nodes, alternate routing of units to avoid congestion, and accelerating or decelerating the processing rate at a node depending on downstream congestion. We introduce a general network process representing the numbers of units at the nodes and derive its equilibrium distribution. This distribution takes the form of a product of functions of vectors in which the arguments of the functions satisfy an interchangeability property. This new type of distribution may apply to other multi-variate processes as well. A basic idea in our approach is a linking of certain micro-level balance properties of the network routing to the processing rates at the nodes. The link is via routing-balance partitions of nodes that are inherent in any network. A byproduct of this approach is a general characterization of blocking of transitions without the restriction that the process is reversible, which had been a standard assumption. We also give necessary and sufficient conditions under which a unit moving in the network sees a time average for the unmoved units (called the MUSTA property). Finally, we discuss when certain flows between nodes in an open network are Poisson processes.This research was sponsored in part by Air Force Office of Scientific Research contract 84-0367.     

Sojourn times in a processor sharing queue with service interruptions

We study the sojourn times of customers in an M/M/1 queue with the processor sharing service discipline and a server that is subject to breakdowns. The lengths of the breakdowns have a general distribution, whereas the on-periods are exponentially distributed. A branching process approach leads to a decomposition of the sojourn time, in which the components are independent of each other and can be investigated separately. We derive the Laplace–Stieltjes transform of the sojourn-time distribution in steady state, and show that the expected sojourn time is not proportional to the service requirement. In the heavy-traffic limit, the sojourn time conditioned on the service requirement and scaled by the traffic load is shown to be exponentially distributed. The results can be used for the performance analysis of elastic traffic in communication networks, in particular, the ABR service class in ATM networks, and best-effort services in IP networks.     

Analysis of a three node queueing network

The three node Jackson queueing network is the simplest acyclic network in which in equilibrium the sojourn times of a customer at each of the nodes are dependent. We show that assuming the individual sojourn times are independent provides a good approximation to the total sojourn time. This is done by simulating the network and showing that the sojourn times generally pass a Kolmogorov-Smirnov test as having come from the approximating distribution. Since the sum of dependent random variables may have the same distribution as the sum of independent random variables with the same marginal distributions, it is conceivable that our approximation is exact. However, we numerically compute upper and lower bounds for the distribution of the total sojourn time; these bounds are so close that the approximating distribution lies outside of the bounds. Thus, the bounds are accurate enough to distinguish between the two distributions even though the Kolmogorov-Smirnov test generally cannot.     

Heavy-traffic analysis of the sojourn time in a three node Jackson network with overtaking

We consider a Jackson network consisting of three first-in-first-out (FIFO)M/M/1 queues. When customers leave the first queue they can be routed to either the second or third queue. Thus, a customer that traverses the network by going from the first to the second to the third queue, can be overtaken by another customer that is routed from the first queue directly to the third. We study the distribution of the sojourn time of a customer through the three node network, in the heavy traffic limit. A three term heavy traffic asymptotic approximation to the sojourn time density is derived. The leading term shows that the nodes decouple in the heavy traffic limit. The next two terms, however, do show the dependence of the sojourn times at the individual nodes and give quantitative measures of the effects of overtaking.     

On the correlation of sojourn times in open networks of exponential multiserver queues

We consider the joint distribution of successive sojourn times of customers traversing a path in Jackson network. It is shown in a general setting that sojourn times exhibit positive dependence. This especially applies to paths which admit overtaking due to the network topology as well as due to the internal node structure.     

“Spatial nodes” in discrete location problems

Discrete location models often assume an underlying network where demands originate at point nodes. To apply these models to planar regions with continuously distributed demand, the region is usually partitioned into zones and the demand from each zone is assumed to originate at a point, usually the zone centroid. Thus, the point node in the underlying network represents a spatial zone with a finite area. This paper examines the effect of approximating these spatial nodes by point nodes. In some problem scenarios, the approximation does not affect the solution. However, especially when the locational criterion includes the consideration of intra-zonal travel cost variances (e.g. travel time variance) and demands may originate anywhere within zones of nonzero area, point nodes do not give an accurate evaluation of the performance of a locational design. To illustrate the application of the concept of spatial nodes, a model is formulated for locatingp (fire-fighting) units in a region having continuously distributed demand with the objective of minimizing a nonlinear function of arrival times of the first and second closest units to any (fire) incident. A heuristic site-substitution procedure is presented that solves the formulated model.     

On passage times in Jackson networks: Two-stations walk and overtake-free paths

We compute the distribution of the time for a two-stations walk in an open Jackson network (which includes the twofold service at one station), and give a simple derivation of the distribution of the sojourn times for overtake-free paths in these networks.     

Queueing networks with dependent nodes and concurrent movements

Classical queueing network processes are useful for modeling the movement of discrete units in a network in which the nodes operate independently, the routing of units is independent of the congestion, only one unit moves at a time and its equilibrium distribution is a well-understood product form. Actual networks, however, typically have dependent nodes and concurrent movement of units. Imagine the dependencies associated with the network movements of telephone calls, manufacturing material, computer data packets, messages in a parallel-processing simulation, etc. A second generation of queueing network processes is beginning to evolve for modeling such “intelligent” networks with dependent nodes and concurrent movements. This paper describes the following fundamental processes that have been developed in this regard:

? A basic queueing network process for dependent nodes and single-unit movements. Examples include the classical Jackson, BCMP, Kelly and Kelly-Whittle networks and networks with interacting subpopulations.

? Reversible queueing network processes for dependent nodes and concurrent movements. An example is a multivariate, compound birth-death process.

? Miscellaneous partially balanced queueing networks. Examples include extensions of the basic network processes and weakly coupled and quasi-reversible networks.

     

Markov Games with Several Ergodic Classes

We consider Markov games of the general form characterized by the property that, for all stationary strategies of players, the set of game states is partitioned into several ergodic sets and a transient set, which may vary depending on the strategies of players. As a criterion, we choose the mean payoff of the first player per unit time. It is proved that the general Markov game with a finite set of states and decisions of both players has a value, and both players have -optimal stationary strategies. The correctness of this statement is demonstrated on the well-known Blackwell's example (Big Match).     

Networks of queues in discrete time

We study a new class of networks of queues whose nodes operate in round-robin fashion and other ways of interest to computer science. We compute a stationary law of product form for the Markov process describing the state of the network. Moreover, we obtain the conditional expected travel time of a job given the job's requested processing times at particular nodes along its route.     

First passage times for Markov renewal processes and applications

This paper proposes a uniformly convergent algorithm for the joint transform of the first passage time and the first passage number of steps for general Markov renewal processes with any initial state probability vector. The uniformly convergent algorithm with arbitrarily prescribed error can be efficiently applied to compute busy periods, busy cycles, waiting times, sojourn times, and relevant indices of various generic queueing systems and queueing networks. This paper also conducts a numerical experiment to implement the proposed algorithm.     

Dependence structure of sojourn times via partition separated ordering

We investigate the joint distribution of successive sojourn times of a customer traversing a path in a Jackson network. It is shown in a general setting that sojourn times for a broad class of such paths exhibit positive dependence. This applies to paths which admit overtaking due to the network topology as well as due to the internal node structure. Our proofs utilize the concept of partition separated orders on multidimensional ordered spaces.     

The cyclic queue and the tandem queue

We consider a closed queueing network, consisting of two FCFS single server queues in series: a queue with general service times and a queue with exponential service times. A fixed number \(N\) of customers cycle through this network. We determine the joint sojourn time distribution of a tagged customer in, first, the general queue and, then, the exponential queue. Subsequently, we indicate how the approach toward this closed system also allows us to study the joint sojourn time distribution of a tagged customer in the equivalent open two-queue system, consisting of FCFS single server queues with general and exponential service times, respectively, in the case that the input process to the first queue is a Poisson process.     

A splitting technique for Harris recurrent Markov chains

Summary A technique is presented, which enables the state space of a Harris recurrent Markov chain to be split in a way, which introduces into the split state space an atom. Hence the full force of renewal theory can be used in the analysis of Markov chains on a general state space. As a first illustration of the method we show how Derman's construction for the invariant measure works in the general state space. The Splitting Technique is also applied to the study of sums of transition probabilities.     

Queues as Harris recurrent Markov chains

We present a framework for representing a queue at arrival epochs as a Harris recurrent Markov chain (HRMC). The input to the queue is a marked point process governed by a HRMC and the queue dynamics are formulated by a general recursion. Such inputs include the cases of i.i.d, regenerative, Markov modulated, Markov renewal and the output from some queues as well. Since a HRMC is regenerative, the queue inherits the regenerative structure. As examples, we consider split & match, tandem, G/G/c and more general skip forward networks. In the case of i.i.d. input, we show the existence of regeneration points for a Jackson type open network having general service and interarrivai time distributions.A revised version of the author's winning paper of the 1986 George E. Nicholson Prize (awarded by the Operations Research Society of America).     

Individual customer’s behaviour in networks with state-dependent arrival rates

We consider Jackson networks with state-dependent arrival and service rates which show product form or nearly product form steady-states and come up with examples of load-dependent admission control. For these networks we prove an arrival theorem for external as well as for internal arrivals. In case of open tandem systems with state-independent service rates we compute the joint distribution of the sojourn times of a customer in the nodes and the distribution of the customer’s end-to-end-delay. This revised version was published online in June 2006 with corrections to the Cover Date.     

Path comparisons for a priori and time-adaptive decisions in stochastic,time-varying networks

Travel times in congested transportation networks are time-varying quantities that can at best be known a priori probabilistically. In such networks, the arc weights (travel times) are represented by random variables whose probability distribution functions vary with time. These networks are referred to herein as stochastic, time-varying, or STV, networks. The determination of “least time” routes in STV networks is more difficult than in deterministic networks, in part because, for a given departure time, more than one path may exist between an origin and destination, each with a positive probability of having the least travel time. In this paper, measures for comparing time-varying, random path travel times over a time period are given for both a priori optimization and time-adaptive choices (where a driver may react to revealed arrival times at intermediate nodes). The resulting measures are central to the development of methodologies for determining “optimal” paths in STV networks.     

Hierarchical routing over dynamic wireless networks

The topology of a mobile wireless network changes over time. Maintaining routes between all nodes requires the continuous transmission of control information, which consumes precious power and bandwidth resources. Many routing protocols have been developed, trading off control overhead and route quality. In this paper, we ask whether there exist low‐overhead schemes that produce low‐stretch routes, even in large networks where all the nodes are mobile. We present a scheme that maintains a hierarchical structure within which constant‐stretch routes can be efficiently computed between every pair of nodes. The scheme rebuilds each level of the hierarchy periodically, at a rate that decreases exponentially with the level of the hierarchy. We prove that this scheme achieves constant stretch under a mild smoothness condition on the nodal mobility processes. Furthermore, we prove tight bounds for the network‐wide control overhead under the additional assumption of the connectivity graph forming a doubling metric space. Specifically, we show that for a connectivity model combining the random geometric graph with obstacles, constant‐stretch routes can be maintained with a total overhead of bits of control information per time unit. © 2015 Wiley Periodicals, Inc. Random Struct. Alg., 47, 669–709, 2015     

Designing Least-Cost Survivable Wireless Backhaul Networks

This paper presents a new heuristic algorithm for designing least-cost telecommunications networks to carry cell site traffic to wireless switches while meeting survivability, capacity, and technical compatibility constraints. This requires solving the following combinatorial optimization problems simultaneously: (1) Select a least-cost subset of locations (network nodes) as hubs where traffic is to be aggregated and switched, and choose the type of hub (high-capacity DS3 vs. lower-capacity DS1 hub) for each location; (2) Optimally assign traffic from other nodes to these hubs, so that the traffic entering the network at these nodes is routed to the assigned hubs while respecting capacity constraints on the links and routing-diversity constraints on the hubs to assure survivability; and (3) Optimally choose the types of links to be used in interconnecting the nodes and hubs based on the capacities and costs associated with each link type. Each of these optimization problems must be solved while accounting for its impacts on the other two. This paper introduces a short term Tabu Search (STTS) meta-heuristic, with embedded knapsack and network flow sub-problems, that has proved highly effective in designing such backhaul networks for carrying personal communications services (PCS) traffic. It solves problems that are challenging for conventional branch-and-bound solvers in minutes instead of hours and finds lower-cost solutions. Applied to real-world network design problems, the heuristic has successfully identified designs that save over 20% compared to the best previously known designs.