Controlling the GI/M/1 queue by conditional acceptance of customers

In this paper we consider the problem of controlling the arrival of customers into a GI/M/1 service station. It is known that when the decisions controlling the system are made only at arrival epochs, the optimal acceptance strategy is of a control-limit type, i.e., an arrival is accepted if and only if fewer than n customers are present in the system. The question is whether exercising conditional acceptance can further increase the expected long run average profit of a firm which operates the system. To reveal the relevance of conditional acceptance we consider an extension of the control-limit rule in which the nth customer is conditionally admitted to the queue. This customer may later be rejected if neither service completion nor arrival has occurred within a given time period since the last arrival epoch. We model the system as a semi-Markov decision process, and develop conditions under which such a policy is preferable to the simple control-limit rule.     

Computation of the Optimal Policy for the Control of a Compound Immigration Process through Total Catastrophes

In this paper we consider a Markov decision model introduced by Economou (2003), in which it was proved that the optimal policy in the problem of controlling a compound immigration process through total catastrophes is of control-limit type. We show that the average cost of a control-limit policy is unimodal as a function of the critical point. This result enables us to design very efficient algorithms for the computation of the optimal policy as the bisection procedure and a special-purpose policy iteration algorithm that operates on the class of control-limit policies.AMS 2000 Subject Classification: Primary 9OC40; Secondary 6OJ25     

Optimal maintenance of a production-inventory system with idle periods

In this paper we consider a production-inventory system in which an input generating installation supplies a buffer with a raw material and a production unit pulls the raw material from the buffer with constant rate. The installation deteriorates in time and the problem of its optimal preventive maintenance is considered. It is assumed that the installation after the completion of its maintenance remains idle until the buffer is evacuated. Under a suitable cost structure it is shown that the average-cost optimal policy for fixed buffer content is of control-limit type, i.e. it prescribes a preventive maintenance of the installation if and only if its degree of deterioration is greater than or equal to a critical level. Using the usual regenerative argument, the average cost of a control-limit policy is computed exactly and then, the optimal control-limit policy is determined. Furthermore, the stationary probabilities of the system under the optimal policy are computed.     

Optimal preventive maintenance of a production system with an intermediate buffer

In this paper we consider a model consisting of a deteriorating installation that transfers a raw material to a production unit and a buffer which has been built between the installation and the production unit. The deterioration process of the installation is considered to be nonstationary, i.e. the transition probabilities may depend not only on the working conditions of the installation but on its age as well. The problem of the optimal preventive maintenance of the installation is considered. Under a suitable cost structure it is shown that, for fixed age of the installation and fixed buffer level, the optimal policy is of control-limit type. When the deterioration process is stationary, an efficient Markov decision algorithm operating on the class of control-limit policies is developed. There is strong numerical evidence that the algorithm converges to the optimal policy. Two generalizations of this model are also discussed.     

Monotone optimal replacement policies for a Markovian deteriorating system in a controllable environment

We consider the optimal replacement of a periodically inspected system under Markov deterioration that operates in a controlled environment. Provided are sufficient conditions that characterize an optimal control-limit replacement policy with respect to the system’s condition and its environment. The structure of the optimal policy is illustrated by a numerical example.     

A note on replacement policy for a system subject to non-homogeneous pure birth shocks

A system is subject to shocks that arrive according to a non-homogeneous pure birth process. As shocks occur, the system has two types of failures. Type-I failure (minor failure) is removed by a general repair, whereas type-II failure (catastrophic failure) is removed by an unplanned replacement. The occurrence of the failure type is based on some random mechanism which depends on the number of shocks occurred since the last replacement. Under an age replacement policy, a planned (or scheduled) replacement happens whenever an operating system reaches age T. The aim of this note is to derive the expected cost functions and characterize the structure of the optimal replacement policy for such a general setting. We show that many previous models are special cases of our general model. A numerical example is presented to show the application of the algorithm and several useful insights.     

On an optimal harvesting problem

Kaplan [1972] treated a harvesting problem as a discrete time stochastic control model with independent disturbances and with decreasing mean value increase depending either on the value or on the age of the asset. We consider the more general model where the mean value increase depends on the value and also on the age. As main result we obtain the existence of optimal control-limit policies with respect to three different natural orders in state space. A basic role play additional convexity and boundedness assumptions. Our findings extend and correct the main result of Kaplan. The paper contains further detailed information about the solution.     

Extended optimal age-replacement policy with minimal repair of a system subject to shocks

An operating system is subject to shocks that arrive according to a non-homogeneous Poisson process. As shocks occur the system has two types of failure: type I failure (minor) or type II failure (catastrophic). A generalization of the age replacement policy for such a system is proposed and analyzed in this study. Under such a policy, if an operating system suffers a shock and fails at age y (⩽t), it is either replaced by a new system (type II failure) or it undergoes minimal repair (type I failure). Otherwise, the system is replaced when the first shock after t arrives, or the total operating time reaches age T (0 ⩽ t ⩽ T), whichever occurs first. The occurrence of those two possible actions occurring during the period [0, t] is based on some random mechanism which depends on the number of shocks suffered since the last replacement. The aim of this paper is to find the optimal pair (t¹, T¹) that minimizes the long-run expected cost per unit time of this policy. Various special cases are included, and a numerical example is given.     

On the value of customer information for an independent supplier in a continuous review inventory system

We consider the inventory control problem of an independent supplier in a continuous review system. The supplier faces demand from a single customer who in turn faces Poisson demand and follows a continuous review (R, Q) policy. If no information about the inventory levels at the customer is available, reviews and ordering are usually carried out by the supplier only at points in time when a customer demand occurs. It is common to apply an installation stock reorder point policy. However, as the demand faced by the supplier is not Markovian, this policy can be improved by allowing placement of orders at any point in time. We develop a time delay policy for the supplier, wherein the supplier waits until time t after occurrence of the customer demand to place his next order. If the next customer demand occurs before this time delay, then the supplier places an order immediately. We develop an algorithm to determine the optimal time delay policy. We then evaluate the value of information about the customer’s inventory level. Our numerical study shows that if the supplier were to use the optimal time delay policy instead of the installation stock policy then the value of the customer’s inventory information is not very significant.     

A deteriorating cold standby repairable system with priority in use

In this paper, a cold standby repairable system consisting of two dissimilar components and one repairman is studied. In this system, it is assumed that the working time distributions and the repair time distributions of the two components are both exponential and component 1 is given priority in use. After repair, component 2 is “as good as new” while component 1 follows a geometric process repair. Under these assumptions, using the geometric process and a supplementary variable technique, some important reliability indices such as the system availability, reliability, mean time to first failure (MTTFF), rate of occurrence of failure (ROCOF) and the idle probability of the repairman are derived. A numerical example for the system reliability R(t) is given. And it is considered that a repair-replacement policy based on the working age T of component 1 under which the system is replaced when the working age of component 1 reaches T. Our problem is to determine an optimal policy T^∗ such that the long-run average cost per unit time of the system is minimized. The explicit expression for the long-run average cost per unit time of the system is evaluated, and the corresponding optimal replacement policy T^∗ can be found analytically or numerically. Another numerical example for replacement model is also given.     

Optimal replacement policy for a deteriorating system with increasing repair times

This paper considers an optimal maintenance policy for a practical and reparable deteriorating system subject to random shocks. Modeling the repair time by a geometric process and the failure mechanism by a generalized δ-shock process, we develop an explicit expression of the long-term average cost per time unit for the system under a threshold-type replacement policy. Based on this average cost function, we propose a finite search algorithm to locate the optimal replacement policy N^∗ to minimize the average cost rate. We further prove that the optimal policy N^∗ is unique and present some numerical examples. Many practical systems fit the model developed in this paper.     

Efficiency consideration of generalized age replacement policy

Under the generalized age replacement policy, the system is replaced either at the predetermined age or upon failure if its corresponding repair time exceeds the threshold, whichever comes first. In this paper, we investigate the optimal choice of the pre‐determined preventive replacement age for a nonwarranted system, which minimizes the expected cost rate during the life cycle of the system from the customer's perspective under certain cost structures. Furthermore, we discuss several properties of such a generalized age replacement policy in comparison with the traditional age replacement policy. An efficiency, which represents the fractional time that the system is on, is defined under the proposed generalized age replacement policy and its monotonicity properties are investigated as well. The main objective of this study is to investigate the advantageous features of the generalized age replacement policy over the traditional age replacement policy with regard to the availability of the repairable system. Assuming that the system deteriorates with age, we illustrate our proposed optimal policies numerically and observe the impact of relevant parameters on the optimal preventive replacement age.     

Optimal replacement policies with minimal repair and age-dependent costs

In this paper, we study a modified minimal repair/replacement problem that is formulated as a Markov decision process. The operating cost is assumed to be a nondecreasing function of the system's age. The specific maintenance actions for a manufacturing system to be considered are whether to have replacement, minimal repair or keep it operating. It is shown that a control limit policy, or in particular a (t, T) policy, is optimal over the space of all possible policies under the discounted cost criterion. A computational algorithm for the optimal (t, T) policy is suggested based on the total expected discounted cost.     

An extended optimal replacement model of systems subject to shocks

A system is subject to shocks that arrive according to a non-homogeneous Poisson process. As shocks occur a system has two types of failures: type I failure (minor failure) is rectified by a minimal repair, whereas type II failure (catastrophic failure) is removed by replacement. The probability of a type II failure is permitted to depend on the number of shocks since the last replacement. This paper proposes a generalized replacement policy where a system is replaced at the nth type I failure or first type II failure or at age T, whichever occurs first. The cost of the minimal repair of the system at age t depends on the random part C(t) and deterministic paper c(t). The expected cost rate is obtained. The optimal n¹ and optimal T¹ which would minimize the cost rate are derived and discussed. Various special cases are considered and detailed.     

On optimal operating conditions for a data processing system: A stochastic approach

In this paper, we consider the problem of determining optimal operating conditions for a data processing system. The system is burned‐in for a fixed burn‐in time before it is put into field operation and, in field operation, it has a work size and follows an age‐replacement policy. Assuming that the underlying lifetime distribution of the system has a bathtub‐shaped failure rate function, the properties of optimal burn‐in time, optimal work size and optimal age‐replacement policy will be derived. It can be seen that this model is a generalization of those considered in the previous works, and it yields a better optimal operating conditions. This paper presents an analytical method for three‐dimensional optimization problem. An algorithm for determining optimal operating conditions is also given. Copyright © 2007 John Wiley & Sons, Ltd.     

Models and algorithms for skip-free Markov decision processes on trees

We introduce a class of models for multidimensional control problems that we call skip-free Markov decision processes on trees. We describe and analyse an algorithm applicable to Markov decision processes of this type that are skip-free in the negative direction. Starting with the finite average cost case, we show that the algorithm combines the advantages of both value iteration and policy iteration—it is guaranteed to converge to an optimal policy and optimal value function after a finite number of iterations but the computational effort required for each iteration step is comparable with that for value iteration. We show that the algorithm can also be used to solve discounted cost models and continuous-time models, and that a suitably modified algorithm can be used to solve communicating models.     

Value iteration and optimization of multiclass queueing networks

This paper considers in parallel the scheduling problem for multiclass queueing networks, and optimization of Markov decision processes. It is shown that the value iteration algorithm may perform poorly when the algorithm is not initialized properly. The most typical case where the initial value function is taken to be zero may be a particularly bad choice. In contrast, if the value iteration algorithm is initialized with a stochastic Lyapunov function, then the following hold: (i) a stochastic Lyapunov function exists for each intermediate policy, and hence each policy is regular (a strong stability condition), (ii) intermediate costs converge to the optimal cost, and (iii) any limiting policy is average cost optimal. It is argued that a natural choice for the initial value function is the value function for the associated deterministic control problem based upon a fluid model, or the approximate solution to Poisson’s equation obtained from the LP of Kumar and Meyn. Numerical studies show that either choice may lead to fast convergence to an optimal policy. This revised version was published online in June 2006 with corrections to the Cover Date.     

Joint-optimization of inventory policies on a multi-product multi-echelon pharmaceutical system with batching and ordering constraints

This paper presents a methodology to find near-optimal joint inventory control policies for the real case of a one-warehouse, n-retailer distribution system of infusion solutions at a University Medical Center in France. We consider stochastic demand, batching and order-up-to level policies as well as aspects particular to the healthcare setting such as emergency deliveries, required service level rates and a new constraint on the ordering policy that fits best the hospital’s interests instead of abstract ordering costs. The system is modeled as a Markov chain with an objective to minimize the stock-on-hand value for the overall system. We provide the analytical structure of the model to show that the optimal reorder point of the policy at both echelons is easily derived from a simple probability calculation. We also show that the optimal policy at the care units is to set the order-up-to level one unit higher than the reorder point. We further demonstrate that optimizing the care units in isolation is optimal for the joint multi-echelon, n-retailer problem. A heuristic algorithm is presented to find the near-optimal order-up-to level of the policy of each product at the central pharmacy; all other policy parameters are guaranteed optimal via the structure provided by the model. Comparison of our methodology versus that currently in place at the hospital showed a reduction of approximately 45% in the stock-on-hand value while still respecting the service level requirements.     

Optimization issues in k-out-of-n systems

The k-out-of-n system is a system consisting of n independent components such that the system works if and only if at least k of these n components are successfully running. Each component of the system is subject to shocks which arrive according to a nonhomogeneous Poisson process. When a shock takes place, the component is either minimally repaired (type 1 failure) or lying idle (type 2 failure). Assume that the probability of type 1 failure or type 2 failure depends on age. First, we investigate a general age replacement policy for a k-out-of-n system that incorporates minimal repair, shortage and excess costs. Under such a policy, the system is replaced at age T or at the occurrence of the (n-k + 1)th idle component, whichever occurs first. Moreover, we consider another model; we assume that the system operates some successive projects without interruptions. The replacement could not be performed at age T. In this case, the system is replaced at the completion of the Nth project or at the occurrence of the (n-k + 1)th idle component, whichever occurs first. For each model, we develop the long term expected cost per unit time and theoretically present the corresponding optimum replacement schedule. Finally, we give a numerical example illustrating the models we proposed. The proposed models include more realistic factors and extend many existing models.