Notes on equivalent stationary policies in Markov decision processes with total rewards

We construct examples of Markov Decision Processes for which, for a given initial state and for a given nonstationary transient policy, there is no equivalent (randomized) stationary policy, i.e. there is no stationary policy which occupation measure is equal to the occupation measure of a given policy. We also investigate the relation between the existence of equivalent stationary policies in special models and the existence of equivalent strategies in various classes of nonstationary policies in general models.     

Extended optimal inspection policies for a system in storage

A system such as missiles and spare parts of aircraft has to perform a normal operation in a severe environment at any time when it is used. However, the system is in storage for a long time from the delivery to the usage and its reliability goes down with time. Thus, a system in storage should be inspected and maintained at periodic times to hold a higher reliability than is prespecified.The following inspection model is considered: A system has three types of units, where unit 1 is maintained, unit 21 is not maintained but is replaced and unit 22 is neither maintained nor replaced. The system is overhauled if its reliability becomes lower than a prespecified probability. The number of replacements and time until overhaul are derived. Using these results, the average cost is obtained and both an optimal inspection time and an optimal replacement time to minimize it are numerically discussed.     

Warranty cost analysis for products with a dormant state

Most building services products are installed while a building is constructed, but they are not operated until the building is commissioned. The warranty of the products may cover the time starting from their installation to the end of the warranty period. Prior to the commissioning of the building, the products are at a dormant mode (i.e., not operated) but protected by the warranty. For such products, both the usage intensity and the failure patterns are different from those with continuous usage intensity and failure patterns. This paper develops warranty cost models for repairable products with a dormant mode from both the manufacturer’s and buyer’s perspectives. Relationships between the failure patterns at the dormant mode and at the operational mode are also discussed. Numerical examples and sensitivity analysis are used to demonstrate the applicability of the methodology derived in the paper.     

On the optimal control of a two-queue polling model

This paper deals with two M/M/1 queues served by a single server with threshold switching. Our main goal is to solve the Poisson equation and, as a result, give expressions for the long-run expected average cost of holding units and switching actions of the server, and the bias vector.     

Asymptotically optimal production policies in dynamic stochastic jobshops with limited buffers

We consider a production planning problem for a jobshop with unreliable machines producing a number of products. There are upper and lower bounds on intermediate parts and an upper bound on finished parts. The machine capacities are modelled as finite state Markov chains. The objective is to choose the rate of production so as to minimize the total discounted cost of inventory and production. Finding an optimal control policy for this problem is difficult. Instead, we derive an asymptotic approximation by letting the rates of change of the machine states approach infinity. The asymptotic analysis leads to a limiting problem in which the stochastic machine capacities are replaced by their equilibrium mean capacities. The value function for the original problem is shown to converge to the value function of the limiting problem. The convergence rate of the value function together with the error estimate for the constructed asymptotic optimal production policies are established.     

Asymptotic normality of discounted random series with applications in reliability and queueing

It is proved that a certain kind of randomly discounted random sums is asymptotically normal as the discount constant tends to zero. For replaceable systems with random lifetime, these sums represent the total discounted cost of policies of the age-replacement type; other applications to queueing and related areas are also indicated.     

基于实物期权的油气开采许可证策略分析

石油与天然气作为我国最重要的矿产资源,其矿权管理采用许可证制度.基于我国的石油与天然气许可证体系结构,分析了石油天然气勘查与开采许可证的实物期权特性,采用实物期权思想对油气勘探开发项目进行战略经济评价,针对传统方法在颁发开采许可证时忽略了时间灵活性价值这一问题,提出了估算颁发开采许可证价值和对投资开发时机进行选择的办法.为资源管理部门和石油公司的决策提供科学依据.最后,应用实物期权法对一个算例进行了分析.     

A ratio‐dependent predator–prey model with stage structure and optimal harvesting policy

In this paper, a ratio‐dependent predator–prey model with stage structure and harvesting is investigated. Mathematical analyses of the model equations with regard to boundedness of solutions, nature of equilibria, permanence and stability are performed. By constructing appropriate Lyapunov functions, a set of easily verifiable sufficient conditions are obtained for the global asymptotic stability of nonnegative equilibria of the model. The existence possibilities of bioeconomic equilibria have been examined. An optimal harvesting policy is also given by using Pontryagin's maximal principle. Copyright © 2007 John Wiley & Sons, Ltd.     

An economic model for investing in sulfur oxide control technology development

The model estimates, subject to restrictions, the functional relationship between national sulfur dioxide discharges and investment in control technology development. Estimates may be made for different product demands, allowed cost increases, and clean fuel availabilities. Control technology development may be accelerated by increased investment. Both optimal and sub-optimal solutions are calculated for an example problem. A general computing method is provided.The model was presented at the Industrial Process Design for Pollution Control Workshop, Chicago, Illinois, Oct. 1973; copyright released by the American Institute of Chemical Engineers for publication for the first time in the monographNormative Analysis in Policy Decisions: Public and Private.