A discrete time optimal control model with uncertainty for dynamic machine allocation problem and its application to manufacturing and construction industries

This paper proposed a discrete time optimal control model in which machine failure time is modeled assuming a Weibull distribution and machine productivity is regarded as a fuzzy variable for dealing with a dynamic machine allocation problem (DMAP) in manufacturing and construction industries. The aim is to maximize total production or construction throughput when uncertainties such as machine breakdowns are taken into account. A failure probability-work time equation is presented to describe the relationship between machine failure probability and mean time to work. To transform the uncertain optimal control model into a deterministic one, the expected value model (EVM) was introduced for forming an equivalent crisp model. The fuzzy variables in the model are also defuzzified by using an expected value operator with an optimistic–pessimistic index. Then a number of lemmas and theorems are presented and proved to formulate the theoretical algorithm so that the crisp model of the DMAP can be solved. Three actual construction and production projects are used as practical application examples. The theoretical algorithm results for the three project examples are compared with a particle swarm optimization approach and a genetic algorithm method, which demonstrates the practicality and efficiency of our optimization method.     

基于改进灰色GM(1,1)马尔柯夫模型的煤矿事故百万吨死亡率预测

煤矿安全事故预防和控制是煤矿安全评价和决策的基础.灰色预测适合于时间短、数据量少和波动不大的系统对象,而马尔可夫链理论适用于预测随机波动大的动态过程.结合灰色预测GM(1,1)模型和马尔可夫链理论的优点,提出了一种改进的灰色马尔可夫GM(1,1)模型.利用改进的GM(1,1)模型进一步拟合煤矿人因失误事故的发展变化趋势,并以此为基础进行马尔柯夫预测,提高预测效果.以2000-2010年全国煤矿事故百万吨死亡率为例进行了预测分析,结果表明模型既能揭示煤矿人因失误事故百万吨死亡率变化的总体趋势,又能克服随机波动性数据对预测精度的影响,具有较强的工程实用性,并对煤矿人因失误安全事故的预测和控制有一定的实际意义.     

Quantum Bayesian computation

Quantum Bayesian computation is an emerging field that levers the computational gains available from quantum computers. They promise to provide an exponential speed-up in Bayesian computation. Our article adds to the literature in three ways. First, we describe how quantum von Neumann measurement provides quantum versions of popular machine learning algorithms such as Markov chain Monte Carlo and deep learning that are fundamental to Bayesian learning. Second, we describe quantum data encoding methods needed to implement quantum machine learning including the counterparts to traditional feature extraction and kernel embeddings methods. Third, we show how quantum algorithms naturally calculate Bayesian quantities of interest such as posterior distributions and marginal likelihoods. Our goal then is to show how quantum algorithms solve statistical machine learning problems. On the theoretical side, we provide quantum versions of high dimensional regression, Gaussian processes and stochastic gradient descent. On the empirical side, we apply a quantum FFT algorithm to Chicago house price data. Finally, we conclude with directions for future research.     

Nearly-Optimal Asset Allocation in Hybrid Stock Investment Models

This work develops a class of stock-investment models that are hybrid in nature and involve continuous dynamics and discrete-event interventions. In lieu of the usual geometric Brownian motion formulation, hybrid geometric Brownian motion models are proposed, in which both the expected return and the volatility depend on a finite-state Markov chain. Our objective is to find nearly-optimal asset allocation strategies so as to maximize the expected returns. The use of the Markov chain stems from the motivation of capturing the market trends as well as various economic factors. To incorporate these economic factors into the models, the underlying Markov chain inevitably has a large state space. To reduce the complexity, a hierarchical approach is suggested, which leads to singularly-perturbed switching diffusion processes. By aggregating the states of the Markov chains in each weakly irreducible class into a single state, limit switching diffusion processes are obtained. Using such asymptotic properties, nearly-optimal asset allocation policies are developed.     

Accelerated modified policy iteration algorithms for Markov decision processes

We propose a new approach to accelerate the convergence of the modified policy iteration method for Markov decision processes with the total expected discounted reward. In the new policy iteration an additional operator is applied to the iterate generated by Markov operator, resulting in a bigger improvement in each iteration.     

A contribution to the stochastic flow shop scheduling problem

This paper deals with performance evaluation and scheduling problems in m machine stochastic flow shop with unlimited buffers. The processing time of each job on each machine is a random variable exponentially distributed with a known rate. We consider permutation flow shop. The objective is to find a job schedule which minimizes the expected makespan. A classification of works about stochastic flow shop with random processing times is first given. In order to solve the performance evaluation problem, we propose a recursive algorithm based on a Markov chain to compute the expected makespan and a discrete event simulation model to evaluate the expected makespan. The recursive algorithm is a generalization of a method proposed in the literature for the two machine flow shop problem to the m machine flow shop problem with unlimited buffers. In deterministic context, heuristics (like CDS [Management Science 16 (10) (1970) B630] and Rapid Access [Management Science 23 (11) (1977) 1174]) and metaheuristics (like simulated annealing) provide good results. We propose to adapt and to test this kind of methods for the stochastic scheduling problem. Combinations between heuristics or metaheuristics and the performance evaluation models are proposed. One of the objectives of this paper is to compare the methods together. Our methods are tested on problems from the OR-Library and give good results: for the two machine problems, we obtain the optimal solution and for the m machine problems, the methods are mutually validated.     

A periodic review production and maintenance model with random demand,deteriorating equipment,and binomial yield

In many environments, product yield is heavily influenced by equipment condition. Despite this fact, previous research has either focused on the issue of maintenance, ignoring the effect of equipment condition on yield, or has focused on the issue of production, omitting the possibility of actively changing the machine state. We formulate a Markov decision process model of a single-stage production system in which demand is random. The product yield has a binomial distribution that depends on the equipment condition, which deteriorates over time. The objective is to choose simultaneously the equipment maintenance schedule as well as the quantity to produce in a way that minimizes the sum of expected production, backorder, and holding costs. After proving some results about the structural properties of the optimal policy, numerical problems are used to compare this method to the typical approach of solving the maintenance and production problems sequentially. The results show that the simultaneous solution provides substantial gains over the sequential approach. In the cases studied, the proposed method resulted in an average cost savings of approximately 18%.     

Single-machine time-dependent scheduling problems with fixed rate-modifying activities and resumable jobs

In this paper, we deal with single machine scheduling problems subject to time dependent effects. The main point in our models is that we do not assume a constant processing rate during job processing time. Rather, processing rate changes according to a fixed schedule of activities, such as replacing a human operator by a less skilled operator. The contribution of this paper is threefold. First, we devise a time-dependent piecewise constant processing rate model and show how to compute processing time for a resumable job. Second, we prove that any time-dependent continuous piecewise linear processing time model can be generated by the proposed rate model. Finally, we propose polynomial-time algorithms for some single machine problems with job independent rate function. In these procedures the job-independent rate effect does not imply any restriction on the number of breakpoints for the corresponding continuous piecewise linear processing time model. This is a clear element of novelty with respect to the polynomial-time algorithms proposed in previous contributions for time-dependent scheduling problems.     

Stability and optimality of a multi-product production and storage system under demand uncertainty

This work develops a discrete event model for a multi-product multi-stage production and storage (P&S) problem subject to random demand. The intervention problem consists of three types of possible decisions made at the end of one stage, which depend on the observed demand (or lack of) for each item: (i) to proceed further with the production of the same product, (ii) to proceed with the production of another product or (iii) to halt the production. The intervention problem is formulated in terms of dynamic programming (DP) operators and each possible solution induces an homogeneous Markov chain that characterizes the dynamics. However, solving directly the DP problem is not a viable task in situations involving a moderately large number of products with many production stages, and the idea of the paper is to detach from strict optimality with monitored precision, and rely on stability. The notion of stochastic stability brought to bear requires a finite set of positive recurrent states and the paper derives necessary and sufficient conditions for a policy to induce such a set in the studied P&S problem. An approximate value iteration algorithm is proposed, which applies to the broader class of control problems described by homogeneous Markov chains that satisfy a structural condition pointed out in the paper. This procedure iterates in a finite subset of the state space, circumventing the computational burden of standard dynamic programming. To benchmark the approach, the proposed algorithm is applied to a simple two-product P&S system.     

Service level robustness in stochastic production planning under random machine breakdowns

In this paper, we consider a multi-period, multi-product production planning problem where the production rate and the customer service level are random variables due to machine breakdowns. In order to determine robust production plans, constraints are introduced in the stochastic capacitated lot-sizing problem to ensure that a pre-specified customer service level is met with high probability. The probability of meeting a service level is evaluated by using the first passage time theory of a Wiener process to a boundary. A two-step optimization approach is proposed to solve the developed model. In the first step, the mean-value deterministic model is solved. Then, a method is proposed in the second step to improve the probability of meeting service level. The resulting approach has the advantage of not being a scenario-based one. It is shown that substantial improvements in service level robustness are often possible with minimal increases in expected cost.     

Multi-objective open shop scheduling by considering human error and preventive maintenance

This study presents an open shop scheduling model by considering human error and preventive maintenance. The proposed mathematical model takes into account conflicting objective functions including makespan, human error and machine availability. In order to find the optimum scheduling, human error, maintenance and production factors are considered, simultaneously. Human error is measured by Human Error Assessment and Reduction Technique (HEART). Three metaheuristic methods including non-dominated sorting genetic algorithm-II (NSGA-II), multi-objective particle swarm optimization (MOPSO) and strength Pareto evolutionary algorithm II (SPEA-II) are developed to find near-optimal solution. The Taguchi method is applied by adjusting parameters of metaheuristic algorithms. Several illustrative examples and a real case study (auto spare parts manufacturer) are applied to show the applicability of the multi-objective mixed integer nonlinear programming model. The proposed approach of this study may be used for similar open shop problems with minor modifications.     

Estimating Hidden Semi-Markov Chains From Discrete Sequences

This article addresses the estimation of hidden semi-Markov chains from nonstationary discrete sequences. Hidden semi-Markov chains are particularly useful to model the succession of homogeneous zones or segments along sequences. A discrete hidden semi-Markov chain is composed of a nonobservable state process, which is a semi-Markov chain, and a discrete output process. Hidden semi-Markov chains generalize hidden Markov chains and enable the modeling of various durational structures. From an algorithmic point of view, a new forward-backward algorithm is proposed whose complexity is similar to that of the Viterbi algorithm in terms of sequence length (quadratic in the worst case in time and linear in space). This opens the way to the maximum likelihood estimation of hidden semi-Markov chains from long sequences. This statistical modeling approach is illustrated by the analysis of branching and flowering patterns in plants.     

Integration of the human operator into responsive discrete production management systems

The need for production systems that can react or respond to dynamic changes is continuously increasing because of the reduction of product life cycle time and the rise of competition. To improve responsiveness, we show that integrating the intelligence of the human operator into the system helps to face complexity. However, little work has been done about the optimization of such integration considering production system constraints (real time decision making, observability, etc.) and human operator constraints (mental workload, trust in management system, self-confidence, etc.). This paper aims at discussing ways to take account of the cognitive abilities of the human operator and offers some advice on how to take accurately into account the integration of the human operator by the proposal of a set of global specifications. To illustrate how it is possible to contribute to the optimized design of a system based upon such specifications, we propose the concept of “distributed production management system”. We first address the specific interface issue. A classical example of “advanced display” designed for continuous systems, that is, the Ecological Interface Design (EID) approach, is applied to discrete production systems. We show that such an approach is coherent with parts of the introduced specifications but can be adapted to large complex and discrete systems with difficulty. To solve this issue, we propose the development of a distributed DSS where each local DSS integrates an advanced display and manages a set of production resources in cooperation with an operator, which reduces the global complexity.     

Stochastic analysis of machine interference of a production system wit[hdot] identical machines

This paper examines the transient and steady–state interference characteristics of a production system with one operator and n identical, semi–automatic and reliable machines. A stochastic model is developed to describe the system and analytical expressions for the percentage interference and mean output rate permachine with arbitrary distribution of concurrent time and exponential distribution of procssing time of each machine, have been obtained by using a state–space method and the regeneration point technique. A particular case is investigated and numerical results are presented illustrating some features of this machine inter ference problem     

A perspective on artificial intelligence: Learning to learn

The classical approach to the acquisition of knowledge in artificial intelligence has been to program the intelligence into the machine in the form of specific rules for the application of the knowledge: expert systems. Unfortunately, the amount of time and resources required to program an expert system with sufficient knowledge for non-trivial problem-solving is prohibitively large. An alternative approach is to allow the machine tolearn the rules based upon trial-and-error interaction with the environment, much as humans do. This will require endowing the machine with a sophisticated set of sensors for the perception of the external world, the ability to generate trial actions based upon this perceived information, and a dynamic evaluation policy to allow it to measure the effectiveness of its trial actions and modify its repertoire accordingly. The principles underlying this paradigm, known ascollective learning systems theory, have already been applied to sophisticated gaming problems, demonstrating robust learning and dynamic adaptivity.The fundamental building block of a collective learning system is thelearning cell, which may be embedded in a massively parallel, hierarchical data communications network. Such a network comprising 100 million learning cells will approach the intelligence capacity of the human cortex. In the not-too-distant future, it may be possible to build a race of robotic slaves to perform a wide variety of tasks in our culture. This goal, while irresistibly attractive, is most certainly fraught with severe social, political, moral, and economic difficulties.This paper was given as an invited talk on the 12th Symposium on Operations Research, University of Passau, September 1987.     

Bayesian estimation and inference for log-ACD models

This paper adapts Bayesian Markov chain Monte Carlo methods for application to some auto-regressive conditional duration models. Subsequently, the properties of these estimators are examined and assessed across a range of possible conditional error distributions and dynamic specifications, including under error mis-specification. A novel model error distribution, employing a truncated skewed Student-t distribution is proposed and the Bayesian estimator assessed for it. The results of an extensive simulation study reveal that favourable estimation properties are achieved under a range of possible error distributions, but that the generalised gamma distribution assumption is most robust and best preserves these properties, including when it is incorrectly specified. The results indicate that the powerful numerical methods underlying the Bayesian estimator allow more efficiency than the (quasi-) maximum likelihood estimator for the cases considered.     

带有缓冲器串行生产线的Harris链结构分析

本文以随机过程中的一类特殊的Ｍａｒｋｏｖ链（Ｈａｒｒｉｓ常返Ｍａｒｋｏｖ链）为工具，研究离散事件动态系统（ＤＥＤＳ）中的典型情况之一：带有缓冲器的串行生产线．求得了各缓冲器中产品数的联合稳态分布，产品在各台机器上受阻时间的联合稳态分布，以及受阻时间的强大数定律和产品在各台机器加工完时刻的极限行为．     

Asymptotically optimal production policies in dynamic stochastic jobshops with limited buffers

We consider a production planning problem for a jobshop with unreliable machines producing a number of products. There are upper and lower bounds on intermediate parts and an upper bound on finished parts. The machine capacities are modelled as finite state Markov chains. The objective is to choose the rate of production so as to minimize the total discounted cost of inventory and production. Finding an optimal control policy for this problem is difficult. Instead, we derive an asymptotic approximation by letting the rates of change of the machine states approach infinity. The asymptotic analysis leads to a limiting problem in which the stochastic machine capacities are replaced by their equilibrium mean capacities. The value function for the original problem is shown to converge to the value function of the limiting problem. The convergence rate of the value function together with the error estimate for the constructed asymptotic optimal production policies are established.     

Modeling and SPM-dependent control of multi-rate networked control system with long time delay

In this paper, an efficient approach of modeling and control is presented for Multi-Rate Networked Control System (MRNCS) with considering long time delay. Firstly, the system is modeled as a switched system with a random switching signal which is subject to random networked-induced delay. For this, time delay is defined as a Markov chain and the model of MRNCS is obtained as a Markovian jump linear system. Afterward, a dynamic output feedback controller is designed for output tracking as well as stabilization of closed-loop system. The modeling and control of MRNCS are presented for two structures. At first, a new model of single-side MRNCS is proposed and a mode-independent controller is designed for stabilizing the system. Then the proposed modeling method is generalized to double-side MRNCS and by introducing the Set of Possible Modes (SPM) concept, an SPM-dependent controller is proposed for double-side MRNCS. To show the effectiveness of the proposed methods, some numerical results are provided on the quadruple-tank process.     

Spanning Forests of Digraphs and Limiting Probabilities of Markov Chains

A maximum out forest of a digraph is its spanning subgraph that consists of disjoint diverging trees and has the maximum possible number of arcs. For an arbitrary weighted digraph, we consider a matrix of specific weights of maximum out forests and demonstrate how this matrix can be used to get a graph-theoretic interpretation for the limiting probabilities of Markov chains. For a special (nonclassical) correspondence between Markov chains and weighted digraphs, the matrix of Cesáro limiting transition probabilities of any finite homogeneous Markov chain coincides with the normalized matrix of maximum out forests of the corresponding digraphs. This provides a finite (combinatorial) method to calculate the limiting probabilities of Markov chains and thus their stationary distributions. On the other hand, the Markov chain technique provides the proofs to some statements about digraphs.