Generalization of simulation results practicality of statistical methods

The major purpose of this paper is to evaluate the practical use of statistical techniques in both the generalization or analysis of simulation results, and the design of simulation experiments. This problem is investigated with the help of a real-life system, namely the container terminus of ECT in Rotterdam. This system is modeled by a simulation program. The relationship between the simulation response and its input variables is modeled by a linear regression model: metamodel or auxiliary model. The paper summarizes regression analysis including generalized least squares which might be used for simulation responses with non-constant variances. The validity of the postulated regression metamodel is tested statistically: F- and t-statistics. The selection of the situations to be simulated, is done through experimental design methodology, permitting both quantitative and qualitative factors. The statistical techniques apply not only to simulation but also to real-life experiments.     

Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping

This paper proposes a novel method to select an experimental design for interpolation in random simulation, especially discrete event simulation. (Though the paper focuses on Kriging, this design approach may also apply to other types of metamodels such as non-linear regression models and splines.) Assuming that simulation requires much computer time, it is important to select a design with a small number of observations (or simulation runs). The proposed method is therefore sequential. Its novelty is that it accounts for the specific input/output behavior (or response function) of the particular simulation at hand; i.e., the method is customized or application-driven. A tool for this customization is bootstrapping, which enables the estimation of the variances of predictions for inputs not yet simulated. The method is tested through two classic simulation models, namely the expected steady-state waiting time of the M/M/1 queuing model, and the mean costs of a terminating (s, S) inventory simulation. For these two simulation models the novel design indeed gives better results than a popular alternative design, namely Latin Hypercube Sampling (LHS) with a prefixed sample.     

Exploring alternative routes to realising the benefits of simulation in healthcare

Discrete event simulation should offer numerous benefits in designing healthcare systems but the reality is often problematic. Healthcare modelling faces particular challenges: genuine, fundamental variations in practice and an opposition to any suggestion of standardisation from some professional groups. This paper compares the experiences of developing a new simulation in an Accident and Emergency (A&E) Department, a subsequent adaptation for modelling an outpatient clinic and applications of a generic A&E simulation. These studies provide examples of three distinct approaches to realising the potential benefits of simulation: the bespoke, the reuse and the generic route. Reuse has many advantages: it is relatively efficient in exploiting previous modelling experience, delivering timely results although providing scope for adaptations to local practice. Explicitly demonstrating this willingness to adapt to local conditions and engaging with stakeholders is particularly important in healthcare simulation.     

Validation,verification, and testing techniques throughout the life cycle of a simulation study

Life cycle validation, verification, and testing (VV&T) is extremely important for the success of a simulation study. This paper surveys current software VV&T techniques and current simulation model VV&T techniques and describes how they can all be applied throughout the life cycle of a simulation study. The processes and credibility assessment stages of the life cycle are described and the applicability of the VV&T techniques for each stage is stated. A glossary is provided to explicitly define important terms and VV&T techniques.     

Applications of simulation within the healthcare context

A large number of studies have applied simulation to a multitude of issues relating to healthcare. These studies have been published in a number of unrelated publishing outlets, which may hamper the widespread reference and use of such resources. In this paper, we analyse existing research in healthcare simulation in order to categorise and synthesise it in a meaningful manner. Hence, the aim of this paper is to conduct a review of the literature pertaining to simulation research within healthcare in order to ascertain its current development. A review of approximately 250 high-quality journal papers published between 1970 and 2007 on healthcare-related simulation research was conducted. The results present a classification of the healthcare publications according to the simulation techniques they employ; the impact of published literature in healthcare simulation; a report on demonstration and implementation of the studies’ results; the sources of funding; and the software used. Healthcare planners and researchers will benefit from this study by having ready access to an indicative article collection of simulation techniques applied to healthcare problems that are clustered under meaningful headings. This study facilitates the understanding of the potential of different simulation techniques in solving diverse healthcare problems.     

A Trajectory for Validating Computational Emulation Models of Organizations

Validation of complex simulation models is a challenging problem in computational organization theory research. In this paper, we describe a validation strategy suitable for emulation simulation systems, and show how a comprehensive validation consists of a sequence of steps that evaluate different aspects of a computational organizational simulation model. We demonstrate how this strategy can be applied to the evaluation of the Virtual Team Alliance (VTA), an emulation simulation system designed to guide managers charged with organizational change. VTA required a &201C;trajectory&201D; of successive validation experiments, before managers where willing to follow the recommendations of VTA. Ultimately, we believe this validation approach can be applied to a wide range of different simulation systems, and will make identification of the strengths and weaknesses of organizational simulations easier.     

A simulation optimization method that considers uncertainty and multiple performance measures

Simulation optimization provides a structured approach to system design and configuration when analytical expressions for input/output relationships are unavailable. This research focuses on the development of a new simulation optimization technique applicable to systems having multiple performance measures. The aim of this research is to incorporate a simulation end user’s preference towards risk and uncertainty into the search process for the best decision alternative. Automation of the optimization procedure is a necessity. Therefore, this paper proposes a simulation optimization method that involves a preference model, specifically adapted for decision making with simulation models.     

Applying simulation optimization to the asset allocation of a property–casualty insurer

Proper asset allocations are vital for property–casualty insurers to be competitive and solvent. Theories of finance offer little practical guidance in constructing such asset allocations however. This research integrates simulation models with a newly developed evolutionary algorithm for the multi-period asset allocation problem of a property–casualty insurer. We first construct a simulation model to simulate operations of a property–casualty insurer. Then we develop multi-phase evolution strategies (MPES) to be used with the simulation model to search for promising asset allocations for the insurer. A thorough experiment is conducted to evaluate the performance of our simulation optimization approach. Computational results show that MPES is an effective search algorithm. It dominates the grid search method by a significant margin. The re-allocation strategy resulting from MPES outperforms re-balancing strategies significantly. This research further demonstrates that the simulation optimization approach can be used to study economic issues related to multi-period asset allocation problems in practical settings.     

Quantile regression metamodeling: Toward improved responsiveness in the high-tech electronics manufacturing industry

Both technology and market demands within the high-tech electronics manufacturing industry change rapidly. Accurate and efficient estimation of cycle-time (CT) distribution remains a critical driver of on-time delivery and associated customer satisfaction metrics in these complex manufacturing systems. Simulation models are often used to emulate these systems in order to estimate parameters of the CT distribution. However, execution time of such simulation models can be excessively long limiting the number of simulation runs that can be executed for quantifying the impact of potential future operational changes. One solution is the use of simulation metamodeling which is to build a closed-form mathematical expression to approximate the input–output relationship implied by the simulation model based on simulation experiments run at selected design points in advance. Metamodels can be easily evaluated in a spreadsheet environment “on demand” to answer what-if questions without needing to run lengthy simulations. The majority of previous simulation metamodeling approaches have focused on estimating mean CT as a function of a single input variable (i.e., throughput). In this paper, we demonstrate the feasibility of a quantile regression based metamodeling approach. This method allows estimation of CT quantiles as a function of multiple input variables (e.g., throughput, product mix, and various distributional parameters of time-between-failures, repair time, setup time, loading and unloading times). Empirical results are provided to demonstrate the efficacy of the approach in a realistic simulation model representative of a semiconductor manufacturing system.     

Some Examples of the Use of Simulation in U.S. and Canadian Agriculture

Mathematical simulation is a technique widely used by agricultural engineers to determine systems interactions, to evaluate technical feasibility, to determine economic feasibility, to design complex structures and to help with managing production. In the past, simulation has been used one level removed from the farmers who manage operation on a day-to-day basis. This is changing.Three completely different types of simulations are presented: (1) a static and deterministic simulation of on-farm ethanol production that models the conversion process and determines economic feasibility; (2) a dynamic and deterministic simulation of maize production, including harvest and drying that is weather dependent; (3) a static and stochastic simulation that uses Monte Carlo techniques to obtain information about strength and performance of glued-laminated beams.We believe that inexpensive computing capability will tremendously increase the use of simulation models, such as the three described, that will help farmers better manage their production systems.     

How to choose the simulation model for computer experiments: a local approach

In many scientific areas, non‐stochastic simulation models such as finite element simulations replace real experiments. A common approach is to fit a meta‐model, for example a Gaussian process model, a radial basis function interpolation, or a kernel interpolation, to computer experiments conducted with the simulation model. This article deals with situations where more than one simulation model is available for the same real experiment, with none being the best over all possible input combinations. From fitted models for a real experiment as well as for computer experiments using the different simulation models, a criterion is derived to identify the locally best one. Applying this criterion to a number of design points allows the design space to be split into areas where the individual simulation models are locally superior. An example from sheet metal forming is analyzed, where three different simulation models are available. In this application and many similar problems, the new approach provides valuable assistance with the choice of the simulation model to be used. Copyright © 2011 John Wiley & Sons, Ltd.     

Mean-absolute deviation portfolio optimization for mortgage-backed securities

We develop an integrated simulation/optimization model for managing portfolios of mortgage-backed securities. The mortgage portfolio problem is viewed in the same spirit of models used for the management of portfolios of equities. That is, it trades off rates of return with a suitable measure of risk. In this respect we employ amean-absolute deviation model which is consistent with the asymmetric distribution of returns of mortgage securities and derivative products. We develop a simulation procedure to compute holding period returns of the mortgage securities under a range of interest rate scenarios. The simulation explicitly takes into account the stylized facts of mortgage securities: the propensity of homeowners to prepay their mortgages, and theoption adjusted premia associated with these securities. Details of both the simulation and optimization models are presented. The model is then applied to the funding of a typical insurance liability stream, and it is shown to generate superior results than the standardportfolio immunization approach.     

Facilitated modelling with discrete-event simulation: Reality or myth?

Is it possible for discrete-event simulation to be used in a facilitated workshop environment? Over the last decade there have been various attempts to use simulation in this way, but we argue here that none have been successful in achieving a fully facilitated mode where the model is both developed and used in the workshop. We attempt to use a discrete-event simulation in a facilitated mode as part of a lean improvement workshop in a hospital setting. The model was successfully developed and used within the 3 day period of the workshop. Although the intervention was successful, we still had to build the model in the ‘back-office’, meaning that a fully facilitated mode was not achieved. The paper concludes by discussing how fully facilitated modelling with discrete-event simulation might be made possible; the answer is more about changing mind-sets than about technological challenge.     

A philosophy for the modelling of realistic nonlinear systems

A nonlinear dynamical system is modelled as a nonlinear mapping from a set of input signals into a corresponding set of output signals. Each signal is specified by a set of real number parameters, but such sets may be uncountably infinite. For numerical simulation of the system each signal must be represented by a finite parameter set and the mapping must be defined by a finite arithmetical process. Nevertheless the numerical simulation should be a good approximation to the mathematical model. We discuss the representation of realistic dynamical systems and establish a stable approximation theorem for numerical simulation of such systems.

     

Key performance indicators for successful simulation projects

There are many factors that may contribute to the successful delivery of a simulation project. To provide a structured approach to assessing the impact various factors have on project success, we propose a top-down framework whereby 15 Key Performance Indicators (KPI) are developed that represent the level of successfulness of simulation projects from various perspectives. They are linked to a set of Critical Success Factors (CSF) as reported in the simulation literature. A single measure called Project’s Success Measure (PSM), which represents the project’s total success level, is proposed. The framework is tested against 9 simulation exemplar cases in healthcare and this provides support for its reliability. The results suggest that responsiveness to the customer’s needs and expectations, when compared with other factors, holds the strongest association with the overall success of simulation projects. The findings highlight some patterns about the significance of individual CSFs, and how the KPIs are used to identify problem areas in simulation projects.     

Extended dynamic partial-overlapping batch means estimators for steady-state simulations

Estimating the variance of the sample mean from a stochastic process is essential in assessing the quality of using the sample mean to estimate the population mean, which is the fundamental question in simulation experiments. Most existing studies for estimating the variance of the sample mean from simulation output assume that the simulation run length is known in advance. An interesting and open question is how to estimate the variance of the sample mean with limited memory space, reasonable computation time, and good statistical properties such as small mean-squared-error (mse), without knowing the simulation run length a priori. This paper proposes a finite-memory algorithm that satisfies the above good estimation criteria. Our findings show that the proposed algorithm improves over its competitors in terms of the mse criterion.     

Context in social simulation: why it can’t be wished away

Context is everywhere in the human social and cognitive spheres but it is often implicit and unnoticed. However, when one is involved in trying to understand and model the social and cognitive realms context becomes an important factor. This paper is an analysis of the role and effects of context on social simulation and a call for it to be squarely faced by the social simulation community. It briefly looks at some different kinds of context, and discussed the difficulty of talking about context, before looking at the “context heuristic” that seems to be used in human cognition. This allows for rich and fuzzy context recognition to be combined with crisp ‘foreground’ belief update and reasoning. Such a heuristic allows for causality to make sense, and limits the phenomena of causal spread—it is thus at the root of the modelling enterprise. This analysis is then applied to simulation modelling, considering the context of a simulation, and its ramifications, in particular, why generalisation is so hard.     

Run length not required: Optimal-mse dynamic batch means estimators for steady-state simulations

This paper addresses the estimation of the variance of the sample mean from steady-state simulations without requiring the knowledge of simulation run length a priori. Dynamic batch means is a new and useful approach to implementing the traditional batch means in limited memory without the knowledge of the simulation run length. However, existing dynamic batch means estimators do not allow one to control the value of batch size, which is the performance parameter of the batch means estimators. In this work, an algorithm is proposed based on two dynamic batch means estimators to dynamically estimate the optimal batch size as the simulation runs. The simulation results show that the proposed algorithm requires reasonable computation time and possesses good statistical properties such as small mean-squared-error (mse).     

Simulation and the Micro

Simulation is established as a proven and effective aid to decision-making in business and industry, providing a means of investigation where to experiment on the `real world' would be costly or disruptive. This paper first traces the pioneering developments carried out in simulation by a B.S.C. O.R. team, in particular in:

• - simulation packages, making model construction sounder and easier,
• - the use of visual displays to communicate model behaviour as a simulation run progresses,
• - interactive simulation (`gaming'), involving operational personnel, such as line management and planners, in the simulation process.

The paper goes on to describe how the advent of the microprocessor, in desk-top computers and colour graphics, has enabled more power and flexibility to be brought to these areas. It also discusses the further benefits arising from the use of micros in simulation and identifies for comparison the benefits of using a mainframe or mini-computer. The paper also describes how the early development areas have culminated in the FORSSIGHT computer-aided decision-making package.