Application-driven sequential designs for simulation experiments: Kriging metamodelling

This paper proposes a novel method to select an experimental design for interpolation in simulation. Although the paper focuses on Kriging in deterministic simulation, the method also applies to other types of metamodels (besides Kriging), and to stochastic simulation. The paper focuses on simulations that require much computer time, so it is important to select a design with a small number of observations. The proposed method is therefore sequential. The novelty of the method is that it accounts for the specific input/output function of the particular simulation model at hand; that is, the method is application-driven or customized. This customization is achieved through cross-validation and jackknifing. The new method is tested through two academic applications, which demonstrate that the method indeed gives better results than either sequential designs based on an approximate Kriging prediction variance formula or designs with prefixed sample sizes.     

Controlled sequential factorial design for simulation factor screening

Screening experiments are performed to eliminate unimportant factors efficiently so that the remaining important factors can be studied more thoroughly in later experiments. This paper proposes controlled sequential factorial design (CSFD) for discrete-event simulation experiments. It combines a sequential hypothesis testing procedure with a traditional (fractional) factorial design to control the Type I error and power for each factor under heterogeneous variance conditions. We compare CSFD with other sequential screening methods with similar error control properties. CSFD requires few assumptions and demonstrates robust performance with different system conditions. The method is appropriate for systems with a moderate number of factors and large variances.     

Quadrature Methods for Bayesian Optimal Design of Experiments With Nonnormal Prior Distributions

Many optimal experimental designs depend on one or more unknown model parameters. In such cases, it is common to use Bayesian optimal design procedures to seek designs that perform well over an entire prior distribution of the unknown model parameter(s). Generally, Bayesian optimal design procedures are viewed as computationally intensive. This is because they require numerical integration techniques to approximate the Bayesian optimality criterion at hand. The most common numerical integration technique involves pseudo Monte Carlo draws from the prior distribution(s). For a good approximation of the Bayesian optimality criterion, a large number of pseudo Monte Carlo draws is required. This results in long computation times. As an alternative to the pseudo Monte Carlo approach, we propose using computationally efficient Gaussian quadrature techniques. Since, for normal prior distributions, suitable quadrature techniques have already been used in the context of optimal experimental design, we focus on quadrature techniques for nonnormal prior distributions. Such prior distributions are appropriate for variance components, correlation coefficients, and any other parameters that are strictly positive or have upper and lower bounds. In this article, we demonstrate the added value of the quadrature techniques we advocate by means of the Bayesian D-optimality criterion in the context of split-plot experiments, but we want to stress that the techniques can be applied to other optimality criteria and other types of experimental designs as well. Supplementary materials for this article are available online.     

二维序贯均匀设计方法的拓展研究

将二维序贯均匀设计方法拓展到三维空间,扩展到三维的序贯均匀设计可用于优化的三因子试验设计问题,从而达到既减少试验次数,又提高试验精度的目的.     

Construction of uniform designs without replications

A uniform design scatters its design points evenly on the experimental domain according to some discrepancy measure. In this paper all the design points of a full factorial design can be split into two subdesigns. One is called the complementary design of the other. The complementary design theories of characterizing one design through the other under the four commonly used discrepancy measures are investigated. Based on these complementary design theories, some general rules for searching uniform designs through their complementary designs are proposed. An efficient method to check if a design has repeated points is introduced and a modified threshold-accepting algorithm is proposed to search uniform or nearly uniform designs without replications. The new algorithm is shown to be more efficient by comparing with other existing methods. Many new uniform or nearly uniform designs without replications are tabulated and compared.     

How to choose the simulation model for computer experiments: a local approach

In many scientific areas, non‐stochastic simulation models such as finite element simulations replace real experiments. A common approach is to fit a meta‐model, for example a Gaussian process model, a radial basis function interpolation, or a kernel interpolation, to computer experiments conducted with the simulation model. This article deals with situations where more than one simulation model is available for the same real experiment, with none being the best over all possible input combinations. From fitted models for a real experiment as well as for computer experiments using the different simulation models, a criterion is derived to identify the locally best one. Applying this criterion to a number of design points allows the design space to be split into areas where the individual simulation models are locally superior. An example from sheet metal forming is analyzed, where three different simulation models are available. In this application and many similar problems, the new approach provides valuable assistance with the choice of the simulation model to be used. Copyright © 2011 John Wiley & Sons, Ltd.     

Aggregation and disaggregation in hierarchical production planning

This paper reports on a simulation study of hierarchical planning procedures, which can support a material requirements planning system. Data for this study have been obtained from a Swedish manufacturing company. The three final products considered in the simulations represent a major part of this company.An aggregate plan in terms of product groups and machine groups is derived with the aid of an aggregate model. This plan is then disaggregated by changing order release times obtained from material requirements planning, and by distributing extra capacity among individual machines.The results indicate that our methods in general perform significantly better than a comparable reference case without the supporting hierarchical planning process. In our simulation experiments we evaluate different design features, like disaggregation procedures and methods for aggregating items and machines.     

Multi-objective Geometry Optimization of a Gas Cyclone Using Triple-Fidelity Co-Kriging Surrogate Models

Cyclone separators are widely used in a variety of industrial applications. A low-mass loading gas cyclone is characterized by two performance parameters, namely the Euler and Stokes numbers. These parameters are highly sensitive to the geometrical design parameters defining the cyclone. Optimizing the cyclone geometry therefore is a complex problem. Testing a large number of cyclone geometries is impractical due to time constraints. Experimental data and even computational fluid dynamics simulations are time-consuming to perform, with a single simulation or experiment taking several weeks. Simpler analytical models are therefore often used to expedite the design process. However, this comes at the cost of model accuracy. Existing techniques used for cyclone shape optimization in literature do not take multiple fidelities into account. This work combines cheap-to-evaluate well-known mathematical models of cyclones, available data from computational fluid dynamics simulations and experimental data to build a triple-fidelity recursive co-Kriging model. This model can be used as a surrogate with a multi-objective optimization algorithm to identify a Pareto set of a finite number of solutions. The proposed scheme is applied to optimize the cyclone geometry, parametrized by seven design variables.     

Economical experiments: Bayesian efficient experimental design

We propose and implement a Bayesian optimal design procedure. Our procedure takes as its primitives a class of parametric models of strategic behavior, a class of games (experimental designs), and priors on the behavioral parameters. We select the experimental design that maximizes the information from the experiment. We sequentially sample with the given design and models until only one of the models has viable posterior odds. A model which has low posterior odds in a small collection of models will have an even lower posterior odds when compared to a larger class, and hence we can dismiss it. The procedure can be used sequentially by introducing new models and comparing them to the models that survived earlier rounds of experiments. The emphasis is not on running as many experiments as possible, but rather on choosing experimental designs to distinguish between models in the shortest possible time period. We illustrate this procedure with a simple experimental game with one-sided incomplete information.We acknowledge the financial support from NSF grant #SES-9223701 to the California Institute of Technology. We also acknowledge the research assistance of Eugene Grayver who wrote the software for the experiments.     

混水平均匀设计的构造

我们用离散偏差来度量部分因子设计的均匀性,本文的目的在于寻找一些构造混水平均匀设计的方法,这些方法比文献中已有的方法更简单且计算成本更低.我们得到了离散偏差的一个下界,如果一个U 型设计的离散偏差值达到这个下界,那么该设计是—个均匀设计.我们建立了均匀设计与组合设计理论中一致可分解设计之间的联系.通过一致可分解设计,我们提出了一些构造均匀设计的新方法,同时也给出了许多均匀设计存在的无穷类.     

混料切片试验设计

混料试验设计在众多领域中都有广泛的应用,有时试验者不仅仅需要考虑各混料成分所占比例对响应变量的影响,同时还关心其它被称为过程变量的因素.在实际中,对于这类问题通常使用的设计方案是混料设计和因子设计的组合设计.这种组合设计在过程变量的不同水平组合下,使用的是相同的设计阵,因此空间填充性较差.基于混料球体堆积设计,文章提出了一类新的混料设计,称之为混料切片设计,它的整体设计和所有子设计(过程变量的每一水平组合对应的混料设计)都具有很好的空间填充性,从而比组合设计有更好的模型稳健性.基于同余子群的陪集分解方法,针对过程变量水平组合数的不同情况提出了相应的简单快速的构造算法,文章最后的数值例子解释了算法的可行性和设计的有效性.     

Gradient-based approximation methods applied to the optimal design of vehicle suspension systems using computational models with severe inherent noise

The principal aim of this paper is to evaluate the feasibility of using gradient-based approximation methods for the optimisation of the spring and damper characteristics of an off-road vehicle, for both ride comfort and handling. The Sequential Quadratic Programming algorithm and the relatively new Dynamic-Q method are the two successive approximation methods used for the optimisation. The determination of the objective function value is performed using computationally expensive numerical simulations that exhibit severe inherent numerical noise. The use of forward finite differences and central finite differences for the determination of the gradients of the objective function within Dynamic-Q is also investigated. This is done in investigating methods for overcoming the difficulties associated with the optimisation of noisy objective functions.A recreational off-road vehicle is modelled in ADAMS, and coupled to MATLAB for the execution of the optimisation process. The full vehicle ADAMS model includes suspension kinematics, a load-dependent tyre model, as well as non-linear springs and dampers. Up to four design variables are considered in modelling the suspension characteristics.It is found that both algorithms perform well in optimising handling. However, difficulties are encountered in obtaining improvements in the design process when ride comfort is considered. Nevertheless, meaningful design configurations are still achievable through the proposed optimisation process, at a relatively low cost in terms of the number of simulations that have to be performed.     

Construction of optimal supersaturated designs by the packing method

A supersaturated design is essentially a factorial design with the equal occurrence of levels property and no fully aliased factors in which the number of main effects is greater than the number of runs. It has received much recent interest because of its potential in factor screening experiments. A packing design is an important object in combinatorial design theory. In this paper, a strong link between the two apparently unrelated kinds of designs is shown. Several criteria for comparing supersaturated designs are proposed, their properties and connections with other existing criteria are discussed. A combinatorial approach, called the packing method, for constructing optimal supersaturated designs is presented, and properties of the resulting designs are also investigated. Comparisons between the new designs and other existing designs are given, which show that our construction method and the newly constructed designs have good properties.     

Parameter Identification for Inelastic Constitutive Models

The identification of material parameters of constitutive models is based on identification experiments. Since even specimens from the same lot show high deviations in the experimental data, the identification of the material parameters leads to different results for one and the same material. The number of identification experiments is usually not large enough for a statistical analysis of the deviations in the identified parameters. In order to overcome this problem we present a method of stochastic simulation which is based on time series analysis for generating artificial data with the same stochastic behaviour as the experimental data. The stochastic simulations allow an investigation of the confidence in the fits of the material parameters. We validate the stochastic simulations by comparing the results of the parameter identification from experimental data with the results from artificial data. The presented simulation method applied here turns out to be a suitable tool for generating artificial data for various kinds of analysis purposes. However, it is very important to take into account that the machines which perform the experiments do not maintain constant strain rates in the loading history of the tension and cyclic experiments.     

Improving shock-free compressible RANS solvers for LES on unstructured meshes

Standard numerical methods used to solve the Reynolds averaged Navier–Stokes equations are known to be too dissipative to carry out large eddy simulations since the artificial dissipation they introduce to stabilize the discretization of the convection term usually interacts strongly with the subgrid scale model. A possible solution is to resort to non-dissipative central schemes. Unfortunately, these schemes are in general unstable. A way to reach stability is to select a central scheme that conserves the discrete kinetic energy. To that purpose, a family of kinetic energy conserving schemes is developed to perform simulations of compressible shock-free flows on unstructured grids. A direct numerical simulation of the flow past a sphere at a Reynolds number of 300 and a large eddy simulation at a Reynolds number of 10,000 are performed to validate the methodology.     

Comparing different fractions of a factorial design: a metal cutting case study

Full factorial designs of a significant size are very rarely performed in industry due to the number of trials involved and unavailable time and resources. The data in this paper were obtained from a six‐factor full factorial (2⁶) designed experiment that was conducted to determine the optimum operating conditions for a steel milling operation. Fractional‐factorial designs 2 (one‐eighth) and 2 (one‐fourth, using a fold‐over from the one‐eighth) are compared with the full 2⁶ design. Four of the 2 are de‐aliased by adding four more runs. In addition, two 12‐run Plackett–Burman experiments and their combination into a fold‐over 24‐run experiment are considered. Many of the one‐eighth fractional‐factorial designs reveal some significant effects, but the size of the estimates varies much due to aliasing. Adding four more runs improves the estimation considerably. The one‐quarter fraction designs yield satisfactory results, compared to the full factorial, if the ‘correct’ parameterization is assumed. The Plackett–Burman experiments, estimating all main effects, always perform worse than the equivalent regular designs (which have fewer runs). When considering a reduced model many of the different designs are more or less identical. The paper provides empirical evidence for managers and engineers that the choice of an experimental design is very important and highlights how designs of a minimal size may not always result in productive findings. Copyright © 2006 John Wiley & Sons, Ltd.     

关于离散型试验的最优设计

本文对线性模型中离散型试验的设计阵提出了一致最优与分组E最优的标准,找到了一致最优设计阵存在的充要条件,证明了正交设计与BIB区组设计是分组E最优设计和E最优设计.本文中的方法可以用于非离散型的试验,得到相应的结果.     

关于试验设计的效率及有关问题

效率是试验设计的根本出发点和追求目标,但实际上它是一个抽象和模糊的概念,容易误解和产生偏颇,至今还很少有文献对它专门进行过很好的阐述和研究。常见的各种试验设计方法所追求的效率一般都是单一地在某种数学模型及相应的数学准则下要求实现最优而已。本文将探讨试验设计的基本原理,解释和研究到底什么是试验设计的效率?应该怎样来追求试验设计的效率?并指出现有试验设计学科的一些不足和问题,提出一个改进的试验设计框架和学说。     

Efficient Computation of Bayesian Optimal Discriminating Designs

An efficient algorithm for the determination of Bayesian optimal discriminating designs for competing regression models is developed, where the main focus is on models with general distributional assumptions beyond the “classical” case of normally distributed homoscedastic errors. For this purpose, we consider a Bayesian version of the Kullback–Leibler (KL). Discretizing the prior distribution leads to local KL-optimal discriminating design problems for a large number of competing models. All currently available methods either require a large amount of computation time or fail to calculate the optimal discriminating design, because they can only deal efficiently with a few model comparisons. In this article, we develop a new algorithm for the determination of Bayesian optimal discriminating designs with respect to the Kullback–Leibler criterion. It is demonstrated that the new algorithm is able to calculate the optimal discriminating designs with reasonable accuracy and computational time in situations where all currently available procedures are either slow or fail.