Robust optimization using computer experiments

During metamodel-based optimization three types of implicit errors are typically made. The first error is the simulation-model error, which is defined by the difference between reality and the computer model. The second error is the metamodel error, which is defined by the difference between the computer model and the metamodel. The third is the implementation error. This paper presents new ideas on how to cope with these errors during optimization, in such a way that the final solution is robust with respect to these errors. We apply the robust counterpart theory of Ben-Tal and Nemirovsky to the most frequently used metamodels: linear regression and Kriging models. The methods proposed are applied to the design of two parts of the TV tube. The simulation-model errors receive little attention in the literature, while in practice these errors may have a significant impact due to propagation of such errors.     

On estimation of surrogate models for multivariate computer experiments

Estimation of surrogate models for computer experiments leads to nonparametric regression estimation problems without noise in the dependent variable. In this paper, we propose an empirical maximal deviation minimization principle to construct estimates in this context and analyze the rate of convergence of corresponding quantile estimates. As an application, we consider estimation of computer experiments with moderately high dimension by neural networks and show that here we can circumvent the so-called curse of dimensionality by imposing rather general assumptions on the structure of the regression function. The estimates are illustrated by applying them to simulated data and to a simulation model in mechanical engineering.

     

Application-driven sequential designs for simulation experiments: Kriging metamodelling

This paper proposes a novel method to select an experimental design for interpolation in simulation. Although the paper focuses on Kriging in deterministic simulation, the method also applies to other types of metamodels (besides Kriging), and to stochastic simulation. The paper focuses on simulations that require much computer time, so it is important to select a design with a small number of observations. The proposed method is therefore sequential. The novelty of the method is that it accounts for the specific input/output function of the particular simulation model at hand; that is, the method is application-driven or customized. This customization is achieved through cross-validation and jackknifing. The new method is tested through two academic applications, which demonstrate that the method indeed gives better results than either sequential designs based on an approximate Kriging prediction variance formula or designs with prefixed sample sizes.     

Flexible sliced designs for computer experiments

Sliced Latin hypercube designs are popularly adopted for computer experiments with qualitative factors. Previous constructions require the sizes of different slices to be identical. Here we construct sliced designs with flexible sizes of slices. Besides achieving desirable one-dimensional uniformity, flexible sliced designs (FSDs) constructed in this paper accommodate arbitrary sizes for different slices and cover ordinary sliced Latin hypercube designs as special cases. The sampling properties of FSDs are derived and a central limit theorem is established. It shows that any linear combination of the sample means from different models on slices follows an asymptotic normal distribution. Some simulations compare FSDs with other sliced designs in collective evaluations of multiple computer models.     

Multivariate versus univariate Kriging metamodels for multi-response simulation models

To analyze the input/output behavior of simulation models with multiple responses, we may apply either univariate or multivariate Kriging (Gaussian process) metamodels. In multivariate Kriging we face a major problem: the covariance matrix of all responses should remain positive-definite; we therefore use the recently proposed “nonseparable dependence” model. To evaluate the performance of univariate and multivariate Kriging, we perform several Monte Carlo experiments that simulate Gaussian processes. These Monte Carlo results suggest that the simpler univariate Kriging gives smaller mean square error.     

Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping

This paper proposes a novel method to select an experimental design for interpolation in random simulation, especially discrete event simulation. (Though the paper focuses on Kriging, this design approach may also apply to other types of metamodels such as non-linear regression models and splines.) Assuming that simulation requires much computer time, it is important to select a design with a small number of observations (or simulation runs). The proposed method is therefore sequential. Its novelty is that it accounts for the specific input/output behavior (or response function) of the particular simulation at hand; i.e., the method is customized or application-driven. A tool for this customization is bootstrapping, which enables the estimation of the variances of predictions for inputs not yet simulated. The method is tested through two classic simulation models, namely the expected steady-state waiting time of the M/M/1 queuing model, and the mean costs of a terminating (s, S) inventory simulation. For these two simulation models the novel design indeed gives better results than a popular alternative design, namely Latin Hypercube Sampling (LHS) with a prefixed sample.     

Robustness of Kriging when interpolating in random simulation with heterogeneous variances: Some experiments

This paper investigates the use of Kriging in random simulation when the simulation output variances are not constant. Kriging gives a response surface or metamodel that can be used for interpolation. Because Ordinary Kriging assumes constant variances, this paper also applies Detrended Kriging to estimate a non-constant signal function, and then standardizes the residual noise through the heterogeneous variances estimated from replicated simulation runs. Numerical examples, however, suggest that Ordinary Kriging is a robust interpolation method.     

Robust economic order quantity models

In this paper, we study the classical economic order quantity (EOQ) model under significant. In particular, the problem under consideration is the economic order quantity model with the input data of the demand rate, the order cost, and the holding cost rate being uncertain. A robustness approach based on scenario characterization of the input data is adopted to describe the uncertainties. The aim of the approach is to find an inventory policy that performs well under all realizable input data scenarios. An efficient linear time algorithm is devised to find the robust decisions. Analytical results are obtained for the case where input data are defined in continuous intervals. Comparisons on performances between the robust decisions and the stochastic optimization decisions are conducted. The results demonstrate the advantages of robustness approach.     

Robust estimation of nonlinear structural models

The problem is considered of identification of nonlinear error-in-variables models under conditions of a heavy contamination of the sample, including the presence of outliers (i.e., anomalous observations). On the basis of robust estimation methods, we propose a development of the algorithms of adjusted and total least squares. This has enabled us to improve the accuracy of the response prediction in the presence of outliers in the sample. The algorithms are used for constructing the Engel curve from the budget survey data. In result, we draw better conclusions about the regularities of the household behavior when the income changes.     

Robust algorithms for multiphase regression models

This paper proposes a robust procedure for solving multiphase regression problems that is efficient enough to deal with data contaminated by atypical observations due to measurement errors or those drawn from heavy-tailed distributions. Incorporating the expectation and maximization algorithm with the M-estimation technique, we simultaneously derive robust estimates of the change-points and regression parameters, yet as the proposed method is still not resistant to high leverage outliers we further suggest a modified version by first moderately trimming those outliers and then implementing the new procedure for the trimmed data. This study sets up two robust algorithms using the Huber loss function and Tukey's biweight function to respectively replace the least squares criterion in the normality-based expectation and maximization algorithm, illustrating the effectiveness and superiority of the proposed algorithms through extensive simulations and sensitivity analyses. Experimental results show the ability of the proposed method to withstand outliers and heavy-tailed distributions. Moreover, as resistance to high leverage outliers is particularly important due to their devastating effect on fitting a regression model to data, various real-world applications show the practicability of this approach.     

Category-theoretic models of algebraic computer systems

A computer system is said to be algebraic if it contains nodes that implement unconventional computation paradigms based on universal algebra. A category-based approach to modeling such systems that provides a theoretical basis for mapping tasks to these systems’ architecture is proposed. The construction of algebraic models of general-purpose computations involving conditional statements and overflow control is formally described by a reflector in an appropriate category of algebras. It is proved that this reflector takes the modulo ring whose operations are implemented in the conventional arithmetic processors to the ?ukasiewicz logic matrix. Enrichments of the set of ring operations that form bases in the ?ukasiewicz logic matrix are found.     

Robust estimation for varying index coefficient models

Varying index coefficient models (VICMs) proposed by Ma and Song (J Am Stat Assoc, 2014. doi: 10.1080/01621459.2014.903185) are a new class of semiparametric models, which encompass most of the existing semiparametric models. So far, only the profile least squares method and local linear fitting were developed for the VICM, which are very sensitive to the outliers and will lose efficiency for the heavy tailed error distributions. In this paper, we propose an efficient and robust estimation procedure for the VICM based on modal regression which depends on a bandwidth. We establish the consistency and asymptotic normality of proposed estimators for index coefficients by utilizing profile spline modal regression method. The oracle property of estimators for the nonparametric functions is also established by utilizing a two-step spline backfitted local linear modal regression approach. In addition, we discuss the bandwidth selection for achieving better robustness and efficiency and propose a modified expectation–maximization-type algorithm for the proposed estimation procedure. Finally, simulation studies and a real data analysis are carried out to assess the finite sample performance of the proposed method.     

Robust designs for Haar wavelet approximation models

In this paper, we discuss the construction of robust designs for heteroscedastic wavelet regression models when the assumed models are possibly contaminated over two different neighbourhoods: G ₁ and G ₂ . Our main findings are: (1) A recursive formula for constructing D‐optimal designs under G ₁ ; (2) Equivalency of Q‐optimal and A‐optimal designs under both G ₁ and G ₂ ; (3) D‐optimal robust designs under G ₂ ; and (4) Analytic forms for A‐ and Q‐optimal robust design densities under G ₂ . Several examples are given for the comparison, and the results demonstrate that our designs are efficient. Copyright © 2010 John Wiley & Sons, Ltd.     

Robust test for structural instability in dynamic factor models

Annals of the Institute of Statistical Mathematics - In this paper, we consider a robust test for structural breaks in dynamic factor models. The proposed framework considers structural changes...     

Generalized Kolmogorov models for probabilistic quantum-type experiments

Doklady Mathematics -     

Global stability of nonresident computer virus models

In this paper, applying Lyapunov functional techniques to nonresident computer virus models, we establish global dynamics of the model whose threshold parameter is the basic reproduction number R₀ such that the virus‐free equilibrium is globally asymptotically stable when R₀ ≤ 1, and the infected equilibrium is globally asymptotically stable when R₀ > 1 under the same restricted condition on a parameter, which appeared in the literature on delayed susceptible‐infected‐recovered‐susceptible (SIRS) epidemic models. We use new techniques on permanence and global stability of this model for R₀ > 1. Copyright © 2014 John Wiley & Sons, Ltd.     

Robust cutting parameters optimization for production time via computer experiment

This study intends to determine the optimal cutting parameters required to minimize the cutting time while maintaining an acceptable quality level. Usually, the equation for predicting cutting time is unknown during the early stages of cutting operations. This equation can be determined by studying the output cutting times vs. input cutting parameters through CATIA software, with assistance from the statistical method, response surface methodology (RSM). Then, the equation is formulated as an objective function in the form of mathematical programming (MP) to determine the optimal cutting parameters so that the cutting time is minimized. The formulation in MP also includes the constraints of feasible ranges for process capability consideration and surface roughness for quality concerns. The important ranking obtained from the statistical method in cooperation with the optimal solutions found from MP can also be used as references for the possibility of robust design improvements.     

Using adaptive multi-accurate function evaluations in a surrogate-assisted method for computer experiments

In many computer experiments, surrogates are used to assist in searching for certain target points. If the surrogates are defined by response function values evaluated by costly iterative processes, the computational burdens may impede the efficiency of regular surrogate-assisted methods. Instead of computing the fully convergent response function values, we propose to control the function evaluation iterations dynamically to save time on function evaluations without degrading the overall performance. Our new algorithms adaptively determine whether each of the function evaluation iterations should be paused, kept running, or restarted; we then use the approximate function values with various levels of accuracy to construct the surrogates. The numerical results show that the proposed algorithms achieve significant savings when solving super-level set searching problems that involve identifying positive Lyapunov exponents of a dynamical system.     

Robust estimation in generalized semiparametric mixed models for longitudinal data

In this paper, we consider robust generalized estimating equations for the analysis of semiparametric generalized partial linear mixed models (GPLMMs) for longitudinal data. We approximate the non-parametric function in the GPLMM by a regression spline, and make use of bounded scores and leverage-based weights in the estimating equation to achieve robustness against outliers and influential data points, respectively. Under some regularity conditions, the asymptotic properties of the robust estimators are investigated. To avoid the computational problems involving high-dimensional integrals in our estimators, we adopt a robust Monte Carlo Newton-Raphson (RMCNR) algorithm for fitting GPLMMs. Small simulations are carried out to study the behavior of the robust estimates in the presence of outliers, and these estimates are also compared to their corresponding non-robust estimates. The proposed robust method is illustrated in the analysis of two real data sets.     

Robust and accurate inference for generalized linear models

In the framework of generalized linear models, the nonrobustness of classical estimators and tests for the parameters is a well known problem, and alternative methods have been proposed in the literature. These methods are robust and can cope with deviations from the assumed distribution. However, they are based on first order asymptotic theory, and their accuracy in moderate to small samples is still an open question. In this paper, we propose a test statistic which combines robustness and good accuracy for moderate to small sample sizes. We combine results from Cantoni and Ronchetti [E. Cantoni, E. Ronchetti, Robust inference for generalized linear models, Journal of the American Statistical Association 96 (2001) 1022–1030] and Robinson, Ronchetti and Young [J. Robinson, E. Ronchetti, G.A. Young, Saddlepoint approximations and tests based on multivariate M-estimators, The Annals of Statistics 31 (2003) 1154–1169] to obtain a robust test statistic for hypothesis testing and variable selection, which is asymptotically χ²-distributed as the three classical tests but with a relative error of order O(n⁻¹). This leads to reliable inference in the presence of small deviations from the assumed model distribution, and to accurate testing and variable selection, even in moderate to small samples.