Statistical testing of optimality conditions in multiresponse simulation-based optimization

This article studies simulation-based optimization with multiple outputs. It assumes that the simulation model has one random objective function and must satisfy given constraints on the other random outputs. It presents a statistical procedure for testing whether a specific input combination (proposed by some optimization heuristic) satisfies the Karush–Kuhn–Tucker (KKT) first-order optimality conditions. The article focuses on “expensive” simulations, which have small sample sizes. The article applies the classic t test to check whether the specific input combination is feasible, and whether any constraints are binding; next, it applies bootstrapping (resampling) to test the estimated gradients in the KKT conditions. The new methodology is applied to three examples, which gives encouraging empirical results.     

A computer‐aided methodology for the optimization of electrostatic separation processes in recycling

The rapid growth of technological products has led to an increasing volume of waste electrical and electronic equipments (WEEE), which could represent a valuable source of critical raw materials. However, current mechanical separation processes for recycling are typically poorly operated, making it impossible to modify the process parameters as a function of the materials under treatment, thus resulting in untapped separation potentials. Corona electrostatic separation (CES) is one of the most popular processes for separating fine metal and nonmetal particles derived from WEEE. In order to optimize the process operating conditions (i.e., variables) for a given multi‐material mixture under treatment, several technological and economical criteria should be jointly considered. This translates into a complex optimization problem that can be hardly solved by a purely experimental approach. As a result, practitioners tend to assign process parameters by few experiments based on a small material sample and to keep these parameters fixed during the process life‐cycle. The use of computer experiments for parameter optimization is a mostly unexplored area in this field. In this work, a computer‐aided approach is proposed to the problem of optimizing the operational parameters in CES processes. Three metamodels, developed starting from a multi‐body simulation model of the process physics, are presented and compared by means of a numerical and simulation study. Our approach proves to be an effective framework to optimize the CES process performance. Furthermore, by comparing the predicted response surfaces of the metamodels, additional insight into the process behavior over the operating region is obtained. Copyright © 2015 John Wiley & Sons, Ltd.     

Global optimization method using adaptive and parallel ensemble of surrogates for engineering design optimization

As a robust and efficient technique for global optimization, surrogate-based optimization method has been widely used in dealing with the complicated and computation-intensive engineering design optimization problems. It’s hard to select an appropriate surrogate model without knowing the behaviour of the real system a priori in most cases. To overcome this difficulty, a global optimization method using an adaptive and parallel ensemble of surrogates combining three representative surrogate models with optimized weight factors has been proposed. The selection of weight factors is treated as an optimization problem with the desired solution being one that minimizes the generalized mean square cross-validation error. The proposed optimization method is tested by considering several well-known numerical examples and one industrial problem compared with other optimization methods. The results show that the proposed optimization method can be a robust and efficient approach in surrogate-based optimization for locating the global optimum.     

Parallel radial basis function methods for the global optimization of expensive functions

We introduce a master–worker framework for parallel global optimization of computationally expensive functions using response surface models. In particular, we parallelize two radial basis function (RBF) methods for global optimization, namely, the RBF method by Gutmann [Gutmann, H.M., 2001a. A radial basis function method for global optimization. Journal of Global Optimization 19(3), 201–227] (Gutmann-RBF) and the RBF method by Regis and Shoemaker [Regis, R.G., Shoemaker, C.A., 2005. Constrained global optimization of expensive black box functions using radial basis functions, Journal of Global Optimization 31, 153–171] (CORS-RBF). We modify these algorithms so that they can generate multiple points for simultaneous evaluation in parallel. We compare the performance of the two parallel RBF methods with a parallel multistart derivative-based algorithm, a parallel multistart derivative-free trust-region algorithm, and a parallel evolutionary algorithm on eleven test problems and on a 6-dimensional groundwater bioremediation application. The results indicate that the two parallel RBF algorithms are generally better than the other three alternatives on most of the test problems. Moreover, the two parallel RBF algorithms have comparable performances on the test problems considered. Finally, we report good speedups for both parallel RBF algorithms when using a small number of processors.     

Using parallel function evaluations to improve hessian approximation for unconstrained optimization

This paper presents a new class of methods for solving unconstrained optimization problems on parallel computers. The methods are intended to solve small to moderate dimensional problems where function and derivative evaluation is the dominant cost. They utilize multiple processors to evaluate the function, (finite difference) gradient, and a portion of the finite difference Hessian simultaneously at each iterate. We introduce three types of new methods, which all utilize the new finite difference Hessian information in forming the new Hessian approximation at each iteration; they differ in whether and how they utilize the standard secant information from the current step as well. We present theoretical analyses of the rate of convergence of several of these methods. We also present computational results which illustrate their performance on parallel computers when function evaluation is expensive.Research supported by AFOSR grant AFOSR-85-0251, ARO contract DAAG 29-84-K-0140, NSF grant DCR-8403483, and NFS cooperative agreement DCR -8420944.     

Computation of Confidence Regions for Optimal Factor Levels in Constrained Response Surface Problems

This article presents an improved approach for computing the confidence regions for the optimal factor settings obtained from optimizing a general response surface model. The approach has a better computational efficiency and improved accuracy compared to existing methodology. A three-factor mixture experiment was used for the performance comparison. The coverage rate properties of the resulting confidence regions were assessed through an extensive simulation study.     

Improved Strategies for Radial basis Function Methods for Global Optimization

We propose some strategies that can be shown to improve the performance of the radial basis function (RBF) method by Gutmann [J. Global optim. 19(3), 201–227 (2001a)] (Gutmann-RBF) and the RBF method by Regis and Shoemaker [J. Global optim. 31, 153–171 (2005)] (CORS–RBF) on some test problems when they are initialized by symmetric Latin hypercube designs (SLHDs). Both methods are designed for the global optimization of computationally expensive functions with multiple local optima. We demonstrate how the original implementation of Gutmann-RBF can sometimes converge slowly to the global minimum on some test problems because of its failure to do local search. We then propose Controlled Gutmann-RBF (CG-RBF), which is a modification of Gutmann-RBF where the function evaluation point in each iteration is restricted to a subregion of the domain centered around a global minimizer of the current RBF model. By varying the size of this subregion in different iterations, we ensure a better balance between local and global search. Moreover, we propose a complete restart strategy for CG-RBF and CORS-RBF whenever the algorithm fails to make any substantial progress after some threshold number of consecutive iterations. Computational experiments on the seven Dixon and Szegö [Towards Global optimization, pp. 1–13. North-Holland, Amsterdam (1978)] test problems and on nine Schoen [J. Global optim. 3, 133–137 (1993)] test problems indicate that the proposed strategies yield significantly better performance on some problems. The results also indicate that, for some fixed setting of the restart parameters, the two modified RBF algorithms, namely CG-RBF-Restart and CORS-RBF-Restart, are comparable on the test problems considered. Finally, we examine the sensitivity of CG-RBF-Restart and CORS-RBF-Restart to the restart parameters.     

Optimizing the DMFC operating conditions using a response surface method

The power density of a Direct Methanol Fuel Cell (DMFC) as a function of temperature, methanol concentration, air flow rate, methanol flow rate and air relative humidity was studied using a Response Surface Methodology (RSM). For a DMFC equipped with a membrane of Nafion 112, it was observed that only the temperature, methanol concentration and air flow rate were relevant factors or operating variables. A new design of experiments was done for a narrower range of these variables and the operating values that optimise the power density were obtained using the software JMP 7.0 (SAS). The predicted power density values were in agreement with the experimental results obtained for the optimized operating conditions. Then, the RSM was applied to membranes with different thicknesses, Nafion 112, Nafion 1135 and Nafion 117, and as a function of the temperature and methanol concentration. The DMFC was characterized for the open circuit voltage (OCV), methanol crossover at the OC, power density and global efficiency. The membrane showing the best compromise between power density and efficiency was Nafion 117.     

CFD modeling and multi-objective optimization of cyclone geometry using desirability function,artificial neural networks and genetic algorithms

The low-mass loading gas cyclone separator has two performance parameters, the pressure drop and the collection efficiency (cut-off diameter). In this paper, a multi-objective optimization study of a gas cyclone separator has been performed using the response surface methodology (RSM) and CFD data. The effects of the inlet height, the inlet width, the vortex finder diameter and the cyclone total height on the cyclone performance have been investigated. The analysis of design of experiment shows a strong interaction between the inlet dimensions and the vortex finder diameter. No interaction between the cyclone height and the other three factors was observed. The desirability function approach has been used for the multi-objective optimization. A new set of geometrical ratios (design) has been obtained to achieve the best performance. A numerical comparison between the new design and the Stairmand design confirms the superior performance of the new design. As an alternative approach for applying RSM as a meta-model, two radial basis function neural networks (RBFNNs) have been used. Furthermore, the genetic algorithms technique has been used instead of the desirability function approach. A multi-objective optimization study using NSGA-II technique has been performed to obtain the Pareto front for the best performance cyclone separator.     

Global optimization over a box via canonical dual function

In this paper, we study global concave optimization by the canonical dual function. A differential flow on the dual feasible space is introduced. We show that the flow reaches a global minimizer of the concave function over a box. An example is illustrated.     

An adaptive radial basis algorithm (ARBF) for expensive black-box global optimization

Powerful response surface methods based on kriging and radial basis function (RBF) interpolation have been developed for expensive, i.e. computationally costly, global nonconvex optimization. We have implemented some of these methods in the solvers rbfSolve and EGO in the TOMLAB Optimization Environment (http://www.tomopt.com/tomlab/). In this paper we study algorithms based on RBF interpolation. The practical performance of the RBF algorithm is sensitive to the initial experimental design, and to the static choice of target values. A new adaptive radial basis interpolation (ARBF) algorithm, suitable for parallel implementation, is presented. The algorithm is described in detail and its efficiency is analyzed on the standard test problem set of Dixon–Szegö. Results show that it outperforms the published results of rbfSolve and several other solvers.     

Efficient Global Optimization of Expensive Black-Box Functions

In many engineering optimization problems, the number of function evaluations is severely limited by time or cost. These problems pose a special challenge to the field of global optimization, since existing methods often require more function evaluations than can be comfortably afforded. One way to address this challenge is to fit response surfaces to data collected by evaluating the objective and constraint functions at a few points. These surfaces can then be used for visualization, tradeoff analysis, and optimization. In this paper, we introduce the reader to a response surface methodology that is especially good at modeling the nonlinear, multimodal functions that often occur in engineering. We then show how these approximating functions can be used to construct an efficient global optimization algorithm with a credible stopping rule. The key to using response surfaces for global optimization lies in balancing the need to exploit the approximating surface (by sampling where it is minimized) with the need to improve the approximation (by sampling where prediction error may be high). Striking this balance requires solving certain auxiliary problems which have previously been considered intractable, but we show how these computational obstacles can be overcome.     

Global and local optimization using radial basis function response surface models

The focus of this paper is the optimization of complex multi-parameter systems. We consider systems in which the objective function is not known explicitly, and can only be evaluated through computationally intensive numerical simulation or through costly physical experiments. The objective function may also contain many local extrema which may be of interest. Given objective function values at a scattered set of parameter values, we develop a response surface model that can dramatically reduce the required computation time for parameter optimization runs. The response surface model is developed using radial basis functions, producing a model whose objective function values match those of the original system at all sampled data points. Interpolation to any other point is easily accomplished and generates a model which represents the system over the entire parameter space. This paper presents the details of the use of radial basis functions to transform scattered data points, obtained from a complex continuum mechanics simulation of explosive materials, into a response surface model of a function over the given parameter space. Response surface methodology and radial basis functions are discussed in general and are applied to a global optimization problem for an explosive oil well penetrator.     

Outer approximation algorithm for nondifferentiable optimization problems

Omissions from the list of references of Ref. 1 are corrected.     

Outer approximation algorithm for nondifferentiable optimization problems

It is known that the problem of minimizing a convex functionf(x) over a compact subsetX of ⁿ can be expressed as minimizing max{g(x, y)|y X}, whereg is a support function forf[f(x) g(x, y), for ally X andf(x)=g(x, x)]. Standard outer-approximation theory can then be employed to obtain outer-approximation algorithms with procedures for dropping previous cuts. It is shown here how this methodology can be extended to nonconvex nondifferentiable functions.This research was supported by the Science and Engineering Research Council, UK, and by the National Science Foundation under Grant No. ECS-79-13148.     

An approximation technique for robust nonlinear optimization

Nonlinear equality and inequality constrained optimization problems with uncertain parameters can be addressed by a robust worst-case formulation that is, however, difficult to treat computationally. In this paper we propose and investigate an approximate robust formulation that employs a linearization of the uncertainty set. In case of any norm bounded parameter uncertainty, this formulation leads to penalty terms employing the respective dual norm of first order derivatives of the constraints. The main advance of the paper is to present two sparsity preserving ways for efficient computation of these derivatives in the case of large scale problems, one similar to the forward mode, the other similar to the reverse mode of automatic differentiation. We show how to generalize the techniques to optimal control problems, and discuss how even infinite dimensional uncertainties can be treated efficiently. Finally, we present optimization results for an example from process engineering, a batch distillation.     

Global polynomial approximation for Symm's equation on polygons

Summary. To solve 1D linear integral equations on bounded intervals with nonsmooth input functions and solutions, we have recently proposed a quite general procedure, that is essentially based on the introduction of a nonlinear smoothing change of variable into the integral equation and on the approximation of the transformed solution by global algebraic polynomials. In particular, the new procedure has been applied to weakly singular equations of the second kind and to solve the generalized air foil equation for an airfoil with a flap. In these cases we have obtained arbitrarily high orders of convergence through the solution of very-well conditioned linear systems. In this paper, to enlarge the domain of applicability of our technique, we show how the above procedure can be successfully used also to solve the classical Symm's equation on a piecewise smooth curve. The collocation method we propose, applied to the transformed equation and based on Chebyshev polynomials of the first kind, has shown to be stable and convergent. A comparison with some recent numerical methods using splines or trigonometric polynomials shows that our method is highly competitive. Received October 1, 1998 / Revised version received September 27, 1999 / Published online June 21, 2000     

Neural network metamodeling for cycle time-throughput profiles in manufacturing

This paper proposed a neural network (NN) metamodeling method to generate the cycle time (CT)–throughput (TH) profiles for single/multi-product manufacturing environments. Such CT–TH profiles illustrate the trade-off relationship between CT and TH, the two critical performance measures, and hence provide a comprehensive performance evaluation of a manufacturing system. The proposed methods distinct from the existing NN metamodeling work in three major aspects: First, instead of treating an NN as a black box, the geometry of NN is examined and utilized; second, a progressive model-fitting strategy is developed to obtain the simplest-structured NN that is adequate to capture the CT–TH relationship; third, an experiment design method, particularly suitable to NN modeling, is developed to sequentially collect simulation data for the efficient estimation of the NN models.