Spatial Regression Models,Response Surfaces,and Process Optimization

Abstract

Spatial regression models are developed as a complementary alternative to second-order polynomial response surfaces in the context of process optimization. These models provide estimates of design variable effects and smooth, data-faithful approximations to the unknown response function over the design space. The predicted response surfaces are driven by the covariance structures of the models. Several structures, isotropic and anisotropic, are considered and connections with thin plate splines are reviewed. Estimation of covariance parameters is achieved via maximum likelihood and residual maximum likelihood. A feature of the spatial regression approach is the visually appealing graphical summaries that are produced. These allow rapid and intuitive identification of process windows on the design space for which the response achieves target performance. Relevant design issues are briefly discussed and spatial designs, such as the packing designs available in Gosset, are suggested as a suitable design complement. The spatial regression models also perform well with no global design, for example with data obtained from series of designs on the same space of design variables. The approach is illustrated with an example involving the optimization of components in a DNA amplification assay. A Monte Carlo comparison of the spatial models with both thin plate splines and second-order polynomial response surfaces for a scenario motivated by the example is also given. This shows superior performance of the spatial models to the second-order polynomials with respect to both prediction over the complete design space and for cross-validation prediction error in the region of the optimum. An anisotropic spatial regression model performs best for a high noise case and both this model and the thin plate spline for a low noise case. Spatial regression is recommended for construction of response surfaces in all process optimization applications.     

Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models

This paper proposes a new method that extends the efficient global optimization to address stochastic black-box systems. The method is based on a kriging meta-model that provides a global prediction of the objective values and a measure of prediction uncertainty at every point. The criterion for the infill sample selection is an augmented expected improvement function with desirable properties for stochastic responses. The method is empirically compared with the revised simplex search, the simultaneous perturbation stochastic approximation, and the DIRECT methods using six test problems from the literature. An application case study on an inventory system is also documented. The results suggest that the proposed method has excellent consistency and efficiency in finding global optimal solutions, and is particularly useful for expensive systems.     

全局优化问题的一个无参数填充函数算法

填充函数法是求解全局优化问题的一种有效的确定性算法,方法的关键在于填充函数的构造.对于一般无约束优化问题提出了一个新的无参数填充函数,通过定义证明了此填充函数能保持填充性质.利用其理论性质设计了相应的算法并对几个经典的算例进行了数值实验,实验结果表明算法有效可行.     

A Trust-Region Algorithm for Global Optimization

We consider the global minimization of a bound-constrained function with a so-called funnel structure. We develop a two-phase procedure that uses sampling, local optimization, and Gaussian smoothing to construct a smooth model of the underlying funnel. The procedure is embedded in a trust-region framework that avoids the pitfalls of a fixed sampling radius. We present a numerical comparison to three popular methods and show that the new algorithm is robust and uses up to 20 times fewer local minimizations steps.     

一类新的求全局优化问题的填充函数

填充函数方法是一种寻找全局极小解的有效方法.本文首先对现有的填充函数进行研究分析,然后构造出一类新的填充函数,设计算法,并通过数值试验验证了该函数和算法的有效性.     

求无约束连续全局优化问题的单参数填充函数法

填充函数法是求解全局优化问题的一个重要的确定性算法,这种方法的关键是构造具有良好性质的填充函数.构造了一个新的求解无约束全局优化问题的填充函数.函数连续可微且只包含一个参数.通过分析该函数的相关性质,设计了相应的算法.数值实验表明该算法简单有效.     

A Smoothing Method of Global Optimization that Preserves Global Minima

A new smoothing method of global optimization is proposed in the present paper, which prevents shifting of global minima. In this method, smoothed functions are solutions of a heat diffusion equation with external heat source. The source helps to control the diffusion such that a global minimum of the smoothed function is again a global minimum of the cost function. This property, and the existence and uniqueness of the solution are proved using results in theory of viscosity solutions. Moreover, we devise an iterative equation by which smoothed functions can be obtained analytically for a class of cost functions. The effectiveness and potential of our method are then demonstrated with some experimental results.     

Optimization Design of Holding Poles Based on the Response Surface Methodology and the Improved Arithmetic Optimization Algorithm北大核心CSCD

抱杆优化设计需要耗费大量有限元分析计算时间,难以确定可行域.该文采用响应面法(response surface method,RSM)来模拟抱杆结构的真实响应,提出了改进的算术优化算法(improved arithmetic optimization algorithm,IAOA)对抱杆结构进行优化设计.将分数阶积分引入算术优化算法(arithmetic optimization algorithm,AOA),改善了算法的开发能力.采用拉丁超立方抽样,选取抱杆结构杆件截面试验样本,利用最小二乘法对样本点进行分析,构建了抱杆结构应力和位移关于杆件截面尺寸的二阶响应面代理模型.建立以抱杆质量最小化为优化目标,许用应力和位移为约束条件的优化模型,采用IAOA对其进行求解.结果表明:二阶响应面模型能够准确预测抱杆结构的响应值,IAOA的求解精度得到显著提升,代理模型可大幅降低有限元分析所需的计算代价,优化后抱杆结构质量减轻了8.2%.联合使用RSM和IAOA可有效求解大型空间杆系结构的优化设计问题.     

A New Version of the Price's Algorithm for Global Optimization

We present an algorithm for finding a global minimum of a multimodal,multivariate function whose evaluation is very expensive, affected by noise andwhose derivatives are not available. The proposed algorithm is a new version ofthe well known Price's algorithm and its distinguishing feature is that ittries to employ as much as possible the information about the objectivefunction obtained at previous iterates. The algorithm has been tested on alarge set of standard test problems and it has shown a satisfactorycomputational behaviour. The proposed algorithm has been used to solveefficiently some difficult optimization problems deriving from the study ofeclipsing binary star light curves.     

Optimal Multisections in Interval Branch-and-Bound Methods of Global Optimization

In this paper we define multisections of intervals that yield sharp lower bounds in branch-and-bound type methods for interval global optimization. A so called 'generalized kite', defined for differentiable univariate functions, is built simultaneously with linear boundary forms and suitably chosen centered forms. Proofs for existence and uniqueness of optimal cuts are given. The method described may be used either as an accelerating device or in a global optimization algorithm with an efficient pruning effect. A more general principle for decomposition of boxes is suggested.     

一种求约束总极值的水平值估计方法

给出了一种求约束总极值的水平值估计方法,说明了修正的方差方程的根与原始问题的最优值之间的等价性,给出了一种基于牛顿法的水平值估计算法并证明了实现算法的收敛性．初步的计算例子表明所给算法是有效的．     

一个全局最优化问题的填充函数

本文给出了一个非线性全局最优化问题的填充函数定义,此定义不同于以前已有的填充函数定义。根据此定义,本文提出了一簇单参数填充函数和相应的填充函数算法．对几个算例的数据测试表明,该填充函数法是可行和有效的．     

一类非光滑总体极值的区间算法

本文利用区间分析知识 ,构造了一类 n维非光滑函数总体极值的区间算法 ,理论分析和实例计算均表明本文算法安全可靠 ;能求出全部总体极小点 ;收敛速度也比以前方法[1] 明显加快     

Efficient Global Optimization of Expensive Black-Box Functions

In many engineering optimization problems, the number of function evaluations is severely limited by time or cost. These problems pose a special challenge to the field of global optimization, since existing methods often require more function evaluations than can be comfortably afforded. One way to address this challenge is to fit response surfaces to data collected by evaluating the objective and constraint functions at a few points. These surfaces can then be used for visualization, tradeoff analysis, and optimization. In this paper, we introduce the reader to a response surface methodology that is especially good at modeling the nonlinear, multimodal functions that often occur in engineering. We then show how these approximating functions can be used to construct an efficient global optimization algorithm with a credible stopping rule. The key to using response surfaces for global optimization lies in balancing the need to exploit the approximating surface (by sampling where it is minimized) with the need to improve the approximation (by sampling where prediction error may be high). Striking this balance requires solving certain auxiliary problems which have previously been considered intractable, but we show how these computational obstacles can be overcome.     

A Simple Multistart Algorithm for Global Optimization

1.IntroductionConsidertheunconstrainedoptimizationproblem:findx*suchthatf(x*)~caf(x),(1)wheref(x)isanonlinearfllnctiondefinedonW"andXCR".Ourobjectiveistofindtheglobalminimizeroff(x)inthefeasibleset.Withoutassuminganyconditionsonf(x)globaloptimizationproblemsareunsolvableinthefollowingsensefnoalgorithmcanbeguaranteedtofindaglobalminimizerofageneralnonlinearfunctionwithinfinitelymanyiterations.Supposethatanalgorithmappliedtoanonlinearfunctionf(x)producesiteratesxlandterminatesafterKiterations.…     

A Non-myopic Utility Function for Statistical Global Optimization Algorithms

The high cost of providing worst-case solutions to global optimization problems has motivated the development of average-case algorithms that rely on a statistical model of the objective function. The critical role of the statistical model is to guide the search for the optimum. The standard approach is to define a utility function u(x) that in a certain sense reflects the benefit of evaluating the function at x. A proper utility function needs to strike a balance between the immediate benefit of evaluating the function at x – a myopic consideration; and the overall effect of this choice on the performance of the algorithm – a global criterion. The utility functions currently used in this context are heuristically modified versions of some myopic utility functions. We propose using a new utility function that is provably a globally optimal utility function in a non-adaptive context (where the model of the function values remains unchanged). In the adaptive context, this utility function is not necessarily optimal, however, given its global nature, we expect that its use will lead to the improved performance of statistical global optimization algorithms. To illustrate the approach, and to test the above assertion, we apply this utility function to an existing adaptive multi-dimensional statistical global optimization algorithm and provide experimental comparisons with the original algorithm.     

大规模系统的全局优化

本文讨论了可分非凸大规模系统的全局优化控制问题 .提出了一种 3级递阶优化算法 .该算法首先把原问题转化为可分的多目标优化问题 ,然后凸化非劣前沿 ,再从非劣解集中挑出原问题的全局最优解 .建立了算法的理论基础 ,证明了算法的收敛性 .仿真结果表明算法是有效的 .     

Stochastic Methods for Practical Global Optimization

Engineering design problems often involve global optimization of functions that are supplied as black box functions. These functions may be nonconvex, nondifferentiable and even discontinuous. In addition, the decision variables may be a combination of discrete and continuous variables. The functions are usually computationally expensive, and may involve finite element methods. An engineering example of this type of problem is to minimize the weight of a structure, while limiting strain to be below a certain threshold. This type of global optimization problem is very difficult to solve, yet design engineers must find some solution to their problem – even if it is a suboptimal one. Sometimes the most difficult part of the problem is finding any feasible solution. Stochastic methods, including sequential random search and simulated annealing, are finding many applications to this type of practical global optimization problem. Improving Hit-and-Run (IHR) is a sequential random search method that has been successfully used in several engineering design applications, such as the optimal design of composite structures. A motivation to IHR is discussed as well as several enhancements. The enhancements include allowing both continuous and discrete variables in the problem formulation. This has many practical advantages, because design variables often involve a mixture of continuous and discrete values. IHR and several variations have been applied to the composites design problem. Some of this practical experience is discussed.     

Prediction of Oligopeptide Conformations via Deterministic Global Optimization

A deterministic global optimization method is described for identifying the global minimum potential energy conformation of oligopeptides. The ECEPP/3 detailed potential energy model is utilized for describing the energetics of the atomic interactions posed in the space of the peptide dihedral angles. Based on previous work on the microcluster and molecular structure determination [21, 22, 23, 24], a procedure for deriving convex lower bounding functions for the total potential energy function is developed. A procedure that allows the exclusion of domains of the (ø, ) space based on the analysis of experimentally determined native protein structures is presented. The reduced disjoint sub-domains are appropriately combined thus defining the starting regions for the search. The proposed approach provides valuable information on (i) the global minimum potential energy conformation, (ii) upper and lower bounds of the global minimum energy structure and (iii) low energy conformers close to the global minimum one. The proposed approach is illustrated with Ac-Ala4-Pro-NHMe, Met-enkephalin, Leu-enkephalin, and Decaglycine.     

无约束全局优化的一个可调参数的填充函数

A filled function with adjustable parameters is suggested in this paper for finding a global minimum point of a general class of nonlinear programming problems with a bounded and closed domain. This function has two adjustable parameters. We will discuss the properties of the proposed filled function. Conditions on this function and on the values of parameters are given so that the constructed function has the desired properties of traditional filled function.