Efficient Global Optimization of Expensive Black-Box Functions

In many engineering optimization problems, the number of function evaluations is severely limited by time or cost. These problems pose a special challenge to the field of global optimization, since existing methods often require more function evaluations than can be comfortably afforded. One way to address this challenge is to fit response surfaces to data collected by evaluating the objective and constraint functions at a few points. These surfaces can then be used for visualization, tradeoff analysis, and optimization. In this paper, we introduce the reader to a response surface methodology that is especially good at modeling the nonlinear, multimodal functions that often occur in engineering. We then show how these approximating functions can be used to construct an efficient global optimization algorithm with a credible stopping rule. The key to using response surfaces for global optimization lies in balancing the need to exploit the approximating surface (by sampling where it is minimized) with the need to improve the approximation (by sampling where prediction error may be high). Striking this balance requires solving certain auxiliary problems which have previously been considered intractable, but we show how these computational obstacles can be overcome.     

大规模系统的全局优化

本文讨论了可分非凸大规模系统的全局优化控制问题 .提出了一种 3级递阶优化算法 .该算法首先把原问题转化为可分的多目标优化问题 ,然后凸化非劣前沿 ,再从非劣解集中挑出原问题的全局最优解 .建立了算法的理论基础 ,证明了算法的收敛性 .仿真结果表明算法是有效的 .     

A Taxonomy of Global Optimization Methods Based on Response Surfaces

This paper presents a taxonomy of existing approaches for using response surfaces for global optimization. Each method is illustrated with a simple numerical example that brings out its advantages and disadvantages. The central theme is that methods that seem quite reasonable often have non-obvious failure modes. Understanding these failure modes is essential for the development of practical algorithms that fulfill the intuitive promise of the response surface approach.     

Global Solution of Optimization Problems with Parameter-Embedded Linear Dynamic Systems

This paper develops a theory for the global solution of nonconvex optimization problems with parameter-embedded linear dynamic systems. A quite general problem formulation is introduced and a solution is shown to exists. A convexity theory for integrals is then developed to construct convex relaxations for utilization in a branch-and-bound framework to calculate a global minimum. Interval analysis is employed to generate bounds on the state variables implied by the bounds on the embedded parameters. These bounds, along with basic integration theory, are used to prove convergence of the branch-and-bound algorithm to the global minimum of the optimization problem. The implementation of the algorithm is then considered and several numerical case studies are examined thoroughly     

Prediction of Oligopeptide Conformations via Deterministic Global Optimization

A deterministic global optimization method is described for identifying the global minimum potential energy conformation of oligopeptides. The ECEPP/3 detailed potential energy model is utilized for describing the energetics of the atomic interactions posed in the space of the peptide dihedral angles. Based on previous work on the microcluster and molecular structure determination [21, 22, 23, 24], a procedure for deriving convex lower bounding functions for the total potential energy function is developed. A procedure that allows the exclusion of domains of the (ø, ) space based on the analysis of experimentally determined native protein structures is presented. The reduced disjoint sub-domains are appropriately combined thus defining the starting regions for the search. The proposed approach provides valuable information on (i) the global minimum potential energy conformation, (ii) upper and lower bounds of the global minimum energy structure and (iii) low energy conformers close to the global minimum one. The proposed approach is illustrated with Ac-Ala4-Pro-NHMe, Met-enkephalin, Leu-enkephalin, and Decaglycine.     

基于偏最小二乘变换与Kriging模型的全局优化方法

针对传统Kriging模型在多变量(高维)输入全局优化中因超参数过多而引发收敛速度慢,精度低,建模效率不高问题,提出了基于偏最小二乘变换技术和Kriging模型的有效全局优化方法.首先,构造偏最小二乘高斯核函数;其次,借助差分进化算法寻找满足期望改进准则最大化条件的新样本点;然后,将不同核函数和期望改进准则组合,构建四种有效全局优化算法并进行比较;最后,数值算例结果表明,基于偏最小二乘变换的Kriging全局优化方法在解决高维全局优化问题方面相比于标准的全局优化算法在收敛精度及收敛速度方面更具优势.     

Almost Surely Convergent Global Optimization Algorithm Using Noise-Corrupted Observations

A new recursive algorithm for searching the global minimizer of a function is proposed when the function is observed with noise. The algorithm is based on switches between the stochastic approximation and the random search. The combination of SA with RS is not a new idea in such combination, the difficulty consists in creating a good switching rule and in designing an efficient method to reduce the noise effect. The proposed switching rule is easily realizable, the noise reducing method is effective, and the whole recursive optimization algorithm is simply calculated. It is proved that the algorithm a.s. converges to the global minimizer and is asymptotically normal. In comparison with existing methods, the proposed algorithm not only requires much weaker conditions, but also is more efficient as shown by simulation.     

Application of Deterministic Low-Discrepancy Sequences in Global Optimization

It has been recognized through theory and practice that uniformly distributed deterministic sequences provide more accurate results than purely random sequences. A quasi Monte Carlo (QMC) variant of a multi level single linkage (MLSL) algorithm for global optimization is compared with an original stochastic MLSL algorithm for a number of test problems of various complexities. An emphasis is made on high dimensional problems. Two different low-discrepancy sequences (LDS) are used and their efficiency is analysed. It is shown that application of LDS can significantly increase the efficiency of MLSL. The dependence of the sample size required for locating global minima on the number of variables is examined. It is found that higher confidence in the obtained solution and possibly a reduction in the computational time can be achieved by the increase of the total sample size N. N should also be increased as the dimensionality of problems grows. For high dimensional problems clustering methods become inefficient. For such problems a multistart method can be more computationally expedient.     

A Stochastic/Perturbation Global Optimization Algorithm for Distance Geometry Problems

We present a new global optimization approach for solving exactly or inexactly constrained distance geometry problems. Distance geometry problems are concerned with determining spatial structures from measurements of internal distances. They arise in the structural interpretation of nuclear magnetic resonance data and in the prediction of protein structure. These problems can be naturally formulated as global optimization problems which generally are large and difficult. The global optimization method that we present is related to our previous stochastic/perturbation global optimization methods for finding minimum energy configurations, but has several key differences that are important to its success. Our computational results show that the method readily solves a set of artificial problems introduced by Moré and Wu that have up to 343 atoms. On a set of considerably more difficult protein fragment problems introduced by Hendrickson, the method solves all the problems with up to 377 atoms exactly, and finds nearly exact solution for all the remaining problems which have up to 777 atoms. These preliminary results indicate that this approach has very good promise for helping to solve distance geometry problems.     

Global Optimization using a Dynamical Systems Approach

We develop new algorithms for global optimization by combining well known branch and bound methods with multilevel subdivision techniques for the computation of invariant sets of dynamical systems. The basic idea is to view iteration schemes for local optimization problems – e.g. Newton’s method or conjugate gradient methods – as dynamical systems and to compute set coverings of their fixed points. The combination with bounding techniques allow for the computation of coverings of the global optima only. We show convergence of the new algorithms and present a particular implementation. Michael Dellnitz - Research of the authors is partially supported by the Deutsche Forschungsgemeinschaft within the Sonderforschungsbereich 376     

复合随机时滞系统的全局稳定

本文主要是研究了具有时滞随机复合系统的反馈律和全局稳定,及其所需要的充分条件.主要的方法是:引入一个测度函数u,使得关于ξ的随机系统稳定,再通过附加条件,从而达到整个复合系统的稳定.     

非线性极大极小系统全局优化算法的分析

非线性极大极小系统的全局优化可用于柔性制造和智能交通的决策与控制.实现了非线性极大极小系统的全局优化算法的仿真,并进行了计算时间分析.数值实验表明了全局优化算法的可行性.算法的计算时间主要由系统的优化极大射影矩阵数目决定,而优化极大射影矩阵数目与系统解析式中单极大式的系数紧密相关,系数取值越分散,简约极大射影矩阵的效果越好,计算效率越高.     

Global Attractors for Multivalued Random Dynamical Systems Generated by Random Differential Inclusions with Multiplicative Noise

In this paper we consider a stochastic differential inclusion with multiplicative noise. It is shown that it generates a multivalued random dynamical system for which there also exists a global random attractor.     

随机混合动态系统的状态反馈控制

针对一类以有限齐次马氏链δ(k)作为切换信号的随机混合系统,首先,通过构造随机混合Lyapunov函数,得到整个随机混合系统渐近稳定的充分条件.然后,引入可调转移概率等相关概念,通过对有限齐次马氏链δ(k)及各子系统加入控制,以实现状态反馈控制.进一步,得到随机混合闭环系统渐近稳定的充分条件.     

Linearly Constrained Global Optimization and Stochastic Differential Equations

A stochastic algorithm is proposed for the global optimization of nonconvex functions subject to linear constraints. Our method follows the trajectory of an appropriately defined Stochastic Differential Equation (SDE). The feasible set is assumed to be comprised of linear equality constraints, and possibly box constraints. Feasibility of the trajectory is achieved by projecting its dynamics onto the set defined by the linear equality constraints. A barrier term is used for the purpose of forcing the trajectory to stay within the box constraints. Using Laplace’s method we give a characterization of a probability measure (Π) that is defined on the set of global minima of the problem. We then study the transition density associated with the projected diffusion process and show that its weak limit is given by Π. Numerical experiments using standard test problems from the literature are reported. Our results suggest that the method is robust and applicable to large-scale problems.     

Development of the Kriging Method with Application

This paper describes a modification of the kriging method for working with the square root transformation of a spatial random process. We have developed this method for the situation where the spatial process observed is not supposed to be stationary but the assumption is that its square root is a second order stationary spatial random process. Consequently this method is developed for estimating the integral of the process observed and finally some application of the method is given to data from an environmental radioactivity survey.     

Controllability Properties of Linear Mean-Field Stochastic Systems

We study some controllability properties for linear stochastic systems of mean-field type. First, we give necessary and sufficient criteria for exact terminal-controllability. Second, we characterize the approximate and approximate null-controllability via duality techniques. Using Riccati equations associated to linear quadratic problems in the control of mean-field systems, we provide a (conditional) viability criterion for approximate null-controllability. In the classical diffusion framework, approximate and approximate null-controllability are equivalent. This is no longer the case for mean-field systems. We provide sufficient (algebraic) invariance conditions implying approximate null-controllability. We also present a general class of systems for which our criterion is equivalent to approximate null-controllability property. We also introduce some rank conditions under which approximate and approximate null-controllability are equivalent. Several examples and counter-examples as well as a partial algorithm are provided.     

On Stability and Stabilizability of Singular Stochastic Systems with Delays

This paper deals with the class of continuous-time singular linear systems with Markovian jump parameters and time delays. Sufficient conditions on the stochastic stability and stochastic stabilizability are developed. A design algorithm for a state feedback controller which guarantees that the closed-loop dynamics will be regular, impulse free, and stochastically stable is proposed in terms of the solutions to linear matrix inequalities. The research of this author was supported by NSERC Grant RGPIN36444-02. The research of this author was supported by the Program for a New Century of Excellent Talents in the Universities and by the Foundation for the Authors of National Excellent Doctoral Dissertations of P. R. China, Grant 200240. The research of this author was supported by HKU Grant RGC 7029/05P.     

On the Design of Optimization Strategies Based on Global Response Surface Approximation Models

Striking the correct balance between global exploration of search spaces and local exploitation of promising basins of attraction is one of the principal concerns in the design of global optimization algorithms. This is true in the case of techniques based on global response surface approximation models as well. After constructing such a model using some initial database of designs it is far from obvious how to select further points to examine so that the appropriate mix of exploration and exploitation is achieved. In this paper we propose a selection criterion based on the expected improvement measure, which allows relatively precise control of the scope of the search. We investigate its behavior through a set of artificial test functions and two structural optimization problems. We also look at another aspect of setting up search heuristics of this type: the choice of the size of the database that the initial approximation is built upon.     

原木随机存贮策略的最优化问题

本主要介绍原木经销商在周期内原木变价销售情况下的最佳期初贮存量的决策方法，在建立期望机会成本的数学模型基础上，探讨原木随机存贮策略的最优化问题。