Partitioned Random Search for Global Optimization with Sampling Cost and Discounting Factor

The method of partitioned random search has been proposed in recent years to obtain an as good as possible solution for the global optimization problem (1). A practical algorithm has been developed and applied to real-life problems. However, the design of this algorithm was based mainly on intuition. The theoretical foundation of the method is an important issue in the development of efficient algorithms for such problems. In this paper, we generalize previous theoretical results and propose a sequential sampling policy for the partitioned random search for global optimization with sampling cost and discounting factor. A proof of the optimality of the proposed sequential sampling policy is given by using the theory of optimal stopping.     

Optimal Sequential Sampling Policy of Partitioned Random Search and Its Approximation

The paper studies the optimal sequential sampling policy of the partitioned random search (PRS) and its approximation. The PRS is a recently proposed approach for function optimization. It takes explicitly into consideration computation time or cost, assuming that there exist both a cost for each function evaluation and a finite total computation time constraint. It is also motivated at improving efficiency of the widely used crude random search. In particular, the PRS considers partitioning the search region of an objective function into K subregions and employing an independent and identically distributed random sampling scheme for each of K subregions. A sampling policy decides when to terminate the sampling process or which subregion to be sampled next.     

Parallelizing simulated annealing algorithms based on high-performance computer

We implemented five conversions of simulated annealing (SA) algorithm from sequential-to-parallel forms on high-performance computers and applied them to a set of standard function optimization problems in order to test their performances. According to the experimental results, we eventually found that the traditional approach to parallelizing simulated annealing, namely, parallelizing moves in sequential SA, difficultly handled very difficult problem instances. Divide-and-conquer decomposition strategy used in a search space sometimes might find the global optimum function value, but it frequently resulted in great time cost if the random search space was considerably expanded. The most effective way we found in identifying the global optimum solution is to introduce genetic algorithm (GA) and build a highly hybrid GA+SA algorithm. In this approach, GA has been applied to each cooling temperature stage. Additionally, the performance analyses of the best algorithm among the five implemented algorithms have been done on the IBM Beowulf PCs Cluster and some comparisons have been made with some recent global optimization algorithms in terms of the number of functional evaluations needed to obtain a global minimum, success rate and solution quality.     

A survey on the global optimization problem: General theory and computational approaches

Several different approaches have been suggested for the numerical solution of the global optimization problem: space covering methods, trajectory methods, random sampling, random search and methods based on a stochastic model of the objective function are considered in this paper and their relative computational effectiveness is discussed. A closer analysis is performed of random sampling methods along with cluster analysis of sampled data and of Bayesian nonparametric stopping rules.     

SEQUENTIAL QUADRATIC PROGRAMMING METHODS FOR OPTIMAL CONTROL PROBLEMS WITH STATE CONSTRAINTS

A Kind of direct methods is presented for the solution of optimal control problems with state constraints.These methods are sequential quadratic programming methods.At every iteration a quadratic programming which is obtained by quadratic approximation to Lagrangian function and Linear approximations to constraints is solved to get a search direction for a merit function.The merit function is formulated by augmenting the Lagrangian funetion with a penalty term.A line search is carried out along the search direction to determine a step length such that the merit function is decreased.The methods presented in this paper include continuous sequential quadratic programming methods and discreate sequential quadrade programming methods.     

Comparison of independent,stratified and random covering sample schemes in optimization problems

We consider two schemes of global optimization algorithms based on the use of grids. Our main goal is to compare the so-called independent sampling (IS), stratified sampling (SS) and random covering (RC) grids implemented to the estimation problem of the global maximum of a function. The results give an insight on how a decrease of randomness in selection rules for the trial points improves efficiency of global random search algorithms.     

一个修正的SQP方法-滤子方法

序列二次规划方法（SQP）是解决非线性规划问题最有效的算法之一，但是当QP子问题不可行时算法可能会失败．而且线搜索中的罚参数的选择通常比较困难．在文献[1]中，SQP方法得到了修正，使得QP子问题可行．在本文中，我们利用滤子技术避免了罚函数的使用同时提出了带线搜索的滤子方法，最终保证了SQP方法总是可行的，而且得到了方法的全局收敛性．     

Genetic and Random Search Methods in Optimal Shape Design Problems

We describe the application of two global optimization methods, namely of genetic and random search type algorithms in shape optimization. When the so-called fictitious domain approaches are used for the numerical realization of state problems, the resulting minimized function is non-differentiable and stair-wise, in general. Such complicated behaviour excludes the use of classical local methods. Specific modifications of the above-mentioned global methods for our class of problems are described. Numerical results of several model examples computed by different variants of genetic and random search type algorithms are discussed.     

求解全局最优问题的多重点样本水平值估计的相对熵算法

研究有界闭箱约束下的全局最优化问题,利用相对熵及广义方差函数方程的最大根与全局最小值之间的等价关系,设计求解全局最优值的积分型水平值估计算法.对采用重点样本采样技巧产生的函数值按一定规则进行聚类,从而在各聚类中产生的若干新重点样本,结合相对熵算法,构造出多重点样本进行全局搜索的新算法.该算法的优点在于每次迭代选用当前较好的函数值信息,以达到随机搜索到更好的函数值信息.同时多重点样本可有利挖掘出更好的全局信息.一系列的数值实验表明该算法是非常有效的.     

Stochastic Methods for Practical Global Optimization

Engineering design problems often involve global optimization of functions that are supplied as black box functions. These functions may be nonconvex, nondifferentiable and even discontinuous. In addition, the decision variables may be a combination of discrete and continuous variables. The functions are usually computationally expensive, and may involve finite element methods. An engineering example of this type of problem is to minimize the weight of a structure, while limiting strain to be below a certain threshold. This type of global optimization problem is very difficult to solve, yet design engineers must find some solution to their problem – even if it is a suboptimal one. Sometimes the most difficult part of the problem is finding any feasible solution. Stochastic methods, including sequential random search and simulated annealing, are finding many applications to this type of practical global optimization problem. Improving Hit-and-Run (IHR) is a sequential random search method that has been successfully used in several engineering design applications, such as the optimal design of composite structures. A motivation to IHR is discussed as well as several enhancements. The enhancements include allowing both continuous and discrete variables in the problem formulation. This has many practical advantages, because design variables often involve a mixture of continuous and discrete values. IHR and several variations have been applied to the composites design problem. Some of this practical experience is discussed.     

Semiparametric statistical inference in global random search

The multivariate multiextremal optimization problem is considered. Various statistical procedures based on the use of the asymptotic theory of extreme order statistics are thoroughly described. These procedures are used to infer about the maximal value of a function by its values at random points. A class of global random search methods underlying the procedures is considered. These methods generalize the well-known branch and bound methods. The article is mainly of a survey nature. It also contains new results.     

总体随机搜索算法的收敛性及计算效益

本文从理论上揭示相当广泛的一类总体随机搜索算法的计算效益差的问题. 设F(x)在R~n上连续,?是一族n维概率分布;设已有当步点x_k,算法在当步迭代开始时从?中任意取一个分布,自该分布随机产生向量ξ作为当步搜索方向,根据x_k和ξ决定下一点x_(k+1).这样可形成极广的一类算法.又设水平集     

Random tunneling by means of acceptance-rejection sampling for global optimization

Any global minimization algorithm is made by several local searches performed sequentially. In the classical multistart algorithm, the starting point for each new local search is selected at random uniformly in the region of interest. In the tunneling algorithm, such a starting point is required to have the same function value obtained by the last local minimization. We introduce the class of acceptance-rejection based algorithms in order to investigate intermediate procedures. A particular instance is to choose at random the new point approximately according to a Boltzmann distribution, whose temperatureT is updated during the algorithm. AsT 0, such distribution peaks around the global minima of the cost function, producing a kind of random tunneling effect. The motivation for such an approach comes from recent works on the simulated annealing approach in global optimization. The resulting algorithm has been tested on several examples proposed in the literature.     

Optimising Noisy Objective Functions

This discussion paper considers the use of stochastic algorithms for solving global optimisation problems in which function evaluations are subject to random noise. An idea is outlined for discussion at the forthcoming Stochastic Global Optimisation 2001 workshop in Hanmer in June; we propose that a noisy version of pure random search be studied.     

一种改进的进化规划算法

本对于全局优化问题提出一个改进的进化规划算法，该算法以概率p接收基于电磁理论求出合力方向作为随机搜索方向，以概率1-p接收按正态分布产生的随机搜索方向。改进算法不仅克服了传统进化规划算法随机搜索的盲目性，而且保留了传统进化规划算法全局搜索性。本算法应用于几个典型例题，数值结果表明本算法是可行的，有效的。     

Global search algorithms for minimum concave-cost network flow problems

We present algorithms for the single-source uncapacitated version of the minimum concave cost network flow problem. Each algorithm exploits the fact that an extreme feasible solution corresponds to a sub-tree of the original network. A global search heuristic based on random extreme feasible initial solutions and local search is developed. The algorithm is used to evaluate the complexity of the randomly generated test problems. An exact global search algorithm is developed, based on enumerative search of rooted subtrees. This exact technique is extended to bound the search based on cost properties and linear underestimation. The technique is accelerated by exploiting the network structure.     

A Theoretical Approach to Restart in Global Optimization

While searching for the global minimum of a cost function we haveoftento decide if a restart from a different initial point would bemoreadvantageous than continuing current optimization. This is aparticularcase of the efficiency comparison between repeatedminimizations and singleextended search having the same total length.A theoretical approach forthe treatment of this general problem formsthe subject of the present paper.A fundamental role is played by theprobability of reaching the globalminimum, whose asymptoticalbehavior allows to provide useful information onthe efficiency ofrepeated trials.The second part of this work is devoted toa detailed analysis of threeoptimization algorithms whose evolution isindependent of the costfunction to be minimized: pure random search, grid search and randomwalk. These three examples give an interesting validationof thetheoretical results and provide a general procedure which can beemployed in the study of more complex optimization problems.     

基于正弦控制因子的Lateral变异鲨鱼优化算法

针对传统鲨鱼优化算法在求解高维目标函数时,易早熟收敛,陷入局部最优的缺陷.提出一种基于正弦控制因子的Lateral变异鲨鱼优化算法.通过正弦曲线的特性和自适应惯性权重,改善了传统鲨鱼优化算法中由于随机选取控制因子数值大小可能导致算法在迭代后期全局搜索能力降低的问题,提高了算法在迭代后期的全局收敛能力,并对最佳鲨鱼位置引入Lateral变异策略,加强了算法跳出局部最优的可能性.改进后的算法对多个shifted单峰,多峰以及固定维测试函数进行求解,实验结果表明,对比多种不同优化算法而言,本文所提LSSO算法具有更高的收敛精度和搜索速度.     

Almost Surely Convergent Global Optimization Algorithm Using Noise-Corrupted Observations

A new recursive algorithm for searching the global minimizer of a function is proposed when the function is observed with noise. The algorithm is based on switches between the stochastic approximation and the random search. The combination of SA with RS is not a new idea in such combination, the difficulty consists in creating a good switching rule and in designing an efficient method to reduce the noise effect. The proposed switching rule is easily realizable, the noise reducing method is effective, and the whole recursive optimization algorithm is simply calculated. It is proved that the algorithm a.s. converges to the global minimizer and is asymptotically normal. In comparison with existing methods, the proposed algorithm not only requires much weaker conditions, but also is more efficient as shown by simulation.     

Continuous functions minimization by dynamic random search technique

Random search technique is the simplest one of the heuristic algorithms. It is stated in the literature that the probability of finding global minimum is equal to 1 by using the basic random search technique, but it takes too much time to reach the global minimum. Improving the basic random search technique may decrease the solution time. In this study, in order to obtain the global minimum fastly, a new random search algorithm is suggested. This algorithm is called as the Dynamic Random Search Technique (DRASET). DRASET consists of two phases, which are general search and local search based on general solution. Knowledge related to the best solution found in the process of general search is kept and then that knowledge is used as initial value of local search. DRASET’s performance was experimented with 15 test problems and satisfactory results were obtained.