A search technique for global optimization in a chaotic environment

We describe a new algorithm which uses the trajectories of a discrete dynamical system to sample the domain of an unconstrained objective function in search of global minima. The algorithm is unusually adept at avoiding nonoptimal local minima and successfully converging to a global minimum. Trajectories generated by the algorithm for objective functions with many local minima exhibit chaotic behavior, in the sense that they are extremely sensitive to changes in initial conditions and system parameters. In this context, chaos seems to have a beneficial effect: failure to converge to a global minimum from a given initial point can often be rectified by making arbitrarily small changes in the system parameters.     

A hybrid multiagent approach for global trajectory optimization

In this paper we consider a global optimization method for space trajectory design problems. The method, which actually aims at finding not only the global minimizer but a whole set of low-lying local minimizers (corresponding to a set of different design options), is based on a domain decomposition technique where each subdomain is evaluated through a procedure based on the evolution of a population of agents. The method is applied to two space trajectory design problems and compared with existing deterministic and stochastic global optimization methods.     

Pure adaptive search for finite global optimization

Pure Adaptive Search is a stochastic algorithm which has been analyzed for continuous global optimization. When a uniform distribution is used in PAS, it has been shown to have complexity which is linear in dimension. We define strong and weak variations of PAS in the setting of finite global optimization and prove analogous results. In particular, for then-dimensional lattice {1,,k} ⁿ , the expected number of iterations to find the global optimum is linear inn. Many discrete combinatorial optimization problems, although having intractably large domains, have quite small ranges. The strong version of PAS for all problems, and the weak version of PAS for a limited class of problems, has complexity the order of the size of the range.The authors would like to thank the Department of Mathematics and Statistics at the University of Canterbury for support of this research.     

A Metropolis algorithm combined with Hooke-Jeeves local search method applied to global optimization

A hybridization of a recently introduced Metropolis algorithm named the Particle Collision Algorithm (PCA) and the Hooke-Jeeves local search method is applied to a testbed of global optimization functions and to real-world chemical equilibrium nonlinear systems. The results obtained by this method, called HJPCA, are compared against those achieved by two state-of-the-art global optimization methods, C-GRASP and GLOBAL. HJPCA performs better than both algorithms, thus demonstrating its potential for other applications.     

A deterministic algorithm for global optimization

We present an algorithm for finding the global maximum of a multimodal, multivariate function for which derivatives are available. The algorithm assumes a bound on the second derivatives of the function and uses this to construct an upper envelope. Successive function evaluations lower this envelope until the value of the global maximum is known to the required degree of accuracy. The algorithm has been implemented in RATFOR and execution times for standard test functions are presented at the end of the paper.Partially supported by NSF DMS-8718362.     

Computational comparison of two methods for constrained global optimization

For constrained concave global minimization problems, two very different solution techniques have been investigated. The first such method is a stochastic mulitstart approach which typically finds, with high probability, all local minima for the problem. The second method is deterministic and guarantees a global minimum solution to within any user specified tolerance. It is the purpose of this paper to make a careful comparison of these two methods on a range of test problems using separable concave objectives over compact polyhedral sets, and to investigate in this way the advantages and disadvantages of each method. A direct computational comparison, on the same set of over 140 problems, is presented.     

A particle swarm pattern search method for bound constrained global optimization

In this paper we develop, analyze, and test a new algorithm for the global minimization of a function subject to simple bounds without the use of derivatives. The underlying algorithm is a pattern search method, more specifically a coordinate search method, which guarantees convergence to stationary points from arbitrary starting points. In the optional search phase of pattern search we apply a particle swarm scheme to globally explore the possible nonconvexity of the objective function. Our extensive numerical experiments showed that the resulting algorithm is highly competitive with other global optimization methods also based on function values. Support for Luís N. Vicente was provided by Centro de Matemática da Universidade de Coimbra and by FCT under grant POCI/MAT/59442/2004.     

Enhancing PSO methods for global optimization

The Particle Swarm Optimization (PSO) method is a well-established technique for global optimization. During the past years several variations of the original PSO have been proposed in the relevant literature. Because of the increasing necessity in global optimization methods in almost all fields of science there is a great demand for efficient and fast implementations of relative algorithms. In this work we propose three modifications of the original PSO method in order to increase the speed and its efficiency that can be applied independently in almost every PSO variant. These modifications are: (a) a new stopping rule, (b) a similarity check and (c) a conditional application of some local search method. The proposed were tested using three popular PSO variants and a variety test functions. We have found that the application of these modifications resulted in significant gain in speed and efficiency.     

Parallel algorithms for global optimization

In this paper, we report results of implementations of algorithms designed (i) to solve the global optimization problem (GOP) and (ii) to run on a parallel network of transputers. There have always been two alternative approaches to the solution of the GOP, probabilistic and deterministic. Interval methods can be implemented on our network of transputers using Concurrent ADA, and a secondary objective of the tests reported was to investigate the relative computer times required by parallel interval algorithms compared to probabilistic methods.     

Generalized primal-relaxed dual approach for global optimization

A generalized primal-relaxed dual algorithm for global optimization is proposed and its convergence is proved. The (GOP) algorithm of Floudas and Visweswaran (Refs. 1–2) is shown to be a special case of this general algorithm. Within the proposed framework, the algorithm of Floudas and Visweswaran (Refs. 1–2) is further extended to the nonsmooth case. A penalty implementation of the extended (GOP) algorithm is studied to improve its efficiency.     

Nonautonomous stochastic search for global minimum in continuous optimization

Various iterative stochastic optimization schemes can be represented as discrete-time Markov processes defined by the nonautonomous equation _Xt+1=_Tt(_Xt,_Yt)$X_{t+1}=T_t(X_t,Y_t)$, where _Yt$Y_t$ is an independent sequence and _Tt$T_t$ is a sequence of mappings. This paper presents a general framework for the study of the stability and convergence of such optimization processes. Some applications are given: the mathematical convergence analysis of two optimization methods, the elitist evolution strategy (μ+λ)$(\mu +\lambda )$ and the grenade explosion method, is presented.     

An analytical approach to global optimization

Global optimization problems with a few variables and constraints arise in numerous applications but are seldom solved exactly. Most often only a local optimum is found, or if a global optimum is detected no proof is provided that it is one. We study here the extent to which such global optimization problems can be solved exactly using analytical methods. To this effect, we propose a series of tests, similar to those of combinatorial optimization, organized in a branch-and-bound framework. The first complete solution of two difficult test problems illustrates the efficiency of the resulting algorithm. Computational experience with the programbagop, which uses the computer algebra systemmacsyma, is reported on. Many test problems from the compendiums of Hock and Schittkowski and others sources have been solved.The research of the first and the third authors has been supported by AFOSR grants #0271 and #0066 to Rutgers University. Research of the second author has been supported by NSERC grant #GP0036426 and FCAR grants #89EQ4144 and #90NC0305.     

Implementing pure adaptive search for global optimization using Markov chain sampling

The Pure Adaptive Search (PAS) algorithm for global optimization yields a sequence of points, each of which is uniformly distributed in the level set corresponding to its predecessor. This algorithm has the highly desirable property of solving a large class of global optimization problems using a number of iterations that increases at most linearly in the dimension of the problem. Unfortunately, PAS has remained of mostly theoretical interest due to the difficulty of generating, in each iteration, a point uniformly distributed in the improving feasible region. In this article, we derive a coupling equivalence between generating an approximately uniformly distributed point using Markov chain sampling, and generating an exactly uniformly distributed point with a certain probability. This result is used to characterize the complexity of a PAS-implementation as a function of (a) the number of iterations required by PAS to achieve a certain solution quality guarantee, and (b) the complexity of the sampling algorithm used. As an application, we use this equivalence to show that PAS, using the so-called Random ball walk Markov chain sampling method for generating nearly uniform points in a convex region, can be used to solve most convex programming problems in polynomial time.     

Deletion-by-infeasibility rule for DC-constrained global optimization

In Ref. 1, a general class of branch-and-bound methods was proposed by Horst for solving global optimization problems. One of the main contributions of Ref. 1 was the opportunity of handling partition elements whose feasibility is not known. Deletion-by-infeasibility rules were presented for problems where the feasible set is convex, is defined by finitely many convex and reverse convex constraints, or is defined by Lipschitzian inequalities. In this note, we propose a new deletion-by-infeasibility rule for problems whose feasible set is defined by functions representable as differences of convex functions.This research was supported in part by the Hungarian National Research Foundation, Grant OTKA No. 2568.     

A global optimization procedure for the location of a median line in the three-dimensional space

A global optimization procedure is proposed to find a line in the Euclidean three-dimensional space which minimizes the sum of distances to a given finite set of three-dimensional data points.Although we are using similar techniques as for location problems in two dimensions, it is shown that the problem becomes much harder to solve. However, a problem parameterization as well as lower bounds are suggested whereby we succeeded in solving medium-size instances in a reasonable amount of computing time.     

Accelerations for global optimization covering methods using second derivatives

Two improvements for the algorithm of Breiman and Cutler are presented. Better envelopes can be built up using positive quadratic forms. Better utilization of first and second derivative information is attained by combining both global aspects of curvature and local aspects near the global optimum. The basis of the results is the geometric viewpoint developed by the first author and can be applied to a number of covering type methods. Improvements in convergence rates are demonstrated empirically on standard test functions.Partially supported by an University of Canterbury Erskine grant.     

Stochastic global optimization methods part I: Clustering methods

In this stochastic approach to global optimization, clustering techniques are applied to identify local minima of a real valued objective function that are potentially global. Three different methods of this type are described; their accuracy and efficiency are analyzed in detail.     

Bayesian stopping rules for multistart global optimization methods

By far the most efficient methods for global optimization are based on starting a local optimization routine from an appropriate subset of uniformly distributed starting points. As the number of local optima is frequently unknown in advance, it is a crucial problem when to stop the sequence of sampling and searching. By viewing a set of observed minima as a sample from a generalized multinomial distribution whose cells correspond to the local optima of the objective function, we obtain the posterior distribution of the number of local optima and of the relative size of their regions of attraction. This information is used to construct sequential Bayesian stopping rules which find the optimal trade off between reliability and computational effort.     

Scatter search for chemical and bio-process optimization

Scatter search is a population-based method that has recently been shown to yield promising outcomes for solving combinatorial and nonlinear optimization problems. Based on formulations originally proposed in 1960s for combining decision rules and problem constraints such as the surrogate constraint method, scatter search uses strategies for combining solution vectors that have proved effective in a variety of problem settings. In this paper, we develop a general purpose heuristic for a class of nonlinear optimization problems. The procedure is based on the scatter search methodology and treats the objective function evaluation as a black box, making the search algorithm context-independent. Most optimization problems in the chemical and bio-chemical industries are highly nonlinear in either the objective function or the constraints. Moreover, they usually present differential-algebraic systems of constraints. In this type of problem, the evaluation of a solution or even the feasibility test of a set of values for the decision variables is a time-consuming operation. In this context, the solution method is limited to a reduced number of solution examinations. We have implemented a scatter search procedure in Matlab (Mathworks, 2004) for this special class of difficult optimization problems. Our development goes beyond a simple exercise of applying scatter search to this class of problems, but presents innovative mechanisms to obtain a good balance between intensification and diversification in a short-term search horizon. Computational comparisons with other recent methods over a set of benchmark problems favor the proposed procedure.     

Properties of the random search in global optimization

From theorems which we prove about the behavior of gaps in a set ofN uniformly random points on the interval [0, 1], we determine properties of the random search procedure in one-dimensional global optimization. In particular, we show that the uniform grid search is better than the random search when the optimum is chosen using the deterministic strategy, that a significant proportion of large gaps are contained in the uniformly random search, and that the error in the determination of the point at which the optimum occurs, assuming that it is unique, will on the average be twice as large using the uniformly random search compared with the uniform grid. In addition, some of the properties of the largest gap are verified numerically, and some extensions to higher dimensions are discussed. The latter show that not all of the conclusions derived concerning the inadequacies of the one-dimensional random search extend to higher dimensions, and thaton average the random search is better than the uniform grid for dimensions greater than 6.This paper is based on work started in the Statistics Department of Princeton University when the first author was visiting as a Research Associate. Part of this research was supported by the Office of Naval Research, Contract No. 0014-67-A-0151-0017, and by the US Army Research Office—Durham, Contract No. DA-31-124-ARO-D-215.2 The authors wish to thank B. Omodei for his careful work in preparing the programs for the results of Figs. 1–2 and Table 1. The computations were performed on the IBM 360/50 of the Australian National University's Computer Centre. Thanks are also due to R. Miles for suggestions regarding the extension of the results to multidimensional regions, and to P. A. P. Moran and R. Brent for suggestions regarding the evaluation of the integral ₀ ¹ ... ^0/1 (x ₁ ² + ... +x _n ^/2 )^1/2 dx ₁ ...dx _n.