Conditions for Global Optimality 2

In this paper bearing the same title as our earlier survey-paper [11] we pursue the goal of characterizing the global solutions of an optimization problem, i.e. getting at necessary and sufficient conditions for a feasible point to be a global minimizer (or maximizer) of the objective function. We emphasize nonconvex optimization problems presenting some specific structures like convex-anticonvex ones or quadratic ones.     

A Multivariate Global Optimization Using Linear Bounding Functions

Recently linear bounding functions (LBFs) were proposed and used to find -global minima. This paper presents an LBF-based algorithm for multivariate global optimization problems. The algorithm consists of three phases. In the global phase, big subregions not containing a solution are quickly eliminated and those which possibly contain the solution are detected. An efficient scheme for the local phase is developed using our previous local minimization algorithm, which is globally convergent with superlinear/quadratic rate and does not require evaluation of gradients and Hessian matrices. To ensure that the found minimizers are indeed the global solutions or save computation effort, a third phase called the verification phase is often needed. Under adequate conditions the algorithm finds the -global solution(s) within finite steps. Numerical testing results illustrate how the algorithm works, and demonstrate its potential and feasibility.     

The continuous reactive tabu search: Blending combinatorial optimization and stochastic search for global optimization

A novel algorithm for the global optimization of functions (C-RTS) is presented, in which a combinatorial optimization method cooperates with a stochastic local minimizer. The combinatorial optimization component, based on the Reactive Tabu Search recently proposed by the authors, locates the most promising boxes, in which starting points for the local minimizer are generated. In order to cover a wide spectrum of possible applications without user intervention, the method is designed with adaptive mechanisms: the box size is adapted to the local structure of the function to be optimized, the search parameters are adapted to obtain a proper balance of diversification and intensification. The algorithm is compared with some existing algorithms, and the experimental results are presented for a variety of benchmark tasks.     

Global Interval Methods for Local Nonsmooth Optimization

An interval method for determining local solutions of nonsmooth unconstrained optimization problems is discussed. The objective function is assumed to be locally Lipschitz and to have appropriate interval inclusions. The method consists of two parts, a local search and a global continuation and termination. The local search consists of a globally convergent descent algorithm showing similarities to -bundle methods. While -bundle methods use polytopes as inner approximations of the -subdifferentials, which are the main tools of almost all bundle concepts, our method uses axes parallel boxes as outer approximations of the -subdifferentials. The boxes are determined almost automatically with inclusion techniques of interval arithmetic. The dimension of the boxes is equal to the dimension of the problem and remains constant during the whole computation. The application of boxes does not suffer from the necessity to invest methodical and computational efforts to adapt the polytopes to the latest state of the computation as well as to simplify them when the number of vertices becomes too large, as is the case with the polytopes. The second part of the method applies interval techniques of global optimization to the approximative local solution obtained from the search of the first part in order to determine guaranteed error bounds or to improve the solution if necessary. We present prototype algorithms for both parts of the method as well as a complete convergence theory for them and demonstrate how outer approximations can be obtained.     

A Numerical Comparison of Some Modified Controlled Random Search Algorithms

In this paper we propose a new version of the Controlled Random Search(CRS) algorithm of Price. The new algorithmhas been tested on thirteen global optimization test problems. Numericalexperiments indicate that the resulting algorithm performs considerablybetter than the earlier versions of the CRS algorithms. The algorithm,therefore, could offer a reasonable alternative to many currently availablestochastic algorithms, especially for problems requiring direct searchtype methods. Also a classification of the CRS algorithms is made based onglobal technique – local technique and the relative performance ofclasses is numerically explored.     

Global optimization conditions for certain nonconvex minimization problems

By means of suitable dual problems to the following global optimization problems: extremum{f(x): x M X}, wheref is a proper convex and lower-semicontinuous function andM a nonempty, arbitrary subset of a reflexive Banach spaceX, we derive necessary and sufficient optimality conditions for a global minimizer. The method is also applicable to other nonconvex problems and leads to at least necessary global optimality conditions.     

Global optimization and stochastic differential equations

Let ⁿ be then-dimensional real Euclidean space,x=(x ₁,x ₂, ...,x _n)^{T n} , and letf: ⁿ R be a real-valued function. We consider the problem of finding the global minimizers off. A new method to compute numerically the global minimizers by following the paths of a system of stochastic differential equations is proposed. This method is motivated by quantum mechanics. Some numerical experience on a set of test problems is presented. The method compares favorably with other existing methods for global optimization.This research has been supported by the European Research Office of the US Army under Contract No. DAJA-37-81-C-0740.The third author gratefully acknowledges Prof. A. Rinnooy Kan for bringing to his attention Ref. 4.     

Epitaxial Growth Without Slope Selection: Energetics,Coarsening, and Dynamic Scaling

We study a continuum model for epitaxial growth of thin films in which the slope of mound structure of film surface increases. This model is a diffusion equation for the surface height profile h which is assumed to satisfy the periodic boundary condition. The equation happens to possess a Liapunov or free-energy functional. This functional consists of the term | h|², which represents the surface diffusion, and - log (1 + | h|²), which describes the effect of kinetic asymmetry in the adatom attachment-detachment. We first prove for large time t that the interface width---the standard deviation of the height profile---is bounded above by O(t^1/2), the averaged gradient is bounded above by O(t^1/4), and the averaged energy is bounded below by O(- log t). We then consider a small coefficient ² of | h|² with = 1/L and L the linear size of the underlying system, and study the energy asymptotics in the large system limit 0. We show that global minimizers of the free-energy functional exist for each > 0, the L²-norm of the gradient of any global minimizer scales as O(1/), and the global minimum energy scales as O( log ). The existence of global energy minimizers and a scaling argument are used to construct a sequence of equilibrium solutions with different wavelengths. Finally, we apply our minimum energy estimates to derive bounds in terms of the linear system size L for the saturation interface width and the corresponding saturation time.     

A cut-peak function method for global optimization

A new method is proposed for solving box constrained global optimization problems. The basic idea of the method is described as follows: Constructing a so-called cut-peak function and a choice function for each present minimizer, the original problem of finding a global solution is converted into an auxiliary minimization problem of finding local minimizers of the choice function, whose objective function values are smaller than the previous ones. For a local minimum solution of auxiliary problems this procedure is repeated until no new minimizer with a smaller objective function value could be found for the last minimizer. Construction of auxiliary problems and choice of parameters are relatively simple, so the algorithm is relatively easy to implement, and the results of the numerical tests are satisfactory compared to other methods.     

Studying the Complexity of Global Verification for NP-Hard Discrete Optimization Problems

This paper examines the complexity of global verification for MAX-SAT, MAX-k-SAT (for k3), Vertex Cover, and Traveling Salesman Problem. These results are obtained by adaptations of the transformations that prove such problems to be NP-complete. The class of problems PGS is defined to be those discrete optimization problems for which there exists a polynomial time algorithm such that given any solution , either a solution can be found with a better objective function value or it can be concluded that no such solution exists and is a global optimum. This paper demonstrates that if any one of MAX-SAT, MAX-k-SAT (for k3), Vertex Cover, or Traveling Salesman Problem are in PGS, then P=NP.     

The cluster problem in multivariate global optimization

We consider branch and bound methods for enclosing all unconstrained global minimizers of a nonconvex nonlinear twice-continuously differentiable objective function. In particular, we consider bounds obtained with interval arithmetic, with the midpoint test, but no acceleration procedures. Unless the lower bound is exact, the algorithm without acceleration procedures in general gives an undesirable cluster of boxes around each minimizer. In a previous paper, we analyzed this problem for univariate objective functions. In this paper, we generalize that analysis to multi-dimensional objective functions. As in the univariate case, the results show that the problem is highly related to the behavior of the objective function near the global minimizers and to the order of the corresponding interval extension.This work was partially funded by National Science Foundation grant # CCR-9203730.     

On the complexity of approximating a KKT point of quadratic programming

We present a potential reduction algorithm to approximate a Karush—Kuhn—Tucker (KKT) point of general quadratic programming (QP). We show that the algorithm is a fully polynomial-time approximation scheme, and its running-time dependency on accuracy (0, 1) is O((l/) log(l/) log(log(l/))), compared to the previously best-known result O((l/)²). Furthermore, the limit of the KKT point satisfies the second-order necessary optimality condition of being a local minimizer. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.Research support in part by NSF grants DDM-9207347 and DMI-9522507, and the Iowa Business School Summer Grant.     

Family of Projected Descent Methods for Optimization Problems with Simple Bounds

This paper presents a family of projected descent direction algorithms with inexact line search for solving large-scale minimization problems subject to simple bounds on the decision variables. The global convergence of algorithms in this family is ensured by conditions on the descent directions and line search. Whenever a sequence constructed by an algorithm in this family enters a sufficiently small neighborhood of a local minimizer satisfying standard second-order sufficiency conditions, it gets trapped and converges to this local minimizer. Furthermore, in this case, the active constraint set at is identified in a finite number of iterations. This fact is used to ensure that the rate of convergence to a local minimizer, satisfying standard second-order sufficiency conditions, depends only on the behavior of the algorithm in the unconstrained subspace. As a particular example, we present projected versions of the modified Polak–Ribière conjugate gradient method and the limited-memory BFGS quasi-Newton method that retain the convergence properties associated with those algorithms applied to unconstrained problems.     

Numerical Experience with a Class of Self-Scaling Quasi-Newton Algorithms

Self-scaling quasi-Newton methods for unconstrained optimization depend upon updating the Hessian approximation by a formula which depends on two parameters (say, and ) such that = 1, = 0, and = 1 yield the unscaled Broyden family, the BFGS update, and the DFP update, respectively. In previous work, conditions were obtained on these parameters that imply global and superlinear convergence for self-scaling methods on convex objective functions. This paper discusses the practical performance of several new algorithms designed to satisfy these conditions.     

Global optimization of univariate Lipschitz functions: II. New algorithms and computational comparison

We consider the following global optimization problems for a Lipschitz functionf implicitly defined on an interval [a, b]. Problem P: find a globally-optimal value off and a corresponding point; Problem Q: find a set of disjoint subintervals of [a, b] containing only points with a globally-optimal value and the union of which contains all globally optimal points. A two-phase algorithm is proposed for Problem P. In phase I, this algorithm obtains rapidly a solution which is often globally-optimal. Moreover, a sufficient condition onf for this to be the case is given. In phase II, the algorithm proves the-optimality of the solution obtained in phase I or finds a sequence of points of increasing value containing one with a globally-optimal value. The new algorithm is empirically compared (on twenty problems from the literature) with a best possible algorithm (for which the optimal value is assumed to be known), with a passive algorithm and with the algorithms of Evtushenko, Galperin, Shen and Zhu, Piyavskii, Timonov and Schoen. For small, the new algorithm requires only a few percent more function evaluations than the best possible one. An extended version of Piyavskii's algorithm is proposed for problem Q. A sufficient condition onf is given for the globally optimal points to be in one-to-one correspondance with the obtained intervals. This result is achieved for all twenty test problems.The research of the authors has been supported by AFOSR grants 0271 and 0066 to Rutgers University. Research of the second author has been also supported by NSERC grant GP0036426, FCAR grant 89EQ4144 and partially by AFOSR grant 0066. We thank Nicole Paradis for her help in drawing the figures.     

Parallel continuation-based global optimization for molecular conformation and protein folding

This paper presents our recent work on developing parallel algorithms and software for solving the global minimization problem for molecular conformation, especially protein folding. Global minimization problems are difficult to solve when the objective functions have many local minimizers, such as the energy functions for protein folding. In our approach, to avoid directly minimizing a difficult function, a special integral transformation is introduced to transform the function into a class of gradually deformed, but smoother or easier functions. An optimization procedure is then applied to the new functions successively, to trace their solutions back to the original function. The method can be applied to a large class of nonlinear partially separable functions including energy functions for molecular conformation and protein folding. Mathematical theory for the method, as a special continuation approach to global optimization, is established. Algorithms with different solution tracing strategies are developed. Different levels of parallelism are exploited for the implementation of the algorithms on massively parallel architectures.     

Optimization over the efficient set: overview

Over the past several decades, the optimization over the efficient set has seen a substantial development. The aim of this paper is to provide a state-of-the-art survey of the development. Given p linear criteria c ¹x,,c^p x and a feasible region X of R ⁿ, the linear multicriteria problem is to find a point x of X such that no point x' of X satisfies (c¹ x',,c^p x')(c¹ x,,c^p x) and (c¹x',,c^p x')q (c¹ x ,,c^p x). Such a point is called an efficient point. The optimization over the efficient set is the maximization of a given function over the set of efficient points. The difficulty of this problem is mainly due to the nonconvexity of this set. The existing algorithms for solving this problem could be classified into several groups such as adjacent vertex search algorithm, nonadjacent vertex search algorithm, branch-and-bound based algorithm, Lagrangian relaxation based algorithm, dual approach and bisection algorithm. In this paper we review a typical algorithm from each group and compare them from the computational point of view.     

On Logarithmic Smoothing of the Maximum Function

We consider the maximum function f resulting from a finite number of smooth functions. The logarithmic barrier function of the epigraph of f gives rise to a smooth approximation g of f itself, where >0 denotes the approximation parameter. The one-parametric family g converges – relative to a compact subset – uniformly to the function f as tends to zero. Under nondegeneracy assumptions we show that the stationary points of g and f correspond to each other, and that their respective Morse indices coincide. The latter correspondence is obtained by establishing smooth curves x() of stationary points for g , where each x() converges to the corresponding stationary point of f as tends to zero. In case of a strongly unique local minimizer, we show that the nondegeneracy assumption may be relaxed in order to obtain a smooth curve x().     

αBB: A global optimization method for general constrained nonconvex problems

A branch and bound global optimization method,BB, for general continuous optimization problems involving nonconvexities in the objective function and/or constraints is presented. The nonconvexities are categorized as being either of special structure or generic. A convex relaxation of the original nonconvex problem is obtained by (i) replacing all nonconvex terms of special structure (i.e. bilinear, fractional, signomial) with customized tight convex lower bounding functions and (ii) by utilizing the parameter as defined in [17] to underestimate nonconvex terms of generic structure. The proposed branch and bound type algorithm attains finite-convergence to the global minimum through the successive subdivision of the original region and the subsequent solution of a series of nonlinear convex minimization problems. The global optimization method,BB, is implemented in C and tested on a variety of example problems.     

Constraint decomposition algorithms in global optimization

Many global optimization problems can be formulated in the form min{c(x, y): x X, y Y, (x, y) Z, y G} where X, Y are polytopes in ^p , ⁿ , respectively, Z is a closed convex set in ^p+n, while G is the complement of an open convex set in ⁿ . The function c: ^p+n is assumed to be linear. Using the fact that the nonconvex constraints depend only upon they-variables, we modify and combine basic global optimization techniques such that some new decomposition methods result which involve global optimization procedures only in ⁿ . Computational experiments show that the resulting algorithms work well for problems with smalln.