Global optimization of univariate Lipschitz functions: I. Survey and properties

We consider the following global optimization problems for a univariate Lipschitz functionf defined on an interval [a, b]: Problem P: find a globally optimal value off and a corresponding point; Problem P: find a globally-optimal value off and a corresponding point; Problem Q: localize all globally optimal points; Problem Q: find a set of disjoint subintervals of small length whose union contains all globally optimal points; Problem Q: find a set of disjoint subintervals containing only points with a globally-optimal value and whose union contains all globally optimal points.We present necessary conditions onf for finite convergence in Problem P and Problem Q, recall the concepts necessary for a worst-case and an empirical study of algorithms (i.e., those ofpassive and ofbest possible algorithms), summarize and discuss algorithms of Evtushenko, Piyavskii-Shubert, Timonov, Schoen, Galperin, Shen and Zhu, presenting them in a simplified and uniform way, in a high-level computer language. We address in particular the problems of using an approximation for the Lipschitz constant, reducing as much as possible the expected length of the region of indeterminacy which contains all globally optimal points and avoiding remaining subintervals without points with a globally-optimal value. New algorithms for Problems P and Q and an extensive computational comparison of algorithms are presented in a companion paper.The research of the authors has been supported by AFOSR grants 0271 and 0066 to Rutgers University. Research of the second author has been also supported by NSERC grant GP0036426 and FCAR grant 89EQ4144. We thank N. Paradis for drawing some of the figures.     

On Timonov's algorithm for global optimization of univariate Lipschitz functions

Timonov proposes an algorithm for global maximization of univariate Lipschitz functions in which successive evaluation points are chosen in order to ensure at each iteration a maximal expected reduction of the region of indeterminacy, which contains all globally optimal points. It is shown that such an algorithm does not necessarily converge to a global optimum.     

On using estimates of Lipschitz constants in global optimization

Several authors have proposed estimating Lipschitz constants in global optimization by a multiple of the largest slope (in absolute value) between successive evaluation points. A class of univariate functions is exhibited for which the global optimum will be missed when using such a procedure, even if the multiple is arbitrarily large.Research of the first and third authors was supported by AFOSR Grants 0271 and 0066 to Rutgers University. Research of the second author was supported by NSERC Grant GP0036426 and FCAR Grant 90NC0305.This research was done while the first author was Professor and the third author was Graduate Student at RUTCOR, Rutgers University.     

Efficient domain partitioning algorithms for global optimization of rational and Lipschitz continuous functions

A domain partitioning algorithm for minimizing or maximizing a Lipschitz continuous function is enhanced to yield two new, more efficient algorithms. The use of interval arithmetic in the case of rational functions and the estimates of Lipschitz constants valid in subsets of the domain in the case of others and the addition of local optimization have resulted in an algorithm which, in tests on standard functions, performs well.     

An algorithm for global optimization of Lipschitz continuous functions

An algorithm is presented which locates the global minimum or maximum of a function satisfying a Lipschitz condition. The algorithm uses lower bound functions defined on a partitioned domain to generate a sequence of lower bounds for the global minimum. Convergence is proved, and some numerical results are presented.     

A global minimization algorithm for Lipschitz functions

The global optimization problem with and f(x) satisfying the Lipschitz condition , is considered. To solve it a region-search algorithm is introduced. This combines a local minimum algorithm with a procedure that at the ith iteration finds a region S _i where the global minimum has to be searched for. Specifically, by making use of the Lipschitz condition, S _i , which is a sequence of intervals, is constructed by leaving out from S _i-1 an interval where the global minimum cannot be located. A convergence property of the algorithm is given. Further, the ratio between the measure of the initial feasible region and that of the unexplored region may be used as stop rule. Numerical experiments are carried out; these show that the algorithm works well in finding and reducing the measure of the unexplored region.     

Global optimization of Hölder functions

We propose a branch-and-bound framework for the global optimization of unconstrained Hölder functions. The general framework is used to derive two algorithms. The first one is a generalization of Piyavskii's algorithm for univariate Lipschitz functions. The second algorithm, using a piecewise constant upper-bounding function, is designed for multivariate Hölder functions. A proof of convergence is provided for both algorithms. Computational experience is reported on several test functions from the literature.     

New LP bound in multivariate Lipschitz optimization: Theory and applications

Most numerically promising methods for solving multivariate unconstrained Lipschitz optimization problems of dimension greater than two use rectangular or simplicial branch-and-bound techniques with computationally cheap but rather crude lower bounds.Generalizations to constrained problems, however, require additional devices to detect sufficiently many infeasible partition sets. In this article, a new lower bounding procedure is proposed for simplicial methods yielding considerably better bounds at the expense of two linear programs in each iteration. Moreover, the resulting approach can solve easily linearly constrained problems, since in this case infeasible partition sets are automatically detected by the lower bounding procedure.Finally, it is shown that the lower bounds can be further improved when the method is applied to solve systems of inequalities. Implementation issues, numerical experiments, and comparisons are discussed in some detail.The authors are indebted to the Editor-in-Chief of this journal for his valuable suggestions which have considerably improved the final version of this article.     

A nonsmooth version of the univariate optimization algorithm for locating the nearest extremum (locating extremum in nonsmooth univariate optimization)

An algorithm for univariate optimization using a linear lower bounding function is extended to a nonsmooth case by using the generalized gradient instead of the derivative. A convergence theorem is proved under the condition of semismoothness. This approach gives a globally superlinear convergence of algorithm, which is a generalized Newton-type method.      

A univariate global search working with a set of Lipschitz constants for the first derivative

In the paper, a global optimization problem is considered where the objective function f (x) is univariate, black-box, and its first derivative f ′(x) satisfies the Lipschitz condition with an unknown Lipschitz constant K. In the literature, there exist methods solving this problem by using an a priori given estimate of K, its adaptive estimates, and adaptive estimates of local Lipschitz constants. Algorithms working with a number of Lipschitz constants for f ′(x) chosen from a set of possible values are not known in spite of the fact that a method working in this way with Lipschitz objective functions, DIRECT, has been proposed in 1993. A new geometric method evolving its ideas to the case of the objective function having a Lipschitz derivative is introduced and studied in this paper. Numerical experiments executed on a number of test functions show that the usage of derivatives allows one to obtain, as it is expected, an acceleration in comparison with the DIRECT algorithm. This research was supported by the RFBR grant 07-01-00467-a and the grant 4694.2008.9 for supporting the leading research groups awarded by the President of the Russian Federation.     

Cord-slope form of Taylor's expansion in univariate global optimization

Interval arithmetic and Taylor's formula can be used to bound the slope of the cord of a univariate function at a given point. This leads in turn to bounding the values of the function itself. Computing such bounds for the function, its first and second derviatives, allows the determination of intervals in which this function cannot have a global minimum. Exploiting this information together with a simple branching rule yields an efficient algorithm for global minimization of univariate functions. Computational experience is reported.The first and second authors have been supported by FCAR (Fonds pour la Formation de Chercheurs et l'Aide à la Recherche) Grant 92EQ1048 and AFOSR Grant 90-0008 to Rutgers University. The first author has also been supported by NSERC (Natural Sciences and Engineering Research Council of Canada) Grant to HEC and NSERC Grant GP0105574. The second author has been supported by NSERC Grant GP0036426, FCAR Grant 90NC0305, and a NSF Visiting Professorship for Women in Science at Princeton University. Work of the third author was done in part while he was a graduate student at the Department of Mathematics, Rutgers University, New Brunswick, New Jersey, USA and during a visit to GERAD, June–August 1991.     

Global maximization of UTA functions in multi-objective optimization

The UTAs (UTilité Additives) type methods for constructing nondecreasing additive utility functions were first proposed by Jacquet-Lagrèze and Siskos in 1982 for handling decision problems of multicriteria ranking. In this article, by UTA functions, we mean functions which are constructed by the UTA type methods. Our purpose is to propose an algorithm for globally maximizing UTA functions of a class of linear/convex multiple objective programming problems. The algorithm is established based on a branch and bound scheme, in which the branching procedure is performed by a so-called I-rectangular bisection in the objective (outcome) space, and the bounding procedure by some convex or linear programs. Preliminary computational experiments show that this algorithm can work well for the case where the number of objective functions in the multiple objective optimization problem under consideration is much smaller than the number of variables.     

Index branch-and-bound algorithm for Lipschitz univariate global optimization with multiextremal constraints

In this paper, Lipschitz univariate constrained global optimization problems where both the objective function and constraints can be multiextremal are considered. The constrained problem is reduced to a discontinuous unconstrained problem by the index scheme without introducing additional parameters or variables. A Branch-and-Bound method that does not use derivatives for solving the reduced problem is proposed. The method either determines the infeasibility of the original problem or finds lower and upper bounds for the global solution. Not all the constraints are evaluated during every iteration of the algorithm, providing a significant acceleration of the search. Convergence conditions of the new method are established. Extensive numerical experiments are presented.     

An improved univariate global optimization algorithm with improved linear lower bounding functions

Recently linear lower bounding functions (LLBF's) were proposed and used to find -global minima. Basically an LLBF over an interval is a linear function which lies below a given function over the interval and matches the function value at one end point. By comparing it with the best function value found, it can be used to eliminate subregions which do not contain -global minima. To develop a more efficient LLBF algorithm, two important issues need to be addressed: how to construct a better LLBF and how to use it efficiently. In this paper, an improved LLBF for factorable functions overn-dimensional boxes is derived, in the sense that the new LLBF is always better than those in [3] for continuously differentiable functions. Exploration of the properties of the LLBF enables us to develop a new LLBF-based univariate global optimization algorithm, which is again better than those in [3]. Numerical results on some standard test functions indicate the high potential of our algorithm.This work was supported in part by VLSI Technology Inc. and Tyecin Systems Inc. through the University of California MICRO proram with grant number 92-024.     

Inclusion functions and global optimization II

This paper discusses algorithms of Moore, Skelboe, Ichida, Fujii and Hansen for solving the global unconstrained optimization problem. These algorithms have been tried on computers, but a thorough theoretical discussion of their convergence properties has been missing. The discussion was started in part I of this paper (Mathematical Programming 33 (1985) 300–317) where the convergence to the global minimum was studied. The present paper is concerned with the different behaviours of these algorithms when they are used for the determination of global minimum points. The solution sets of the algorithms can be a subset of the set of global minimum points,G, a superset ofG, or exactlyG. The algorithms are applicable to a very general class of functions: functions which are continuous, and have suitable inclusion functions. The number of global minimum points can be infinite.This work was supported by the Deutsche Forschungsgemeinschaft.     

New filled functions for nonsmooth global optimization

The filled function method is an effective approach to find a global minimizer for a general class of nonsmooth programming problems with a closed bounded domain. This paper gives a new definition for the filled function, which overcomes some drawbacks of the previous definition. It proposes a two-parameter filled function and a one-parameter filled function to improve the efficiency of numerical computation. Based on these analyses, two corresponding filled function algorithms are presented. They are global optimization methods which modify the objective function as a filled function, and which find a better local minimizer gradually by optimizing the filled function constructed on the minimizer previously found. Numerical results obtained indicate the efficiency and reliability of the proposed filled function methods.     

On the Lipschitz perturbation of monotonic functions

Summary A real valued function <InlineEquation ID=IE"5"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"6"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"7"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"8"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"9"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"10"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"11"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"12"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"13"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"14"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"15"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"16"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"17"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"18"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"19"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"20"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"21"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"22"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"23"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"24"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"25"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"26"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"27"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"28"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"29"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"30"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"31"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"32"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"33"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"34"><EquationSource Format="TEX"><![CDATA[$]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>f$ defined on a real interval $I$ is called \emph{$d$-Lipschitz} if it satisfies $|\ell(x)- \ell(y)| \le d(x,y)$ for $x,y\in I$. In this paper, we investigate when a function $p\: I \to \bR$ can be decomposed in the form $p=q+ \ell$, where $q$ is increasing and $\ell$ is $d$-Lipschitz. In the general case when $d\: I^{2} \to \bR$ is an arbitrary semimetric, a function $p\: I \to \bR$ can be written in the form $p=q+ \ell$ if and only if \vspace{-4pt} <InlineEquation ID=IE"1"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"2"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"3"><EquationSource Format="TEX"><![CDATA[<InlineEquation ID=IE"4"><EquationSource Format="TEX"><![CDATA[$$]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation>]]></EquationSource></InlineEquation> \sum_{i=1}^{n}{\big(p(s_{i})-p(t_{i})-d(t_{i},s_{i}) \big)^{+}} \le \sum_{j=1}^{m}{\big(p(v_{j})-p(u_{j})+d(u_{j},v_{j}) \big)} \vspace{-4pt} $$ is fulfilled for all real numbers $t_{1}<s_{1}, \dots, t_{n}<s_{n}$ and $u_{1}<v_{1}, \dots, u_{m}<v_{m}$ in $I$ satisfying the condition \vspace{-4pt} $$ \sum_{i=1}^{n} 1_{\left]t_i,s_i\right]}= \sum_{j=1}^{m} 1_{\left]u_j,v_j\right]}, \vspace{-4pt} $$ where $1_{\left]a,b\right]}$ denotes the characteristic function of the interval $\left]a,b\right]$. In the particular case when $d\: I^{2} \to R$ is a so-called concave semimetric, a function $p\: I \to \bR$ is of the form $p=q+ \ell$ if and only if \vspace{-4pt} $$ 0 \le \sum_{k=1}^{n}{d(x_{2k-1},x_{2k})} + d(x_0,x_{2n+1}) + \sum_{k=0}^{n}{\big(p(x_{2k+1})-p(x_{2k})\big)} \vspace{-4pt} $$ holds for all $x_0\le x_1\ki \cdots\ki x_{2n}\le x_{2n+1}$ in $I$.     

Global optimization for sum of geometric fractional functions

This paper presents an efficient branch and bound algorithm for globally solving sum of geometric fractional functions under geometric constraints, which arise in various practical problems. By using an equivalent transformation and a new linear relaxation technique, a linear relaxation programming problem of the equivalent problem is obtained. The proposed algorithm is convergent to the global optimal solution by means of the subsequent solutions of a series of linear programming problems. Numerical results are reported to show the feasibility of our algorithm.     

New constructions for local approximation of Lipschitz functions.II

In this paper, some properties of the set-valued mapping _Dαf(.)$D_{\alpha }f(.)$ connected with the new approximation method of a function f(.)$f(.)$ defined in the first part of the article are given. Continuity and Lipschitz properties of _Dαf(.)$D_{\alpha }f(.)$ are formulated. A continuous extension of the Clarke subdifferential of any function represented as a difference of two convex functions is given. For the convex case, the set-valued mapping _Dαf(.)$D_{\alpha }f(.)$ is similar to the ε$\epsilon$-subdifferential mapping.     

Integrability of subdifferentials of directionally Lipschitz functions

Using a quantitative version of the subdifferential characterization of directionally Lipschitz functions, we study the integrability of subdifferentials of such functions over arbitrary Banach space.