Variational optimization via chattering

We develop a chattering approach to solve variational problems that lack traditional properties such as differentiable everywhere and convexity conditions. We prove that our chattering approximation approaches the true relaxed solution as the intervals get smaller. Our chattering approach suggests a nonlinear optimization problem that can be easily solved to recover the optimal trajectory. A numerical example demonstrates our approach.     

Stochastic Methods for Practical Global Optimization

Engineering design problems often involve global optimization of functions that are supplied as black box functions. These functions may be nonconvex, nondifferentiable and even discontinuous. In addition, the decision variables may be a combination of discrete and continuous variables. The functions are usually computationally expensive, and may involve finite element methods. An engineering example of this type of problem is to minimize the weight of a structure, while limiting strain to be below a certain threshold. This type of global optimization problem is very difficult to solve, yet design engineers must find some solution to their problem – even if it is a suboptimal one. Sometimes the most difficult part of the problem is finding any feasible solution. Stochastic methods, including sequential random search and simulated annealing, are finding many applications to this type of practical global optimization problem. Improving Hit-and-Run (IHR) is a sequential random search method that has been successfully used in several engineering design applications, such as the optimal design of composite structures. A motivation to IHR is discussed as well as several enhancements. The enhancements include allowing both continuous and discrete variables in the problem formulation. This has many practical advantages, because design variables often involve a mixture of continuous and discrete values. IHR and several variations have been applied to the composites design problem. Some of this practical experience is discussed.     

Improving Hit-and-Run for global optimization

Improving Hit-and-Run is a random search algorithm for global optimization that at each iteration generates a candidate point for improvement that is uniformly distributed along a randomly chosen direction within the feasible region. The candidate point is accepted as the next iterate if it offers an improvement over the current iterate. We show that for positive definite quadratic programs, the expected number of function evaluations needed to arbitrarily well approximate the optimal solution is at most O(n^5/2) wheren is the dimension of the problem. Improving Hit-and-Run when applied to global optimization problems can therefore be expected to converge polynomially fast as it approaches the global optimum.Paper presented at the II. IIASA-Workshop on Global Optimization, December 9–14, 1990, Sopron (Hungary).     

A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems

There is a need for a methodology to fairly compare and present evaluation study results of stochastic global optimization algorithms. This need raises two important questions of (i) an appropriate set of benchmark test problems that the algorithms may be tested upon and (ii) a methodology to compactly and completely present the results. To address the first question, we compiled a collection of test problems, some are better known than others. Although the compilation is not exhaustive, it provides an easily accessible collection of standard test problems for continuous global optimization. Five different stochastic global optimization algorithms have been tested on these problems and a performance profile plot based on the improvement of objective function values is constructed to investigate the macroscopic behavior of the algorithms. The paper also investigates the microscopic behavior of the algorithms through quartile sequential plots, and contrasts the information gained from these two kinds of plots. The effect of the length of run is explored by using three maximum numbers of function evaluations and it is shown to significantly impact the behavior of the algorithms.     

An analytically derived cooling schedule for simulated annealing

We present an analytically derived cooling schedule for a simulated annealing algorithm applicable to both continuous and discrete global optimization problems. An adaptive search algorithm is used to model an idealized version of simulated annealing which is viewed as consisting of a series of Boltzmann distributed sample points. Our choice of cooling schedule ensures linearity in the expected number of sample points needed to become arbitrarily close to a global optimum.     

Pattern discrete and mixed Hit-and-Run for global optimization

We develop new Markov chain Monte Carlo samplers for neighborhood generation in global optimization algorithms based on Hit-and-Run. The success of Hit-and-Run as a sampler on continuous domains motivated Discrete Hit-and-Run with random biwalk for discrete domains. However, the potential for efficiencies in the implementation, which requires a randomization at each move to create the biwalk, lead us to a different approach that uses fixed patterns in generating the biwalks. We define Sphere and Box Biwalks that are pattern-based and easily implemented for discrete and mixed continuous/discrete domains. The pattern-based Hit-and-Run Markov chains preserve the convergence properties of Hit-and-Run to a target distribution. They also converge to continuous Hit-and-Run as the mesh of the discretized variables becomes finer, approaching a continuum. Moreover, we provide bounds on the finite time performance for the discrete cases of Sphere and Box Biwalks. We embed our samplers in an Improving Hit-and-Run global optimization algorithm and test their performance on a number of global optimization test problems.     

Towards Pure Adaptive Search

The algorithm known as Pure Adaptive Search is a global optimisation ideal with desirable complexity. In this paper we temper it to a framework we term Somewhat Adaptive Search. This retains the desirable complexity, but allows scope for a practical realisation. We introduce a new algorithm termed Pure Localisation Search which attempts to reach the practical ideal. For a certain class of one variable functions the gap is bridged.     

Optimization techniques for Available Transfer Capability (ATC) and market calculations

The recent movement towards an open, competitive market environmentintroduced new optimization problems such as market clearingmechanism, bidding decision and Available Transfer Capability(ATC) calculation. These optimization problems are characterizedby the complexity of power systems and the uncertainties inthe electricity market. Accurate evaluation of the transfercapability of a transmission system is required to maximizethe utilization of the existing transmission systems in a competitivemarket environment. The transfer capability of the transmissionnetworks can be limited by various system constraints such asthermal, voltage and stability limits. The ability to incorporatesuch limits into the optimization problem is a challenge inthe ATC calculation from an engineering point of view. In thecompetitive market environment, a power supplier needs to findan optimal strategy that maximizes its own profits under variousuncertainties such as electricity prices and load. On the otherhand, an efficient market clearing mechanism is needed to increasethe social welfare, i.e. the sum of the consumers’ andproducers’ surplus. The need to maximize the social welfaresubject to system operational constraints is also a major challengefrom a societal point of view. This paper presents new optimizationtechniques motivated by the competitive electricity market environment.Numerical simulation results are presented to demonstrate theperformance of the proposed optimization techniques.     

Hesitant adaptive search: the distribution of the number of iterations to convergence

Hesitant adaptive search is a stochastic optimisation procedure which accommodates hesitation, or pausing, at objective function values. It lies between pure adaptive search (which strictly improves at each iteration) and simulated annealing with constant temperature (which allows backtracking, or the acceptance of worse function values). In this paper we build on an earlier work and make two contributions; first, we link hesitant adaptive search to standard counting process theory, and second, we use this to derive the exact distribution of the number of iterations of hesitant adaptive search to termination. Received: November 17, 1997 / Accepted: July 9, 1999?Published online December 15, 2000