Variational optimization via chattering

We develop a chattering approach to solve variational problems that lack traditional properties such as differentiable everywhere and convexity conditions. We prove that our chattering approximation approaches the true relaxed solution as the intervals get smaller. Our chattering approach suggests a nonlinear optimization problem that can be easily solved to recover the optimal trajectory. A numerical example demonstrates our approach.     

Pure adaptive search for finite global optimization

Pure Adaptive Search is a stochastic algorithm which has been analyzed for continuous global optimization. When a uniform distribution is used in PAS, it has been shown to have complexity which is linear in dimension. We define strong and weak variations of PAS in the setting of finite global optimization and prove analogous results. In particular, for then-dimensional lattice {1,,k} ⁿ , the expected number of iterations to find the global optimum is linear inn. Many discrete combinatorial optimization problems, although having intractably large domains, have quite small ranges. The strong version of PAS for all problems, and the weak version of PAS for a limited class of problems, has complexity the order of the size of the range.The authors would like to thank the Department of Mathematics and Statistics at the University of Canterbury for support of this research.     

A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems

There is a need for a methodology to fairly compare and present evaluation study results of stochastic global optimization algorithms. This need raises two important questions of (i) an appropriate set of benchmark test problems that the algorithms may be tested upon and (ii) a methodology to compactly and completely present the results. To address the first question, we compiled a collection of test problems, some are better known than others. Although the compilation is not exhaustive, it provides an easily accessible collection of standard test problems for continuous global optimization. Five different stochastic global optimization algorithms have been tested on these problems and a performance profile plot based on the improvement of objective function values is constructed to investigate the macroscopic behavior of the algorithms. The paper also investigates the microscopic behavior of the algorithms through quartile sequential plots, and contrasts the information gained from these two kinds of plots. The effect of the length of run is explored by using three maximum numbers of function evaluations and it is shown to significantly impact the behavior of the algorithms.     

Stochastic Methods for Practical Global Optimization

Engineering design problems often involve global optimization of functions that are supplied as black box functions. These functions may be nonconvex, nondifferentiable and even discontinuous. In addition, the decision variables may be a combination of discrete and continuous variables. The functions are usually computationally expensive, and may involve finite element methods. An engineering example of this type of problem is to minimize the weight of a structure, while limiting strain to be below a certain threshold. This type of global optimization problem is very difficult to solve, yet design engineers must find some solution to their problem – even if it is a suboptimal one. Sometimes the most difficult part of the problem is finding any feasible solution. Stochastic methods, including sequential random search and simulated annealing, are finding many applications to this type of practical global optimization problem. Improving Hit-and-Run (IHR) is a sequential random search method that has been successfully used in several engineering design applications, such as the optimal design of composite structures. A motivation to IHR is discussed as well as several enhancements. The enhancements include allowing both continuous and discrete variables in the problem formulation. This has many practical advantages, because design variables often involve a mixture of continuous and discrete values. IHR and several variations have been applied to the composites design problem. Some of this practical experience is discussed.     

Improving Hit-and-Run for global optimization

Improving Hit-and-Run is a random search algorithm for global optimization that at each iteration generates a candidate point for improvement that is uniformly distributed along a randomly chosen direction within the feasible region. The candidate point is accepted as the next iterate if it offers an improvement over the current iterate. We show that for positive definite quadratic programs, the expected number of function evaluations needed to arbitrarily well approximate the optimal solution is at most O(n^5/2) wheren is the dimension of the problem. Improving Hit-and-Run when applied to global optimization problems can therefore be expected to converge polynomially fast as it approaches the global optimum.Paper presented at the II. IIASA-Workshop on Global Optimization, December 9–14, 1990, Sopron (Hungary).     

Towards Pure Adaptive Search

The algorithm known as Pure Adaptive Search is a global optimisation ideal with desirable complexity. In this paper we temper it to a framework we term Somewhat Adaptive Search. This retains the desirable complexity, but allows scope for a practical realisation. We introduce a new algorithm termed Pure Localisation Search which attempts to reach the practical ideal. For a certain class of one variable functions the gap is bridged.     

Optimization techniques for Available Transfer Capability (ATC) and market calculations

The recent movement towards an open, competitive market environmentintroduced new optimization problems such as market clearingmechanism, bidding decision and Available Transfer Capability(ATC) calculation. These optimization problems are characterizedby the complexity of power systems and the uncertainties inthe electricity market. Accurate evaluation of the transfercapability of a transmission system is required to maximizethe utilization of the existing transmission systems in a competitivemarket environment. The transfer capability of the transmissionnetworks can be limited by various system constraints such asthermal, voltage and stability limits. The ability to incorporatesuch limits into the optimization problem is a challenge inthe ATC calculation from an engineering point of view. In thecompetitive market environment, a power supplier needs to findan optimal strategy that maximizes its own profits under variousuncertainties such as electricity prices and load. On the otherhand, an efficient market clearing mechanism is needed to increasethe social welfare, i.e. the sum of the consumers’ andproducers’ surplus. The need to maximize the social welfaresubject to system operational constraints is also a major challengefrom a societal point of view. This paper presents new optimizationtechniques motivated by the competitive electricity market environment.Numerical simulation results are presented to demonstrate theperformance of the proposed optimization techniques.     

Hesitant adaptive search: the distribution of the number of iterations to convergence

Hesitant adaptive search is a stochastic optimisation procedure which accommodates hesitation, or pausing, at objective function values. It lies between pure adaptive search (which strictly improves at each iteration) and simulated annealing with constant temperature (which allows backtracking, or the acceptance of worse function values). In this paper we build on an earlier work and make two contributions; first, we link hesitant adaptive search to standard counting process theory, and second, we use this to derive the exact distribution of the number of iterations of hesitant adaptive search to termination. Received: November 17, 1997 / Accepted: July 9, 1999?Published online December 15, 2000