Information sharing impact of stochastic diffusion search on differential evolution algorithm

This work details the research aimed at applying the powerful resource allocation mechanism deployed in stochastic diffusion search (SDS) to the differential evolution (DE), effectively merging a nature inspired swarm intelligence algorithm with a biologically inspired evolutionary algorithm. The results reported herein suggest that the hybrid algorithm, exploiting information sharing between the population elements, has the potential to improve the optimisation capability of classical DE algorithms. This claim is verified by running several experiments using state-of-the-art benchmarks. Additionally, the significance of the frequency within which SDS introduces communication and information exchange is also investigated.     

Parallel local search

We present a survey of parallel local search algorithms in which we review the concepts that can be used to incorporate parallelism into local search. For this purpose we distinguish between single-walk and multiple-walk parallel local search and between asynchronous and synchronous parallelism. Within the class of single-walk algorithms we differentiate between multiple-step and single-step parallelism. To describe parallel local search we introduce the concepts of hyper neighborhood structures and distributed neighborhood structures. Furthermore, we present templates that capture most of the parallel local search algorithms proposed in the literature. Finally, we discuss some complexity issues related to parallel local search.     

Multiple optima in local search

We consider the application of local search methods to the maximum independent set problem. These methods employ a relation R on the power set of the graph's vertices that identifies a set of vertices, U, with a collection of subsets of vertices, R(U), called the neighbors of U. If each set U has only polynomially many neighbors then we say that R is polynomially bounded. Further, given a graph, G, we call a permutation of G, φ(G), to be any graph that arises from G by relabeling the vertices. Our main result is slightly stronger than the following: we construct a graph G such that, for all polynomially bounded relations, R, most permutations φ(G) of the graph contain exponentially many strict local optima that are not global optima. That is, a single graph (up to relabelling of the vertices) exists that soundly defeats all polynomially bounded relations R. Corollaries follow for 0–1 integer programming and other hard optimization problems.     

Mechanisms for local search

Local search methods for combinatorial optimization make a series of steps, at each stage improving the current solution by moving to a neighbouring solution. This is usually done by considering the neighbouring solutions one at a time and moving to the first one which gives an improvement (a first-improving method). In this paper we consider whether there are circumstances in which some other strategy will have better performance. In exploring this question we begin by giving a theoretical treatment of a simple model with random objective values at each solution point. We carry out some experiments on the Travelling Salesman Problem and the Quadratic Assignment Problem using varying values of a spread parameter, k. The value of k corresponds to the number of improving solutions looked at before making a move. We also make some conjectures relating the overall performance of the local search method to the average number of solutions which are evaluated before a local minimum is reached.     

Exploiting relaxation in local search for LABS

Branch-and-bound uses relaxation to prune search trees but sometimes scales poorly to large problems. Conversely, local search often scales well but may be unable to find optimal solutions. Both phenomena occur in the construction of low-autocorrelation binary sequences (LABS), a problem arising in communication engineering. This paper proposes a hybrid approach to optimization: using relaxation to prune local search spaces. An implementation gives very competitive results, showing the feasibility of the approach.     

The island confinement method for reducing search space in local search methods

Typically local search methods for solving constraint satisfaction problems such as GSAT, WalkSAT, DLM, and ESG treat the problem as an optimisation problem. Each constraint contributes part of a penalty function in assessing trial valuations. Local search examines the neighbours of the current valuation, using the penalty function to determine a “better” neighbour valuation to move to, until finally a solution which satisfies all the constraints is found. In this paper we investigate using some of the constraints as “hard” constraints, that are always satisfied by every trial valuation visited, rather than as part of a penalty function. In this way these constraints reduce the possible neighbours in each move and also the overall search space. The treating of some constraints as hard requires that the space of valuations that are satisfied is “connected” in order to guarantee that a solution can be found from any starting position within the region; thus the concept of islands and the name “island confinement method” arises. Treating some constraints as hard provides new difficulties for the search mechanism since the search space becomes more jagged, and there are more deep local minima. A new escape strategy is needed. To demonstrate the feasibility and generality of our approach, we show how the island confinement method can be incorporated in, and significantly improve, the search performance of two successful local search procedures, DLM and ESG, on SAT problems arising from binary CSPs. A preliminary version of this paper appeared in AAAI’2002.     

Bare bones differential evolution

The barebones differential evolution (BBDE) is a new, almost parameter-free optimization algorithm that is a hybrid of the barebones particle swarm optimizer and differential evolution. Differential evolution is used to mutate, for each particle, the attractor associated with that particle, defined as a weighted average of its personal and neighborhood best positions. The performance of the proposed approach is investigated and compared with differential evolution, a Von Neumann particle swarm optimizer and a barebones particle swarm optimizer. The experiments conducted show that the BBDE provides excellent results with the added advantage of little, almost no parameter tuning. Moreover, the performance of the barebones differential evolution using the ring and Von Neumann neighborhood topologies is investigated. Finally, the application of the BBDE to the real-world problem of unsupervised image classification is investigated. Experimental results show that the proposed approach performs very well compared to other state-of-the-art clustering algorithms in all measured criteria.     

Fast local search and guided local search and their application to British Telecom's workforce scheduling problem

This paper reports a fast local search (FLS) algorithm which helps to improve the efficiency of hill climbing and a guided local search (GLS) algorithm which was developed to help local search to escape local optima and distribute search effort. To illustrate how these algorithms work, this paper describes their application to British Telecom's workforce scheduling problem, which is a hard real life problem. The effectiveness of FLS and GLS are demonstrated by the fact that they both outperform all the methods applied to this problem so far, which include simulated annealing, genetic algorithms and constraint logic programming.     

Statistical analysis of local search landscapes

This paper discusses the application of some statistical estimation tools in trying to understand the nature of the combinatorial landscapes induced by local search methods. One interesting property of a landscape is the number of optima that are present. In this paper we show that it is possible to compute a confidence interval on the number of independent local searches needed to find all optima. By extension, this also expresses the confidence that the global optimum has been found. In many cases, this confidence may be too low to be acceptable, but it is also possible to estimate the number of optima that exist. Theoretical analysis and empirical studies are discussed, which show that it may be possible to obtain a fairly accurate picture of this property of a combinatorial landscape. The approach is illustrated by analysis of an instance of the flowshop scheduling problem.     

Automated code generation by local search

There are many successful evolutionary computation techniques for automatic program generation, with the best known, perhaps, being genetic programming. Genetic programming has obtained human competitive results, even infringing on patented inventions. The majority of the scientific literature on automatic program generation employs such population-based search approaches, to allow a computer system to search a space of programs. In this paper, we present an alternative approach based on local search. There are many local search methodologies that allow successful search of a solution space, based on maintaining a single incumbent solution and searching its neighbourhood. However, use of these methodologies in searching a space of programs has not yet been systematically investigated. The contribution of this paper is to show that a local search of programs can be more successful at automatic program generation than current nature inspired evolutionary computation methodologies.     

Boosting local search with Lagrangian relaxation

Local search algorithms play an essential role in solving large-scale combinatorial optimization problems. Traditionally, the local search procedure is guided mainly by the objective function of the problem. Hence, the greedy improvement paradigm poses the potential threat of prematurely getting trapped in low quality attraction basins. In this study, we intend to utilize the information extracted from the relaxed problem, to enhance the performance of the local search process. Considering the Lin-Kernighan-based local search (LK-search) for the p-median problem as a case study, we propose the Lagrangian relaxation Assisted Neighborhood Search (LANS). In the proposed algorithm, two new mechanisms, namely the neighborhood reduction and the redundancy detection, are developed. The two mechanisms exploit the information gathered from the relaxed problem, to avoid the search from prematurely targeting low quality directions, and to cut off the non-promising searching procedure, respectively. Extensive numerical results over the benchmark instances demonstrate that LANS performs favorably to LK-search, which is among the state-of-the-art local search algorithms for the p-median problem. Furthermore, by embedding LANS into other heuristics, the best known upper bounds over several benchmark instances could be updated. Besides, run-time distribution analysis is also employed to investigate the reason why LANS works. The findings of this study confirm that the idea of improving local search by leveraging the information induced from relaxed problem is feasible and practical, and might be generalized to a broad class of combinatorial optimization problems.     

On parallel local search for permutations

We investigate some ways in which massively parallel computing devices can be exploited in local search algorithms. We show that the substantial speedups that can be gained from parallel neighbourhood evaluation enables an efficient best improvement local search, and that this in turn enables further speedups through selection and parallel application of a set of independent, improving moves. Our experiments demonstrate a total speedup of up to several hundred times compared to a classical, sequential best improvement search. We also demonstrate how an exchange of good partial solutions between the incumbent and best found solutions improves the efficiency of the Iterated Local Search algorithm.     

Image registration with iterated local search

This contribution is devoted to the application of iterated local search to image registration, a very complex, real-world problem in the field of image processing. To do so, we first re-define this parameter estimation problem as a combinatorial optimization problem, then analyze the use of image-specific information to guide the search in the form of an heuristic function, and finally propose its solution by iterated local search. Our algorithm is tested by comparing its performance to that of two different baseline algorithms: iterative closest point, a well-known, image registration technique, a hybrid algorithm including the latter technique within a simulated annealing approach, a multi-start local search procedure, that allows us to check the influence of the search scheme considered in the problem solving, and a real coded genetic algorithm. Four different problem instances are tackled in the experimental study, resulting from two images and two transformations applied on them. Three parameter settings are analyzed in our approach in order to check three heuristic information scenarios where the heuristic is not used at all, is partially used or almost completely guides the search process, as well as two different number of iterations in the algorithms outer-inner loops. This work was partially supported by the Spanish Ministerio de Ciencia y Tecnología under project TIC2003-00877 (including FEDER fundings) and under Network HEUR TIC2002-10866-E.     

Smoothed performance guarantees for local search

We study popular local search and greedy algorithms for standard machine scheduling problems. The performance guarantee of these algorithms is well understood, but the worst-case lower bounds seem somewhat contrived and it is questionable whether they arise in practical applications. To find out how robust these bounds are, we study the algorithms in the framework of smoothed analysis, in which instances are subject to some degree of random noise. While the lower bounds for all scheduling variants with restricted machines are rather robust, we find out that the bounds are fragile for unrestricted machines. In particular, we show that the smoothed performance guarantee of the jump and the lex-jump algorithm are (in contrast to the worst case) independent of the number of machines. They are  \(\Theta (\phi )\) and  \(\Theta (\log \phi )\) , respectively, where  \(1/\phi \) is a parameter measuring the magnitude of the perturbation. The latter immediately implies that also the smoothed price of anarchy is  \(\Theta (\log \phi )\) for routing games on parallel links. Additionally, we show that for unrestricted machines also the greedy list scheduling algorithm has an approximation guarantee of   \(\Theta (\log \phi )\) .     

Computational bounds for local search in combinatorial optimization

The results related to finding local optima in combinatorial optimization are overviewed. The class of polynomial-time local search problems (class PLS) is considered. By analogy with Cook’s theorem, the existence of most complicated problems in this class is established. The number of steps in local descent algorithms is estimated in the worst and average cases. The local search determination of exact and approximate solutions with guaranteed error bounds is discussed.     

Maximally differential ideals in regular local rings

It is shown that ifA is a regular local ring andI is a maximally differential ideal inA, thenI is generated by anA-sequence.     

Problem space local search for number partitioning

We show how simple and effective metaheuristics can be developed for the number partitioning problem using the problem space approach [1]. In a previous application of local search to number partitioning [2], it was found that the performance of simulated annealing used in conjunction with swap neighborhoods was disappointing relative to the differencing heuristic of Karmarkar and Karp [3]. Using problem space neighborhoods as an alternative to swapping, we empirically demonstrate several orders of magnitude improvement over the differencing algorithm, albeit with greater computation time. This improvement in performance comes as little surprise since a modified version of the differencing heuristic is explicitly embedded in the problem space algorithm.     

Combining simulated annealing with local search heuristics

We introduce a meta-heuristic to combine simulated annealing with local search methods for CO problems. This new class of Markov chains leads to significantly more powerful optimization methods than either simulated annealing or local search. The main idea is to embed deterministic local search techniques into simulated annealing so that the chain explores only local optima. It makes large, global changes, even at low temperatures, thus overcoming large barriers in configuration space. We have tested this meta-heuristic for the traveling salesman and graph partitioning problems. Tests on instances from public libraries and random ensembles quantify the power of the method. Our algorithm is able to solve large instances to optimality, improving upon local search methods very significantly. For the traveling salesman problem with randomly distributed cities, in a square, the procedure improves on 3-opt by 1.6%, and on Lin-Kernighan local search by 1.3%. For the partitioning of sparse random graphs of average degree equal to 5, the improvement over Kernighan-Lin local search is 8.9%. For both CO problems, we obtain new best heuristics.     

Hybridizing local search algorithms for global optimization

In this paper, we combine two types of local search algorithms for global optimization of continuous functions. In the literature, most of the hybrid algorithms are produced by combination of a global optimization algorithm with a local search algorithm and the local search is used to improve the solution quality, not to explore the search space to find independently the global optimum. The focus of this research is on some simple and efficient hybrid algorithms by combining the Nelder–Mead simplex (NM) variants and the bidirectional random optimization (BRO) methods for optimization of continuous functions. The NM explores the whole search space to find some promising areas and then the BRO local search is entered to exploit optimal solution as accurately as possible. Also a new strategy for shrinkage stage borrowed from differential evolution (DE) is incorporated in the NM variants. To examine the efficiency of proposed algorithms, those are evaluated by 25 benchmark functions designed for the special session on real-parameter optimization of CEC2005. A comparison study between the hybrid algorithms and some DE algorithms and non-parametric analysis of obtained results demonstrate that the proposed algorithms outperform most of other algorithms and their difference in most cases is statistically considerable. In a later part of the comparative experiments, a comparison of the proposed algorithms with some other evolutionary algorithms reported in the CEC2005 confirms a better performance of our proposed algorithms.