Fitness based Differential Evolution

Differential Evolution (DE) is a well known and simple population based probabilistic approach for global optimization. It has reportedly outperformed a few Evolutionary Algorithms and other search heuristics like Particle Swarm Optimization when tested over both benchmark and real world problems. But, DE, like other probabilistic optimization algorithms, sometimes exhibits premature convergence and stagnates at suboptimal point. In order to avoid stagnation behavior while maintaining a good convergence speed, a new position update process is introduced, named fitness based position update process in DE. In the proposed strategy, position of the solutions are updated in two phases. In the first phase all the solutions update their positions using the basic DE and in the second phase, all the solutions update their positions based on their fitness. In this way, a better solution participates more times in the position update process. The position update equation is inspired from the Artificial Bee Colony algorithm. The proposed strategy is named as Fitness Based Differential Evolution ( $FBDE$ ). To prove efficiency and efficacy of $FBDE$ , it is tested over 22 benchmark optimization problems. A comparative analysis has also been carried out among proposed FBDE, basic DE, Simulated Annealing Differential Evolution and Scale Factor Local Search Differential Evolution. Further, $FBDE$ is also applied to solve a well known electrical engineering problem called Model Order Reduction problem for Single Input Single Output Systems.     

Cognitive learning in differential evolution and its application to model order reduction problem for single-input single-output systems

Differential evolution (DE) is a well known and simple population based probabilistic approach for global optimization over continuous spaces. It has reportedly outperformed a few evolutionary algorithms and other search heuristics like the particle swarm optimization when tested over both benchmark and real world problems. DE, like other probabilistic optimization algorithms, has inherent drawback of premature convergence and stagnation. Therefore, in order to find a trade-off between exploration and exploitation capability of DE algorithm, a new parameter namely, cognitive learning factor (CLF) is introduced in the mutation process. Cognitive learning is a powerful mechanism that adjust the current position of individuals by the means of some specified knowledge (previous experience of individuals). The proposed strategy is named as cognitive learning in differential evolution (CLDE). To prove the efficiency of various approaches of CLF in DE,?CLDE is tested over 25 benchmark problems. Further, to establish the wide applicability of CLF,?CLDE is applied to two advanced DE variants. CLDE is also applied to solve a well known electrical engineering problem called model order reduction problem for single input single output systems.     

Step-size Adaptation in a Multi-objective Differential Evolution Algorithm

If Differential Evolution (DE) is applied to multi-objective optimization problems, some of its features like automatic problem-specific step-size adaptation gets lost. This can be resolved by modifying the DE equations. Three strategies are proposed and compared with existing algorithms. It is shown, that the proposed strategies deliver a superior convergence and preserve the positive characteristics of DE known from solving mono-objective optimization problems. (© 2016 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

改进种群多样性的双变异差分进化算法

差分进化算法(DE)是一种基于种群的启发式随机搜索技术,对于解决连续性优化问题具有较强的鲁棒性.然而传统差分进化算法存在种群多样性和收敛速度之间的矛盾,一种改进种群多样性的双变异差分进化算法(DADE),通过引入BFS-best机制(基于排序的可行解选取递减策略)改进变异算子"DE/current-to-best",将其与DE/rand/1构成双变异策略来改善DE算法中种群多样性减少的问题.同时,每个个体的控制参数基于排序自适应更新.最后,利用多个CEC2013标准测试函数对改进算法进行测试,实验结果表明,改进后的算法能有效改善种群多样性,较好地提高了算法的全局收敛能力和收敛速度.     

An efficient Differential Evolution based algorithm for solving multi-objective optimization problems

In the present study, a modified variant of Differential Evolution (DE) algorithm for solving multi-objective optimization problems is presented. The proposed algorithm, named Multi-Objective Differential Evolution Algorithm (MODEA) utilizes the advantages of Opposition-Based Learning for generating an initial population of potential candidates and the concept of random localization in mutation step. Finally, it introduces a new selection mechanism for generating a well distributed Pareto optimal front. The performance of proposed algorithm is investigated on a set of nine bi-objective and five tri-objective benchmark test functions and the results are compared with some recently modified versions of DE for MOPs and some other Multi Objective Evolutionary Algorithms (MOEAs). The empirical analysis of the numerical results shows the efficiency of the proposed algorithm.     

Self-adaptive randomized and rank-based differential evolution for multimodal problems

Differential Evolution (DE) is a widely used successful evolutionary algorithm (EA) based on a population of individuals, which is especially well suited to solve problems that have non-linear, multimodal cost functions. However, for a given population, the set of possible new populations is finite and a true subset of the cost function domain. Furthermore, the update formula of DE does not use any information about the fitness of the population. This paper presents a novel extension of DE called Randomized and Rank-based Differential Evolution (R2DE) and its self-adaptive version SAR2DE to improve robustness and global convergence speed on multimodal problems by introducing two multiplicative terms in the DE update formula. The first term is based on a random variate of a Cauchy distribution, which leads to a randomization. The second term is based on ranking of individuals, so that R2DE exploits additional information provided by the population fitness. In extensive experiments conducted with a wide range of complexity settings, we show that the proposed heuristics lead to an overall improvement in robustness and speed of convergence compared to several global optimization techniques, including DE, Opposition based Differential Evolution (ODE), DE with Random Scale Factor (DERSF) and the self-adaptive Cauchy distribution based DE (NSDE).     

Differential Evolution algorithm with Separated Groups for multi-dimensional optimization problems

The classical Differential Evolution (DE) algorithm, one of population-based Evolutionary Computation methods, proved to be a successful approach for relatively simple problems, but does not perform well for difficult multi-dimensional non-convex functions. A number of significant modifications of DE have been proposed in recent years, including very few approaches referring to the idea of distributed Evolutionary Algorithms. The present paper presents a new algorithm to improve optimization performance, namely DE with Separated Groups (DE-SG), which distributes population into small groups, defines rules of exchange of information and individuals between the groups and uses two different strategies to keep balance between exploration and exploitation capabilities. The performance of DE-SG is compared to that of eight algorithms belonging to the class of Evolutionary Strategies (Covariance Matrix Adaptation ES), Particle Swarm Optimization (Comprehensive Learning PSO and Efficient Population Utilization Strategy PSO), Differential Evolution (Distributed DE with explorative-exploitative population families, Self-adaptive DE, DE with global and local neighbours and Grouping Differential Evolution) and multi-algorithms (AMALGAM). The comparison is carried out for a set of 10-, 30- and 50-dimensional rotated test problems of varying difficulty, including 10- and 30-dimensional composition functions from CEC2005. Although slow for simple functions, the proposed DE-SG algorithm achieves a great success rate for more difficult 30- and 50-dimensional problems.     

Distributed mixed variant differential evolution algorithms for unconstrained global optimization

This paper proposes a novel distributed differential evolution algorithm called Distributed Mixed Variant Differential Evolution (dmvDE). To alleviate the time consuming trial-and-error selection of appropriate Differential Evolution (DE) variant to solve a given optimization problem, dmvDE proposes to mix effective DE variants with diverse characteristics in a distributed framework. The novelty of dmvDEs lies in mixing different DE variants in an island based distributed framework. The 19 dmvDE algorithms, discussed in this paper, constitute various proportions and combinations of four DE variants (DE/rand/1/bin, DE/rand/2/bin, DE/best/2/bin and DE/rand-to-best/1/bin) as subpopulations with each variant evolving independently but also exchanging information amongst others to co-operatively enhance the efficacy of the distributed DE as a whole. The dmvDE algorithms have been run on a set of test problems and compared to the distributed versions of the constituent DE variants. Simulation results show that dmvDEs display a consistent overall improvement in performance than that of distributed DEs. The best of dmvDE algorithms has also been benchmarked against five distributed differential evolution algorithms. Simulation results reiterate the superior performance of the mixing of the DE variants in a distributed frame work. The best of dmvDE algorithms outperforms, on average, all five algorithms considered.     

A Differential Evolution algorithm to deal with box, linear and quadratic-convex constraints for boundary optimization

We both propose and test an implicit strategy that is based on changing the search space from points to directions, which in combination with the Differential Evolution (DE) algorithm, is easily implemented for solving boundary optimization of a generic continuous function. In particular, we see that the DE method can be efficiently implemented to find solutions on the boundary of a convex and bounded feasible set resulting when the constraints are bounds on the variables, linear inequalities and quadratic convex inequalities. The computational results are performed on different classes of boundary minimization problems. The proposed technique is compared with the Generalized Differential Evolution method.     

Empirical analysis of self-adaptive differential evolution

Differential evolution (DE) is generally considered as a reliable, accurate, robust and fast optimization technique. DE has been successfully applied to solve a wide range of numerical optimization problems. However, the user is required to set the values of the control parameters of DE for each problem. Such parameter tuning is a time consuming task. In this paper, a self-adaptive DE (SDE) algorithm which eliminates the need for manual tuning of control parameters is empirically analyzed. The performance of SDE is investigated and compared with other well-known approaches. The experiments conducted show that SDE generally outperform other DE algorithms in all the benchmark functions. Moreover, the performance of SDE using the ring neighborhood topology is investigated.     

New mutation schemes for differential evolution algorithm and their application to the optimization of directional over-current relay settings

Differential evolution is a novel evolutionary approach capable of handling non-differentiable, nonlinear and multimodal objective functions. It has been consistently ranked as one of the best search algorithm for solving global optimization problems in several case studies. In the present study we propose five new mutation schemes for the basic DE algorithm. The corresponding versions are termed as MDE1, MDE2, MDE3, MDE4 and MDE5. These new schemes make use of the absolute weighted difference between the two points and instead of using a fixed scaling factor F, use a scaling factor following the Laplace distribution. The performance of the proposed schemes is validated empirically on a suit of ten benchmark problems having box constraints. Numerical analysis of results shows that the proposed schemes improves the convergence rate of the DE algorithm and also maintains the quality of solution. Efficiency of the proposed schemes is further validated by applying it to a real life electrical engineering problem dealing with the optimization of directional over-current relay settings. It is a highly constrained nonlinear optimization problem. A constraint handling mechanism based on repair methods is used for handling the constraints. Once again the simulation results show the compatibility of the proposed schemes for solving the real life problem.     

A novel multi-objective memetic algorithm based on opposition-based self-adaptive differential evolution

Under the framework of evolutionary paradigms, many evolutionary algorithms have been designed for handling multi-objective optimization problems. Each of the different algorithms may display exceptionally good performance in certain optimization problems, but none of them can be completely superior over one another. As such, different evolutionary algorithms are being synthesized to complement each other in view of their strengths and the limitations inherent in them. In this study, the novel memetic algorithm known as the Opposition-based Self-adaptive Hybridized Differential Evolution algorithm (OSADE) is being comprehensively investigated through a comparative study with some state-of-the-art algorithms, such as NSGA-II, non-dominated sorting Differential Evolution (NSDE), MOEA/D-SBX, MOEA/D-DE and the Multi-objective Evolutionary Gradient Search (MO-EGS) by using a suite of different benchmark problems. Through the experimental results that are presented by employing the Inverted Generational Distance (IGD) and the Hausdorff Distance performance indicators, it is seen that OSADE is able to achieve competitive, if not better, performance when compared to the other algorithms in this study.     

A memetic Differential Evolution approach in noisy optimization

This paper proposes a memetic approach for solving complex optimization problems characterized by a noisy fitness function. The proposed approach aims at solving highly multivariate and multi-modal landscapes which are also affected by a pernicious noise. The proposed algorithm employs a Differential Evolution framework and combines within this three additional algorithmic components. A controlled randomization of scale factor and crossover rate are employed which should better handle uncertainties of the problem and generally enhance performance of the Differential Evolution. Two combined local search algorithms applied to the scale factor, during offspring generation, should enhance performance of the Differential Evolution framework in the case of multi-modal and high dimensional problems. An on-line statistical test aims at assuring that only strictly necessary samples are taken and that all pairwise selections are properly performed. The proposed algorithm has been tested on a various set of test problems and its behavior has been studied, dependent on the dimensionality and noise level. A comparative analysis with a standard Differential Evolution, a modern version of Differential Evolution employing randomization of the control parameters and four metaheuristics tailored to optimization in a noisy environment has been carried out. One of these metaheuristics is a classical algorithm for noisy optimization while the other three are modern Differential Evolution based algorithms for noisy optimization which well represent the state-of-the-art in the field. Numerical results show that the proposed memetic approach is an efficient and robust alternative for various and complex multivariate noisy problems and can be exported to real-world problems affected by a noise whose distribution can be approximated by a Gaussian distribution.     

Global optimization of higher order moments in portfolio selection

We discuss the global optimization of the higher order moments of a portfolio of financial assets. The proposed model is an extension of the celebrated mean variance model of Markowitz. Asset returns typically exhibit excess kurtosis and are often skewed. Moreover investors would prefer positive skewness and try to reduce kurtosis of their portfolio returns. Therefore the mean variance model (assuming either normally distributed returns or quadratic utility functions) might be too simplifying. The inclusion of higher order moments has therefore been proposed as a possible augmentation of the classical model in order to make it more widely applicable. The resulting problem is non-convex, large scale, and highly relevant in financial optimization. We discuss the solution of the model using two stochastic algorithms. The first algorithm is Differential Evolution (DE). DE is a population based metaheuristic originally designed for continuous optimization problems. New solutions are generated by combining up to four existing solutions plus noise, and acceptance is based on evolutionary principles. The second algorithm is based on the asymptotic behavior of a suitably defined Stochastic Differential Equation (SDE). The SDE consists of three terms. The first term tries to reduce the value of the objective function, the second enforces feasibility of the iterates, while the third adds noise in order to enable the trajectory to climb hills.     

Massively parallel differential evolution—pattern search optimization with graphics hardware acceleration: an investigation on bound constrained optimization problems

This paper presents a novel parallel Differential Evolution (DE) algorithm with local search for solving function optimization problems, utilizing graphics hardware acceleration. As a population-based meta-heuristic, DE was originally designed for continuous function optimization. Graphics Processing Units (GPU) computing is an emerging desktop parallel computing technology that is becoming popular with its wide availability in many personal computers. In this paper, the classical DE was adapted in the data-parallel CPU-GPU heterogeneous computing platform featuring Single Instruction-Multiple Thread (SIMT) execution. The global optimal search of the DE was enhanced by the classical local Pattern Search (PS) method. The hybrid DE–PS method was implemented in the GPU environment and compared to a similar implementation in the common computing environment with a Central Processing Unit (CPU). Computational results indicate that the GPU-accelerated SIMT-DE-PS method is orders of magnitude faster than the corresponding CPU implementation. The main contribution of this paper is the parallelization analysis and performance analysis of the hybrid DE–PS with GPU acceleration. The research results demonstrate a promising direction for high speed optimization with desktop parallel computing on a personal computer.     

基于精英协同的混洗差分进化算法及其应用

提出了基于精英协同的混洗差分进化算法(Shuffled Differential Evolution,SDE)。该算法引入反向学习的初始化机制,并对设置的普通群和虚拟精英群采用不同的差分策略,进而将精英个体作为信息通道实现种群间的信息交流;同时,借助定期混洗机制实现种群间的文化交流,从而达到协同进化的目的;此外,对长期停滞的个体进行跳变操作,以充分挖掘种群的搜索潜能,增强搜索的有效性。通过函数仿真,并与PSO及其它差分进化算法比较,结果表明该算法具有较好的寻优能力。     

Global optimization based on novel heuristics,low-discrepancy sequences and genetic algorithms

In this paper a new heuristic hybrid technique for bound-constrained global optimization is proposed. We developed iterative algorithm called GLP_τS that uses genetic algorithms, LP_τ low-discrepancy sequences of points and heuristic rules to find regions of attraction when searching a global minimum of an objective function. Subsequently Nelder–Mead Simplex local search technique is used to refine the solution. The combination of the three techniques (Genetic algorithms, LP_τO Low-discrepancy search and Simplex search) provides a powerful hybrid heuristic optimization method which is tested on a number of benchmark multimodal functions with 10–150 dimensions, and the method properties – applicability, convergence, consistency and stability are discussed in detail.     

混合变异粒子群优化算法及其应用

为改善粒子群优化算法在解决复杂优化问题时收敛质量不高的不足,提出了一种改进的粒子群优化算法,即混合变异粒子群优化算法(HMPSO).HMPSO算法采用了带有随机因子的惯性权重取值更新策略,降低了标准粒子群优化算法中由于粒子飞行速度过大而错过最优解的概率,从而加速了算法的收敛速度.此外,通过混合变异进化环节的引入,缓解了...     

PSOHS: an efficient two-stage approach for data clustering

Cluster analysis is an important task in data mining and refers to group a set of objects such that the similarities among objects within the same group are maximal while similarities among objects from different groups are minimal. The particle swarm optimization algorithm (PSO) is one of the famous metaheuristic optimization algorithms, which has been successfully applied to solve the clustering problem. However, it has two major shortcomings. The PSO algorithm converges rapidly during the initial stages of the search process, but near global optimum, the convergence speed will become very slow. Moreover, it may get trapped in local optimum if the global best and local best values are equal to the particle’s position over a certain number of iterations. In this paper we hybridized the PSO with a heuristic search algorithm to overcome the shortcomings of the PSO algorithm. In the proposed algorithm, called PSOHS, the particle swarm optimization is used to produce an initial solution to the clustering problem and then a heuristic search algorithm is applied to improve the quality of this solution by searching around it. The superiority of the proposed PSOHS clustering method, as compared to other popular methods for clustering problem is established for seven benchmark and real datasets including Iris, Wine, Crude Oil, Cancer, CMC, Glass and Vowel.     

A combined local search and integer programming approach to the traveling tournament problem

The traveling tournament problem is a well-known combinatorial optimization problem with direct applications to sport leagues scheduling, that sparked intensive algorithmic research over the last decade. With the Challenge Traveling Tournament Instances as an established benchmark, the most successful approaches to the problem use meta-heuristics like tabu search or simulated annealing, partially heavily parallelized. Integer programming based methods on the other hand are hardly able to tackle larger benchmark instances. In this work we present a hybrid approach that draws on the power of commercial integer programming solvers as well as the speed of local search heuristics. Our proposed method feeds the solution of one algorithm phase to the other one, until no further improvements can be made. The applicability of this method is demonstrated experimentally on the galaxy instance set, resulting in currently best known solutions for most of the considered instances.