Particle swarm optimization: Hybridization perspectives and experimental illustrations

Metaheuristic optimization algorithms have become popular choice for solving complex and intricate problems which are otherwise difficult to solve by traditional methods. In the present study an attempt is made to review the hybrid optimization techniques in which one main algorithm is a well known metaheuristic; particle swarm optimization or PSO. Hybridization is a method of combining two (or more) techniques in a judicious manner such that the resulting algorithm contains the positive features of both (or all) the algorithms. Depending on the algorithm/s used we made three classifications as (i) Hybridization of PSO and genetic algorithms (ii) Hybridization of PSO with differential evolution and (iii) Hybridization of PSO with other techniques. Where, other techniques include various local and global search methods. Besides giving the review we also show a comparison of three hybrid PSO algorithms; hybrid differential evolution particle swarm optimization (DE-PSO), adaptive mutation particle swarm optimization (AMPSO) and hybrid genetic algorithm particle swarm optimization (GA-PSO) on a test suite of nine conventional benchmark problems.     

A hybrid simplex search and particle swarm optimization for unconstrained optimization

This paper proposes the hybrid NM-PSO algorithm based on the Nelder–Mead (NM) simplex search method and particle swarm optimization (PSO) for unconstrained optimization. NM-PSO is very easy to implement in practice since it does not require gradient computation. The modification of both the Nelder–Mead simplex search method and particle swarm optimization intends to produce faster and more accurate convergence. The main purpose of the paper is to demonstrate how the standard particle swarm optimizers can be improved by incorporating a hybridization strategy. In a suite of 20 test function problems taken from the literature, computational results via a comprehensive experimental study, preceded by the investigation of parameter selection, show that the hybrid NM-PSO approach outperforms other three relevant search techniques (i.e., the original NM simplex search method, the original PSO and the guaranteed convergence particle swarm optimization (GCPSO)) in terms of solution quality and convergence rate. In a later part of the comparative experiment, the NM-PSO algorithm is compared to various most up-to-date cooperative PSO (CPSO) procedures appearing in the literature. The comparison report still largely favors the NM-PSO algorithm in the performance of accuracy, robustness and function evaluation. As evidenced by the overall assessment based on two kinds of computational experience, the new algorithm has demonstrated to be extremely effective and efficient at locating best-practice optimal solutions for unconstrained optimization.     

Chaotic catfish particle swarm optimization for solving global numerical optimization problems

Chaotic catfish particle swarm optimization (C-CatfishPSO) is a novel optimization algorithm proposed in this paper. C-CatfishPSO introduces chaotic maps into catfish particle swarm optimization (CatfishPSO), which increase the search capability of CatfishPSO via the chaos approach. Simple CatfishPSO relies on the incorporation of catfish particles into particle swarm optimization (PSO). The introduced catfish particles improve the performance of PSO considerably. Unlike other ordinary particles, the catfish particles initialize a new search from extreme points of the search space when the gbest fitness value (global optimum at each iteration) has not changed for a certain number of consecutive iterations. This results in further opportunities of finding better solutions for the swarm by guiding the entire swarm to promising new regions of the search space and accelerating the search. The introduced chaotic maps strengthen the solution quality of PSO and CatfishPSO significantly. The resulting improved PSO and CatfishPSO are called chaotic PSO (C-PSO) and chaotic CatfishPSO (C-CatfishPSO), respectively. PSO, C-PSO, CatfishPSO, C-CatfishPSO, as well as other advanced PSO procedures from the literature were extensively compared on several benchmark test functions. Statistical analysis of the experimental results indicate that the performance of C-CatfishPSO is better than the performance of PSO, C-PSO, CatfishPSO and that C-CatfishPSO is also superior to advanced PSO methods from the literature.     

Integrating particle swarm optimization with genetic algorithms for solving nonlinear optimization problems

Heuristic optimization provides a robust and efficient approach for solving complex real-world problems. The aim of this paper is to introduce a hybrid approach combining two heuristic optimization techniques, particle swarm optimization (PSO) and genetic algorithms (GA). Our approach integrates the merits of both GA and PSO and it has two characteristic features. Firstly, the algorithm is initialized by a set of random particles which travel through the search space. During this travel an evolution of these particles is performed by integrating PSO and GA. Secondly, to restrict velocity of the particles and control it, we introduce a modified constriction factor. Finally, the results of various experimental studies using a suite of multimodal test functions taken from the literature have demonstrated the superiority of the proposed approach to finding the global optimal solution.     

非线性约束优化问题的混合粒子群算法

把处理约束条件的一个外点方法和改进的粒子群优化算法相结合,提出了一种求解非线性约束优化问题的混合粒子群优化算法.该方法兼顾了粒子群优化和外点法的优点,对算法迭代过程中出现不可行粒子,利用外点法处理后产生可行粒子.数值实验表明了提出的新算法具有有效性、通用性和稳健性.     

A hybrid glowworm swarm optimization algorithm to solve constrained multimodal functions optimization

In this paper, a novel hybrid glowworm swarm optimization (HGSO) algorithm is proposed. The HGSO algorithm embeds predatory behaviour of artificial fish swarm algorithm (AFSA) into glowworm swarm optimization (GSO) algorithm and combines the GSO with differential evolution on the basis of a two-population co-evolution mechanism. In addition, to overcome the premature convergence, the local search strategy based on simulated annealing is applied to make the search of GSO approach the true optimum solution gradually. Finally, several benchmark functions show that HGSO has faster convergence efficiency and higher computational precision, and is more effective for solving constrained multi-modal function optimization problems.     

Particle swarm optimization with age-group topology for multimodal functions and data clustering

This paper proposes particle swarm optimization with age-group topology (PSOAG), a novel age-based particle swarm optimization (PSO). In this work, we present a new concept of age to measure the search ability of each particle in local area. To keep population diversity during searching, we separate particles to different age-groups by their age and particles in each age-group can only select the ones in younger groups or their own groups as their neighbourhoods. To allow search escape from local optima, the aging particles are regularly replaced by new and randomly generated ones. In addition, we design an age-group based parameter setting method, where particles in different age-groups have different parameters, to accelerate convergence. This algorithm is applied to nonlinear function optimization and data clustering problems for performance evaluation. In comparison against several PSO variants and other EAs, we find that the proposed algorithm provides significantly better performances on both the function optimization problems and the data clustering tasks.     

Dynamic guiding particle swarm optimization with embedded chaotic search for solving multidimensional problems

The proposed approach incorporated dynamic guiding approach and chaotic search procedure into particle swarm optimization (PSO), named DCPSO. Chaotic search, enjoyed ergodicity, irregularity and pseudo-randomness in PSO, would refine global best position evidently. And, dynamic guiding approach with fluctuating property would easily conduct unpredictable migrations for PSO to break away from evolutionary stagnation. The experiment reports indicated that the proposed DCPSO approach could improve the evolution performance significantly, and present the superiority in solving complex multidimensional problems.     

Handling equality constraints in evolutionary optimization

Over the last few decades several methods have been proposed for handling functional constraints while solving optimization problems using evolutionary algorithms (EAs). However, the presence of equality constraints makes the feasible space very small compared to the entire search space. As a consequence, the handling of equality constraints has long been a difficult issue for evolutionary optimization methods. This paper presents a Hybrid Evolutionary Algorithm (HEA) for solving optimization problems with both equality and inequality constraints. In HEA, we propose a new local search technique with special emphasis on equality constraints. The basic concept of the new technique is to reach a point on the equality constraint from the current position of an individual solution, and then explore on the constraint landscape. We believe this new concept will influence the future research direction for constrained optimization using population based algorithms. The proposed algorithm is tested on a set of standard benchmark problems. The results show that the proposed technique works very well on those benchmark problems.     

Improved particle swarm algorithm for hydrological parameter optimization

In this paper, a new method named MSSE-PSO (master-slave swarms shuffling evolution algorithm based on particle swarm optimization) is proposed. Firstly, a population of points is sampled randomly from the feasible space, and then partitioned into several sub-swarms (one master swarm and other slave swarms). Each slave swarm independently executes PSO or its variants, including the update of particles’ position and velocity. For the master swarm, the particles enhance themselves based on the social knowledge of master swarm and that of slave swarms. At periodic stage in the evolution, the master swarm and the whole slave swarms are forced to mix, and points are then reassigned to several sub-swarms to ensure the share of information. The process is repeated until a user-defined stopping criterion is reached. The tests of numerical simulation and the case study on hydrological model show that MSSE-PSO remarkably improves the accuracy of calibration, reduces the time of computation and enhances the performance of stability. Therefore, it is an effective and efficient global optimization method.     

Balanced fuzzy particle swarm optimization

In the present study an extension of particle swarm optimization (PSO) algorithm which is in conformity with actual nature is introduced for solving combinatorial optimization problems. Development of this algorithm is essentially based on balanced fuzzy sets theory. The classical fuzzy sets theory cannot distinguish differences between positive and negative information of membership functions, while in the new method both kinds of information “positive and negative” about membership function are equally important. The balanced fuzzy particle swarm optimization algorithm is used for fundamental optimization problem entitled traveling salesman problem (TSP). For convergence inspecting of new algorithm, method was used for TSP problems. Convergence curves were represented fast convergence in restricted and low iterations for balanced fuzzy particle swarm optimization algorithm (BF-PSO) comparison with fuzzy particle swarm optimization algorithm (F-PSO).     

Particle swarm optimization with time-varying acceleration coefficients for the multidimensional knapsack problem

The multidimensional knapsack problem (MKP) is a difficult combinatorial optimization problem, which has been proven as NP-hard problems. Various population-based search algorithms are applied to solve these problems. The particle swarm optimization (PSO) technique is adapted in our study, which proposes two novel PSO algorithms, namely, the binary PSO with time-varying acceleration coefficients (BPSOTVAC) and the chaotic binary PSO with time-varying acceleration coefficients (CBPSOTVAC). The two proposed methods were tested using 116 benchmark problems from the OR-Library to validate and demonstrate the efficiency of these algorithms in solving multidimensional knapsack problems. The results were then compared with those in the other two existing PSO algorithms. The simulation and evaluation results showed that the proposed algorithms, BPSOTVAC and CBPSOTVAC, are superior over the other methods according to its success rate, mean absolute deviation, mean absolute percentage error, least error, and standard deviation.     

An improved particle swarm optimization algorithm combined with piecewise linear chaotic map

Particle swarm optimization (PSO) has gained increasing attention in tackling complex optimization problems. Its further superiority when hybridized with other search techniques is also shown. Chaos, with the properties of ergodicity and stochasticity, is definitely a good candidate, but currently only the well-known logistic map is prevalently used. In this paper, the performance and deficiencies of schemes coupling chaotic search into PSO are analyzed. Then, the piecewise linear chaotic map (PWLCM) is introduced to perform the chaotic search. An improved PSO algorithm combined with PWLCM (PWLCPSO) is proposed subsequently, and experimental results verify its great superiority.     

Gbest-guided artificial bee colony algorithm for numerical function optimization

Artificial bee colony (ABC) algorithm invented recently by Karaboga is a biological-inspired optimization algorithm, which has been shown to be competitive with some conventional biological-inspired algorithms, such as genetic algorithm (GA), differential evolution (DE) and particle swarm optimization (PSO). However, there is still an insufficiency in ABC algorithm regarding its solution search equation, which is good at exploration but poor at exploitation. Inspired by PSO, we propose an improved ABC algorithm called gbest-guided ABC (GABC) algorithm by incorporating the information of global best (gbest) solution into the solution search equation to improve the exploitation. The experimental results tested on a set of numerical benchmark functions show that GABC algorithm can outperform ABC algorithm in most of the experiments.     

Tribal particle swarm optimization for neurofuzzy inference systems and its prediction applications

This study presents tribal particle swarm optimization (TPSO) to optimize the parameters of the functional-link-based neurofuzzy inference system (FLNIS) for prediction applications. The proposed TPSO uses particle swarm optimization (PSO) as evolution strategies of the tribes optimization algorithm (TOA) to balance local and global exploration of the search space. The proposed TPSO uses a self-clustering algorithm to divide the particle swarm into multiple tribes, and selects suitable evolution strategies to update each particle. The TPSO also uses a tribal adaptation mechanism to remove and generate particles and reconstruct tribal links. The tribal adaptation mechanism can improve the qualities of the tribe and the tribe adaptation. Finally, the FLNIS model with the proposed TPSO (FLNIS-TPSO) was used in several predictive applications. Experimental results demonstrated that the proposed TPSO method converges quickly and yields a lower RMS error than other current methods.     

A cooperative swarm intelligence algorithm for multi-objective discrete optimization with application to the knapsack problem

We propose a novel cooperative swarm intelligence algorithm to solve multi-objective discrete optimization problems (MODP). Our algorithm combines a firefly algorithm (FA) and a particle swarm optimization (PSO). Basically, we address three main points: the effect of FA and PSO cooperation on the exploration of the search space, the discretization of the two algorithms using a transfer function, and finally, the use of the epsilon dominance relation to manage the size of the external archive and to guarantee the convergence and the diversity of Pareto optimal solutions.We compared the results of our algorithm with the results of five well-known meta-heuristics on nine multi-objective knapsack problem benchmarks. The experiments show clearly the ability of our algorithm to provide a better spread of solutions with a better convergence behavior.     

Particle swarm optimization for the Steiner tree in graph and delay-constrained multicast routing problems

This paper presents the first investigation on applying a particle swarm optimization (PSO) algorithm to both the Steiner tree problem and the delay constrained multicast routing problem. Steiner tree problems, being the underlining models of many applications, have received significant research attention within the meta-heuristics community. The literature on the application of meta-heuristics to multicast routing problems is less extensive but includes several promising approaches. Many interesting research issues still remain to be investigated, for example, the inclusion of different constraints, such as delay bounds, when finding multicast trees with minimum cost. In this paper, we develop a novel PSO algorithm based on the jumping PSO (JPSO) algorithm recently developed by Moreno-Perez et al. (Proc. of the 7th Metaheuristics International Conference, 2007), and also propose two novel local search heuristics within our JPSO framework. A path replacement operator has been used in particle moves to improve the positions of the particle with regard to the structure of the tree. We test the performance of our JPSO algorithm, and the effect of the integrated local search heuristics by an extensive set of experiments on multicast routing benchmark problems and Steiner tree problems from the OR library. The experimental results show the superior performance of the proposed JPSO algorithm over a number of other state-of-the-art approaches.     

A discrete particle swarm optimization method for feature selection in binary classification problems

This paper investigates the feature subset selection problem for the binary classification problem using logistic regression model. We developed a modified discrete particle swarm optimization (PSO) algorithm for the feature subset selection problem. This approach embodies an adaptive feature selection procedure which dynamically accounts for the relevance and dependence of the features included the feature subset. We compare the proposed methodology with the tabu search and scatter search algorithms using publicly available datasets. The results show that the proposed discrete PSO algorithm is competitive in terms of both classification accuracy and computational performance.     

Particle swarm optimization with chaotic opposition-based population initialization and stochastic search technique

Particle swarm optimization (PSO) is a relatively new optimization algorithm that has been applied to a variety of problems. However, it may easily get trapped in a local optima when solving complex multimodal problems. To address this concerning issue, we propose a novel PSO called as CSPSO to improve the performance of PSO on complex multimodal problems in the paper. Specifically, a stochastic search technique is used to execute the exploration in PSO, so as to help the algorithm to jump out of the likely local optima. In addition, to enhance the global convergence, when producing the initial population, both opposition-based learning method and chaotic maps are employed. Moreover, numerical simulation and comparisons with some typical existing algorithms demonstrate the superiority of the proposed algorithm.     

A rank based particle swarm optimization algorithm with dynamic adaptation

The particle swarm optimization (PSO) technique is a powerful stochastic evolutionary algorithm that can be used to find the global optimum solution in a complex search space. This paper presents a variation on the standard PSO algorithm called the rank based particle swarm optimizer, or PSO_rank, employing cooperative behavior of the particles to significantly improve the performance of the original algorithm. In this method, in order to efficiently control the local search and convergence to global optimum solution, the γ best particles are taken to contribute to the updating of the position of a candidate particle. The contribution of each particle is proportional to its strength. The strength is a function of three parameters: strivness, immediacy and number of contributed particles. All particles are sorted according to their fitness values, and only the γ best particles will be selected. The value of γ decreases linearly as the iteration increases. A time-varying inertia weight decreasing non-linearly is introduced to improve the performance. PSO_rank is tested on a commonly used set of optimization problems and is compared to other variants of the PSO algorithm presented in the literature. As a real application, PSO_rank is used for neural network training. The PSO_rank strategy outperformed all the methods considered in this investigation for most of the functions. Experimental results show the suitability of the proposed algorithm in terms of effectiveness and robustness.