Integrating particle swarm optimization with genetic algorithms for solving nonlinear optimization problems

Heuristic optimization provides a robust and efficient approach for solving complex real-world problems. The aim of this paper is to introduce a hybrid approach combining two heuristic optimization techniques, particle swarm optimization (PSO) and genetic algorithms (GA). Our approach integrates the merits of both GA and PSO and it has two characteristic features. Firstly, the algorithm is initialized by a set of random particles which travel through the search space. During this travel an evolution of these particles is performed by integrating PSO and GA. Secondly, to restrict velocity of the particles and control it, we introduce a modified constriction factor. Finally, the results of various experimental studies using a suite of multimodal test functions taken from the literature have demonstrated the superiority of the proposed approach to finding the global optimal solution.     

Particle swarm optimization with time-varying acceleration coefficients for the multidimensional knapsack problem

The multidimensional knapsack problem (MKP) is a difficult combinatorial optimization problem, which has been proven as NP-hard problems. Various population-based search algorithms are applied to solve these problems. The particle swarm optimization (PSO) technique is adapted in our study, which proposes two novel PSO algorithms, namely, the binary PSO with time-varying acceleration coefficients (BPSOTVAC) and the chaotic binary PSO with time-varying acceleration coefficients (CBPSOTVAC). The two proposed methods were tested using 116 benchmark problems from the OR-Library to validate and demonstrate the efficiency of these algorithms in solving multidimensional knapsack problems. The results were then compared with those in the other two existing PSO algorithms. The simulation and evaluation results showed that the proposed algorithms, BPSOTVAC and CBPSOTVAC, are superior over the other methods according to its success rate, mean absolute deviation, mean absolute percentage error, least error, and standard deviation.     

A discrete particle swarm optimization method for feature selection in binary classification problems

This paper investigates the feature subset selection problem for the binary classification problem using logistic regression model. We developed a modified discrete particle swarm optimization (PSO) algorithm for the feature subset selection problem. This approach embodies an adaptive feature selection procedure which dynamically accounts for the relevance and dependence of the features included the feature subset. We compare the proposed methodology with the tabu search and scatter search algorithms using publicly available datasets. The results show that the proposed discrete PSO algorithm is competitive in terms of both classification accuracy and computational performance.     

Particle swarm optimization: Hybridization perspectives and experimental illustrations

Metaheuristic optimization algorithms have become popular choice for solving complex and intricate problems which are otherwise difficult to solve by traditional methods. In the present study an attempt is made to review the hybrid optimization techniques in which one main algorithm is a well known metaheuristic; particle swarm optimization or PSO. Hybridization is a method of combining two (or more) techniques in a judicious manner such that the resulting algorithm contains the positive features of both (or all) the algorithms. Depending on the algorithm/s used we made three classifications as (i) Hybridization of PSO and genetic algorithms (ii) Hybridization of PSO with differential evolution and (iii) Hybridization of PSO with other techniques. Where, other techniques include various local and global search methods. Besides giving the review we also show a comparison of three hybrid PSO algorithms; hybrid differential evolution particle swarm optimization (DE-PSO), adaptive mutation particle swarm optimization (AMPSO) and hybrid genetic algorithm particle swarm optimization (GA-PSO) on a test suite of nine conventional benchmark problems.     

Chaos-enhanced accelerated particle swarm optimization

There are more than two dozen variants of particle swarm optimization (PSO) algorithms in the literature. Recently, a new variant, called accelerated PSO (APSO), shows some extra advantages in convergence for global search. In the present study, we will introduce chaos into the APSO in order to further enhance its global search ability. Firstly, detailed studies are carried out on benchmark problems with twelve different chaotic maps to find out the most efficient one. Then the chaotic APSO (CAPSO) will be compared with some other chaotic PSO algorithms presented in the literature. The performance of the CAPSO algorithm is also validated using three engineering problems. The results show that the CAPSO with an appropriate chaotic map can clearly outperform standard APSO, with very good performance in comparison with other algorithms and in application to a complex problem.     

Optimal multiple-objective resource allocation using hybrid particle swarm optimization and adaptive resource bounds technique

The multiple-objective resource allocation problem (MORAP) seeks for an allocation of resource to a number of activities such that a set of objectives are optimized simultaneously and the resource constraints are satisfied. MORAP has many applications, such as resource distribution, project budgeting, software testing, health care resource allocation, etc. This paper addresses the nonlinear MORAP with integer decision variable constraint. To guarantee that all the resource constraints are satisfied, we devise an adaptive-resource-bound technique to construct feasible solutions. The proposed method employs the particle swarm optimization (PSO) paradigm and presents a hybrid execution plan which embeds a hill-climbing heuristic into the PSO for expediting the convergence. To cope with the optimization problem with multiple objectives, we evaluate the candidate solutions based on dominance relationship and a score function. Experimental results manifest that the hybrid PSO derives solution sets which are very close to the exact Pareto sets. The proposed method also outperforms several representatives of the state-of-the-art algorithms on a simulation data set of the MORAP.     

A competitive and cooperative co-evolutionary approach to multi-objective particle swarm optimization algorithm design

Multi-objective particle swarm optimization (MOPSO) is an optimization technique inspired by bird flocking, which has been steadily gaining attention from the research community because of its high convergence speed. On the other hand, in the face of increasing complexity and dimensionality of today’s application coupled with its tendency of premature convergence due to the high convergence speeds, there is a need to improve the efficiency and effectiveness of MOPSO. In this paper a competitive and cooperative co-evolutionary approach is adapted for multi-objective particle swarm optimization algorithm design, which appears to have considerable potential for solving complex optimization problems by explicitly modeling the co-evolution of competing and cooperating species. The competitive and cooperative co-evolution model helps to produce the reasonable problem decompositions by exploiting any correlation, interdependency between components of the problem. The proposed competitive and cooperative co-evolutionary multi-objective particle swarm optimization algorithm (CCPSO) is validated through comparisons with existing state-of-the-art multi-objective algorithms using established benchmarks and metrics. Simulation results demonstrated that CCPSO shows competitive, if not better, performance as compared to the other algorithms.     

Particle swarm with radial basis function surrogates for expensive black-box optimization

This paper develops the OPUS (Optimization by Particle swarm Using Surrogates) framework for expensive black-box optimization. In each iteration, OPUS considers multiple trial positions for each particle in the swarm and uses a surrogate model to identify the most promising trial position. Moreover, the current overall best position is refined by finding the global minimum of the surrogate in the neighborhood of that position. OPUS is implemented using an RBF surrogate and the resulting OPUS-RBF algorithm is applied to a 36-D groundwater bioremediation problem, a 14-D watershed calibration problem, and ten mostly 30-D test problems. OPUS-RBF is compared with a standard PSO, CMA-ES, two other surrogate-assisted PSO algorithms, and an RBF-assisted evolution strategy. The numerical results suggest that OPUS-RBF is promising for expensive black-box optimization.     

A multi-swarm PSO using charged particles in a partitioned search space for continuous optimization

Particle swarm optimization (PSO) is characterized by a fast convergence, which can lead the algorithms of this class to stagnate in local optima. In this paper, a variant of the standard PSO algorithm is presented, called PSO-2S, based on several initializations in different zones of the search space, using charged particles. This algorithm uses two kinds of swarms, a main one that gathers the best particles of auxiliary ones, initialized several times. The auxiliary swarms are initialized in different areas, then an electrostatic repulsion heuristic is applied in each area to increase its diversity. We analyse the performance of the proposed approach on a testbed made of unimodal and multimodal test functions with and without coordinate rotation and shift. The Lennard-Jones potential problem is also used. The proposed algorithm is compared to several other PSO algorithms on this benchmark. The obtained results show the efficiency of the proposed algorithm.     

Chaotic catfish particle swarm optimization for solving global numerical optimization problems

Chaotic catfish particle swarm optimization (C-CatfishPSO) is a novel optimization algorithm proposed in this paper. C-CatfishPSO introduces chaotic maps into catfish particle swarm optimization (CatfishPSO), which increase the search capability of CatfishPSO via the chaos approach. Simple CatfishPSO relies on the incorporation of catfish particles into particle swarm optimization (PSO). The introduced catfish particles improve the performance of PSO considerably. Unlike other ordinary particles, the catfish particles initialize a new search from extreme points of the search space when the gbest fitness value (global optimum at each iteration) has not changed for a certain number of consecutive iterations. This results in further opportunities of finding better solutions for the swarm by guiding the entire swarm to promising new regions of the search space and accelerating the search. The introduced chaotic maps strengthen the solution quality of PSO and CatfishPSO significantly. The resulting improved PSO and CatfishPSO are called chaotic PSO (C-PSO) and chaotic CatfishPSO (C-CatfishPSO), respectively. PSO, C-PSO, CatfishPSO, C-CatfishPSO, as well as other advanced PSO procedures from the literature were extensively compared on several benchmark test functions. Statistical analysis of the experimental results indicate that the performance of C-CatfishPSO is better than the performance of PSO, C-PSO, CatfishPSO and that C-CatfishPSO is also superior to advanced PSO methods from the literature.     

PSOHS: an efficient two-stage approach for data clustering

Cluster analysis is an important task in data mining and refers to group a set of objects such that the similarities among objects within the same group are maximal while similarities among objects from different groups are minimal. The particle swarm optimization algorithm (PSO) is one of the famous metaheuristic optimization algorithms, which has been successfully applied to solve the clustering problem. However, it has two major shortcomings. The PSO algorithm converges rapidly during the initial stages of the search process, but near global optimum, the convergence speed will become very slow. Moreover, it may get trapped in local optimum if the global best and local best values are equal to the particle’s position over a certain number of iterations. In this paper we hybridized the PSO with a heuristic search algorithm to overcome the shortcomings of the PSO algorithm. In the proposed algorithm, called PSOHS, the particle swarm optimization is used to produce an initial solution to the clustering problem and then a heuristic search algorithm is applied to improve the quality of this solution by searching around it. The superiority of the proposed PSOHS clustering method, as compared to other popular methods for clustering problem is established for seven benchmark and real datasets including Iris, Wine, Crude Oil, Cancer, CMC, Glass and Vowel.     

An evolutionary game based particle swarm optimization algorithm

Particle swarm optimization (PSO) is an evolutionary algorithm used extensively. This paper presented a new particle swarm optimizer based on evolutionary game (EGPSO). We map particles’ finding optimal solution in PSO algorithm to players’ pursuing maximum utility by choosing strategies in evolutionary games, using replicator dynamics to model the behavior of particles. And in order to overcome premature convergence a multi-start technique was introduced. Experimental results show that EGPSO can overcome premature convergence and has great performance of convergence property over traditional PSO.     

Robust portfolio optimization with a hybrid heuristic algorithm

Estimation errors in both the expected returns and the covariance matrix hamper the construction of reliable portfolios within the Markowitz framework. Robust techniques that incorporate the uncertainty about the unknown parameters are suggested in the literature. We propose a modification as well as an extension of such a technique and compare both with another robust approach. In order to eliminate oversimplifications of Markowitz’ portfolio theory, we generalize the optimization framework to better emulate a more realistic investment environment. Because the adjusted optimization problem is no longer solvable with standard algorithms, we employ a hybrid heuristic to tackle this problem. Our empirical analysis is conducted with a moving time window for returns of the German stock index DAX100. The results of all three robust approaches yield more stable portfolio compositions than those of the original Markowitz framework. Moreover, the out-of-sample risk of the robust approaches is lower and less volatile while their returns are not necessarily smaller.     

The block diagram method for designing the particle swarm optimization algorithm

In this paper, the block diagram method of the dispersed control system is proposed for designing or improving the normal particle swarm optimization algorithms (PSO), that is, it uses the Jury-test of the control theory to compare the block diagrams getting from existing particle swarm optimization methods and finds out some defects of the existing particle swarm optimization methods, for example, the premature convergence of PSO algorithm, and so on. Thus a new particle swarm algorithm is also proposed for improving these defects, that is, the speed iteration and position iteration formulas of PSO are revised for both adjusting its convergence speed and jumping out of the local minimum points. To show effectiveness of the proposed method, the simulations of 13 benchmark examples are carried out, as a result, it indicates that the proposed method is very useful.     

Balanced fuzzy particle swarm optimization

In the present study an extension of particle swarm optimization (PSO) algorithm which is in conformity with actual nature is introduced for solving combinatorial optimization problems. Development of this algorithm is essentially based on balanced fuzzy sets theory. The classical fuzzy sets theory cannot distinguish differences between positive and negative information of membership functions, while in the new method both kinds of information “positive and negative” about membership function are equally important. The balanced fuzzy particle swarm optimization algorithm is used for fundamental optimization problem entitled traveling salesman problem (TSP). For convergence inspecting of new algorithm, method was used for TSP problems. Convergence curves were represented fast convergence in restricted and low iterations for balanced fuzzy particle swarm optimization algorithm (BF-PSO) comparison with fuzzy particle swarm optimization algorithm (F-PSO).     

Short-term cascaded hydroelectric system scheduling based on chaotic particle swarm optimization using improved logistic map

In order to solve the model of short-term cascaded hydroelectric system scheduling, a novel chaotic particle swarm optimization (CPSO) algorithm using improved logistic map is introduced, which uses the water discharge as the decision variables combined with the death penalty function. According to the principle of maximum power generation, the proposed approach makes use of the ergodicity, symmetry and stochastic property of improved logistic chaotic map for enhancing the performance of particle swarm optimization (PSO) algorithm. The new hybrid method has been examined and tested on two test functions and a practical cascaded hydroelectric system. The experimental results show that the effectiveness and robustness of the proposed CPSO algorithm in comparison with other traditional algorithms.     

An efficient co-swarm particle swarm optimization for non-linear constrained optimization

This paper proposes a new co-swarm PSO (CSHPSO) for constrained optimization problems, which is obtained by hybridizing the recently proposed shrinking hypersphere PSO (SHPSO) with the differential evolution (DE) approach. The total swarm is subdivided into two sub swarms in such a way that the first sub swarms uses SHPSO and second sub swarms uses DE. Experiments are performed on a state-of-the-art problems proposed in IEEE CEC 2006. The results of the CSHPSO is compared with SHPSO and DE in a variety of fashions. A statistical approach is applied to provide the significance of the numerical experiments. In order to further test the efficacy of the proposed CSHPSO, an economic dispatch (ED) problem with valve points effects for 40 generating units is solved. The results of the problem using CSHPSO is compared with SHPSO, DE and the existing solutions in the literature. It is concluded that CSHPSO is able to give the minimal cost for the ED problem in comparison with the other algorithms considered. Hence, CSHPSO is a promising new co-swarm PSO which can be used to solve any real constrained optimization problem.     

An optimization model for traceable closed-loop supply chain networks

In this paper, a new non-linear mixed-integer mathematical programming problem is proposed to model a stochastic multi-product closed-loop supply chain (CLSC). The radio frequency identification (RFID) system is implemented in the supply chain to decrease product losses and the overall lead time of transportation while computing the profit derived from internet and conventional sales. The resulting traceable CLSC improves upon the existing literature by allowing us to: (1) boost the incorporation of traceability assumptions in mathematical programming problems so as to enhance the efficiency and visibility of a supply chain, (2) analyze the strategic effects that different internet sale formats have on customers’ evaluations and acquisition choices, and (3) account for the environmental and socio-economical dimension by explicitly formalizing employment-based incomes as part of the profit function. Two meta-heuristic algorithms are introduced to solve the proposed optimization problem, namely, the greedy randomized adaptive search procedure (GRASP) and particle swarm optimization (PSO). Twelve test problems of different sizes are generated and solved using these algorithms. The computational results show that GRASP outperforms PSO in terms of both profit and CPU time values. Finally, a case study in the network marketing industry is presented and managerial implications outlined to show the validity of the proposed model and shed more light on its practical implications.     

A hybrid simplex search and particle swarm optimization for unconstrained optimization

This paper proposes the hybrid NM-PSO algorithm based on the Nelder–Mead (NM) simplex search method and particle swarm optimization (PSO) for unconstrained optimization. NM-PSO is very easy to implement in practice since it does not require gradient computation. The modification of both the Nelder–Mead simplex search method and particle swarm optimization intends to produce faster and more accurate convergence. The main purpose of the paper is to demonstrate how the standard particle swarm optimizers can be improved by incorporating a hybridization strategy. In a suite of 20 test function problems taken from the literature, computational results via a comprehensive experimental study, preceded by the investigation of parameter selection, show that the hybrid NM-PSO approach outperforms other three relevant search techniques (i.e., the original NM simplex search method, the original PSO and the guaranteed convergence particle swarm optimization (GCPSO)) in terms of solution quality and convergence rate. In a later part of the comparative experiment, the NM-PSO algorithm is compared to various most up-to-date cooperative PSO (CPSO) procedures appearing in the literature. The comparison report still largely favors the NM-PSO algorithm in the performance of accuracy, robustness and function evaluation. As evidenced by the overall assessment based on two kinds of computational experience, the new algorithm has demonstrated to be extremely effective and efficient at locating best-practice optimal solutions for unconstrained optimization.     

A cooperative swarm intelligence algorithm for multi-objective discrete optimization with application to the knapsack problem

We propose a novel cooperative swarm intelligence algorithm to solve multi-objective discrete optimization problems (MODP). Our algorithm combines a firefly algorithm (FA) and a particle swarm optimization (PSO). Basically, we address three main points: the effect of FA and PSO cooperation on the exploration of the search space, the discretization of the two algorithms using a transfer function, and finally, the use of the epsilon dominance relation to manage the size of the external archive and to guarantee the convergence and the diversity of Pareto optimal solutions.We compared the results of our algorithm with the results of five well-known meta-heuristics on nine multi-objective knapsack problem benchmarks. The experiments show clearly the ability of our algorithm to provide a better spread of solutions with a better convergence behavior.