A tabu search algorithm for the training of neural networks

The most widely used training algorithm of neural networks (NNs) is back propagation (BP), a gradient-based technique that requires significant computational effort. Metaheuristic search techniques such as genetic algorithms, tabu search (TS) and simulated annealing have been recently used to cope with major shortcomings of BP such as the tendency to converge to a local optimal and a slow convergence rate. In this paper, an efficient TS algorithm employing different strategies to provide a balance between intensification and diversification is proposed for the training of NNs. The proposed algorithm is compared with other metaheuristic techniques found in literature using published test problems, and found to outperform them in the majority of the test cases.     

Chaotic harmony search algorithms

Harmony Search (HS) is one of the newest and the easiest to code music inspired heuristics for optimization problems. Like the use of chaos in adjusting note parameters such as pitch, dynamic, rhythm, duration, tempo, instrument selection, attack time, etc. in real music and in sound synthesis and timbre construction, this paper proposes new HS algorithms that use chaotic maps for parameter adaptation in order to improve the convergence characteristics and to prevent the HS to get stuck on local solutions. This has been done by using of chaotic number generators each time a random number is needed by the classical HS algorithm. Seven new chaotic HS algorithms have been proposed and different chaotic maps have been analyzed in the benchmark functions. It has been detected that coupling emergent results in different areas, like those of HS and complex dynamics, can improve the quality of results in some optimization problems. It has been also shown that, some of the proposed methods have somewhat increased the solution quality, that is in some cases they improved the global searching capability by escaping the local solutions.     

Two improved harmony search algorithms for solving engineering optimization problems

This paper describes two new harmony search (HS) meta-heuristic algorithms for engineering optimization problems with continuous design variables. The key difference between these algorithms and traditional (HS) method is in the way of adjusting bandwidth (bw). bw is very important factor for the high efficiency of the harmony search algorithms and can be potentially useful in adjusting convergence rate of algorithms to optimal solution. First algorithm, proposed harmony search (PHS), introduces a new definition of bandwidth (bw). Second algorithm, improving proposed harmony search (IPHS) employs to enhance accuracy and convergence rate of PHS algorithm. In IPHS, non-uniform mutation operation is introduced which is combination of Yang bandwidth and PHS bandwidth. Various engineering optimization problems, including mathematical function minimization problems and structural engineering optimization problems, are presented to demonstrate the effectiveness and robustness of these algorithms. In all cases, the solutions obtained using IPHS are in agreement or better than those obtained from other methods.     

Parameter adaptive harmony search algorithm for unimodal and multimodal optimization problems

This paper presents a parameter adaptive harmony search algorithm (PAHS) for solving optimization problems. The two important parameters of harmony search algorithm namely Harmony Memory Consideration Rate (HMCR) and Pitch Adjusting Rate (PAR), which were either kept constant or the PAR value was dynamically changed while still keeping HMCR fixed, as observed from literature, are both being allowed to change dynamically in this proposed PAHS. This change in the parameters has been done to get the global optimal solution. Four different cases of linear and exponential changes have been explored. The change has been allowed during the process of improvization. The proposed algorithm is evaluated on 15 standard benchmark functions of various characteristics. Its performance is investigated and compared with three existing harmony search algorithms. Experimental results reveal that proposed algorithm outperforms the existing approaches when applied to 15 benchmark functions. The effects of scalability, noise, and harmony memory size have also been investigated on four approaches of HS. The proposed algorithm is also employed for data clustering. Five real life datasets selected from UCI machine learning repository are used. The results show that, for data clustering, the proposed algorithm achieved results better than other algorithms.     

An improved harmony search algorithm for synchronization of discrete-time chaotic systems

The harmony search (HS) algorithm is a recently developed meta-heuristic algorithm, and has been very successful in a wide variety of optimization problems. HS was conceptualized using an analogy with music improvisation process where music players improvise the pitches of their instruments to obtain better harmony. The HS algorithm does not require initial values and uses a random search instead of a gradient search, so derivative information is unnecessary. Furthermore, the HS algorithm is simple in concept, few in parameters, easy in implementation, imposes fewer mathematical requirements, and does not require initial value settings of the decision variables. In recent years, the investigation of synchronization and control problem for discrete chaotic systems has attracted much attention, and many possible applications. The tuning of a proportional–integral–derivative (PID) controller based on an improved HS (IHS) algorithm for synchronization of two identical discrete chaotic systems subject the different initial conditions is investigated in this paper. Simulation results of the IHS to determine the PID parameters to synchronization of two Hénon chaotic systems are compared with other HS approaches including classical HS and global-best HS. Numerical results reveal that the proposed IHS method is a powerful search and controller design optimization tool for synchronization of chaotic systems.     

An evolutionary algorithm with the partial sequential forward floating search mutation for large-scale feature selection problems

Several meta-heuristic algorithms, such as evolutionary algorithms (EAs) and genetic algorithms (GAs), have been developed for solving feature selection problems due to their efficiency for searching feature subset spaces in feature selection problems. Recently, hybrid GAs have been proposed to improve the performance of conventional GAs by embedding a local search operation, or sequential forward floating search mutation, into the GA. Existing hybrid algorithms may damage individuals’ genetic information obtained from genetic operations during the local improvement procedure because of a sequential process of the mutation operation and the local improvement operation. Another issue with a local search operation used in the existing hybrid algorithms is its inappropriateness for large-scale problems. Therefore, we propose a novel approach for solving large-sized feature selection problems, namely, an EA with a partial sequential forward floating search mutation (EAwPS). The proposed approach integrates a local search technique, that is, the partial sequential forward floating search mutation into an EA method. Two algorithms, EAwPS-binary representation (EAwPS-BR) for medium-sized problems and EAwPS-integer representation (EAwPS-IR) for large-sized problems, have been developed. The adaptation of a local improvement method into the EA speeds up the search and directs the search into promising solution areas. We compare the performance of the proposed algorithms with other popular meta-heuristic algorithms using the medium- and large-sized data sets. Experimental results demonstrate that the proposed EAwPS extracts better features within reasonable computational times.     

A diverse human learning optimization algorithm

Human Learning Optimization is a simple but efficient meta-heuristic algorithm in which three learning operators, i.e. the random learning operator, the individual learning operator, and the social learning operator, are developed to efficiently search the optimal solution by imitating the learning mechanisms of human beings. However, HLO assumes that all the individuals possess the same learning ability, which is not true in a real human population as the IQ scores of humans, one of the most important indices of the learning ability of humans, follow Gaussian distribution and increase with the development of society and technology. Inspired by this fact, this paper proposes a Diverse Human Learning Optimization algorithm (DHLO), into which the Gaussian distribution and dynamic adjusting strategy are introduced. By adopting a set of Gaussian distributed parameter values instead of a constant to diversify the learning abilities of DHLO, the robustness of the algorithm is strengthened. In addition, by cooperating with the dynamic updating operation, DHLO can adjust to better parameter values and consequently enhances the global search ability of the algorithm. Finally, DHLO is applied to tackle the CEC05 benchmark functions as well as knapsack problems, and its performance is compared with the standard HLO as well as the other eight meta-heuristics, i.e. the Binary Differential Evolution, Simplified Binary Artificial Fish Swarm Algorithm, Adaptive Binary Harmony Search, Binary Gravitational Search Algorithms, Binary Bat Algorithms, Binary Artificial Bee Colony, Bi-Velocity Discrete Particle Swarm Optimization, and Modified Binary Particle Swarm Optimization. The experimental results show that the presented DHLO outperforms the other algorithms in terms of search accuracy and scalability.     

GHS + LEM: Global-best Harmony Search using learnable evolution models

This paper presents a new optimization algorithm called GHS + LEM, which is based on the Global-best Harmony Search algorithm (GHS) and techniques from the learnable evolution models (LEM) to improve convergence and accuracy of the algorithm. The performance of the algorithm is evaluated with fifteen optimization functions commonly used by the optimization community. In addition, the results obtained are compared against the original Harmony Search algorithm, the Improved Harmony Search algorithm and the Global-best Harmony Search algorithm. The assessment shows that the proposed algorithm (GHS + LEM) improves the accuracy of the results obtained in relation to the other options, producing better results in most situations, but more specifically in problems with high dimensionality, where it offers a faster convergence with fewer iterations.     

A novel hybrid approach combining electromagnetism-like method with Solis and Wets local search for continuous optimization problems

The electromagnetism-like method (EM) is a meta-heuristic algorithm utilizing an attraction-repulsion mechanism to move sample points towards optimality in continuous optimization problems. Traditionally, the EM uses two algorithms known as the original and revised EMs. This paper presents a novel hybrid approach for EM by employing a well-known local search, called Solis and Wets. To show the performance of our proposed hybrid EM, a number of experiments are carried out on a set of well-known test problems and the related results are compared with two forgoing algorithms.     

A hybrid genetic local search algorithm for the permutation flowshop scheduling problem

Traditionally, the permutation flowshop scheduling problem (PFSP) was with the criterion of minimizing makespan. The permutation flowshop scheduling problem to minimize the total flowtime has attracted more attention from researchers in recent years. In this paper, a hybrid genetic local search algorithm is proposed to solve this problem with each of both criteria. The proposed algorithm hybridizes the genetic algorithm and a novel local search scheme that combines two local search methods: the Insertion Search (IS) and the Insertion Search with Cut-and-Repair (ISCR). It employs the genetic algorithm to do the global search and two local search methods to do the local search. Two local search methods play different roles in the search process. The Insertion Search is responsible for searching a small neighborhood while the Insertion Search with Cut-and-Repair is responsible for searching a large neighborhood. Furthermore, the orthogonal-array-based crossover operator is designed to enhance the GA’s capability of intensification. The experimental results show the advantage of combining the two local search methods. The performance of the proposed hybrid genetic algorithm is very competitive. For the PFSP with the total flowtime criterion, it improved 66 out of the 90 current best solutions reported in the literature in short-term search and it also improved all the 20 current best solutions reported in the literature in long-term search. For the PFSP with the makespan criterion, the proposed algorithm also outperforms the other three methods recently reported in the literature.     

Hybrid genetic algorithm for permutation flowshop scheduling problems with total flowtime minimization

In this paper, a HGA (hybrid genetic algorithm) is proposed for permutation flowshop scheduling problems (PFSP) with total flowtime minimization, which are known to be NP-hard. One of the chromosomes in the initial population is constructed by a suitable heuristic and the others are yielded randomly. An artificial chromosome is generated by a weighted simple mining gene structure, with which a new crossover operator is presented. Additionally, two effective heuristics are adopted as local search to improve all generated chromosomes in each generation. The HGA is compared with one of the most effective heuristics and a recent meta-heuristic on 120 benchmark instances. Experimental results show that the HGA outperforms the other two algorithms for all cases. Furthermore, HGA obtains 115 best solutions for the benchmark instances, 92 of which are newly discovered.     

Novel selection schemes for harmony search

Selection is a vital component used in Evolutionary Algorithms (EA) where the fitness value of the solution has influence on the evolution process. Normally, any efficient selection method makes use of the Darwinian principle of natural selection (i.e., survival of the fittest). Harmony search (HS) is a recent EA inspired by musical improvisation process to seek a pleasing harmony. Originally, two selection methods are used in HS: (i) memory consideration selection method where the values of the decision variables are randomly selected from the population (or solutions stored in harmony memory (HM)) to generate a new harmony, and (ii) selecting a new solution in HM whereby a greedy selection is used to update the HM. The memory consideration selection, the focal point of this paper, is not based on natural selection principle which draws heavily on random selection. In this paper, novel selection schemes which replace the random selection scheme in memory consideration are investigated, comprising global-best, fitness-proportional, tournament, linear rank and exponential rank. The proposed selection schemes are individually altered and incorporated in the process of memory consideration and each adoption is realized as a new HS variation. The performance of the proposed HS variations are evaluated and a comparative study is conducted. The experimental results using benchmark functions show that the selection schemes incorporated in memory consideration directly affect the performance of HS algorithm. Finally, a parameter sensitivity analysis of the proposed HS variations is analyzed.     

Hybridizing local search algorithms for global optimization

In this paper, we combine two types of local search algorithms for global optimization of continuous functions. In the literature, most of the hybrid algorithms are produced by combination of a global optimization algorithm with a local search algorithm and the local search is used to improve the solution quality, not to explore the search space to find independently the global optimum. The focus of this research is on some simple and efficient hybrid algorithms by combining the Nelder–Mead simplex (NM) variants and the bidirectional random optimization (BRO) methods for optimization of continuous functions. The NM explores the whole search space to find some promising areas and then the BRO local search is entered to exploit optimal solution as accurately as possible. Also a new strategy for shrinkage stage borrowed from differential evolution (DE) is incorporated in the NM variants. To examine the efficiency of proposed algorithms, those are evaluated by 25 benchmark functions designed for the special session on real-parameter optimization of CEC2005. A comparison study between the hybrid algorithms and some DE algorithms and non-parametric analysis of obtained results demonstrate that the proposed algorithms outperform most of other algorithms and their difference in most cases is statistically considerable. In a later part of the comparative experiments, a comparison of the proposed algorithms with some other evolutionary algorithms reported in the CEC2005 confirms a better performance of our proposed algorithms.     

A Hybrid Symbiosis Organisms Search algorithm and its application to real world problems

In this paper, a new hybrid algorithm, Hybrid Symbiosis Organisms Search (HSOS) has been proposed by combining Symbiosis Organisms Search (SOS) algorithm with Simple Quadratic Interpolation (SQI). The proposed algorithm provides more efficient behavior when dealing with real-world and large scale problems. To verify the performance of this suggested algorithm, 13 (Thirteen) well known benchmark functions, CEC2005 and CEC2010 special session on real-parameter optimization are being considered. The results obtained by the proposed method are compared with other state-of-the-art algorithms and it was observed that the suggested approach provides an effective and efficient solution in regards to the quality of the final result as well as the convergence rate. Moreover, the effect of the common controlling parameters of the algorithm, viz. population size, number of fitness evaluations (number of generations) of the algorithm are also being investigated by considering different population sizes and the number of fitness evaluations (number of generations). Finally, the method endorsed in this paper has been applied to two real life problems and it was inferred that the output of the proposed algorithm is satisfactory.     

A comparison of several nearest neighbor classifier metrics using Tabu Search algorithm for the feature selection problem

The feature selection problem is an interesting and important topic which is relevant for a variety of database applications. This paper utilizes the Tabu Search metaheuristic algorithm to implement a feature subset selection procedure while the nearest neighbor classification method is used for the classification task. Tabu Search is a general metaheuristic procedure that is used in order to guide the search to obtain good solutions in complex solution spaces. Several metrics are used in the nearest neighbor classification method, such as the euclidean distance, the Standardized Euclidean distance, the Mahalanobis distance, the City block metric, the Cosine distance and the Correlation distance, in order to identify the most significant metric for the nearest neighbor classifier. The performance of the proposed algorithms is tested using various benchmark datasets from UCI Machine Learning Repository.     

Navigation of underwater robot based on dynamically adaptive harmony search algorithm

The current research work has employed an evolutionary based novel navigational strategy to trace the collision free near optimal path for underwater robot in a three-dimensional scenario. The population based harmony search algorithm has been dynamically adapted and used to search next global best pose for underwater robot while obstacle is identified near about robot’s current pose. Each pose is evaluated based on their respective value for objective function which incorporates features of path length minimization as well as obstacle avoidance. Dynamic adaptation of control parameters and new perturbation schemes for solution vectors of harmony search has been proposed to strengthen both exploitation and randomization ability of present search process in a balanced manner. Such adaptive tuning process has found to be more effective for avoiding early convergence during underwater motion in comparison with performances of other popular variants of Harmony Search. The proposed path planning method has also shown better navigational performance in comparison with improved version of ant colony optimization and heuristic potential field method for avoiding static obstacles of different shape and sizes during underwater motion. Simulation studies and corresponding experimental verification for three-dimensional navigation are performed to check the accuracy, robustness and efficiency of proposed dynamically adaptive harmony search algorithm.     

Novel hybrid heuristics for an extension of the dynamic relay deployment problem over disaster areas

In this paper, we propose a novel autonomous intelligent tool for the optimum design of a wireless relayed communication network deployed over disaster areas. The so-called dynamic relay deployment problem consists of finding the optimum number of deployed relays and their location aimed at simultaneously maximizing the overall number of mobile nodes covered and minimizing the cost of the deployment. In this paper, we extend the problem by considering diverse relay models characterized by different coverage radii and associated costs. To efficiently tackle this problem we derive a novel hybrid scheme comprising: (1) a Harmony Search (HS)-based global search procedure and (2) a modified version of the well-known K-means clustering algorithm as a local search technique. Single- and bi-objective formulations of the algorithm are proposed for emergency and strategic operational planning, respectively. Monte Carlo simulations are run over a emulated scenario based on real statistical data from the Castilla La Mancha region (center of Spain) to show that, in comparison with a standard implementation of the K-means algorithm followed by a exhaustive search procedure over all relay-model combinations, the proposed scheme renders on average better coverage levels and reduced costs providing, at the same time, an intelligent tool capable of simultaneously determining the number and models of the relays to be deployed.     

A tabu search algorithm for the single machine total weighted tardiness problem

In this study, a tabu search (TS) approach to the single machine total weighted tardiness problem (SMTWT) is presented. The problem consists of a set of independent jobs with distinct processing times, weights and due dates to be scheduled on a single machine to minimize total weighted tardiness. The theoretical foundation of single machine scheduling with due date related objectives reveal that the problem is NP-hard, rendering it a challenging area for meta-heuristic approaches. This paper presents a totally deterministic TS algorithm with a hybrid neighborhood and dynamic tenure structure, and investigates the strength of several candidate list strategies based on problem specific characteristics in increasing the efficiency of the search. The proposed TS approach yields very high quality results for a set of benchmark problems obtained from the literature.     

基于乌鸦喝水的元启发式优化算法

启发式优化算法已成为求解复杂优化问题的一种有效方法,可用于解决传统的优化方法难以求解的问题.受乌鸦喝水寓言故事启发,提出一种新型元启发式优化算法—乌鸦喝水算法,首先建立了乌鸦喝水算法数学模型;其次,给出实现该算法的详细步骤;最后,将该算法用于基准函数优化,并将该算法与乌鸦搜索算法、粒子群优化算法、多元宇宙优化算法、花授粉算法、布谷鸟算法等群智能算法进行了比较.仿真实验结果表明,乌鸦喝水算法优于其他算法.     

GRASP-based heuristic algorithm for the multi-product multi-vehicle inventory routing problem

In this paper, we introduce an improved Greedy Randomized Adaptive Search Procedure (GRASP) based heuristic for the multi-product multi-vehicle inventory routing problem (MMIRP). The inventory routing problem, which combines the vehicle-routing problem and the inventory control decisions, is one of the most important problems in combinatorial optimization field. To deal with the MMIRP, we develop a GRASP-based heuristic (GBH). Each GBH iteration consists of two sequential phases; the first phase is a Greedy Randomized Procedure, in which, the best tradeoff between the inventory holding cost and routing cost is looked. Then, in the second phase, as local search for the GRASP, we use the Tabu search (TS) meta-heuristic to improve the solution found in the first phase. The GBH two phases are repeated until some stopped criterion is met. Our proposed method is evaluated on two benchmark data sets, and successfully compared with two state-of-the-art algorithms.