A novel hybrid algorithm combining hunting search with harmony search algorithm for training neural networks

Neural networks (NNs) are one of the most widely used techniques for pattern classification. Owing to the most common back-propagation training algorithm of NN being extremely computationally intensive and it having some drawbacks, such as converging into local minima, many meta-heuristic algorithms have been applied to training of NNs. This paper presents a novel hybrid algorithm which is the integration of Harmony Search (HS) and Hunting Search (HuS) algorithms, called h_HS-HuS, in order to train Feed-Forward Neural Networks (FFNNs) for pattern classification. HS and HuS algorithms are recently proposed meta-heuristic algorithms inspired from the improvisation process of musicians and hunting of animals, respectively. Harmony search builds up the main structure of the hybrid algorithm, and HuS forms the pitch adjustment phase of the HS algorithm. The performance proposed algorithm is compared to conventional and meta-heuristic algorithms. Empirical tests are carried out by training NNs on nine widely used classification benchmark problems. The experimental results show that the proposed hybrid harmony-hunting algorithm is highly capable of training NNs.     

Optimization of manufacturing systems using a neural network metamodel with a new training approach

In this study, two manufacturing systems, a kanban-controlled system and a multi-stage, multi-server production line in a diamond tool production system, are optimized utilizing neural network metamodels (tst_NNM) trained via tabu search (TS) which was developed previously by the authors. The most widely used training algorithm for neural networks has been back propagation which is based on a gradient technique that requires significant computational effort. To deal with the major shortcomings of back propagation (BP) such as the tendency to converge to a local optimal and a slow convergence rate, the TS metaheuristic method is used for the training of artificial neural networks to improve the performance of the metamodelling approach. The metamodels are analysed based on their ability to predict simulation results versus traditional neural network metamodels that have been trained by BP algorithm (bp_NNM). Computational results show that tst_NNM is superior to bp_NNM for both of the manufacturing systems.     

A novel metaheuristics approach for continuous global optimization

This paper proposes a novel metaheuristics approach to find the global optimum of continuous global optimization problems with box constraints. This approach combines the characteristics of modern metaheuristics such as scatter search (SS), genetic algorithms (GAs), and tabu search (TS) and named as hybrid scatter genetic tabu (HSGT) search. The development of the HSGT search, parameter settings, experimentation, and efficiency of the HSGT search are discussed. The HSGT has been tested against a simulated annealing algorithm, a GA under the name GENOCOP, and a modified version of a hybrid scatter genetic (HSG) search by using 19 well known test functions. Applications to Neural Network training are also examined. From the computational results, the HSGT search proved to be quite effective in identifying the global optimum solution which makes the HSGT search a promising approach to solve the general nonlinear optimization problem.     

Part type selection problem in flexible manufacturing systems: tabu search algorithms

Production planning in flexible manufacturing systems is concerned with the organization of production in order to satisfy a given master production schedule. The planning problem typically gives rise to several hierarchical subproblems which are then solved sequentially or simultaneously. In this paper, we address one of the subproblems: the part type selection problem. The problem is to determine a subset of part types having production requirements for immediate and simultaneous processing over the upcoming period of the planning horizon, subject to the tool magazine and processing time limitation. Several versions of tabu search (TS) algorithm are proposed for solving the problem. A systematic computational test is conducted to test the performance of the TS algorithms. The best TS algorithm developed is compared to a simulated annealing algorithm.     

Continuous ant colony system and tabu search algorithms hybridized for global minimization of continuous multi-minima functions

A new hybrid optimization method, combining Continuous Ant Colony System (CACS) and Tabu Search (TS) is proposed for minimization of continuous multi-minima functions. The new algorithm incorporates the concepts of promising list, tabu list and tabu balls from TS into the framework of CACS. This enables the resultant algorithm to avoid bad regions and to be guided toward the areas more likely to contain the global minimum. New strategies are proposed to dynamically tune the radius of the tabu balls during the execution and also to handle the variable correlations. The promising list is also used to update the pheromone distribution over the search space. The parameters of the new method are tuned based on the results obtained for a set of standard test functions. The results of the proposed scheme are also compared with those of some recent ant based and non-ant based meta-heuristics, showing improvements in terms of accuracy and efficiency.     

Goal programming using multiple objective tabu search

Goal programming (GP) is one of the most commonly used mathematical programming tools to model multiple objective optimisation (MOO) problems. There are numerous MOO problems of various complexity modelled using GP in the literature. One of the main difficulties in the GP is to solve their mathematical formulations optimally. Due to difficulties imposed by the classical solution techniques there is a trend in the literature to solve mathematical programming formulations including goal programmes, using the modern heuristics optimisation techniques, namely genetic algorithms (GA), tabu search (TS) and simulated annealing (SA). This paper uses the multiple objective tabu search (MOTS) algorithm, which was proposed previously by the author to solve GP models. In the proposed approach, GP models are first converted to their classical MOO equivalent by using some simple conversion procedures. Then the problem is solved using the MOTS algorithm. The results obtained from the computational experiment show that MOTS can be considered as a promising candidate tool for solving GP models.     

Nonlinear optimisation using directional step lengths based on RPROP

A search method based on the backpropagation rule commonly used for training neural networks is proposed here for the optimisation of smooth nonlinear functions. The use of the Resilient backPROPagation (RPROP) heuristic rule for local minimisation is described. The details of employing the directional step length determined by RPROP along with a simple restarting scheme are provided. In the approach proposed here direct use of the directional step determined by the heuristic without using any line search conditions takes place. The overall algorithm has been tested on a number of benchmark functions found in the literature with very positive results. The test problems’ dimension ranges from 100 to 50,000. The results obtained show that the suggested search direction method results to a highly efficient algorithm suitable for large scale optimisation.     

A GMDH-based fuzzy modeling approach for constructing TS model

In this paper, a new learning algorithm based on group method of data handling (GMDH) is proposed for the identification of Takagi-Sugeno fuzzy model. Different from existing methods, the new approach, called TS-GMDH, starts from simple elementary TS fuzzy models, and then uses the mechanism of GMDH to produce candidate fuzzy models of growing complexity until the TS model of optimal complexity has been created. The main characteristic of the new approach is its ability to identify the structure of TS model automatically. Experiments on Box-Jenkins gas furnace data and UCI datasets have shown that the proposed method can achieve satisfactory results and is more robust to noise in comparison with other TS modeling techniques such as ANFIS.     

基于智能控制的高速线材轧机水冷控制系统优化

针对武汉钢铁集团公司大型轧钢厂当前在高速线材生产线中存在的水冷控制系统可靠性差,轧线温度波动范围大等问题,应用智能计算理论及方法对上述工业控制系统进行系统辨识、建模以及优化.分析比较了基于梯度下降搜索BP算法、径向基函数网络、Levenberg Marquardt BP算法的前馈神经网络对SMS水冷系统的逼近精度、训练速度.研究了采用Levenberg-Marquardt BP算法的前馈神经网络在样本集和测试集上的表现,建立了基于Levenberg-Marquardt BP算法的前馈神经网络水冷控制系统模型.解决了高速线材水冷控制系统可靠性,温度控制精度问题.     

Application of honey-bee mating optimization algorithm on clustering

Cluster analysis is one of attractive data mining technique that use in many fields. One popular class of data clustering algorithms is the center based clustering algorithm. K-means used as a popular clustering method due to its simplicity and high speed in clustering large datasets. However, K-means has two shortcomings: dependency on the initial state and convergence to local optima and global solutions of large problems cannot found with reasonable amount of computation effort. In order to overcome local optima problem lots of studies done in clustering. Over the last decade, modeling the behavior of social insects, such as ants and bees, for the purpose of search and problem solving has been the context of the emerging area of swarm intelligence. Honey-bees are among the most closely studied social insects. Honey-bee mating may also be considered as a typical swarm-based approach to optimization, in which the search algorithm is inspired by the process of marriage in real honey-bee. Honey-bee has been used to model agent-based systems. In this paper, we proposed application of honeybee mating optimization in clustering (HBMK-means). We compared HBMK-means with other heuristics algorithm in clustering, such as GA, SA, TS, and ACO, by implementing them on several well-known datasets. Our finding shows that the proposed algorithm works than the best one.     

A hybrid tabu search/branch & bound approach to solving the generalized assignment problem

A new approach for solving the generalized assignment problem (GAP) is proposed that combines the exact branch & bound approach with the heuristic strategy of tabu search (TS) to produce a hybrid algorithm for solving GAP. The algorithm described uses commercial software to solve sub-problems generated by the TS guiding strategy. The TS approach makes use of the concept of referent domain optimisation and introduces novel add/drop strategies. In addition, the linear programming relaxation of GAP that forms part of the branch & bound approach is itself helpful in suggesting which variables might take binary values. Computational results on benchmark test instances are presented and compared with results obtained by the standard branch & bound approach and also several other heuristic approaches from the literature. The results show the new algorithm performs competitively against the alternatives and is able to find some new best solutions for several benchmark instances.     

A hybrid method combining continuous tabu search and Nelder–Mead simplex algorithms for the global optimization of multiminima functions

Tabu search (TS) is a metaheuristic, which proved efficient to solve various combinatorial optimization problems. However, few works deal with its application to the global minimization of functions depending on continuous variables. To perform this task, we propose an hybrid method combining tabu search and simplex search (SS). TS allows to cover widely the solution space, to stimulate the search towards solutions far from the current solution, and to avoid the risk of trapping into a local minimum. SS is used to accelerate the convergence towards a minimum. The Nelder–Mead simplex algorithm is a classical very powerful local descent algorithm, making no use of the objective function derivatives. A “simplex” is a geometrical figure consisting, in n-dimensions, of (n+1) points. If any point of a simplex is taken as the origin, the n other points define vector directions that span the n-dimension vector space. Through a sequence of elementary geometric transformations (reflection, contraction and extension), the initial simplex moves, expands or contracts. To select the appropriate transformation, the method only uses the values of the function to be optimized at the vertices of the simplex considered. After each transformation, the current worst vertex is replaced by a better one. Our algorithm called continuous tabu simplex search (CTSS) implemented in two different forms (CTSS_single, CTSS_multiple) is made up of two steps: first, an adaptation of TS to continuous optimization problems, allowing to localize a “promising area”; then, intensification within this promising area, involving SS. The efficiency of CTSS is extensively tested by using analytical test functions of which global and local minima are known. A comparison is proposed with several variants of tabu search, genetic algorithms and simulated annealing. CTSS is applied to the design of a eddy current sensor aimed at non-destructive control.     

Solving a dynamic cell formation problem using metaheuristics

In this paper, solving a cell formation (CF) problem in dynamic condition is going to be discussed by using some traditional metaheuristic methods such as genetic algorithm (GA), simulated annealing (SA) and tabu search (TS). Most of previous researches were done under the static condition. Due to the fact that CF is a NP-hard problem, then solving the model using classical optimization methods needs a long computational time. In this research, a nonlinear integer model of CF is first given and then solved by GA, SA and TS. Then, the results are compared with the optimal solution and the efficiency of the proposed algorithms is discussed.     

Tabu Search directed by direct search methods for nonlinear global optimization

In recent years, there has been a great deal of interest in metaheuristics in the optimization community. Tabu Search (TS) represents a popular class of metaheuristics. However, compared with other metaheuristics like genetic algorithm and simulated annealing, contributions of TS that deals with continuous problems are still very limited. In this paper, we introduce a continuous TS called Directed Tabu Search (DTS) method. In the DTS method, direct-search-based strategies are used to direct a tabu search. These strategies are based on the well-known Nelder–Mead method and a new pattern search procedure called adaptive pattern search. Moreover, we introduce a new tabu list conception with anti-cycling rules called Tabu Regions and Semi-Tabu Regions. In addition, Diversification and Intensification Search schemes are employed. Numerical results show that the proposed method is promising and produces high quality solutions.     

A robust parameter design for multi-response problems

Most real world search and optimization problems naturally involve multiple responses. In this paper we investigate a multiple response problem within desirability function framework and try to determine values of input variables that achieve a target value for each response through three meta-heuristic algorithms such as genetic algorithm (GA), simulated annealing (SA) and tabu search (TS). Each algorithm has some parameters that need to be accurately calibrated to ensure the best performance. For this purpose, a robust calibration is applied to the parameters by means of Taguchi method. The computational results of these three algorithms are compared against each others. The superior performance of SA over TS and TS over GA is inferred from the obtained results in various situations.     

A particle swarm optimization-based hybrid algorithm for minimum concave cost network flow problems

Traditionally, minimum cost transshipment problems have been simplified as linear cost problems, which are not practical in real applications. Some advanced local search algorithms have been developed to solve concave cost bipartite network problems. These have been found to be more effective than the traditional linear approximation methods and local search methods. Recently, a genetic algorithm and an ant colony system algorithm were employed to develop two global search algorithms for solving concave cost transshipment problems. These two global search algorithms were found to be more effective than the advanced local search algorithms for solving concave cost transshipment problems. Although the particle swarm optimization algorithm has been used to obtain good results in many applications, to the best of our knowledge, it has not yet been applied in minimum concave cost network flow problems. Thus, in this study, we employ an arc-based particle swarm optimization algorithm, coupled with some genetic algorithm and threshold accepting method techniques, as well as concave cost network heuristics, to develop a hybrid global search algorithm for efficiently solving minimum cost network flow problems with concave arc costs. The proposed algorithm is evaluated by solving several randomly generated network flow problems. The results indicate that the proposed algorithm is more effective than several other recently designed methods, such as local search algorithms, genetic algorithms and ant colony system algorithms, for solving minimum cost network flow problems with concave arc costs.     

Selecting survivors in genetic algorithm using tabu search strategies

Genetic algorithm (GA) is well-known for its effectiveness in global search and optimization. To balance selection pressure and population diversity is an important issue of designing GA. This paper proposes a novel hybridization of GA and tabu search (TS) to address this issue. The proposed method embeds the key elements of TS—tabu restriction and aspiration criterion—into the survival selection operator of GA. More specifically, the tabu restriction is used to prevent inbreeding for diversity maintenance, and the aspiration criterion is activated to provide moderate selection pressure under the tabu restriction. The interaction of tabu restriction and aspiration criterion enables survivor selection to balance selection pressure and population diversity. The experimental results on numerical and combinatorial optimization problems show that this hybridization can significantly improve GAs in terms of solution quality as well as convergence speed. An empirical analysis further identifies the influences of the TS strategies on the performance of this hybrid GA.     

Improving the Efficiency of Tabu Search for Machine Sequencing Problems

Tabu search (TS) is becoming increasingly recognized as an efficient way of finding high-quality solutions to hard combinatorial problems. It may be described as an intelligent meta-heuristic for controlling simpler local search procedures. However, reported applications have often used a ‘brute-force’ approach without considering the most effective use of the computing effort available. This paper intends firstly to give a basic introduction to the ideas of TS, and then it will report some computational experiments on TS in the context of machine sequencing. These have shown that it is important to define the balance between exploration of the solution space and exploitation of the information obtained. The results will be compared with those obtained from a proven Simulated Annealing (SA) algorithm, which tends to confirm the general opinion that when implemented efficiently, TS is a more effective search paradigm than SA.     

Discrete neural dynamic programming in wheeled mobile robot control

In this paper we propose a discrete algorithm for a tracking control of a two-wheeled mobile robot (WMR), using an advanced Adaptive Critic Design (ACD). We used Dual-Heuristic Programming (DHP) algorithm, that consists of two parametric structures implemented as Neural Networks (NNs): an actor and a critic, both realized in a form of Random Vector Functional Link (RVFL) NNs. In the proposed algorithm the control system consists of the DHP adaptive critic, a PD controller and a supervisory term, derived from the Lyapunov stability theorem. The supervisory term guaranties a stable realization of a tracking movement in a learning phase of the adaptive critic structure and robustness in face of disturbances. The discrete tracking control algorithm works online, uses the WMR model for a state prediction and does not require a preliminary learning. Verification has been conducted to illustrate the performance of the proposed control algorithm, by a series of experiments on the WMR Pioneer 2-DX.     

预制构件流水车间订单接受与调度的集成优化

针对预制构件生产管理过程中订单工期紧和生产能力不足的问题,在充分考虑中断和不可中断工序,串行和并行工序等复杂工况特点的基础上,以最大化净利润为目标,建立了一种订单接受与调度集成优化模型。鉴于问题的NP难性和模型的高度非线性,通过集成问题性质、构造启发式、邻域搜索和破坏-构造机制,提出了一种混合加速迭代贪婪搜索框架。其中,在调度构造阶段,为提高算法求解质量和搜索效率,设计了两种融合订单插入操作性质的加速构造策略。计算结果显示,与混合遗传禁忌搜索算法,遗传算法以及禁忌搜索算法相比,本文所提算法具有更好的求解质量和搜索效率。同时验证了所提出的加速构造策略能够有效减少算法运行时间。该研究有望显著提高预制生产企业净利润和客户满意度。