Hyper-heuristics: a survey of the state of the art

Hyper-heuristics comprise a set of approaches that are motivated (at least in part) by the goal of automating the design of heuristic methods to solve hard computational search problems. An underlying strategic research challenge is to develop more generally applicable search methodologies. The term hyper-heuristic is relatively new; it was first used in 2000 to describe heuristics to choose heuristics in the context of combinatorial optimisation. However, the idea of automating the design of heuristics is not new; it can be traced back to the 1960s. The definition of hyper-heuristics has been recently extended to refer to a search method or learning mechanism for selecting or generating heuristics to solve computational search problems. Two main hyper-heuristic categories can be considered: heuristic selection and heuristic generation. The distinguishing feature of hyper-heuristics is that they operate on a search space of heuristics (or heuristic components) rather than directly on the search space of solutions to the underlying problem that is being addressed. This paper presents a critical discussion of the scientific literature on hyper-heuristics including their origin and intellectual roots, a detailed account of the main types of approaches, and an overview of some related areas. Current research trends and directions for future research are also discussed.     

A joint production and transportation planning problem with heterogeneous vehicles

We consider a manufacturer's planning problem to schedule order production and transportation to respective destinations. The manufacturer in this setting can use two vehicle types for outbound shipments. The first type is available in unlimited numbers. The availability of the second type, which is less expensive, changes over time. Motivated by some industry practices, we present formulations for three different solution approaches: the myopic solution, the hierarchical solution and the coordinated solution. These approaches vary in how the underlying production and transportation subproblems are solved, that is, sequentially versus jointly or heuristically versus optimally. We provide intractability proofs or polynomial-time exact solution procedures for the sub-problems and their special cases. We also compare the three solution approaches over a numerical study to quantify the savings from integration and explicit consideration of transportation availabilities. Our analytical and numerical results set a foundation and a need for a heuristic to solve the integrated problem. We thus propose a tabu search heuristic, which quickly generates near-optimal solutions.     

Design of manufacturing cells in the presence of alternative cell locations and material transporters

This paper addresses two significant issues in the design of cellular manufacturing (CM) systems: (i) the availability of alternative locations for a cell, and (ii) the use of alternative routes to move part loads between cells when the capacity of the material transporter (MT) employed is limited. In addition, several other important factors in the design of CM systems including machine capacity limitations, batches of part demands, non-consecutive operations of parts, and maximum number of machines assigned to a cell are considered. A nonlinear programming model, comprised of binary and general integer variables, is formulated for the research problem. A higher-level heuristic solution algorithm based upon a concept known as ‘tabu search’ is presented for solving industry-size problems. Six different versions of the heuristic are developed to investigate the impact of long-term memory and the use of fixed versus variable tabu-list sizes. Explicit method-based techniques are developed to convert the original nonlinear programming model into an equivalent mixed (binary)-integer linear programming model in order to test the efficacy of the proposed solution technique for solving small problem instances. The solutions obtained from the heuristics have average deviation of less than 3% of the optimal solutions, and require less than a minute in comparison with optimizing methods that needed 1–10 h of computation time. A carefully designed statistical experiment is used to compare the performance of the heuristics by solving three different problem structures, ranging from four to 30 parts, and three to nine locations. The experiment shows that as the problem size increases, the tabu-search-based heuristic using fixed tabu list size and long-term memory based on minimal frequency strategy is preferred over the other heuristics.     

Minimizing the Flow-time Variance in Single-machine Systems

In this paper we deal with an n-job, single-machine scheduling problem. All jobs are available from the start, and the objective is to minimize the variance of job flow-times. A heuristic procedure which is based on the complementary pair-exchange principle is proposed. It has been concluded that this heuristic procedure provides improved results (in terms of objective-function value) when compared with other heuristics. Our heuristic procedure has the complexity of O(n log n).     

No-wait flowshops with bicriteria of makespan and total completion time

We consider the m-machine no-wait flowshop scheduling problem with the objective of minimizing a weighted sum of makespan and total completion time. For the two-machine problem, we develop a dominance relation and embed it within a proposed branch-and-bound algorithm. For the m-machine problem, we propose a heuristic. Computational experiments show that the proposed heuristic outperforms the best existing multi-criteria heuristics and the best single criterion heuristics for makespan and total completion time. The efficiency of the dominance relation and branch-and-bound algorithm is also investigated and shown to be effective.     

New heuristics for flow shop problem to minimize makespan

This paper investigates the flow shop problem with the objective to minimizemakespan. New algorithms are designed: one is off-line heuristic, Single JobFirst, for problemF _m ∥C _max ;and the other is on-line heuristic, Dynamic Single Job First (DSJF), for problemF _m |r _i |C _max and its on-line version. It is proved that the two heuristics are asymptoticallyoptimal when the size of the problem is large enough. In addition, theasymptotical optimality of First-Come, First-Served manner is obtained as abyproduct of the asymptotical analysis of DSJF heuristic. At the end of thepaper, a new lower bound with performance guarantee is provided for problemF _m ∥C _max , andcomputational experiments show the effectiveness of these heuristicalgorithms.     

A basic variable neighborhood search heuristic for the uncapacitated multiple allocation <Emphasis Type="Italic">p</Emphasis>-hub center problem

The uncapacitated multiple allocation p-hub center problem (UMApHCP) consists of choosing p hub locations from a set of nodes with pairwise traffic demands in order to route the traffic between the origin-destination pairs such that the maximum cost between origin-destination pairs is minimum. It is assumed that transportation between non-hub nodes is possible only via chosen hub nodes. In this paper we propose a basic variable neighborhood search (VNS) heuristic for solving this NP hard problem. In addition we apply two mathematical formulations of the UMApHCP in order to detect limitations of the current state-of-the-art solver used for this problem. The heuristics are tested on benchmark instances for p-hub problems. The obtained results reveal the superiority of the proposed basic VNS over the state-of-the-art as well as over a multi-start local search heuristic developed by us in this paper.     

Two new heuristics for the <Emphasis Type="Italic">GI</Emphasis>/<Emphasis Type="Italic">G</Emphasis>/<Emphasis Type="Italic">n</Emphasis>/0 queueing loss system with examples based on the two-phase Coxian distribution

In this paper, we introduce a new heuristic approach for the numerical analysis of queueing systems. In particular, we study the general, multi-server queueing loss system, the GI/G/n/0 queue, with an emphasis on the calculation of steady-state loss probabilities. Two new heuristics are developed, called the GM Heuristic and the MG Heuristic, both of which make use of an exact analysis of the corresponding single-server GI/G/1/0 queue. The GM Heuristic also uses an exact analysis of the GI/M/n/0 queue, while the MG Heuristic uses an exact analysis of the M/G/n/0 queue. Experimental results are based on the use of two-phase Coxian distributions for both the inter-arrival time and the service time; these include an error analysis for each heuristic and the derivation of experimental probability bounds for the loss probability. For the class of problems studied, it is concluded that there are likely to be many situations where the accuracy of the GM Heuristic is adequate for practical purposes. Methods are also developed for combining the GM and MG Heuristics. In some cases, this leads to approximations that are significantly more accurate than those obtained by the individual heuristics.     

A new heuristic approach for the P-median problem

The Euclidean p-median problem is concerned with the decision of the locations for public service centres. Existing methods for the planar Euclidean p-median problems are capable of efficiently solving problems of relatively small scale. This paper proposes two new heuristic algorithms aiming at problems of large scale. Firstly, to reflect the different degrees of proximity to optimality, a new kind of local optimum called level-m optimum is defined. For a level-m optimum of a p-median problem, where m<p, each of its subsets containing m of the p partitions is a global optimum of the corresponding m-median subproblem. Starting from a conventional local optimum, the first new algorithm efficiently improves it to a level-2 optimum by applying an existing exact algorithm for solving the 2-median problem. The second new algorithm further improves it to a level-3 optimum by applying a new exact algorithm for solving the 3-median problem. Comparison based on experimental results confirms that the proposed algorithms are superior to the existing heuristics, especially in terms of solution quality.     

Minimizing total completion time in a no-wait flowshop with sequence-dependent additive changeover times

This paper addresses the problem of minimizing total completion time in a two-machine no-wait flowshop where setup times of the jobs are sequence-dependent. Optimal solutions are obtained for two special flowshops and a dominance relation is developed for the general problem. Several heuristic algorithms with the computational complexity of O(n²) and O(n³) are constructed. The heuristics consist of two phases: in the first phase a starting list is developed and in the second a repeated insertion technique is applied. Computational experience demonstrates that the concept of repeated insertion application is quite useful for any starting list and that solutions for all starting lists converge to about the same value of less than 1% after a few iterations.     

A simple and effective recursive procedure for the manufacturer's pallet loading problem

In this paper we present a simple and effective heuristic to solve the problem of packing the maximum number of rectangles of sizes (l,w) and (w,l) into a larger rectangle (L,W) without overlapping. This problem appears in the loading of identical boxes on pallets, namely the manufacturer's pallet loading (MPL), as well as in package design and truck or rail car loading. Although apparently easy to be optimally solved, the MPL is claimed to be NP-complete and several authors have proposed approximate methods to deal with it. The procedure described in the present paper can be seen as a refinement of Bischoff and Dowsland's heuristic and can easily be implemented on a microcomputer. Using moderate computational resources, the procedure was able to find the optimal solution of 99.9% of more than 20?000 examples analysed.     

Reducing work-in-process movement for multiple products in one-dimensional layout problems

This research describes a method to assign M machines, which are served by a material handling transporter, to M equidistant locations along a track, so that the distance traveled by a given set of jobs is minimized. Traditionally, this problem (commonly known as a machine location problem) has been modeled as a quadratic assignment problem (QAP), which is NP-hard, thus motivating the need for efficient procedures to solve instances with several machines. In this paper we develop a branching heuristic to obtain sub-optimum solutions to the problem; a lower bound on the optimum solution has also been presented. Results obtained from the heuristics are compared with results obtained from other heuristics with similar objectives. It is observed that the results are promising, and justify the usage of developed methods.     

All Colors Shortest Path problem on trees

Given an edge weighted tree T(V, E), rooted at a designated base vertex \(r \in V\), and a color from a set of colors \(C=\{1,\ldots ,k\}\) assigned to every vertex \(v \in V\), All Colors Shortest Path problem on trees (ACSP-t) seeks the shortest, possibly non-simple, path starting from r in T such that at least one node from every distinct color in C is visited. We show that ACSP-t is NP-hard, and also prove that it does not have a constant factor approximation. We give an integer linear programming formulation of ACSP-t. Based on a linear programming relaxation of this formulation, an iterative rounding heuristic is proposed. The paper also explores genetic algorithm and tabu search to develop alternative heuristic solutions for ACSP-t. The performance of all the proposed heuristics are evaluated experimentally for a wide range of trees that are generated parametrically.     

Solving the clique partitioning problem as a maximally diverse grouping problem

In this paper we show that the clique partitioning problem can be reformulated in an equivalent form as the maximally diverse grouping problem (MDGP). We then modify a skewed general variable neighborhood search (SGVNS) heuristic that was first developed to solve the MDGP. Similarly as with the MDGP, significant improvements over the state of the art are obtained when SGVNS is tested on large scale instances. This further confirms the usefulness of a combined approach of diversification afforded with skewed VNS and intensification afforded with the local search in general VNS.     

Fast heuristics for protection networks for dynamic routing

Traffic volatility and network reliability are important issues in the provision of high speed network services. We consider the construction of a second network, the protection network which can carry overload traffic due to the failure or congestion of any two links in the original network. The level of protection against such contingencies can be specified by a traffic requirement matrix. We construct a fully connected protection network, for an n node network, using an O(n²) heuristic based on the largest two traffic requirements for each node. This procedure is then modified to generate a more effective O(n⁴) heuristic, both methods facilitate fast processing for two-hop dynamic routing. We compare the performance of the heuristics with the O(n¹⁵) optimal solution.     

A vehicle routing problem with multi-trips and time windows for circular items

This article presents a vehicle routing problem with time windows, multiple trips, a limited number of vehicles and loading constraints for circular objects. This is a real problem experienced by a home delivery service company. A linear model is proposed to handle small problems and a two-step heuristic method to solve real size instances: the first step builds an initial solution through the modification of the Solomon I1 sequential insertion heuristic, and the second step improves the initial solution through the Tabu search algorithm proposed; in both steps, the problems related to circle packing with different sizes and bin packing are solved jointly with the use of heuristics. Finally, the computing results for two different sets of instances are presented.     

Hybridizations within a graph-based hyper-heuristic framework for university timetabling problems

A significant body of recent literature has explored various research directions in hyper-heuristics (which can be thought as heuristics to choose heuristics). In this paper, we extend our previous work to construct a unified graph-based hyper-heuristic (GHH) framework, under which a number of local search-based algorithms (as the high level heuristics) are studied to search upon sequences of low-level graph colouring heuristics. To gain an in-depth understanding on this new framework, we address some fundamental issues concerning neighbourhood structures and characteristics of the two search spaces (namely, the search spaces of the heuristics and the actual solutions). Furthermore, we investigate efficient hybridizations in GHH with local search methods and address issues concerning the exploration of the high-level search and the exploitation ability of the local search. These, to our knowledge, represent entirely novel directions in hyper-heuristics. The efficient hybrid GHH obtained competitive results compared with the best published results for both benchmark course and exam timetabling problems, demonstrating its efficiency and generality across different problem domains. Possible extensions upon this simple, yet general, GHH framework are also discussed.     

Heuristics for exact nonnegative matrix factorization

The exact nonnegative matrix factorization (exact NMF) problem is the following: given an m-by-n nonnegative matrix X and a factorization rank r, find, if possible, an m-by-r nonnegative matrix W and an r-by-n nonnegative matrix H such that \(X = WH\). In this paper, we propose two heuristics for exact NMF, one inspired from simulated annealing and the other from the greedy randomized adaptive search procedure. We show empirically that these two heuristics are able to compute exact nonnegative factorizations for several classes of nonnegative matrices (namely, linear Euclidean distance matrices, slack matrices, unique-disjointness matrices, and randomly generated matrices) and as such demonstrate their superiority over standard multi-start strategies. We also consider a hybridization between these two heuristics that allows us to combine the advantages of both methods. Finally, we discuss the use of these heuristics to gain insight on the behavior of the nonnegative rank, i.e., the minimum factorization rank such that an exact NMF exists. In particular, we disprove a conjecture on the nonnegative rank of a Kronecker product, propose a new upper bound on the extension complexity of generic n-gons and conjecture the exact value of (i) the extension complexity of regular n-gons and (ii) the nonnegative rank of a submatrix of the slack matrix of the correlation polytope.     

A graph-theoretic approach to interval scheduling on dedicated unrelated parallel machines

We study the problem of scheduling n non-preemptive jobs on m unrelated parallel machines. Each machine can process a specified subset of the jobs. If a job is assigned to a machine, then it occupies a specified time interval on the machine. Each assignment of a job to a machine yields a value. The objective is to find a subset of the jobs and their feasible assignments to the machines such that the total value is maximized. The problem is NP-hard in the strong sense. We reduce the problem to finding a maximum weight clique in a graph and survey available solution methods. Furthermore, based on the peculiar properties of graphs, we propose an exact solution algorithm and five heuristics. We conduct computer experiments to assess the performance of our and other existing heuristics. The computational results show that our heuristics outperform the existing heuristics.     

Irregular polyomino tiling via integer programming with application in phased array antenna design

A polyomino is a generalization of the domino and is created by connecting a fixed number of unit squares along edges. Tiling a region with a given set of polyominoes is a hard combinatorial optimization problem. This paper is motivated by a recent application of irregular polyomino tilings in the design of phased array antennas. Specifically, we formulate the irregular polyomino tiling problem as a nonlinear exact set covering model, where irregularity is incorporated into the objective function using the information-theoretic entropy concept. An exact solution method based on a branch-and-price framework along with novel branching and lower-bounding schemes is proposed. The developed method is shown to be effective for small- and medium-size instances of the problem. For large-size instances, efficient heuristics and approximation algorithms are provided. Encouraging computational results including phased array antenna simulations are reported.