Strategies of node selection in search procedures for solving combinatorial optimization problems: A survey and a general formalization

Currently, most combinatorial optimization problems have to be solved, if the optimum solution is sought, using general techniques to explore the space of feasible solutions and, more specifically, through exploratory enumerative procedures in trees and search graphs. The objective of this work is to propose a survey and a general formalization of the selection strategy of the next node to explore, a feature that is common to all these optimization procedures. This research has been partially supported by TAP98-0494 project     

Heuristics for the flow line problem with setup costs

This paper presents two new heuristics for the flowshop scheduling problem with sequence-dependent setup times (SDSTs) and makespan minimization objective. The first is an extension of a procedure that has been very successful for the general flowshop scheduling problem. The other is a greedy randomized adaptive search procedure (GRASP) which is a technique that has achieved good results on a variety of combinatorial optimization problems. Both heuristics are compared to a previously proposed algorithm based on the traveling salesman problem (TSP). In addition, local search procedures are developed and adapted to each of the heuristics. A two-phase lower bounding scheme is presented as well. The first phase finds a lower bound based on the assignment relaxation for the asymmetric TSP. In phase two, attempts are made to improve the bound by inserting idle time. All procedures are compared for two different classes of randomly generated instances. In the first case where setup times are an order of magnitude smaller than the processing times, the new approaches prove superior to the TSP-based heuristic; for the case where both processing and setup times are identically distributed, the TSP-based heuristic outperforms the proposed procedures.     

Improving the Efficiency of Tabu Search for Machine Sequencing Problems

Tabu search (TS) is becoming increasingly recognized as an efficient way of finding high-quality solutions to hard combinatorial problems. It may be described as an intelligent meta-heuristic for controlling simpler local search procedures. However, reported applications have often used a ‘brute-force’ approach without considering the most effective use of the computing effort available. This paper intends firstly to give a basic introduction to the ideas of TS, and then it will report some computational experiments on TS in the context of machine sequencing. These have shown that it is important to define the balance between exploration of the solution space and exploitation of the information obtained. The results will be compared with those obtained from a proven Simulated Annealing (SA) algorithm, which tends to confirm the general opinion that when implemented efficiently, TS is a more effective search paradigm than SA.     

Unbiased Recursive Partitioning: A Conditional Inference Framework

Recursive binary partitioning is a popular tool for regression analysis. Two fundamental problems of exhaustive search procedures usually applied to fit such models have been known for a long time: overfitting and a selection bias towards covariates with many possible splits or missing values. While pruning procedures are able to solve the overfitting problem, the variable selection bias still seriously affects the interpretability of tree-structured regression models. For some special cases unbiased procedures have been suggested, however lacking a common theoretical foundation. We propose a unified framework for recursive partitioning which embeds tree-structured regression models into a well defined theory of conditional inference procedures. Stopping criteria based on multiple test procedures are implemented and it is shown that the predictive performance of the resulting trees is as good as the performance of established exhaustive search procedures. It turns out that the partitions and therefore the models induced by both approaches are structurally different, confirming the need for an unbiased variable selection. Moreover, it is shown that the prediction accuracy of trees with early stopping is equivalent to the prediction accuracy of pruned trees with unbiased variable selection. The methodology presented here is applicable to all kinds of regression problems, including nominal, ordinal, numeric, censored as well as multivariate response variables and arbitrary measurement scales of the covariates. Data from studies on glaucoma classification, node positive breast cancer survival and mammography experience are re-analyzed.     

Sequential-analysis Based Randomized-regret-methods for Lot-sizing and Scheduling

Lot-sizing and scheduling comprises activities that have to be done repeatedly within MRP-systems. We consider the proportional multi-item, capacitated, dynamic lot-sizing and scheduling problem that is more general than the discrete lot-sizing and scheduling problem, as well as the continuous set-up lot-sizing problem. A greedy randomized algorithm with regret-based biased sampling is presented. We partition the parameter space of the stochastic algorithm and choose subspaces via sequential analysis based on hypothesis testing. The new methods provided in this paper, i.e. the randomized-regret-based backward algorithm, as well as the controlled search via sequential analysis, have three important properties: they are simple, effective and rather general. Computational results are also presented.     

On the convergence of global methods in multiextremal optimization

A general class of derivative-free optimization procedures is presented including the corresponding convergence theory. This theory turns out to be very constructive, in the sense that the convergence conditions not only can be verified easily for many existing algorithms, but also allow one to construct new procedures. It is shown that popular methods such as branch-and-bound concepts, Pintér's general class of procedures, the algorithms of Pijavskii, Shubert, and Mladineo, and the approach of Zheng and Galperin can not only be subsumed under this class of methods, but also partly be improved by regarding them within the framework presented.     

Metaheuristics: A bibliography

Metaheuristics are the most exciting development in approximate optimization techniques of the last two decades. They have had widespread successes in attacking a variety of difficult combinatorial optimization problems that arise in many practical areas. This bibliography provides a classification of a comprehensive list of 1380 references on the theory and application of metaheuristics. Metaheuristics include but are not limited to constraint logic programming; greedy random adaptive search procedures; natural evolutionary computation; neural networks; non-monotonic search strategies; space-search methods; simulated annealing; tabu search; threshold algorithms and their hybrids. References are presented in alphabetical order under a number of subheadings.     

Stopping rules in <Emphasis Type="Italic">k</Emphasis>-adaptive global random search algorithms

In this paper we develop a methodology for defining stopping rules in a general class of global random search algorithms that are based on the use of statistical procedures. To build these stopping rules we reach a compromise between the expected increase in precision of the statistical procedures and the expected waiting time for this increase in precision to occur.     

Experimental studies of variable selection strategies based on constraint weights

Variable ordering heuristics that sample information before or during search in order to inform subsequent decisions have shown better performance and greater robustness than standard heuristics. One such strategy, the “weighted degree heuristic,” is based on weighting constraints according to their involvement in failure during search. A more recent approach uses “random probing” with restarting to gain information less subject to sampling bias. To date, these approaches have not been carefully analysed experimentally. In the present work, several important findings are presented, including a better delineation of the class of events that is sampled, an analysis of the importance of informed choices at the beginning of search, and a demonstration that random probing identifies sources of global contention effectively even when these are not clearly demarcated. These experiments show how empirical analysis can clarify subtle issues in the analysis of heuristic procedures for difficult search problems.     

A survey of solution techniques for the partially observed Markov decision process

We survey several computational procedures for the partially observed Markov decision process (POMDP) that have been developed since the Monahan survey was published in 1982. The POMDP generalizes the standard, completely observed Markov decision process by permitting the possibility that state observations may be noise-corrupted and/or costly. Several computational procedures presented are convergence accelerating variants of, or approximations to, the Smallwood-Sondik algorithm. Finite-memory suboptimal design results are reported, and new research directions involving heuristic search are discussed.This research was supported by NSF Grant ECS-8708183 and ARO Contract DAAG-29-85-K0089.     

On the Use of Exact and Heuristic Cutting Plane Methods for the Quadratic Assignment Problem

This paper uses the formulation of the quadratic assignment problem as that of minimizing a concave quadratic function over the assignment polytope. Cutting plane procedures are investigated for solving this problem. A lower bound derived on the number of cuts needed for termination indicates that conventional cutting plane procedures would require a huge computational effort for the exact solution of the quadratic assignment problems. However, several heuristics which are derived from the cutting planes produce optimal or good quality solutions early on in the search process. An illustrative example and computational results are presented.     

Tabu search for graph partitioning

In this paper, we develop a tabu search procedure for solving the uniform graph partitioning problem. Tabu search, an abstract heuristic search method, has been shown to have promise in solving several NP-hard problems, such as job shop and flow shop scheduling, vehicle routing, quadratic assignment, and maximum satisfiability. We compare tabu search to other heuristic procedures for graph partitioning, and demonstrate that tabu search is superior to other solution approaches for the uniform graph partitioning problem both with respect to solution quality and computational requirements.     

Proofs and Countermodels in Non-Classical Logics

Proofs and countermodels are the two sides of completeness proofs, but, in general, failure to find one does not automatically give the other. The limitation is encountered also for decidable non-classical logics in traditional completeness proofs based on Henkin’s method of maximal consistent sets of formulas. A method is presented that makes it possible to establish completeness in a direct way: For any given sequent either a proof in the given logical system or a countermodel in the corresponding frame class is found. The method is a synthesis of a generation of calculi with internalized relational semantics, a Tait–Schütte–Takeuti style completeness proof, and procedures to finitize the countermodel construction. Finitizations for intuitionistic propositional logic are obtained through the search for a minimal derivation, through pruning of infinite branches in search trees by means of a suitable syntactic counterpart of semantic filtration, or through a proof-theoretic embedding into an appropriate provability logic. A number of examples illustrates the method, its subtleties, challenges, and present scope.     

A general model for cyclic machine scheduling problems

A general framework for modeling and solving cyclic scheduling problems is presented. The objective is to minimize the cycle time. The model covers different cyclic versions of the job-shop problem found in the literature, robotic cell problems, the single hoist scheduling problem and tool transportation between the machines.It is shown that all these problems can be formulated as mixed integer linear programs which have a common structure. Small instances are solved with CPLEX. For larger instances tabu search procedures have been developed. The main ideas of these methods are indicated.     

Tabu search with strategic oscillation for the maximally diverse grouping problem

We propose new heuristic procedures for the maximally diverse grouping problem (MDGP). This NP-hard problem consists of forming maximally diverse groups—of equal or different size—from a given set of elements. The most general formulation, which we address, allows for the size of each group to fall within specified limits. The MDGP has applications in academics, such as creating diverse teams of students, or in training settings where it may be desired to create groups that are as diverse as possible. Search mechanisms, based on the tabu search methodology, are developed for the MDGP, including a strategic oscillation that enables search paths to cross a feasibility boundary. We evaluate construction and improvement mechanisms to configure a solution procedure that is then compared to state-of-the-art solvers for the MDGP. Extensive computational experiments with medium and large instances show the advantages of a solution method that includes strategic oscillation.     

A multiple-tree search procedure for the resource-constrained project scheduling problem

For over three decades, researchers have sought effective solution procedures for PERT/CPM types of scheduling problems under conditions of limited resource availability. This paper presents a procedure for this problem which takes advantage of the emerging technology provided by multiple parallel processors to find and verify an optimal schedule for a project under conditions of multiple resource constraints. In our approach, multiple solutions trees are searched simultaneously in the quest for a minimum duration schedule. Global upper and lower bound information in common memory is shared among processors, enabling one or several processors to prune potentially significant portions of its search tree based upon bounds discovered by a processor using a different search tree. Computational experience is reported both for problems in which resources are available in constant amounts per period, as well as the much more difficult problem in which the resources available are allowed to vary over the schedule horizon (e.g., travel, sick leave, assignment to other tasks or projects, and so forth). The modular multiple-tree search procedure described in this paper is quite general, permitting most types of existing serial search strategies to be adapted to this approach where multiple processors are available.     

Cyclic job shop scheduling problems with blocking

A tabu search algorithm for a cyclic job shop problem with blocking is presented. Operations are blocking if they must stay on a machine after finishing when the next machine is occupied by another job. During this stay the machine is blocked for other jobs. For this problem traditional tabu search moves often lead to infeasible solutions. Recovering procedures are developed which construct nearby feasible solutions. Computational results are presented for the approach.     

Beam search heuristics for quadratic earliness and tardiness scheduling

In this paper, we present beam search heuristics for the single machine scheduling problem with quadratic earliness and tardiness costs, and no machine idle time. These heuristics include classic beam search procedures, as well as filtered and recovering algorithms. We consider three dispatching heuristics as evaluation functions, in order to analyse the effect of different rules on the performance of the beam search procedures. The computational results show that using better dispatching heuristics improves the effectiveness of the beam search algorithms. The performance of the several heuristics is similar for instances with low variability. For high variability instances, however, the detailed, filtered and recovering beam search (RBS) procedures clearly outperform the best existing heuristic. The detailed beam search algorithm performs quite well, and is recommended for small- to medium-sized instances. For larger instances, however, this procedure requires excessive computation times, and the RBS algorithm then becomes the heuristic of choice.     

An Ant Colony Optimization Algorithm for Shop Scheduling Problems

We deal with the application of ant colony optimization to group shop scheduling, which is a general shop scheduling problem that includes, among others, the open shop scheduling problem and the job shop scheduling problem as special cases. The contributions of this paper are twofold. First, we propose a neighborhood structure for this problem by extending the well-known neighborhood structure derived by Nowicki and Smutnicki for the job shop scheduling problem. Then, we develop an ant colony optimization approach, which uses a strong non-delay guidance for constructing solutions and which employs black-box local search procedures to improve the constructed solutions. We compare this algorithm to an adaptation of the tabu search by Nowicki and Smutnicki to group shop scheduling. Despite its general nature, our algorithm works particularly well when applied to open shop scheduling instances, where it improves the best known solutions for 15 of the 28 tested instances. Moreover, our algorithm is the first competitive ant colony optimization approach for job shop scheduling instances.     

A fast swap-based local search procedure for location problems

We present a new implementation of a widely used swap-based local search procedure for the p-median problem, proposed in 1968 by Teitz and Bart. Our method produces the same output as the best alternatives described in the literature and, even though its worst-case complexity is similar, it can be significantly faster in practice: speedups of up to three orders of magnitude were observed. We also show that our method can be easily adapted to handle the facility location problem and to implement related procedures, such as path-relinking and tabu search. R. F. Werneck: The results presented in this paper were obtained while this author was a summer intern at AT&T Labs Research.