Total completion time with makespan constraint in no-wait flowshops with setup times

The m-machine no-wait flowshop scheduling problem with the objective of minimizing total completion time subject to the constraint that the makespan value is not greater than a certain value is addressed in this paper. Setup times are considered non-zero values, and thus, setup times are treated as separate from processing times. Several recent algorithms, an insertion algorithm, two genetic algorithms, three simulated annealing algorithms, two cloud theory-based simulated annealing algorithms, and a differential evolution algorithm are adapted and proposed for the problem. An extensive computational analysis has been conducted for the evaluation of the proposed algorithms. The computational analysis indicates that one of the nine proposed algorithms, one of the simulated annealing algorithms (ISA-2), performs much better than the others under the same computational time. Moreover, the analysis indicates that the algorithm ISA-2 performs significantly better than the earlier existing best algorithm. Specifically, the best performing algorithm, ISA-2, proposed in this paper reduces the error of the existing best algorithm in the literature by at least 90% under the same computational time. All the results have been statistically tested.     

Heuristic Search Algorithms for the Minimum Volume Ellipsoid

Abstract

A method of robust estimation of multivariate location and shape that has attracted a lot of attention recently is Rousseeuw's minimum volume ellipsoid estimator (MVE). This estimator has a high breakdown point but is difficult to compute successfully. In this article, we apply methods of heuristic search to this problem, including simulated annealing, genetic algorithms, and tabu search, and compare the results to the undirected random search algorithm that is often cited. Heuristic search provides several effective algorithms that are far more computationally efficient than random search. Furthermore, random search, as currently implemented, is shown to be ineffective for larger problems.     

A Hybrid Genetic Algorithm with Boltzmann Convergence Properties

Stochastic global search algorithms such as genetic algorithms are used to attack difficult combinatorial optimization problems. However, genetic algorithms suffer from the lack of a convergence proof. This means that it is difficult to establish reliable algorithm braking criteria without extensive a priori knowledge of the solution space. The hybrid genetic algorithm presented here combines a genetic algorithm with simulated annealing in order to overcome the algorithm convergence problem. The genetic algorithm runs inside the simulated annealing algorithm and provides convergence via a Boltzmann cooling process. The hybrid algorithm was used successfully to solve a classical 30-city traveling salesman problem; it consistently outperformed both a conventional genetic algorithm and a conventional simulated annealing algorithm. This work was supported by the University of Colorado at Colorado Springs.     

Generalizing Swendsen–Wang for Image Analysis

Markov chain Monte Carlo (MCMC) methods have been used in many fields (physics, chemistry, biology, and computer science) for simulation, inference, and optimization. In many applications, Markov chains are simulated for sampling from target probabilities π(X) defined on graphs G. The graph vertices represent elements of the system, the edges represent spatial relationships, while X is a vector of variables on the vertices which often take discrete values called labels or colors. Designing efficient Markov chains is a challenging task when the variables are strongly coupled. Because of this, methods such as the single-site Gibbs sampler often experience suboptimal performance. A well-celebrated algorithm, the Swendsen–Wang (SW) method, can address the coupling problem. It clusters the vertices as connected components after turning off some edges probabilistically, and changes the color of one cluster as a whole. It is known to mix rapidly under certain conditions. Unfortunately, the SW method has limited applicability and slows down in the presence of “external fields;” for example, likelihoods in Bayesian inference. In this article, we present a general cluster algorithm that extends the SW algorithm to general Bayesian inference on graphs. We focus on image analysis problems where the graph sizes are in the order of 10³–10⁶ with small connectivity. The edge probabilities for clustering are computed using discriminative probabilities from data. We design versions of the algorithm to work on multi grid and multilevel graphs, and present applications to two typical problems in image analysis, namely image segmentation and motion analysis. In our experiments, the algorithm is at least two orders of magnitude faster (in CPU time) than the single-site Gibbs sampler.     

Investigation of temperature parallel simulated annealing for optimizing continuous functions with application to hyperspectral tomography

The simulated annealing (SA) algorithm is a well-established optimization technique which has found applications in many research areas. However, the SA algorithm is limited in its application due to the high computational cost and the difficulties in determining the annealing schedule. This paper demonstrates that the temperature parallel simulated annealing (TPSA) algorithm, a parallel implementation of the SA algorithm, shows great promise to overcome these limitations when applied to continuous functions. The TPSA algorithm greatly reduces the computational time due to its parallel nature, and avoids the determination of the annealing schedule by fixing the temperatures during the annealing process. The main contributions of this paper are threefold. First, this paper explains a simple and effective way to determine the temperatures by applying the concept of critical temperature (T_C). Second, this paper presents systematic tests of the TPSA algorithm on various continuous functions, demonstrating comparable performance as well-established sequential SA algorithms. Third, this paper demonstrates the application of the TPSA algorithm on a difficult practical inverse problem, namely the hyperspectral tomography problem. The results and conclusions presented in this work provide are expected to be useful for the further development and expanded applications of the TPSA algorithm.     

Bayesian Nonparametric Inference for the Power Likelihood

The aim in this article is to provide a means to undertake Bayesian inference for mixture models when the likelihood function is raised to a power between 0 and 1. The main purpose for doing this is to guarantee a strongly consistent model and hence, make it possible to compare the consistent posterior with the correct posterior, looking for signs of discrepancy. This will be explained in detail in the article. Another purpose would be for simulated annealing algorithms. In particular, for the widely used mixture of Dirichlet process model, it is far from obvious how to undertake inference via Markov chain Monte Carlo methods when the likelihood is raised to a power other than 1. In this article, we demonstrate how posterior sampling can be carried out when using a power likelihood. Matlab code to implement the algorithm is available as supplementary material.     

Calculation of Posterior Bounds Given Convex Sets of Prior Probability Measures and Likelihood Functions

Abstract

This article presents alternatives and improvements to Lavine's algorithm, currently the most popular method for calculation of posterior expectation bounds induced by sets of probability measures. First, methods from probabilistic logic and Walley's and White-Snow's algorithms are reviewed and compared to Lavine's algorithm. Second, the calculation of posterior bounds is reduced to a fractional programming problem. From the unifying perspective of fractional programming, Lavine's algorithm is derived from Dinkelbach's algorithm, and the White-Snow algorithm is shown to be similar to the Charnes-Cooper transformation. From this analysis, a novel algorithm for expectation bounds is derived. This algorithm provides a complete solution for the calculation of expectation bounds from priors and likelihood functions specified as convex sets of measures. This novel algorithm is then extended to handle the situation where several independent identically distributed measurements are available. Examples are analyzed through a software package that performs robust inferences and that is publicly available.     

Image registration with iterated local search

This contribution is devoted to the application of iterated local search to image registration, a very complex, real-world problem in the field of image processing. To do so, we first re-define this parameter estimation problem as a combinatorial optimization problem, then analyze the use of image-specific information to guide the search in the form of an heuristic function, and finally propose its solution by iterated local search. Our algorithm is tested by comparing its performance to that of two different baseline algorithms: iterative closest point, a well-known, image registration technique, a hybrid algorithm including the latter technique within a simulated annealing approach, a multi-start local search procedure, that allows us to check the influence of the search scheme considered in the problem solving, and a real coded genetic algorithm. Four different problem instances are tackled in the experimental study, resulting from two images and two transformations applied on them. Three parameter settings are analyzed in our approach in order to check three heuristic information scenarios where the heuristic is not used at all, is partially used or almost completely guides the search process, as well as two different number of iterations in the algorithms outer-inner loops. This work was partially supported by the Spanish Ministerio de Ciencia y Tecnología under project TIC2003-00877 (including FEDER fundings) and under Network HEUR TIC2002-10866-E.     

Molecular conformation on the CM-5 by parallel two-level simulated annealing

In this paper, we propose a new kind of simulated annealing algorithm calledtwo-level simulated annealing for solving certain class of hard combinatorial optimization problems. This two-level simulated annealing algorithm is less likely to get stuck at a non-global minimizer than conventional simulated annealing algorithms. We also propose a parallel version of our two-level simulated annealing algorithm and discuss its efficiency. This new technique is then applied to the Molecular Conformation problem in 3 dimensional Euclidean space. Extensive computational results on Thinking Machines CM-5 are presented. With the full Lennard-Jones potential function, we were able to get satisfactory results for problems for cluster sizes as large as 100,000. A peak rate of over 0.8 giga flop per second in 64-bit operations was sustained on a partition with 512 processing elements. To the best of our knowledge, ground states of Lennard-Jones clusters of size as large as these have never been reported before.Also a researcher at the Army High Performance Computing Research Center, University of Minnesota, Minneapolis, MN 55415     

Gibbs Sampling Will Fail in Outlier Problems with Strong Masking

Abstract

This article discusses the convergence of the Gibbs sampling algorithm when it is applied to the problem of outlier detection in regression models. Given any vector of initial conditions, theoretically, the algorithm converges to the true posterior distribution. However, the speed of convergence may slow down in a high-dimensional parameter space where the parameters are highly correlated. We show that the effect of the leverage in regression models makes very difficult the convergence of the Gibbs sampling algorithm in sets of data with strong masking. The problem is illustrated with examples.     

Bayesian variable selection in generalized linear models using a combination of stochastic optimization methods

In this paper the usage of a stochastic optimization algorithm as a model search tool is proposed for the Bayesian variable selection problem in generalized linear models. Combining aspects of three well known stochastic optimization algorithms, namely, simulated annealing, genetic algorithm and tabu search, a powerful model search algorithm is produced. After choosing suitable priors, the posterior model probability is used as a criterion function for the algorithm; in cases when it is not analytically tractable Laplace approximation is used. The proposed algorithm is illustrated on normal linear and logistic regression models, for simulated and real-life examples, and it is shown that, with a very low computational cost, it achieves improved performance when compared with popular MCMC algorithms, such as the MCMC model composition, as well as with “vanilla” versions of simulated annealing, genetic algorithm and tabu search.     

Robust priors for smoothing and image restoration

The Bayesian method for restoring an image corrupted by added Gaussian noise uses a Gibbs prior for the unknown clean image. The potential of this Gibbs prior penalizes differences between adjacent grey levels. In this paper we discuss the choice of the form and the parameters of the penalizing potential in a particular example used previously by Ogata (1990,Ann. Inst. Statist. Math.,42, 403–433). In this example the clean image is piecewise constant, but the constant patches and the step sizes at edges are small compared with the noise variance. We find that contrary to results reported in Ogata (1990,Ann. Inst. Statist. Math.,42, 403–433) the Bayesian method performs well provided the potential increases more slowly than a quadratic one and the scale parameter of the potential is sufficiently small. Convex potentials with bounded derivatives perform not much worse than bounded potentials, but are computationally much simpler. For bounded potentials we use a variant of simulated annealing. For quadratic potentials data-driven choices of the smoothing parameter are reviewed and compared. For other potentials the smoothing parameter is determined by considering which deviations from a flat image we would like to smooth out and retain respectively.     

利用模拟退火遗传算法实现图像阈值分割

本文提出了一种利用模拟退火算法和遗传算法相结合的图像阈值分割算法,试验结果表明该算法增强了算法的全局收敛性,加快了算法的收敛速度,提高了图像阈值分割的效率.     

Analysis of Static Simulated Annealing Algorithms

Generalized hill climbing (GHC) algorithms provide a framework for modeling local search algorithms to address intractable discrete optimization problems. This paper introduces a measure for determining the expected number of iterations to visit a predetermined objective function level, given that an inferior objective function level has been reached in a finite number of iterations. A variation of simulated annealing (SA), termed static simulated annealing (S²A), is analyzed using this measure. S²A uses a fixed cooling schedule during the algorithm execution. Though S²A is probably nonconvergent, its finite-time performance can be assessed using the finite-time performance measure defined in this paper.     

基于改进模拟退火算法的推动式生产-配送协调优化

为减小物资生产与配送不协调造成的成本及生产资源浪费,建立了考虑推动式生产调度的物资配送优化模型,并针对标准模拟退火算法受随机因素影响易陷入局部最优的缺点,设计带有回火与缓冷操作的改进模拟退火算法对模型求解,确定了优化的车辆配送路线以及物资生产计划。对比实验结果表明：相对于单纯的物资配送优化模型,考虑推动式生产调度的配送优化模型,能够有效减小物资滞留时间以及配送延误成本;相较于标准模拟退火算法,改进算法搜索到了更优解,且计算结果的标准差减小了93.42%,稳定性更好;同时,改进模拟退火算法具有较低的偏差率,在中小规模算例中求解质量较高,平均偏差率在0.5%以内。     

On the Convergence of Successive Substitution Sampling

Abstract

The problem of finding marginal distributions of multidimensional random quantities has many applications in probability and statistics. Many of the solutions currently in use are very computationally intensive. For example, in a Bayesian inference problem with a hierarchical prior distribution, one is often driven to multidimensional numerical integration to obtain marginal posterior distributions of the model parameters of interest. Recently, however, a group of Monte Carlo integration techniques that fall under the general banner of successive substitution sampling (SSS) have proven to be powerful tools for obtaining approximate answers in a very wide variety of Bayesian modeling situations. Answers may also be obtained at low cost, both in terms of computer power and user sophistication. Important special cases of SSS include the “Gibbs sampler” described by Gelfand and Smith and the “IP algorithm” described by Tanner and Wong. The major problem plaguing users of SSS is the difficulty in ascertaining when “convergence” of the algorithm has been obtained. This problem is compounded by the fact that what is produced by the sampler is not the functional form of the desired marginal posterior distribution, but a random sample from this distribution. This article gives a general proof of the convergence of SSS and the sufficient conditions for both strong and weak convergence, as well as a convergence rate. We explore the connection between higher-order eigenfunctions of the transition operator and accelerated convergence via good initial distributions. We also provide asymptotic results for the sampling component of the error in estimating the distributions of interest. Finally, we give two detailed examples from familiar exponential family settings to illustrate the theory.     

Approximate solution of a resource-constrained scheduling problem

This paper is devoted to the approximate solution of a strongly NP-hard resource-constrained scheduling problem which arises in relation to the operability of certain high availability real time distributed systems. We present an algorithm based on the simulated annealing metaheuristic and, building on previous research on exact solution methods, extensive computational results demonstrating its practical ability to produce acceptable solutions, in a precisely defined sense. Additionally, our experiments are in remarkable agreement with certain theoretical properties of our simulated annealing scheme. The paper concludes with a short discussion on further research. This research was supported in part by Association Nationale de la Recherche Technique grant CIFRE-121/2004.     

Ridgelet bi-frame

This article introduces a new bi-frame called ridgelet bi-frame. The ridgelet bi-frame consists of two ridgelet frames that are dual to each other. The construction of the ridgelet bi-frame starts with a bi-frame built on a biorthogonal wavelet system in the Radon domain. The image of the resulting bi-frame under an isometric map from the Radon domain to L²(R²) is also a bi-frame, which we refer to as the ridgelet bi-frame. The ridgelet bi-frame can be thought of as an extension of the notion of the orthonormal ridgelet, which provides a more flexible and effective tool for image analysis and processing applications. An algorithm for image denoising based on the new bi-frame is developed. Experimental examples have demonstrated that the excellent performance can be achieved when using the ridgelet bi-frame for image denoising.     

Generation of Over-Dispersed and Under-Dispersed Binomial Variates

Abstract

This article proposes an algorithm for generating over-dispersed and under-dispersed binomial variates with specified mean and variance. The over-dispersed/under-dispersed distributions are derived from correlated binary variables with an underlying continuous multivariate distribution. Different multivariate distributions or different correlation matrices result in different over-dispersed (or under-dispersed) distributions. The over-dispersed binomial distributions that are generated from three different correlation matrices of a multivariate normal are compared with the beta-binomial distribution for various mean and over-dispersion parameters by quantile-quantile (Q-Q) plots. The two distributions appear to be similar. The under-dispersed binomial distribution is simulated to model an example data set that exhibits under-dispersed binomial variation.     

Simulated annealing algorithm for finding periodic orbits of multi-electron atomic systems

We adapt the simulated annealing algorithm to the search of periodic orbits for classical multi-electron atomic systems. This is done by minimizing the nth return distance to the initial position on a Poincaré surface of section under an energy constraint. Here we give evidence of the feasibility of the method by applying it to the helium atom in the ground state for one to three spatial dimensions. We examine the structure of the dynamics and connect its organization to the periodic orbits we have found.