Global optimization and simulated annealing

In this paper we are concerned with global optimization, which can be defined as the problem of finding points on a bounded subset of ⁿ in which some real valued functionf assumes its optimal (maximal or minimal) value.We present a stochastic approach which is based on the simulated annealing algorithm. The approach closely follows the formulation of the simulated annealing algorithm as originally given for discrete optimization problems. The mathematical formulation is extended to continuous optimization problems, and we prove asymptotic convergence to the set of global optima. Furthermore, we discuss an implementation of the algorithm and compare its performance with other well-known algorithms. The performance evaluation is carried out for a standard set of test functions from the literature.     

Continuous optimization by a variant of simulated annealing

A variant of the simulated annealing algorithm, based on the generalized method of Bohachevsky et al., is proposed for continuous optimization problems. The algorithm automatically adjusts the step sizes to reflect the local slopes and function values, and it controls the random directions to point favorably toward potential improvements. Computational results on some well known functions show substantial improvements both in solution quality and efficiency.     

A simulated annealing technique for multi-objective simulation optimization

In this paper, we present a simulated annealing algorithm for solving multi-objective simulation optimization problems. The algorithm is based on the idea of simulated annealing with constant temperature, and uses a rule for accepting a candidate solution that depends on the individual estimated objective function values. The algorithm is shown to converge almost surely to an optimal solution. It is applied to a multi-objective inventory problem; the numerical results show that the algorithm converges rapidly.     

Proposal adaptation in simulated annealing for continuous optimization problems

In recent years, adaptive Markov Chain Monte Carlo (MCMC) methods have become a standard tool for Bayesian parameter estimation. In adaptive MCMC, the past iterations are used to tune the proposal distribution of the algorithm. The same adaptation mechanisms can be used in Simulated Annealing (SA), a popular optimization method based on MCMC. The difficulty in using adaptation directly in SA is that the target function changes along the iterations in the annealing process, and the adaptation should keep up with the annealing. In this paper, a mechanism for automatically tuning the proposal distribution in SA is proposed. The approach is based on the Adaptive Metropolis algorithm of Haario et al. (Bernoulli 7(2):223–242, 2001), combined with a weighting mechanism to account for the cooling target. The proposed adaptation mechanism does not add any computational complexity to the problem in terms of objective function evaluations. The effect of adaptation is demonstrated using two benchmark problems, showing that the proposed adaptation mechanism can significantly improve optimization results compared to non-adaptive SA. The approach is presented for continuous optimization problems and generalization to integer and mixed-integer problems is a topic of future research.     

A simulated annealing algorithm for transient optimization in gas networks

In this paper we present a simulated annealing approach for the gas network optimization problem. A gas network consists of a set of pipes to transport the gas from the sources to the sinks whereby gas pressure gets lost due to friction. Further on there are compressors, which increase gas pressure, and valves. The aim is to minimize fuel gas consumption of the compressors whereas demands of consumers have to be satisfied. The problem of transient (time-dependent) optimization of gas networks results in a highly complex mixed integer nonlinear program. We relax the equations describing the gas dynamic in pipes by adding these constraints combined with appropriate penalty factors to the objective function. A suitable neighborhood structure is developed for the relaxed problem where time steps as well as pressure and flow of the gas are decoupled. Our approach convinces with flexibility and very good computational results.     

Modified particle swarm optimization algorithm with simulated annealing behavior and its numerical verification

The hybrid algorithm that combined particle swarm optimization with simulated annealing behavior (SA-PSO) is proposed in this paper. The SA-PSO algorithm takes both of the advantages of good solution quality in simulated annealing and fast searching ability in particle swarm optimization. As stochastic optimization algorithms are sensitive to their parameters, proper procedure for parameters selection is introduced in this paper to improve solution quality. To verify the usability and effectiveness of the proposed algorithm, simulations are performed using 20 different mathematical optimization functions with different dimensions. The comparative works have also been conducted among different algorithms under the criteria of quality of the solution, the efficiency of searching for the solution and the convergence characteristics. According to the results, the SA-PSO could have higher efficiency, better quality and faster convergence speed than compared algorithms.     

Derivative-free simulated annealing and deflecting function technique for global optimization

A hybrid descent method based on simulated annealing (SA) algorithm and one modifying function technique, named deflecting function method, for global optimization is proposed. Unlike some previously proposed algorithms, the designed SA algorithm is executed repeatedly on the transformed function with respect to one prior-obtained local minimum instead of on the original objective function. Meanwhile, large scale searches at the beginning stages and small scale detections in the last stages are adopted. The global convergence is proved. Simulation demonstrates that the new method utilizes the obtained information effectively, so the convergence is significantly sped up and the success rate is greatly improved, compared with other existing methods. As an experimental result, how to combine SA and the deflecting function technique can make the new method more effective is discussed.     

A simulated annealing driven multi-start algorithm for bound constrained global optimization

A derivative-free simulated annealing driven multi-start algorithm for continuous global optimization is presented. We first propose a trial point generation scheme in continuous simulated annealing which eliminates the need for the gradient-based trial point generation. We then suitably embed the multi-start procedure within the simulated annealing algorithm. We modify the derivative-free pattern search method and use it as the local search in the multi-start procedure. We study the convergence properties of the algorithm and test its performance on a set of 50 problems. Numerical results are presented which show the robustness of the algorithm. Numerical comparisons with a gradient-based simulated annealing algorithm and three population-based global optimization algorithms show that the new algorithm could offer a reasonable alternative to many currently available global optimization algorithms, specially for problems requiring ‘direct search’ type algorithm.     

An adaptive simulated annealing algorithm for global optimization over continuous variables

A method is presented for attempting global minimization for a function of continuous variables subject to constraints. The method, calledAdaptive Simulated Annealing (ASA), is distinguished by the fact that the fixed temperature schedules and step generation routines that characterize other implementations are here replaced by heuristic-based methods that effectively eliminate the dependence of the algorithm's overall performance on user-specified control parameters. A parallelprocessing version of ASA that gives increased efficiency is presented and applied to two standard problems for illustration and comparison.This research was supported by the University Research Initiative of the U.S. Army Research Office.     

Asymptotic convergence of a simulated annealing algorithm for multiobjective optimization problems

In this paper we consider a simulated annealing algorithm for multiobjective optimization problems. With a suitable choice of the acceptance probabilities, the algorithm is shown to converge asymptotically, that is, the Markov chain that describes the algorithm converges with probability one to the Pareto optimal set.     

Parallel simulated annealing

This article introduces the notion of restricted parallelism for networks, a generalization of the unlimited parallelism for Boltzmann machines. The convergence of the annealing algorithm in the restricted parallel form is established, for an arbitrary network.     

Investigating a hybrid simulated annealing and local search algorithm for constrained optimization

Constrained Optimization Problems (COP) often take place in many practical applications such as kinematics, chemical process optimization, power systems and so on. These problems are challenging in terms of identifying feasible solutions when constraints are non-linear and non-convex. Therefore, finding the location of the global optimum in the non-convex COP is more difficult as compared to non-convex bound-constrained global optimization problems. This paper proposes a Hybrid Simulated Annealing method (HSA), for solving the general COP. HSA has features that address both feasibility and optimality issues and here, it is supported by a local search procedure, Feasible Sequential Quadratic Programming (FSQP). We develop two versions of HSA. The first version (HSAP) incorporates penalty methods for constraint handling and the second one (HSAD) eliminates the need for imposing penalties in the objective function by tracing feasible and infeasible solution sequences independently. Numerical experiments show that the second version is more reliable in the worst case performance.     

Combining simulated annealing with local search heuristics

We introduce a meta-heuristic to combine simulated annealing with local search methods for CO problems. This new class of Markov chains leads to significantly more powerful optimization methods than either simulated annealing or local search. The main idea is to embed deterministic local search techniques into simulated annealing so that the chain explores only local optima. It makes large, global changes, even at low temperatures, thus overcoming large barriers in configuration space. We have tested this meta-heuristic for the traveling salesman and graph partitioning problems. Tests on instances from public libraries and random ensembles quantify the power of the method. Our algorithm is able to solve large instances to optimality, improving upon local search methods very significantly. For the traveling salesman problem with randomly distributed cities, in a square, the procedure improves on 3-opt by 1.6%, and on Lin-Kernighan local search by 1.3%. For the partitioning of sparse random graphs of average degree equal to 5, the improvement over Kernighan-Lin local search is 8.9%. For both CO problems, we obtain new best heuristics.     

A survey of simulated annealing as a tool for single and multiobjective optimization

This paper presents a comprehensive review of simulated annealing (SA)-based optimization algorithms. SA-based algorithms solve single and multiobjective optimization problems, where a desired global minimum/maximum is hidden among many local minima/maxima. Three single objective optimization algorithms (SA, SA with tabu search and CSA) and five multiobjective optimization algorithms (SMOSA, UMOSA, PSA, WDMOSA and PDMOSA) based on SA have been presented. The algorithms are briefly discussed and are compared. The key step of SA is probability calculation, which involves building the annealing schedule. Annealing schedule is discussed briefly. Computational results and suggestions to improve the performance of SA-based multiobjective algorithms are presented. Finally, future research in the area of SA is suggested.     

Comparative analysis of the cutting angle and simulated annealing methods in global optimization

This article presents a comparative analysis of two methods of global optimization: the simulated annealing method and a method based on a combination of the cutting angle method and a local search. This analysis is carried out using results of numerical experiments. These results demonstrate that the combined method is more effective than the simulated annealing method.     

Customized simulated annealing based decision algorithms for combinatorial optimization in VLSI floorplanning problem

Modern very large scale integration technology is based on fixed-outline floorplan constraints, mostly with an objective of minimizing area and wirelength between the modules. The aim of this work is to minimize the unused area, that is, dead space in the floorplan, in addition to these objectives. In this work, a Simulated Annealing Algorithm (SAA) based heuristic, namely Simulated Spheroidizing Annealing Algorithm (SSAA) has been developed and improvements in the proposed heuristic algorithm is also suggested to improve its performance. Exploration capability of the proposed algorithm is due to the mechanism of reducing the uphill moves made during the initial stage of the algorithm, extended search at each temperature and the improved neighborhood search procedure. The proposed algorithm has been tested using two kinds of benchmarks: Microelectronics Center of North Carolina (MCNC) and Gigascale Systems Research Center (GSRC). The performance of the proposed algorithm is compared with that of other stochastic algorithms reported in the literature and is found to be efficient in producing floorplans with very minimal dead space. The proposed SSAA algorithm is also found more efficient for problems of larger sizes.     

Robust portfolio optimization with copulas

Conditional Value at Risk (CVaR) is widely used in portfolio optimization as a measure of risk. CVaR is clearly dependent on the underlying probability distribution of the portfolio. We show how copulas can be introduced to any problem that involves distributions and how they can provide solutions for the modeling of the portfolio. We use this to provide the copula formulation of the CVaR of a portfolio. Given the critical dependence of CVaR on the underlying distribution, we use a robust framework to extend our approach to Worst Case CVaR (WCVaR). WCVaR is achieved through the use of rival copulas. These rival copulas have the advantage of exploiting a variety of dependence structures, symmetric and not. We compare our model against two other models, Gaussian CVaR and Worst Case Markowitz. Our empirical analysis shows that WCVaR can asses the risk more adequately than the two competitive models during periods of crisis.     

Robust global optimization with polynomials

We consider the optimization problems max_z∈Ω min_x∈K p(z, x) and min_{x ∈ K} max_{z ∈ Ω} p(z, x) where the criterion p is a polynomial, linear in the variables z, the set Ω can be described by LMIs, and K is a basic closed semi-algebraic set. The first problem is a robust analogue of the generic SDP problem max_{z ∈ Ω} p(z), whereas the second problem is a robust analogue of the generic problem min_{x ∈ K} p(x) of minimizing a polynomial over a semi-algebraic set. We show that the optimal values of both robust optimization problems can be approximated as closely as desired, by solving a hierarchy of SDP relaxations. We also relate and compare the SDP relaxations associated with the max-min and the min-max robust optimization problems.     

Improving simulated annealing through derandomization

We propose and study a version of simulated annealing (SA) on continuous state spaces based on \((t,s)_R\)-sequences. The parameter \(R\in \bar{\mathbb {N}}\) regulates the degree of randomness of the input sequence, with the case \(R=0\) corresponding to IID uniform random numbers and the limiting case \(R=\infty \) to (t, s)-sequences. Our main result, obtained for rectangular domains, shows that the resulting optimization method, which we refer to as QMC-SA, converges almost surely to the global optimum of the objective function \(\varphi \) for any \(R\in \mathbb {N}\). When \(\varphi \) is univariate, we are in addition able to show that the completely deterministic version of QMC-SA is convergent. A key property of these results is that they do not require objective-dependent conditions on the cooling schedule. As a corollary of our theoretical analysis, we provide a new almost sure convergence result for SA which shares this property under minimal assumptions on \(\varphi \). We further explain how our results in fact apply to a broader class of optimization methods including for example threshold accepting, for which to our knowledge no convergence results currently exist. We finally illustrate the superiority of QMC-SA over SA algorithms in a numerical study.