A computational study of local search algorithms for Italian high-school timetabling

In this paper we report on a computational experience with a local search algorithm for High-school Timetabling Problems. The timetable has to satisfy “hard” requirements, that are mandatory, and should minimize the violation of “soft” constraints. In our approach, we combine Simulated Annealing with a Very Large-Scale Neighborhood search where the neighborhood is explored by solving an Integer Programming problem. We report on a computational experience validating the usefulness of the proposed approach. Support for I. Vasil’ev was provided by NATO grant CBP.NR.RIG.911258.     

Simulated annealing for discrete optimization with estimation

We extend the basic convergence results for the Simulated Annealing (SA) algorithm to a stochastic optimization problem where the objective function is stochastic and can be evaluated only through Monte Carlo simulation (hence, disturbed with random error). This extension is important when either the objective function cannot be evaluated exactly or such an evaluation is computationally expensive. We present a modified SA algorithm and show that under suitable conditions on the random error, the modified SA algorithm converges in probability to a global optimizer. Computational results and comparisons with other approaches are given to demonstrate the performance of the proposed modified SA algorithm.     

Converging Marriage in Honey-Bees Optimization and Application to Stochastic Dynamic Programming

In this paper, we first refine a recently proposed metaheuristic called “Marriage in Honey-Bees Optimization” (MBO) for solving combinatorial optimization problems with some modifications to formally show that MBO converges to the global optimum value. We then adapt MBO into an algorithm called “Honey-Bees Policy Iteration” (HBPI) for solving infinite horizon-discounted cost stochastic dynamic programming problems and show that HBPI also converges to the optimal value.     

A new algorithm for generalized fractional programs

A new dual problem for convex generalized fractional programs with no duality gap is presented and it is shown how this dual problem can be efficiently solved using a parametric approach. The resulting algorithm can be seen as “dual” to the Dinkelbach-type algorithm for generalized fractional programs since it approximates the optimal objective value of the dual (primal) problem from below. Convergence results for this algorithm are derived and an easy condition to achieve superlinear convergence is also established. Moreover, under some additional assumptions the algorithm also recovers at the same time an optimal solution of the primal problem. We also consider a variant of this new algorithm, based on scaling the “dual” parametric function. The numerical results, in case of quadratic-linear ratios and linear constraints, show that the performance of the new algorithm and its scaled version is superior to that of the Dinkelbach-type algorithms. From the computational results it also appears that contrary to the primal approach, the “dual” approach is less influenced by scaling. This research was carried out at the Econometric Institute, Erasmus University, Rotterdam, the Netherlands and was supported by J.N.I.C.T. (Portugal) under contract BD/707/90-RM.     

A Continuous Genetic Algorithm Designed for the Global Optimization of Multimodal Functions

Genetic algorithms are stochastic search approaches based on randomized operators, such as selection, crossover and mutation, inspired by the natural reproduction and evolution of the living creatures. However, few published works deal with their application to the global optimization of functions depending on continuous variables.A new algorithm called Continuous Genetic Algorithm (CGA) is proposed for the global optimization of multiminima functions. In order to cover a wide domain of possible solutions, our algorithm first takes care over the choice of the initial population. Then it locates the most promising area of the solution space, and continues the search through an intensification inside this area. The selection, the crossover and the mutation are performed by using the decimal code. The efficiency of CGA is tested in detail through a set of benchmark multimodal functions, of which global and local minima are known. CGA is compared to Tabu Search and Simulated Annealing, as alternative algorithms.     

Two “well-known” properties of subgradient optimization

The subgradient method is both a heavily employed and widely studied algorithm for non-differentiable optimization. Nevertheless, there are some basic properties of subgradient optimization that, while “well known” to specialists, seem to be rather poorly known in the larger optimization community. This note concerns two such properties, both applicable to subgradient optimization using the divergent series steplength rule. The first involves convergence of the iterative process, and the second deals with the construction of primal estimates when subgradient optimization is applied to maximize the Lagrangian dual of a linear program. The two topics are related in that convergence of the iterates is required to prove correctness of the primal construction scheme. Dedicated to B.T. Polyak on the occassion of his 70th birthday.     

Simulated Annealing for noisy cost functions

We generalize a classical convergence result for the Simulated Annealing algorithm to a stochastic optimization context, i.e., to the case where cost function observations are disturbed by random noise. It is shown that for a certain class of noise distributions, the convergence assertion remains valid, provided that the standard deviation of the noise is reduced in the successive steps of cost function evaluation (e.g., by repeated observation) with a speed O(k ^- ), where is an arbitrary constant larger than one.     

Multi-armed bandits based on a variant of Simulated Annealing

A variant of Simulated Annealing termed Simulated Annealing with Multiplicative Weights (SAMW) has been proposed in the literature. However, convergence was dependent on a parameter β(T), which was calculated a-priori based on the total iterations T the algorithm would run for. We first show the convergence of SAMW even when a diminishing stepsize β_k → 1 is used, where k is the index of iteration. Using this SAMW as a kernel, a stochastic multi-armed bandit (SMAB) algorithm called SOFTMIX can be improved to obtain the minimum-possible log regret, as compared to log² regret of the original. Another modification of SOFTMIX is proposed which avoids the need for a parameter that is dependent on the reward distribution of the arms. Further, a variant of SOFTMIX that uses a comparison term drawn from another popular SMAB algorithm called UCB1 is then described. It is also shown why the proposed scheme is computationally more efficient over UCB1, and an alternative to this algorithm with simpler stepsizes is also proposed. Numerical simulations for all the proposed algorithms are then presented.     

A novel memetic algorithm and its application to data clustering

In this paper, a novel memetic algorithm (MA) named GS-MPSO is proposed by combining a particle swarm optimization (PSO) with a Gaussian mutation operator and a Simulated Annealing (SA)-based local search operator. In GS-MPSO, the particles are organized as a ring lattice. The Gaussian mutation operator is applied to the stagnant particles to prevent GS-MPSO trapping into local optima. The SA-based local search strategy is developed to combine with the cognition-only PSO model and perform a fine-grained local search around the promising regions. The experimental results show that GS-MPSO is superior to some other variants of PSO with better performance on optimizing the benchmark functions when the computing resource is limited. Data clustering is studied as a real case study to further demonstrate its optimization ability and usability, too.     

On the convergence of “Threshold Accepting”

Simulated Annealing (SA) has become a very popular tool in combinatorial optimization since its introduction in 1982. Recently Dueck and Scheuer proposed another simple modification of local search which they called Threshold Accepting (TA). In this paper some convergence results for TA are presented. The proofs are not constructive and make use of the fact that in a certain sense SA belongs to the convex hull of TA.     

Iterative improvement methods for a multiperiod network design problem

We address an important problem in telecommunications planning: the multiperiod network expansion problem. Our aim is to show that it can be efficiently solved using a new local search approach. To achieve our goal, we first show how to adapt the problem's formulation to meet our local search program's requirements. We then describe GLIT (Generic Local Improvement Template), a new, generic auto-calibrating local search algorithm; the different neighbouring strategies that we designed specifically for this problem; as well as a Genetic algorithm for this problem. We compare and discuss the performance of these algorithms using extensive computational tests on a large range of instances with up to 7500 arcs. These experiments show that GLIT clearly outperforms competitive approaches, especially when it is coupled with Genetic algorithms. We also show that the hybrid algorithms Genetic/Tabu, Genetic/Simulated Annealing, and finally Genetic/GLIT all outperform both pure Genetic and pure local search algorithms.     

Credit risk optimization using factor models

We study portfolio credit risk management using factor models, with a focus on optimal portfolio selection based on the tradeoff of expected return and credit risk. We begin with a discussion of factor models and their known analytic properties, paying particular attention to the asymptotic limit of a large, finely grained portfolio. We recall prior results on the convergence of risk measures in this “large portfolio approximation” which are important for credit risk optimization. We then show how the results on the large portfolio approximation can be used to reduce significantly the computational effort required for credit risk optimization. For example, when determining the fraction of capital to be assigned to particular ratings classes, it is sufficient to solve the optimization problem for the large portfolio approximation, rather than for the actual portfolio. This dramatically reduces the dimensionality of the problem, and the amount of computation required for its solution. Numerical results illustrating the application of this principle are also presented. JEL Classification G11     

Distributed multi-agent optimization with state-dependent communication

We study distributed algorithms for solving global optimization problems in which the objective function is the sum of local objective functions of agents and the constraint set is given by the intersection of local constraint sets of agents. We assume that each agent knows only his own local objective function and constraint set, and exchanges information with the other agents over a randomly varying network topology to update his information state. We assume a state-dependent communication model over this topology: communication is Markovian with respect to the states of the agents and the probability with which the links are available depends on the states of the agents. We study a projected multi-agent subgradient algorithm under state-dependent communication. The state-dependence of the communication introduces significant challenges and couples the study of information exchange with the analysis of subgradient steps and projection errors. We first show that the multi-agent subgradient algorithm when used with a constant stepsize may result in the agent estimates to diverge with probability one. Under some assumptions on the stepsize sequence, we provide convergence rate bounds on a “disagreement metric” between the agent estimates. Our bounds are time-nonhomogeneous in the sense that they depend on the initial starting time. Despite this, we show that agent estimates reach an almost sure consensus and converge to the same optimal solution of the global optimization problem with probability one under different assumptions on the local constraint sets and the stepsize sequence.     

An analytical study of the sensitivity of a discrete stochastic model for sales dynamics

We consider a discrete model for sales dynamics in the case of a stochastic model of the market. The model includes “fast” and “slow” components of the market situation described by a stochastic process of “white noise” type and the correlated stochastic process. By using an integral representation of the main characteristics of the Kalman filter, we obtain expressions for stochastic parameters of additional errors of the estimate that arise in the case where the characteristics of noises are inexact. We make an asymptotical analysis of these expressions and give recommendations for the price-forming strategy in the case of uncertainty of the market situation. Bibliography: 2 titles. Translated fromObchyslyuval'na ta Prykladna Matematyka, No. 81, 1997, pp. 110–116.     

On Simulated Annealing and Nested Annealing

Simulated Annealing is a family of randomized algorithms used to solve many combinatorial optimization problems. In practice they have been applied to solve some presumably hard (e.g., NP-complete) problems. The level of performance obtained has been promising [2,5,6,14]. The success of this heuristic technique has motivated analysis of this algorithm from a theoretical point of view. In particular, people have looked at the convergence of this algorithm. They have shown (see e.g., [10]) that this algorithm converges in the limit to a globally optimal solution with probability 1. However few of these convergence results specify a time limit within which the algorithm is guaranteed to converge (with some high probability, say). We present, for the first time, a simple analysis of SA that will provide a time bound for convergence with overwhelming probability. The analysis will hold no matter what annealing schedule is used. Convergence of Simulated Annealing in the limit will follow as a corollary to our time convergence proof. In this paper we also look at optimization problems for which the cost function has some special properties. We prove that for these problems the convergence is much faster. In particular, we give a simpler and more general proof of convergence for Nested Annealing, a heuristic algorithm developed in [12]. Nested Annealing is based on defining a graph corresponding to the given optimization problem. If this graph is small separable, they [12] show that Nested Annealing will converge faster. For an arbitrary optimization problem, we may not have any knowledge about the separability of its graph. In this paper we give tight bounds for the separability of a random graph. We then use these bounds to analyze the expected behavior of Nested Annealing on an arbitrary optimization problem. The separability bounds we derive in this paper are of independent interest and have the potential of finding other applications.     

Variable metric bundle methods: From conceptual to implementable forms

To minimize a convex function, we combine Moreau-Yosida regularizations, quasi-Newton matrices and bundling mechanisms. First we develop conceptual forms using “reversal” quasi-Newton formulae and we state their global and local convergence. Then, to produce implementable versions, we incorporate a bundle strategy together with a “curve-search”. No convergence results are given for the implementable versions; however some numerical illustrations show their good behaviour even for large-scale problems.     

关于随机优化算法的几点讨论

关于随机优化算法的几点讨论李晓莉，雷功炎（河南驻马店师专，北京大学数学系）ＳＯＭＥＤＩＳＣＵＳＳＩＯＮＳＡＢＯＵＴＳＴＯＣＨＡＳＴＩＣＯＰＴＩＭＩＺＡＴＩＯＮＡＬＧＯＲＩＴＨＭＳ￥ＬｉＸｉａｏ－ｌｉ（ＤｅｐａｒｔｍｅｎｔｏｆＭａｔｈｅｍａｔｉｃｓ，Ｚ...     

End-to-end availability-dependent pricing of network services

We discuss how a new pricing scheme can be integrated within a communication network. The pricing scheme is based on the availability of end-to-end communications, and is an alternative to congestion pricing, which is not applicable when communication capacity is higher than demand (as happens in most communication backbone networks). We also investigate how, based on this scheme, an optimization algorithm for updating the network topology can be applied. The network update problem is modeled as a combinatorial optimization problem, which is approximately solved using a Genetic Algorithm. The good results obtained in a case study show that the method is robust and can be applied even when end-to-end availability measures can only be computed approximately (in this case, using a Monte Carlo method). This research is part of the PAIR associated research project, supported by the INRIA, France, and has also received the support of ECOS-Sud, under Action U03E02. The participation of Pablo Rodríguez was supported by the French Embassy in Uruguay as part of the French Ministère des Affaires étrangères scientific cooperation program; and by the “Programa de Jóvenes Investigadores” of CSIC, UDELAR, Uruguay.     

Attraction probabilities in variable neighborhood search

Empirical evidence demonstrates that when the same local search operator is used, variable neighborhood search consistently outperforms random multistart local search on all types of combinatorial and global optimization problems tested. In this paper we suggest that this superiority in performance may be explained by the distribution of the attraction basins around a current solution as a function of the distance from the solution. We illustrate with a well-known instance of the multisource Weber problem that the “attraction probabilities” for finding better solutions can be orders of magnitude larger in neighborhoods that are close to the current solution. The paper also discusses the global convergence properties of both general methods in the context of attraction probabilities.