Finding preferred subsets of Pareto optimal solutions

Multi-objective optimization algorithms can generate large sets of Pareto optimal (non-dominated) solutions. Identifying the best solutions across a very large number of Pareto optimal solutions can be a challenge. Therefore it is useful for the decision-maker to be able to obtain a small set of preferred Pareto optimal solutions. This paper analyzes a discrete optimization problem introduced to obtain optimal subsets of solutions from large sets of Pareto optimal solutions. This discrete optimization problem is proven to be NP-hard. Two exact algorithms and five heuristics are presented to address this problem. Five test problems are used to compare the performances of these algorithms and heuristics. The results suggest that preferred subset of Pareto optimal solutions can be efficiently obtained using the heuristics, while for smaller problems, exact algorithms can be applied.     

Balanced fuzzy particle swarm optimization

In the present study an extension of particle swarm optimization (PSO) algorithm which is in conformity with actual nature is introduced for solving combinatorial optimization problems. Development of this algorithm is essentially based on balanced fuzzy sets theory. The classical fuzzy sets theory cannot distinguish differences between positive and negative information of membership functions, while in the new method both kinds of information “positive and negative” about membership function are equally important. The balanced fuzzy particle swarm optimization algorithm is used for fundamental optimization problem entitled traveling salesman problem (TSP). For convergence inspecting of new algorithm, method was used for TSP problems. Convergence curves were represented fast convergence in restricted and low iterations for balanced fuzzy particle swarm optimization algorithm (BF-PSO) comparison with fuzzy particle swarm optimization algorithm (F-PSO).     

Performance Analysis of Cyclical Simulated Annealing Algorithms

Generalized hill climbing (GHC) algorithms provide a framework for modeling local search algorithms for addressing intractable discrete optimization problems. Measures for assessing the finite-time performance of GHC algorithms have been developed using this framework, including the expected number of iterations to visit a predetermined objective function value level. This paper analyzes how the expected number of iterations to visit a predetermined objective function value level can be estimated for cyclical simulated annealing. Cyclical simulated annealing uses a cooling schedule that cycles through a set of temperature values. Computational results with traveling salesman problem instances taken from TSPLIB show how the expected number of iterations to visit solutions with predetermined objective function levels can be estimated for cyclical simulated annealing.AMS 2000 Subject Classification 90-08 Computational Methods: Local Search, 90C59 Heuristics: Simulated Annealing     

A framework for analyzing sub-optimal performance of local search algorithms

This paper presents a framework for analyzing and comparing sub-optimal performance of local search algorithms for hard discrete optimization problems. The β-acceptable solution probability is introduced that captures how effectively an algorithm has performed to date and how effectively an algorithm can be expected to perform in the future. Using this probability, the necessary conditions for a local search algorithm to converge in probability to β-acceptable solutions are derived. To evaluate and compare the effectiveness of local search algorithms, two estimators for the expected number of iterations to visit a β-acceptable solution are obtained. Computational experiments are reported with simulated annealing and tabu search applied to four small traveling salesman problem instances, and the Lin-Kernighan-Helsgaun algorithm applied to eight medium to large traveling salesman problem instances (all with known optimal solutions), to illustrate the application of these estimators.     

A variable neighborhood search for the network design problem with relays

Given a set of commodities to be routed over a network, the network design problem with relays involves selecting a route for each commodity and determining the location of relays where the commodities must be reprocessed at certain distance intervals. We propose a hybrid approach based on variable neighborhood search. The variable neighborhood algorithm searches for the route for each commodity and the optimal relay locations for a given set of routes are determined by an implicit enumeration algorithm. We show that dynamic programming can be used to determine the optimal relay locations for a single commodity. Dynamic programming is embedded into the implicit enumeration algorithm to solve the relay location problem optimally for multiple commodities. The special structure of the problem is leveraged for computational efficiency. In the variable neighborhood search algorithm, the routes of the current solution are perturbed and reconstructed to generate neighbor solutions using random and greedy construction heuristics. Computational experiments on three sets of problems (80 instances) show that the variable neighborhood search algorithm with optimal relay allocations outperforms all existing algorithms in the literature.     

Subset simulation for multi-objective optimization

Subset simulation is an efficient Monte Carlo technique originally developed for structural reliability problems, and further modified to solve single-objective optimization problems based on the idea that an extreme event (optimization problem) can be considered as a rare event (reliability problem). In this paper subset simulation is extended to solve multi-objective optimization problems by taking advantages of Markov Chain Monte Carlo and a simple evolutionary strategy. In the optimization process, a non-dominated sorting algorithm is introduced to judge the priority of each sample and handle the constraints. To improve the diversification of samples, a reordering strategy is proposed. A Pareto set can be generated after limited iterations by combining the two sorting algorithms together. Eight numerical multi-objective optimization benchmark problems are solved to demonstrate the efficiency and robustness of the proposed algorithm. A parametric study on the sample size in a simulation level and the proportion of seed samples is performed to investigate the performance of the proposed algorithm. Comparisons are made with three existing algorithms. Finally, the proposed algorithm is applied to the conceptual design optimization of a civil jet.     

Trajectory-following algorithms for min-max optimization problems

In this paper, we consider a class of nonlinear minimum-maximum optimization problems subject to boundedness constraints on the decision vectors. Three algorithms are developed for finding the min-max point using the concept of solving an associated dynamical system. In the first and third algorithms, solutions are obtained by solving systems of differential equations. The second algorithm is a discrete version of the first algorithm. The trajectories generated by the first and second algorithms may move inside or on the boundary of the constraint set, while the third algorithm ensures that any trajectory that begins inside the constraint region remains in its interior. Sufficient conditions for global convergence of the two algorithms are also established. For illustration, four numerical examples are solved.This work was partially supported by a research grant from the Australian Research Committee.     

AN SQP ALGORITHM WITH NONMONOTONE LINE SEARCHFOR GENERAL NONLINEAR CONSTRAINED OPTIMIZATION PROBLEM

1.IntroductionInthispaper,weconsiderthefollowingoptimizationproblem(P):minf(x)(P)e.t.gi(x)~0(j=1,...,m,),gi(x)s0(j==m,+l,...,m),wherex~(xl,'5x.)"EE",f(x),gi(x)(j=1,',m)areallreaLvaluedsmoothfunctions.Inrecentyears)SequentialQuadraticProgramming(SoP)algorithmshavebeenex-tensivelyusedforthesolutionofsuchproblems,andtheyhavebeenwidelyinvestigatedbymanyauthors(see,e.g.[1-5]).AnattractivefeatureoftheSoPmethodisthat,undersomesuitableconditions,asuperlinearconvergencecanbeobtained,providedth…     

Iterative Algorithms for Equilibrium Problems

We consider equilibrium problems in the framework of the formulation proposed by Blum and Oettli, which includes variational inequalities, Nash equilibria in noncooperative games, and vector optimization problems, for instance, as particular cases. We show that such problems are particular instances of convex feasibility problems with infinitely many convex sets, but with additional structure, so that projection algorithms for convex feasibility can be modified in order to improve their convergence properties, mainly achieving global convergence without either compactness or coercivity assumptions. We present a sequential projections algorithm with an approximately most violated constraint control strategy, and two variants where exact orthogonal projections are replaced by approximate ones, using separating hyperplanes generated by subgradients. We include full convergence analysis of these algorithms.     

Benchmarking ordering techniques for nonserial dynamic programming

Five ordering algorithms for the nonserial dynamic programming algorithm for solving sparse discrete optimization problems are compared in this paper. The benchmarking reveals that the ordering of the variables has a significant impact on the run-time of these algorithms. In addition, it is shown that different orderings are most effective for different classes of problems. Finally, it is shown that, amongst the algorithms considered here, heuristics based on maximum cardinality search and minimum fill-in perform best for solving the discrete optimization problems considered in this paper.     

Tutorial on Surrogate Constraint Approaches for Optimization in Graphs

Surrogate constraint methods have been embedded in a variety of mathematical programming applications over the past thirty years, yet their potential uses and underlying principles remain incompletely understood by a large segment of the optimization community. In a number of significant domains of combinatorial optimization, researchers have produced solution strategies without recognizing that they can be derived as special instances of surrogate constraint methods. Once the connection to surrogate constraint ideas is exposed, additional ways to exploit this framework become visible, frequently offering opportunities for improvement.We provide a tutorial on surrogate constraint approaches for optimization in graphs, illustrating the key ideas by reference to independent set and graph coloring problems, including constructions for weighted independent sets which have applications to associated covering and weighted maximum clique problems. In these settings, the surrogate constraints can be generated relative to well-known packing and covering formulations that are convenient for exposing key notions. The surrogate constraint approaches yield widely used heuristics for identifying independent sets as simple special cases, and also afford previously unidentified heuristics that have greater power in these settings. Our tutorial also shows how the use of surrogate constraints can be placed within the context of vocabulary building strategies for independent set and coloring problems, providing a framework for applying surrogate constraints that can be used in other applications.At a higher level, we show how to make use of surrogate constraint information, together with specialized algorithms for solving associated sub-problems, to obtain stronger objective function bounds and improved choice rules for heuristic or exact methods. The theorems that support these developments yield further strategies for exploiting surrogate constraint relaxations, both in graph optimization and integer programming generally.     

Analysis of Static Simulated Annealing Algorithms

Generalized hill climbing (GHC) algorithms provide a framework for modeling local search algorithms to address intractable discrete optimization problems. This paper introduces a measure for determining the expected number of iterations to visit a predetermined objective function level, given that an inferior objective function level has been reached in a finite number of iterations. A variation of simulated annealing (SA), termed static simulated annealing (S²A), is analyzed using this measure. S²A uses a fixed cooling schedule during the algorithm execution. Though S²A is probably nonconvergent, its finite-time performance can be assessed using the finite-time performance measure defined in this paper.     

Decomposition Methods for Differentiable Optimization Problems over Cartesian Product Sets

This paper presents a unified analysis of decomposition algorithms for continuously differentiable optimization problems defined on Cartesian products of convex feasible sets. The decomposition algorithms are analyzed using the framework of cost approx imation algorithms. A convergence analysis is made for three decomposition algorithms: a sequential algorithm which extends the classical Gauss-Seidel scheme, a synchronized parallel algorithm which extends the Jacobi method, and a partially asynchronous parallel algorithm. The analysis validates inexact computations in both the subproblem and line search phases, and includes convergence rate results. The range of feasible step lengths within each algorithm is shown to have a direct correspondence to the increasing degree of parallelism and asynchronism, and the resulting usage of more outdated information in the algorithms.     

An incremental clustering algorithm based on hyperbolic smoothing

Clustering is an important problem in data mining. It can be formulated as a nonsmooth, nonconvex optimization problem. For the most global optimization techniques this problem is challenging even in medium size data sets. In this paper, we propose an approach that allows one to apply local methods of smooth optimization to solve the clustering problems. We apply an incremental approach to generate starting points for cluster centers which enables us to deal with nonconvexity of the problem. The hyperbolic smoothing technique is applied to handle nonsmoothness of the clustering problems and to make it possible application of smooth optimization algorithms to solve them. Results of numerical experiments with eleven real-world data sets and the comparison with state-of-the-art incremental clustering algorithms demonstrate that the smooth optimization algorithms in combination with the incremental approach are powerful alternative to existing clustering algorithms.     

On the convergence of inexact block coordinate descent methods for constrained optimization

We consider the problem of minimizing a smooth function over a feasible set defined as the Cartesian product of convex compact sets. We assume that the dimension of each factor set is huge, so we are interested in studying inexact block coordinate descent methods (possibly combined with column generation strategies). We define a general decomposition framework where different line search based methods can be embedded, and we state global convergence results. Specific decomposition methods based on gradient projection and Frank–Wolfe algorithms are derived from the proposed framework. The numerical results of computational experiments performed on network assignment problems are reported.     

Analyzing the Performance of Generalized Hill Climbing Algorithms

Generalized hill climbing algorithms provide a framework to describe and analyze metaheuristics for addressing intractable discrete optimization problems. The performance of such algorithms can be assessed asymptotically, either through convergence results or by comparison to other algorithms. This paper presents necessary and sufficient convergence conditions for generalized hill climbing algorithms. These conditions are shown to be equivalent to necessary and sufficient convergence conditions for simulated annealing when the generalized hill climbing algorithm is restricted to simulated annealing. Performance measures are also introduced that permit generalized hill climbing algorithms to be compared using random restart local search. These results identify a solution landscape parameter based on the basins of attraction for local optima that determines whether simulated annealing or random restart local search is more effective in visiting a global optimum. The implications and limitations of these results are discussed.     

Local linear convergence of approximate projections onto regularized sets

The numerical properties of algorithms for finding the intersection of sets depend to some extent on the regularity of the sets, but even more importantly on the regularity of the intersection. The alternating projection algorithm of von Neumann has been shown to converge locally at a linear rate dependent on the regularity modulus of the intersection. In many applications, however, the sets in question come from inexact measurements that are matched to idealized models. It is unlikely that any such problems in applications will enjoy metrically regular intersection, let alone set intersection. We explore a regularization strategy that generates an intersection with the desired regularity properties. The regularization, however, can lead to a significant increase in computational complexity. In a further refinement, we investigate and prove linear convergence of an approximate alternating projection algorithm. The analysis provides a regularization strategy that fits naturally with many ill-posed inverse problems, and a mathematically sound stopping criterion for extrapolated, approximate algorithms. The theory is demonstrated on the phase retrieval problem with experimental data. The conventional early termination applied in practice to unregularized, consistent problems in diffraction imaging can be justified fully in the framework of this analysis providing, for the first time, proof of convergence of alternating approximate projections for finite dimensional, consistent phase retrieval problems.     

On branching heuristics for the bi-objective 0/1 unidimensional knapsack problem

This paper focuses on branching strategies that are involved in branch and bound algorithms when solving multi-objective optimization problems. The choice of the branching variable at each node of the search tree constitutes indeed an important component of these algorithms. In this work we focus on multi-objective knapsack problems. In the literature, branching heuristics used for these problems are static, i.e., the order on the variables is determined prior to the execution. This study investigates the benefit of defining more sophisticated branching strategies. We first analyze and compare a representative set of classic branching heuristics and conclude that none can be identified as the best overall heuristic. Using an oracle, we highlight that combining branching heuristics within the same branch and bound algorithm leads to considerably reduced search trees but induces high computational costs. Based on learning adaptive techniques, we propose then dynamic adaptive branching strategies that are able to select the suitable heuristic to apply at each node of the search tree. Experiments are conducted on the bi-objective 0/1 unidimensional knapsack problem.     

An improved artificial bee colony algorithm for directing orbits of chaotic systems

Artificial bee colony algorithm (ABC) is a relatively new optimization technique which has been shown to be competitive to other population-based algorithms. However, there is still an insufficiency in ABC regarding its solution search equation, which is good at exploration but poor at exploitation. To address this concerning issue, we propose an improved ABC (IABC) by using a modified search strategy to generate a new food source in order that the exploration and exploitation can be well balanced and satisfactory optimization performances can be achieved. In addition, to enhance the global convergence, when producing the initial population, both opposition-based learning method and chaotic maps are employed. In this paper, the proposed algorithm is applied to control and synchronization of discrete chaotic systems which can be formulated as both multimodal numerical optimization problems with high dimension. Numerical simulation and comparisons with some typical existing algorithms demonstrate the effectiveness and robustness of the proposed approach.     

Global optimization of truss topology with discrete bar areas—Part II: Implementation and numerical results

A classical problem within the field of structural optimization is to find the stiffest truss design subject to a given external static load and a bound on the total volume. The design variables describe the cross sectional areas of the bars. This class of problems is well-studied for continuous bar areas. We consider here the difficult situation that the truss must be built from pre-produced bars with given areas. This paper together with Part I proposes an algorithmic framework for the calculation of a global optimizer of the underlying non-convex mixed integer design problem. In this paper we use the theory developed in Part I to design a convergent nonlinear branch-and-bound method tailored to solve large-scale instances of the original discrete problem. The problem formulation and the needed theoretical results from Part I are repeated such that this paper is self-contained. We focus on the implementation details but also establish finite convergence of the branch-and-bound method. The algorithm is based on solving a sequence of continuous non-convex relaxations which can be formulated as quadratic programs according to the theory in Part I. The quadratic programs to be treated within the branch-and-bound search all have the same feasible set and differ from each other only in the objective function. This is one reason for making the resulting branch-and-bound method very efficient. The paper closes with several large-scale numerical examples. These examples are, to the knowledge of the authors, by far the largest discrete topology design problems solved by means of global optimization.