Criteria and dimension reduction of linear multiple criteria optimization problems

Two of the main approaches in multiple criteria optimization are optimization over the efficient set and utility function program. These are nonconvex optimization problems in which local optima can be different from global optima. Existing global optimization methods for solving such problems can only work well for problems of moderate dimensions. In this article, we propose some ways to reduce the number of criteria and the dimension of a linear multiple criteria optimization problem. By the concept of so-called representative and extreme criteria, which is motivated by the concept of redundant (or nonessential) objective functions of Gal and Leberling, we can reduce the number of criteria without altering the set of efficient solutions. Furthermore, by using linear independent criteria, the linear multiple criteria optimization problem under consideration can be transformed into an equivalent linear multiple criteria optimization problem in the space of linear independent criteria. This equivalence is understood in a sense that efficient solutions of each problem can be derived from efficient solutions of the other by some affine transformation. As a result, such criteria and dimension reduction techniques could help to increase the efficiency of existing algorithms and to develop new methods for handling global optimization problems arisen from multiple objective optimization.     

Computational bounds for local search in combinatorial optimization

The results related to finding local optima in combinatorial optimization are overviewed. The class of polynomial-time local search problems (class PLS) is considered. By analogy with Cook’s theorem, the existence of most complicated problems in this class is established. The number of steps in local descent algorithms is estimated in the worst and average cases. The local search determination of exact and approximate solutions with guaranteed error bounds is discussed.     

Application of graph-theoretic approaches to the random landscapes of the three-dimensional assignment problem

We study local optima of combinatorial optimization problems. We show that a local search algorithm can be represented as a digraph and apply recent results for spanning forests of a diagraph. We establish a correspondence between the number of local optima and the algebraic multiplicities of eigenvalues of digraph laplacians. We apply our finding to the three-dimensional assignment problem.     

A stopping rule for optimization using randomized search procedures

In many nonconvex programming problems, it is possible to locate local optima, but the global optimum may be difficult to determine. In such cases, a search procedure is often used, with random starting solutions, to find alternate local optima. This search can be terminated by a stopping rule, based upon Bayesian revised probability distributions, which determines the optimal number of iterations. The application of this rule to a resource allocation problem in project scheduling is illustrated.This work was supported in part by grants from the National Science Foundation and the Rochester Gas and Electric Corporation to the Massachusetts Institute of Technology.     

A Comparison of Global Optimization Methods for the Design of a High-speed Civil Transport

The conceptual design of aircraft often entails a large number of nonlinear constraints that result in a nonconvex feasible design space and multiple local optima. The design of the high-speed civil transport (HSCT) is used as an example of a highly complex conceptual design with 26 design variables and 68 constraints. This paper compares three global optimization techniques on the HSCT problem and two test problems containing thousands of local optima and noise: multistart local optimizations using either sequential quadratic programming (SQP) as implemented in the design optimization tools (DOT) program or Snyman's dynamic search method, and a modified form of Jones' DIRECT global optimization algorithm. SQP is a local optimizer, while Snyman's algorithm is capable of moving through shallow local minima. The modified DIRECT algorithm is a global search method based on Lipschitzian optimization that locates small promising regions of design space and then uses a local optimizer to converge to the optimum. DOT and the dynamic search algorithms proved to be superior for finding a single optimum masked by noise of trigonometric form. The modified DIRECT algorithm was found to be better for locating the global optimum of functions with many widely separated true local optima.     

Enhanced speciation in particle swarm optimization for multi-modal problems

In this paper, we present a novel multi-modal optimization algorithm for finding multiple local optima in objective function surfaces. We build from Species-based particle swarm optimization (SPSO) by using deterministic sampling to generate new particles during the optimization process, by implementing proximity-based speciation coupled with speciation of isolated particles, and by including “turbulence regions” around already found solutions to prevent unnecessary function evaluations. Instead of using error threshold values, the new algorithm uses the particle’s experience, geometric mean, and “exclusion factor” to detect local optima and stop the algorithm. The performance of each extension is assessed with leave-it-out tests, and the results are discussed. We use the new algorithm called Isolated-Speciation-based particle swarm optimization (ISPSO) and a benchmark algorithm called Niche particle swarm optimization (NichePSO) to solve a six-dimensional rainfall characterization problem for 192 rain gages across the United States. We show why it is important to find multiple local optima for solving this real-world complex problem by discussing its high multi-modality. Solutions found by both algorithms are compared, and we conclude that ISPSO is more reliable than NichePSO at finding optima with a significantly lower objective function value.     

Global Optimization of Econometric Functions

Estimating the values of the parameter estimates of econometric functions (maximum likelihood functions or nonlinear least squares functions) are often challenging global optimization problems. Determining the global optimum for these functions is necessary to understand economic behavior and to develop effective economic policies. These functions often have flat surfaces or surfaces characterized by many local optima. Classical deterministic optimization methods often do not yield successful results. For that reason, stochastic optimization methods are becoming widely used in econometrics. Selected stochastic methods are applied to two difficult econometric functions to determine if they might be useful in estimating the parameters of these functions.     

Statistical analysis of local search landscapes

This paper discusses the application of some statistical estimation tools in trying to understand the nature of the combinatorial landscapes induced by local search methods. One interesting property of a landscape is the number of optima that are present. In this paper we show that it is possible to compute a confidence interval on the number of independent local searches needed to find all optima. By extension, this also expresses the confidence that the global optimum has been found. In many cases, this confidence may be too low to be acceptable, but it is also possible to estimate the number of optima that exist. Theoretical analysis and empirical studies are discussed, which show that it may be possible to obtain a fairly accurate picture of this property of a combinatorial landscape. The approach is illustrated by analysis of an instance of the flowshop scheduling problem.     

Subset simulation for unconstrained global optimization

Global optimization problem is known to be challenging, for which it is difficult to have an algorithm that performs uniformly efficient for all problems. Stochastic optimization algorithms are suitable for these problems, which are inspired by natural phenomena, such as metal annealing, social behavior of animals, etc. In this paper, subset simulation, which is originally a reliability analysis method, is modified to solve unconstrained global optimization problems by introducing artificial probabilistic assumptions on design variables. The basic idea is to deal with the global optimization problems in the context of reliability analysis. By randomizing the design variables, the objective function maps the multi-dimensional design variable space into a one-dimensional random variable. Although the objective function itself may have many local optima, its cumulative distribution function has only one maximum at its tail, as it is a monotonic, non-decreasing, right-continuous function. It turns out that the searching process of optimal solution(s) of a global optimization problem is equivalent to exploring the process of the tail distribution in a reliability problem. The proposed algorithm is illustrated by two groups of benchmark test problems. The first group is carried out for parametric study and the second group focuses on the statistical performance.     

Global optimization approach to nonlinear optimal control

To determine the optimum in nonlinear optimal control problems, it is proposed to convert the continuous problems into a form suitable for nonlinear programming (NLP). Since the resulting finite-dimensional NLP problems can present multiple local optima, a global optimization approach is developed where random starting conditions are improved by using special line searches. The efficiency, speed, and reliability of the proposed approach is examined by using two examples.Financial support from the Natural Science and Engineering Research Council under Grant A-3515 as well as an Ontario Graduate Scholarship are gratefully acknowledged. All the computations were done with the facilities of the University of Toronto Computer Centre and the Ontario Centre for Large Scale Computations.     

Explicit solution of an optimization problem

A simply statedn-variable constrained optimization problem, useful as a test problem, is explicitly solved. It has a large number of easily described local optima.This work was supported (in part) by the Office of Naval Research under contract number N00014-71-C-0112.     

An algorithm for finding the global maximum of a multimodal,multivariate function

This algorithm for global optimization uses an arbitrary starting point, requires no derivatives, uses comparatively few function evaluations and is not side-tracked by nearby relative optima. The algorithm builds a gradually closer piecewise-differentiable approximation to the objective function. The computer program exhibits a (theoretically expected) strong tendency to cluster around relative optima close to the global. Results of testing with several standard functions are given.     

Analyzing the Complexity of Finding Good Neighborhood Functions for Local Search Algorithms

A drawback to using local search algorithms to address NP-hard discrete optimization problems is that many neighborhood functions have an exponential number of local optima that are not global optima (termed L-locals). A neighborhood function η is said to be stable if the number of L-locals is polynomial. A stable neighborhood function ensures that the number of L-locals does not grow too large as the instance size increases and results in improved performance for many local search algorithms. This paper studies the complexity of stable neighborhood functions for NP-hard discrete optimization problems by introducing neighborhood transformations. Neighborhood transformations between discrete optimization problems consist of a transformation of problem instances and a corresponding transformation of solutions that preserves the ordering imposed by the objective function values. In this paper, MAX Weighted Boolean SAT (MWBS), MAX Clause Weighted SAT (MCWS), and Zero-One Integer Programming (ZOIP) are shown to be NPO-complete with respect to neighborhood transformations. Therefore, if MWBS, MCWS, or ZOIP has a stable neighborhood function, then every problem in NPO has a stable neighborhood function. These results demonstrate the difficulty of finding effective neighborhood functions for NP-hard discrete optimization problems.This research is supported in part by the Air Force Office of Scientific Research (F49620-01-1-0007, FA9550-04-1-0110).     

New Modified Function Method for Global Optimization

In this paper, a class of global optimization problems is considered. Corresponding to each local minimizer obtained, we introduced a new modified function and construct a corresponding optimization subproblem with one constraint. Then, by applying a local search method to the one-constraint optimization subproblem and using the local minimizer as the starting point, we obtain a better local optimal solution. This process is continued iteratively. A termination rule is obtained which can serve as stopping criterion for the iterating process. To demonstrate the efficiency of the proposed approach, numerical examples are solved.This research was partially supported by the National Science Foundation of China, Grant 10271073.     

An improved particle swarm optimizer for parametric optimization of flexible satellite controller

Parametric optimization of flexible satellite controller is an essential for almost all modern satellites. Particle swarm algorithm is a global optimization algorithm but it suffers from two major shortcomings, that of, premature convergence and low searching accuracy. To solve these problems, this paper proposes an improved particle swarm optimization (IPSO) which substitute “poorly-fitted-particles” with a cross operation. Based on decision possibility, the cross operation can interchange local optima between three particles. Thereafter the swarm is split in two halves, and random number (s) get generated by crossing the dimension of particle from both halves. This produces a new swarm. Now the new swarm and old swarm are mixed, and based on relative fitness a half of the particles are selected for the next generation. As a result of the cross operation, IPSO can easily jump out of local optima, has improved searching accuracy and accelerates the convergence speed. Some test functions with different dimensions are used to analyze the performance of IPSO algorithm. Simulation results show that the IPSO has more advantages than standard PSO and Genetic Algorithm PSO (GAPSO). In that it has a more stable performance and lower level of complexity. Thus the IPSO is applied for parametric optimization of flexible satellite control, for a satellite having solar wings and antennae. Simulation results shows that the IPSO can effectively get the best controller parameters vis-a-vis the other optimization methods.     

A magnetic resonance device designed via global optimization techniques

In this paper we are concerned with the design of a small low-cost, low-field multipolar magnet for Magnetic Resonance Imaging with a high field uniformity. By introducing appropriate variables, the considered design problem is converted into a global optimization one. This latter problem is solved by means of a new derivative free global optimization method which is a distributed multi-start type algorithm controlled by means of a simulated annealing criterion. In particular, the proposed method employs, as local search engine, a derivative free procedure. Under reasonable assumptions, we prove that this local algorithm is attracted by global minimum points. Additionally, we show that the simulated annealing strategy is able to produce a suitable starting point in a finite number of steps with probability one.This work was supported by CNR/MIUR Research Program Metodi e sistemi di supporto alle decisioni, Rome, Italy.Mathematics Subject Classification (1991):65K05, 62K05, 90C56     

A differential equation approach to nonlinear programming

A new method is presented for finding a local optimum of the equality constrained nonlinear programming problem. A nonlinear autonomous system is introduced as the base of the theory instead of usual approaches. The relation between critical points and local optima of the original optimization problem is proved. Asymptotic stability of the critical points is also proved. A numerical algorithm which is capable of finding local optima systematically at the quadratic rate of convergence is developed from a detailed analysis of the nature of trajectories and critical points. Some numerical results are given to show the efficiency of the method.     

Fast Global Optimization of Difficult Lennard-Jones Clusters

The minimization of the potential energy function of Lennard-Jones atomic clusters has attracted much theoretical as well as computational research in recent years. One reason for this is the practical importance of discovering low energy configurations of clusters of atoms, in view of applications and extensions to molecular conformation research; another reason of the success of Lennard Jones minimization in the global optimization literature is the fact that this is an extremely easy-to-state problem, yet it poses enormous difficulties for any unbiased global optimization algorithm.In this paper we propose a computational strategy which allowed us to rediscover most putative global optima known in the literature for clusters of up to 80 atoms and for other larger clusters, including the most difficult cluster conformations. The main feature of the proposed approach is the definition of a special purpose local optimization procedure aimed at enlarging the region of attraction of the best atomic configurations. This effect is attained by performing first an optimization of a modified potential function and using the resulting local optimum as a starting point for local optimization of the Lennard Jones potential.Extensive numerical experimentation is presented and discussed, from which it can be immediately inferred that the approach presented in this paper is extremely efficient when applied to the most challenging cluster conformations. Some attempts have also been carried out on larger clusters, which resulted in the discovery of the difficult optimum for the 102 atom cluster and for the very recently discovered new putative optimum for the 98 atom cluster.     

Reverse Convex Programming Approach in the Space of Extreme Criteria for Optimization over Efficient Sets

The problem of minimizing a convex function over the difference of two convex sets is called ‘reverse convex program’. This is a typical problem in global optimization, in which local optima are in general different from global optima. Another typical example in global optimization is the optimization problem over the efficient set of a multiple criteria programming problem. In this article, we investigate some special cases of optimization problems over the efficient set, which can be transformed equivalently into reverse convex programs in the space of so-called extreme criteria of multiple criteria programming problems under consideration. A suitable algorithm of branch and bound type is then established for globally solving resulting problems. Preliminary computational results with the proposed algorithm are reported.     

Topology Optimization of Trusses—Random Cost Method Versus Evolutionary Algorithms

The recently proposed random cost method is applied to the topology optimization of trusses. Its performance is compared to previous genetic algorithm and evolution strategy simulations. Random cost turns out to be an optimization method with attractive features. In comparison to the genetic algorithm approach of Hajela, Lee and Lin, random cost turns out to be simpler and more efficient. Furthermore it is found that in contrast to evolution strategy, the random cost strategy's ability to find optima, is independent of the initial structure. This characteristic is related to the important capacity of escaping from local optima.