Feature extraction algorithms for constrained global optimization I. Mathematical foundation

Nonconvex mixed integer nonlinear programming problems arise quite frequently in engineering decision problems, in general, and in chemical process design synthesis and process scheduling applications, in particular. These problems are characterized by high dimensionality and multiple local optimal solutions. In this work, a novel approach is developed for determining the global optimum in nonlinear continuous and discrete domains. The mathematical foundations of the feature extraction algorithm are presented and the properties of the algorithm discussed in detail. The algorithm uses a partition and search strategy in which the problem domain is successively partitioned and a statistical approximation approach is used to characterize the objective function values and the constraint feasibility over a partition. Specifically, the general joint distribution function representing the objective function values is relaxed to a separable form and approximated using an expansion in terms of Bernstein functions. The coefficients of the expansion are determined by solving a small linear program. Feasibility is established by computing upper and lower bounds for the inequality constraint functions, while equality constraints are explicitly or numerically eliminated. Estimates of the volume averaged values of objective function and constraint feasibility are used to select efficient partitions for further investigation. These are refined successively so as to focus the search on the most promising decision regions. An alternative, constant resolution partitioning strategy is also developed using a suitably modified genetic search algorithm. Illustrative examples are used to demonstrate the key computational features of the method.     

New algorithms for constrained minimax optimization

A constrained minimax problem is converted to minimization of a sequence of unconstrained and continuously differentiable functions in a manner similar to Morrison's method for constrained optimization. One can thus apply any efficient gradient minimization technique to do the unconstrained minimization at each step of the sequence. Based on this approach, two algorithms are proposed, where the first one is simpler to program, and the second one is faster in general. To show the efficiency of the algorithms even for unconstrained problems, examples are taken to compare the two algorithms with recent methods in the literature. It is found that the second algorithm converges faster with respect to the other methods. Several constrained examples are also tried and the results are presented.     

Convergent stepsizes for constrained optimization algorithms

A fundamental problem in constrained nonlinear optimization algorithms is the design of a satisfactory stepsize strategy which converges to unity. In this paper, we discuss stepsize strategies for Newton or quasi-Newton algorithms which require the solution of quadratic optimization subproblems. Five stepsize strategies are considered for three different subproblems, and the conditions under which the stepsizes will converge to unity are established. It is shown that these conditions depend critically on the convergence of the Hessian approximations used in the algorithms. The stepsize strategies are constructed using basic principles from which the conditions to unit stepsizes follow. Numerical results are discussed in an Appendix.Paper presented to the XI Symposium on Mathematical Programming, Bonn, Germany, 1982.This work was completed while the author was visiting the European University in Florence where, in particular, Professors Fitoussi and Velupillai provided the opportunity for its completion. The author is grateful to Dr. L. C. W. Dixon for his helpful comments and criticisms on numerous versions of the paper, and to R. G. Becker for programming the algorithms in Section 3 and for helpful discussions concerning these algorithms.     

Parallel algorithms for global optimization

In this paper, we report results of implementations of algorithms designed (i) to solve the global optimization problem (GOP) and (ii) to run on a parallel network of transputers. There have always been two alternative approaches to the solution of the GOP, probabilistic and deterministic. Interval methods can be implemented on our network of transputers using Concurrent ADA, and a secondary objective of the tests reported was to investigate the relative computer times required by parallel interval algorithms compared to probabilistic methods.     

Interior-point algorithms for global optimization

We describe recent developments in interior-point algorithms for global optimization. We will focus on the algorithmic research for nonconvex quadratic programming, linear complementarity problem, and integer programming. We also outline directions in which future progress might be made.     

Two algorithms for global optimization

In this paper, we develop and compare two methods for solving the problem of determining the global maximum of a function over a feasible set. The two methods begin with a random sample of points over the feasible set. Both methods then seek to combine these points into “regions of attraction” which represent subsets of the points which will yield the same local maximums when an optimization procedure is applied to points in the subset. The first method for constructing regions of attraction is based on approximating the function by a mixture of normal distributions over the feasible region and the second involves attempts to apply cluster analysis to form regions of attraction. The two methods are then compared on a set of well-known test problems.     

Simulated annealing for constrained global optimization

Hide-and-Seek is a powerful yet simple and easily implemented continuous simulated annealing algorithm for finding the maximum of a continuous function over an arbitrary closed, bounded and full-dimensional body. The function may be nondifferentiable and the feasible region may be nonconvex or even disconnected. The algorithm begins with any feasible interior point. In each iteration it generates a candidate successor point by generating a uniformly distributed point along a direction chosen at random from the current iteration point. In contrast to the discrete case, a single step of this algorithm may generateany point in the feasible region as a candidate point. The candidate point is then accepted as the next iteration point according to the Metropolis criterion parametrized by anadaptive cooling schedule. Again in contrast to discrete simulated annealing, the sequence of iteration points converges in probability to a global optimum regardless of how rapidly the temperatures converge to zero. Empirical comparisons with other algorithms suggest competitive performance by Hide-and-Seek.This material is based on work supported by a NATO Collaborative Research Grant, no. 0119/89.     

Genetic algorithms in constrained optimization

The behavior of the two-point crossover operator, on candidate solutions to an optimization problem that is restricted to integer values and by some set of constraints, is investigated theoretically. This leads to the development of new genetic operators for the case in which the constraint system is linear.The computational difficulty asserted by many optimization problems has lead to exploration of a class of randomized algorithms based on biological adaption. The considerable interest that surrounds these evolutionary algorithms is largely centered on problems that have defied satisfactory illation by traditional means because of badly behaved or noisy objective functions, high dimensionality, or intractable algorithmic complexity. Under such conditions, these alternative methods have often proved invaluable.Despite their attraction, the applicability of evolutionary algorithms has been limited by a deficiency of general techniques to manage constraints, and the difficulty is compounded when the decision variables are discrete. Several new genetic operators are presented here that are guaranteed to preserve the feasibility of discrete aspirant solutions with respect to a system of linear constraints.To avoid performance degradation as the probability of finding a feasible and meaningful information exchange between two candidate solutions decreases, relaxations of the modified genetic crossover operator are also proposed. The effective utilization of these also suggests a manipulation of the genetic algorithm itself, in which the population is evanescently permitted to grow beyond its normal size.     

An interval algorithm for constrained global optimization

An interval algorithm for bounding the solutions of a constrained global optimization problem is described. The problem functions are assumed only to be continuous. It is shown how the computational cost of bounding a set which satisfies equality constraints can often be reduced if the equality constraint functions are assumed to be continuously differentiable. Numerical results are presented.     

New interval methods for constrained global optimization

Interval analysis is a powerful tool which allows to design branch-and-bound algorithms able to solve many global optimization problems. In this paper we present new adaptive multisection rules which enable the algorithm to choose the proper multisection type depending on simple heuristic decision rules. Moreover, for the selection of the next box to be subdivided, we investigate new criteria. Both the adaptive multisection and the subinterval selection rules seem to be specially suitable for being used in inequality constrained global optimization problems. The usefulness of these new techniques is shown by computational studies.     

Efficient interval partitioning for constrained global optimization

A new efficient interval partitioning approach to solve constrained global optimization problems is proposed. This involves a new parallel subdivision direction selection method as well as an adaptive tree search. The latter explores nodes (intervals in variable domains) using a restricted hybrid depth-first and best-first branching strategy. This hybrid approach is also used for activating local search to identify feasible stationary points. The new tree search management technique results in improved performance across standard solution and computational indicators when compared to previously proposed techniques. On the other hand, the new parallel subdivision direction selection rule detects infeasible and suboptimal boxes earlier than existing rules, and this contributes to performance by enabling earlier reliable deletion of such subintervals from the search space.     

优先序约束的排序问题：基于最大匹配的近似算法

本文研究具有加工次序约束的单位工件开放作业和流水作业排序问题,目标函数为极小化工件最大完工时间。工件之间的加工次序约束关系可以用一个被称为优先图的有向无圈图来刻画。当机器数作为输入时,两类问题在一般优先图上都是强NP-困难的,而在入树的优先图上都是可解的。我们利用工件之间的许可对数获得了问题的新下界,并基于许可工件之间的最大匹配设计近似算法,其中匹配的许可工件对均能同时在不同机器上加工。对于一般优先图的开放作业问题和脊柱型优先图的流水作业问题,我们在理论上证明了算法的近似比为$2-frac 2m$,其中$m$是机器数目。     

优先序约束的排序问题：基于最大匹配的近似算法

本文研究具有加工次序约束的单位工件开放作业和流水作业排序问题,目标函数为极小化工件最大完工时间。工件之间的加工次序约束关系可以用一个被称为优先图的有向无圈图来刻画。当机器数作为输入时,两类问题在一般优先图上都是强NP-困难的,而在入树的优先图上都是可解的。我们利用工件之间的许可对数获得了问题的新下界,并基于许可工件之间的最大匹配设计近似算法,其中匹配的许可工件对均能同时在不同机器上加工。对于一般优先图的开放作业问题和脊柱型优先图的流水作业问题,我们在理论上证明了算法的近似比为$2-\frac 2m$,其中$m$是机器数目。     

Direct solution to constrained tropical optimization problems with application to project scheduling

We examine a new optimization problem formulated in the tropical mathematics setting as a further extension of certain known problems. The problem is to minimize a nonlinear objective function, which is defined on vectors over an idempotent semifield by using multiplicative conjugate transposition, subject to inequality constraints. As compared to the known problems, the new one has a more general objective function and additional constraints. We provide a complete solution in an explicit form to the problem by using an approach that introduces an auxiliary variable to represent the values of the objective function, and then reduces the initial problem to a parametrized vector inequality. The minimum of the objective function is evaluated by applying the existence conditions for the solution of this inequality. A complete solution to the problem is given by solving the parametrized inequality, provided the parameter is set to the minimum value. As a consequence, we obtain solutions to new special cases of the general problem. To illustrate the application of the results, we solve a real-world problem drawn from time-constrained project scheduling, and offer a representative numerical example.     

Geometric branch-and-bound methods for constrained global optimization problems

Geometric branch-and-bound methods are popular solution algorithms in deterministic global optimization to solve problems in small dimensions. The aim of this paper is to formulate a geometric branch-and-bound method for constrained global optimization problems which allows the use of arbitrary bounding operations. In particular, our main goal is to prove the convergence of the suggested method using the concept of the rate of convergence in geometric branch-and-bound methods as introduced in some recent publications. Furthermore, some efficient further discarding tests using necessary conditions for optimality are derived and illustrated numerically on an obnoxious facility location problem.     

Potential transformation methods for large-scale constrained global optimization

Several techniques for global optimization treat the objective functionf as a force-field potential. In the simplest case, trajectories of the differential equationmx=–f sample regions of low potential while retaining the energy to surmount passes which might block the way to regions of even lower local minima. Apotential transformation is an increasing functionV:. It determines a new potentialg=V(f), with the same minimizers asf and new trajectories satisfying . We discuss a class of potential transformations that greatly increase the attractiveness of low local minima.These methods can be applied to constrained problems through the use of Lagrange multipliers. We discuss several methods for efficiently computing approximate Lagrange multipliers, making this approach practical.     

A filled function method for constrained global optimization

In this paper, a filled function method for solving constrained global optimization problems is proposed. A filled function is proposed for escaping the current local minimizer of a constrained global optimization problem by combining the idea of filled function in unconstrained global optimization and the idea of penalty function in constrained optimization. Then a filled function method for obtaining a global minimizer or an approximate global minimizer of the constrained global optimization problem is presented. Some numerical results demonstrate the efficiency of this global optimization method for solving constrained global optimization problems.