Theoretical and numerical comparison of relaxation methods for mathematical programs with complementarity constraints

Mathematical programs with equilibrium constraints (MPECs) are difficult optimization problems whose feasible sets do not satisfy most of the standard constraint qualifications. Hence MPECs cause difficulties both from a theoretical and a numerical point of view. As a consequence, a number of MPEC-tailored solution methods have been suggested during the last decade which are known to converge under suitable assumptions. Among these MPEC-tailored solution schemes, the relaxation methods are certainly one of the most prominent class of solution methods. Several different relaxation schemes are available in the meantime, and the aim of this paper is to provide a theoretical and numerical comparison of these schemes. More precisely, in the theoretical part, we improve the convergence theorems of several existing relaxation methods. There, we also take a closer look at the properties of the feasible sets of the relaxed problems and show which standard constraint qualifications are satisfied for these relaxed problems. Finally, the numerical comparison is based on the MacMPEC test problem collection.     

Generating Functions in the Knapsack Problem

The knapsack problem with Boolean variables and a single constraint is considered. Combinatorial formulas for calculating and estimating the cardinality of the set of feasible solutions and the values of the functional in various cases depending on given parameters of the problem are derived. The coefficients of the objective function and of the constraint vector and the knapsack size are used as parameters. The baseline technique is the classical method of generating functions. The results obtained can be used to estimate the complexity of enumeration and decomposition methods for solving the problem and can also be used as auxiliary procedures in developing such methods.     

Constraint classification in mathematical programming

Consider a set of algebraic inequality constraints defining either an empty or a nonempty feasible region. It is known that each constraint can be classified as either absolutely strongly redundant, relatively strongly redundant, absolutely weakly redundant, relatively weakly redundant, or necessary. We show that is is worth making a distinction between weakly necessary constraints and strongly necessary constraints. We also present afeasible set cover method which can detect both weakly and strongly necessary constraints.The main interest in constraint classification is due to the advantages gained by the removal of redundant constraints. Since classification errors are likely to occur, we examine how the removal of a single constraint can affect the classification of the remaining constraints.     

Differential evolution with multi-constraint consensus methods for constrained optimization

Constrained optimization is an important research topic that assists in quality planning and decision making. To solve such problems, one of the important aspects is to improve upon any constraint violation, and thus bring infeasible individuals to the feasible region. To achieve this goal, different constraint consensus methods have been introduced, but no single method performs well for all types of problems. Hence, in this research, for solving constrained optimization problems, we introduce different variants of the Differential Evolution algorithm, with multiple constraint consensus methods. The proposed algorithms are tested and analyzed by solving a set of well-known bench mark problems. For further improvements, a local search is applied to the best variant. We have compared our algorithms among themselves, as well as with other state of the art algorithms. Those comparisons show similar, if not better performance, while also using significantly lower computational time.     

Asymptotic optimality conditions for linear semi-infinite programming

In this paper, the classical KKT, complementarity and Lagrangian saddle-point conditions are generalized to obtain equivalent conditions characterizing the optimality of a feasible solution to a general linear semi-infinite programming problem without constraint qualifications. The method of this paper differs from the usual convex analysis methods and its main idea is rooted in some fundamental properties of linear programming.     

Failure of global convergence for a class of interior point methods for nonlinear programming

Using a simple analytical example, we demonstrate that a class of interior point methods for general nonlinear programming, including some current methods, is not globally convergent. It is shown that those algorithms produce limit points that are neither feasible nor stationary points of some measure of the constraint violation, when applied to a well-posed problem. Received: December 1999 / Accepted: May 2000?Published online August 18, 2000     

Some heuristic methods for solving p-median problems with a coverage constraint

The aim of this paper is to solve p-median problems with an additional coverage constraint. These problems arise in location applications, when the trade-off between distance and coverage is being calculated. Three kinds of heuristic algorithms are developed. First, local search procedures are designed both for constructing and improving feasible solutions. Second, a multistart GRASP heuristic is developed, based on the previous local search methods. Third, by employing Lagrangean relaxation methods, a very efficient Lagrangean heuristic algorithm is designed, which extends the well known algorithm of Handler and Zang, for constrained shortest path problems, to constrained p-median problems. Finally, a comparison of the computational efficiency of the developed methods is made between a variety of problems of different sizes.     

On the Relation between Constant Positive Linear Dependence Condition and Quasinormality Constraint Qualification

The constant positive linear dependence (CPLD) condition for feasible points of nonlinear programming problems was introduced by Qi and Wei (Ref. 1) and used in the analysis of SQP methods. In that paper, the authors conjectured that the CPLD could be a constraint qualification. This conjecture is proven in the present paper. Moreover, it is shown that the CPLD condition implies the quasinormality constraint qualification, but that the reciprocal is not true. Relations with other constraint qualifications are given.This research has been supported by PRONEX-Optimization Grant 76.79.1008-00, by FAPESP Grants 01-04597-4 and 02-00832-1, and by CNPq. The authors are indebted to two anonymous referees for useful comments and to Prof. Liqun Qi for encouragement.     

Global convergence of modified augmented Lagrangian methods for nonlinear semidefinite programming

We investigate in this paper global convergence properties of the augmented Lagrangian method for nonlinear semidefinite programming (NLSDP). Four modified augmented Lagrangian methods for solving NLSDP based on different algorithmic strategies are proposed. Possibly infeasible limit points of the proposed methods are characterized. It is proved that feasible limit points that satisfy the Mangasarian-Fromovitz constraint qualification are KKT points of NLSDP without requiring the boundedness condition of the multipliers. Preliminary numerical results are reported to compare the performance of the modified augmented Lagrangian methods.     

Efficient and Adaptive Lagrange-Multiplier Methods for Nonlinear Continuous Global Optimization

Lagrangian methods are popular in solving continuous constrained optimization problems. In this paper, we address three important issues in applying Lagrangian methods to solve optimization problems with inequality constraints.First, we study methods to transform inequality constraints into equality constraints. An existing method, called the slack-variable method, adds a slack variable to each inequality constraint in order to transform it into an equality constraint. Its disadvantage is that when the search trajectory is inside a feasible region, some satisfied constraints may still pose some effect on the Lagrangian function, leading to possible oscillations and divergence when a local minimum lies on the boundary of the feasible region. To overcome this problem, we propose the MaxQ method that carries no effect on satisfied constraints. Hence, minimizing the Lagrangian function in a feasible region always leads to a local minimum of the objective function. We also study some strategies to speed up its convergence.Second, we study methods to improve the convergence speed of Lagrangian methods without affecting the solution quality. This is done by an adaptive-control strategy that dynamically adjusts the relative weights between the objective and the Lagrangian part, leading to better balance between the two and faster convergence.Third, we study a trace-based method to pull the search trajectory from one saddle point to another in a continuous fashion without restarts. This overcomes one of the problems in existing Lagrangian methods that converges only to one saddle point and requires random restarts to look for new saddle points, often missing good saddle points in the vicinity of saddle points already found.Finally, we describe a prototype Novel (Nonlinear Optimization via External Lead) that implements our proposed strategies and present improved solutions in solving a collection of benchmarks.     

Generalized semi-infinite programming: A tutorial

This tutorial presents an introduction to generalized semi-infinite programming (GSIP) which in recent years became a vivid field of active research in mathematical programming. A GSIP problem is characterized by an infinite number of inequality constraints, and the corresponding index set depends additionally on the decision variables. There exist a wide range of applications which give rise to GSIP models; some of them are discussed in the present paper. Furthermore, geometric and topological properties of the feasible set and, in particular, the difference to the standard semi-infinite case are analyzed. By using first-order approximations of the feasible set corresponding constraint qualifications are developed. Then, necessary and sufficient first- and second-order optimality conditions are presented where directional differentiability properties of the optimal value function of the so-called lower level problem are used. Finally, an overview of numerical methods is given.     

两个约束条件下允许外购的多产品报童问题

研究在资源和预算约束条件下允许外购的多产品报童问题.为解决因资源有限而引起的缺货问题,允许外购,外购的最大可得到量是给定的.建立了在非零延迟时间外购情况下以求总体花费最小值为目标函数的的优化模型,分析了模型的结构特征,结合最优化理论,利用可行方向方法对模型进行了求解.数值结果表明了模型的合理性和算法的有效性.     

A regularization of the Frank—Wolfe method and unification of certain nonlinear programming methods

The Frank—Wolfe linearization technique is a popular feasible direction algorithm for the solution of certain linearly constrained nonlinear problems. The popularity of this technique is due in part to its ability to exploit special constraint structures, such as network structures, and in part to the fact that it decomposes nonseparable problems over Cartesian product sets. However, the linearization which induces these advantages is also the source of the main disadvantages of the method: a sublinear rate of convergence and zigzagging behaviour. In order to avoid these disadvantages, a regularization penalty term is added to the objective of the direction generating subproblem. This results in a generic feasible direction method which also includes certain known nonlinear programming methods.     

Linear programs with an additional reverse convex constraint

A constraintg(x)0 is said to be a reverse convex constraint if the functiong is continuous and strictly quasi-convex. The feasible regions for linear programs with an additional reverse convex constraint are generally non-convex and disconnected. It is shown that the convex hull of the feasible region is a convex polytope and, as a result, there is an optimal solution on an edge of the polytope defined by only the linear constraints. The only possible edges which can contain such an optimal solution are characterized in relation to the best feasible vertex of the polytope defined by only the linear constraints. This characterization then provides a finite algorithm for finding a globally optimal solution.Research supported by NSF Grant ENG76-12250 and ONR Contract N00014-78-C-0428.     

A Smoothing Method for Zero–One Constrained Extremum Problems

In this paper, the zero–one constrained extremum problem is reformulated as an equivalent smooth mathematical program with complementarity constraints (MPCC), and then as a smooth ordinary nonlinear programming problem with the help of the Fischer–Burmeister function. The augmented Lagrangian method is adopted to solve the resulting problem, during which the non-smoothness may be introduced as a consequence of the possible inequality constraints. This paper incorporates the aggregate constraint method to construct a uniform smooth approximation to the original constraint set, with approximation controlled by only one parameter. Convergence results are established, showing that under reasonable conditions the limit point of the sequence of stationary points generated by the algorithm is a strongly stationary point of the original problem and satisfies the second order necessary conditions of the original problem. Unlike other penalty type methods for MPCC, the proposed algorithm can guarantee that the limit point of the sequence is feasible to the original problem.     

Stability in disjunctive linear optimization I: continuity of the feasible set

The aim of this paper is to study the continuous dependence of the feasible set of a disjunctive semi-infinite linear optimization problem on all involved parameters (matrix and right-hand side). The feasible set of such an optimization problem is the union of (a. possible infinite number of) convex sets, which each is described by a finite or an infinite number of strict and non-strict linear inequalities. We derive necessary and sufficient conditions for the upper- and lower-semi-continuity, and the closedness of the feasible-set-mapping Z Especially, the compactness of the boundary of the feasible set and the closedness of Z are equivalent to the upper-semi-continuity of Zwhile the lower semi-continuity of Z is equivalent to a certain constraint qualification. This constraint qualification is a strengthened kind of Slater condition, rrom tuese investigations, we derive known results in parametric semi-infinite optimization and parametric integer programming.     

On redundancy in multi criteria decision making

The concept of redundancy is accepted in Operations Research and Information Theory. In Linear Programming, a constraint is said to be redundant if the feasible decision space is identical with or without the constraint. In Information Theory, redundancy is used as a measure of the stability against noise in transmission. Analogies with Multi Criteria Decision Making (MCDM) are indicated and it is argued that the redundancy concept should be used as a regular feature in conditioning and analysis of Multi Criteria Programs. Properties of a proposed conflict-based characterisation are stated and some existence results are derived. Redundancy is here intended for interactive methods, when the efficient set is progressively explored. A new redundancy test for the linear case is formulated from the framework. A probabilistic method based on correlation is proposed and tested for the non-linear case. Finally, some general guidelines are given concerning the redundancy problem.     

A Kriging-based constrained global optimization algorithm for expensive black-box functions with infeasible initial points

In many engineering optimization problems, the objective and the constraints which come from complex analytical models are often black-box functions with extensive computational effort. In this case, it is necessary for optimization process to use sampling data to fit surrogate models so as to reduce the number of objective and constraint evaluations as soon as possible. In addition, it is sometimes difficult for the constrained optimization problems based on surrogate models to find a feasible point, which is the premise of further searching for a global optimal feasible solution. For this purpose, a new Kriging-based Constrained Global Optimization (KCGO) algorithm is proposed. Unlike previous Kriging-based methods, this algorithm can dispose black-box constrained optimization problem even if all initial sampling points are infeasible. There are two pivotal phases in KCGO algorithm. The main task of the first phase is to find a feasible point when there is no feasible data in the initial sample. And the aim of the second phase is to obtain a better feasible point under the circumstances of fewer expensive function evaluations. Several numerical problems and three design problems are tested to illustrate the feasibility, stability and effectiveness of the proposed method.