Reduction of indefinite quadratic programs to bilinear programs

Indefinite quadratic programs with quadratic constraints can be reduced to bilinear programs with bilinear constraints by duplication of variables. Such reductions are studied in which: (i) the number of additional variables is minimum or (ii) the number of complicating variables, i.e., variables to be fixed in order to obtain a linear program, in the resulting bilinear program is minimum. These two problems are shown to be equivalent to a maximum bipartite subgraph and a maximum stable set problem respectively in a graph associated with the quadratic program. Non-polynomial but practically efficient algorithms for both reductions are thus obtaine.d Reduction of more general global optimization problems than quadratic programs to bilinear programs is also briefly discussed.     

Mixed-integer quadratic programming

This paper considers mixed-integer quadratic programs in which the objective function is quadratic in the integer and in the continuous variables, and the constraints are linear in the variables of both types. The generalized Benders' decomposition is a suitable approach for solving such programs. However, the program does not become more tractable if this method is used, since Benders' cuts are quadratic in the integer variables. A new equivalent formulation that renders the program tractable is developed, under which the dual objective function is linear in the integer variables and the dual constraint set is independent of these variables. Benders' cuts that are derived from the new formulation are linear in the integer variables, and the original problem is decomposed into a series of integer linear master problems and standard quadratic subproblems. The new formulation does not introduce new primary variables or new constraints into the computational steps of the decomposition algorithm.The author wishes to thank two anonymous referees for their helpful comments and suggestions for revising the paper.     

A Benders decomposition method for discretely-constrained mathematical programs with equilibrium constraints

We present a new methodology to solve discretely-constrained mathematical programs with equilibrium constraints (DC-MPECs). Typically these problems include an upper planning-level optimization with some discrete decision variables (eg, build/don’t build) as well as a lower operations-level problem often described by an optimization or nonlinear complementarity problem. This lower-level problem may also include some discrete variables. MPECs are very challenging problems to solve and the inclusion of integrality constraints makes this class of problems even more computationally difficult. We develop a new variant of the Benders algorithm combined with a heuristic procedure that decomposes the domain of the upper-level discrete variables to solve the resulting DC-MPECs. We provide convergence theory as well as a number of numerical examples, some derived from energy applications, to validate the new method. It should be noted that the convergence theory applies if the heuristic procedure correctly identifies a decomposition of the domain so that the lower-level problem's optimal value function is convex. This is challenging but our numerical results are positive.     

A multiobjective optimization model for processing and distributing biosolids to reuse fields

Advanced wastewater treatment plants remove deleterious nutrients, chemicals, and microorganisms from wastewater and produce biosolids products to be reused at farms and other sites. These biosolids are carefully regulated by environmental restrictions but still may be malodorous to the local populations. In this paper, we develop a multiobjective optimization model to simultaneously minimize the biosolids odours as well as processing and distribution costs. The model employs a linear odour function and bilinear costs; the latter being approximated via Schur's decomposition and special ordered set (SOS) type 2 variables resulting in a mixed integer linear multiobjective optimization problem. Such a model can be used proactively by these plants to produce the least malodorous product at minimal costs. We demonstrate use of the model with a case study for the Blue Plains advanced wastewater treatment plant run by the DC Water and Sewer Authority in Washington, DC.     

Lower bounds for nonlinear assignment problems using many body interactions

This paper concerns lower bounding techniques for the general α-adic assignment problem. The nonlinear objective function is linearized by the introduction of additional variables and constraints, thus yielding a mixed integer linear programming formulation of the problem. The concept of many body interactions is introduced to strengthen this formulation and incorporated in a modified formulation obtained by lifting the original representation to a higher dimensional space. This process involves two steps — (i) addition of new variables and constraints and (ii) incorporation of the new variables in the objective function. If this lifting process is repeated β times on an α-adic assignment problem along with the incorporation of higher order interactions, it results in the mixed-integer formulation of an equivalent (α + β)-adic assignment problem. The incorporation of many body interactions in the higher dimensional formulation improves its degeneracy properties and is also critical to the derivation of decomposition methods for the solution of these large scale mathematical programs in the higher dimensional space. It is shown that a lower bound to the optimal solution of the corresponding linear programming relaxation can be obtained by dualizing a subset of constraints in this formulation and solving O(N^2(α+β−1)) linear assignment problems, whose coefficients depend on the dual values. Moreover, it is proved that the optimal solution to the LP relaxation is obtained if we use the optimal duals for the solution of the linear assignment problems. This concept of many body interactions could be applied in designing algorithms for the solution of formulations obtained by lifting general MILP's. We illustrate all these concepts on the quadratic assignment problems With these decomposition bounds, we have found the provably optimal solutions of two unsolved QAP's of size 32 and have also improved upon existing lower bounds for other QAP's.     

Branch-and-cut for linear programs with overlapping SOS1 constraints

SOS1 constraints require that at most one of a given set of variables is nonzero. In this article, we investigate a branch-and-cut algorithm to solve linear programs with SOS1 constraints. We focus on the case in which the SOS1 constraints overlap. The corresponding conflict graph can algorithmically be exploited, for instance, for improved branching rules, preprocessing, primal heuristics, and cutting planes. In an extensive computational study, we evaluate the components of our implementation on instances for three different applications. We also demonstrate the effectiveness of this approach by comparing it to the solution of a mixed-integer programming formulation, if the variables appearing in SOS1 constraints ar bounded.     

A note on reduction of quadratic and bilinear programs with equality constraints

Reduction of some classes of global optimization programs to bilinear programs may be done in various ways, and the choice of method clearly influences the ease of solution of the resulting problem. In this note we show how linear equality constraints may be used together with graph theoretic tools to reduce a bilinear program, i.e., eliminate variables from quadratic terms to minimize the number of complicating variables. The method is illustrated on an example. Computer results are reported on known test problems.     

A Lagrangian decomposition approach to computing feasible solutions for quadratic binary programs

In this paper, we develop a Lagrangian decomposition based heuristic method for general quadratic binary programs (QBPs) with linear constraints. We extend the idea of Lagrangian decomposition by Chardaire and Sutter (Manag Sci 41(4):704–712, 1995) and Billionnet and Soutif (Eur J Oper Res 157(3):565–575, 2004a, Inf J Comput 16(2):188–197, 2004b) in which the quadratic objective is converted to a bilinear function by introducing auxiliary variables to duplicate the original complicating variables in the problem. Instead of using linear constraints to assure the equity between the two types of decision variables, we introduce generalized quadratic constraints and relax them with Lagrangian multipliers. Instead of computing an upper bound for a maximization problem, we focus on lower bounding with Lagrangian decomposition based heuristic. We take advantage of the decomposability presented in the Lagrangian subproblems to speed up the heuristic and identify one feasible solution at each iteration of the subgradient optimization procedure. With numerical studies on several classes of representative QBPs, we investigate the sensitivity of lower-bounding performance on parameters of the additional quadratic constraints. We also demonstrate the potentially improved quality of preprocessing in comparison with the use of a QBP solver.     

Decomposition approach for the global minimization of biconcave functions over polytopes

A decomposition approach is proposed for minimizing biconcave functions over polytopes. Important special cases include concave minimization, bilinear and indefinite quadratic programming for which new algorithms result. The approach introduces a new polyhedral partition and combines branch-and-bound techniques, outer approximation, and projection of polytopes in a suitable way.The authors are indebted to two anonymous reviewers for suggestions which have considerably improved this article.     

Asymmetry in the Converse of Schur's Lemma

We show that the converse of Schur's Lemma can hold in the category of right modules, but not the category of left modules, over an appropriate ring. We exhibit classes of rings over which this left-right asymmetry does not occur, and provide new constructions of rings over whose module categories the converse of Schur's Lemma holds. We propose various open problems and avenues for further research concomitant to our work.     

Univariate parameterization for global optimization of mixed-integer polynomial problems

This paper presents a new relaxation technique to globally optimize mixed-integer polynomial programming problems that arise in many engineering and management contexts. Using a bilinear term as the basic building block, the underlying idea involves the discretization of one of the variables up to a chosen accuracy level (Teles, J.P., Castro, P.M., Matos, H.A. (2013). Multiparametric disaggregation technique for global optimization of polynomial programming problems. J. Glob. Optim. 55, 227–251), by means of a radix-based numeric representation system, coupled with a residual variable to effectively make its domain continuous. Binary variables are added to the formulation to choose the appropriate digit for each position together with new sets of continuous variables and constraints leading to the transformation of the original mixed-integer non-linear problem into a larger one of the mixed-integer linear programming type. The new underestimation approach can be made as tight as desired and is shown capable of providing considerably better lower bounds than a widely used global optimization solver for a specific class of design problems involving bilinear terms.     

A stable approach to Newton's method for general mathematical programming problems inR n

The usual approach to Newton's method for mathematical programming problems with equality constraints leads to the solution of linear systems ofn +m equations inn +m unknowns, wheren is the dimension of the space andm is the number of constraints. Moreover, these linear systems are never positive definite. It is our feeling that this approach is somewhat artificial, since in the unconstrained case the linear systems are very often positive definite. With this in mind, we present an alternate Newton-like approach for the constrained problem in which all the linear systems are of order less than or equal ton. Furthermore, when the Hessian of the Lagrangian at the solution is positive definite (a situation frequently occurring), all our systems will be positive definite. Hence, in all cases, our Newton-like method offers greater numerical stability. We demonstrate that the convergence properties of this Newton-like method are superior to those of the standard approach to Newton's method. The operation count for the new method using Gaussian elimination is of the same order as the operation count for the standard method. However, if the Hessian of the Lagrangian at the solution is positive definite and we use Cholesky decomposition, then the order of the operation count for the new method is half that for the standard approach to Newton's method. This theory is generalized to problems with both equality and inequality constraints.     

Implementations of special ordered sets in MP software

Special ordered sets (SOS) have been introduced as a practical device for efficiently handling special classes of nonconvex optimization problems. They are now implemented in most commercial codes for mathematical programming (MP software). The paper gives a survey of possible applications as multiple choice restrictions, conditional multiple choice restrictions, discrete variables, discontinuous variables and piecewise linear functions, global optimization of separable programming problems, alternative right-hand sides, overlapping special ordered sets and the solution of quadratic programming problems. Alternative problem formulations are discussed. Since special ordered sets are not defined uniquely modelling facilities depend on the definition of a special orderedset in a code. The paper demonstrates the superiority of SOS to the application of binary variables if they are treated judiciously.     

Thevenin decomposition and large-scale optimization

The Thevenin theorem, one of the most celebrated results of electric circuit theory, provides a two-parameter characterization of the behavior of an arbitrarily large circuit, as seen from two of its terminals. We interpret the theorem as a sensitivity result in an associated minimum energy/network flow problem, and we abstract its main idea to develop a decomposition method for convex quadratic programming problems with linear equality constraints, of the type arising in a variety of contexts such as the Newton method, interior point methods, and least squares estimation. Like the Thevenin theorem, our method is particularly useful in problems involving a system consisting of several subssystems, connected to each other with a small number of coupling variables.This research was supported by NSF under Grant CCR-91-03804.     

Modeling disjunctive constraints with a logarithmic number of binary variables and constraints

Many combinatorial constraints over continuous variables such as SOS1 and SOS2 constraints can be interpreted as disjunctive constraints that restrict the variables to lie in the union of a finite number of specially structured polyhedra. Known mixed integer binary formulations for these constraints have a number of binary variables and extra constraints linear in the number of polyhedra. We give sufficient conditions for constructing formulations for these constraints with a number of binary variables and extra constraints logarithmic in the number of polyhedra. Using these conditions we introduce mixed integer binary formulations for SOS1 and SOS2 constraints that have a number of binary variables and extra constraints logarithmic in the number of continuous variables. We also introduce the first mixed integer binary formulations for piecewise linear functions of one and two variables that use a number of binary variables and extra constraints logarithmic in the number of linear pieces of the functions. We prove that the new formulations for piecewise linear functions have favorable tightness properties and present computational results showing that they can significantly outperform other mixed integer binary formulations.     

A semismooth sequential quadratic programming method for lifted mathematical programs with vanishing constraints

Mathematical programs with vanishing constraints are a difficult class of optimization problems with important applications to optimal topology design problems of mechanical structures. Recently, they have attracted increasingly more attention of experts. The basic difficulty in the analysis and numerical solution of such problems is that their constraints are usually nonregular at the solution. In this paper, a new approach to the numerical solution of these problems is proposed. It is based on their reduction to the so-called lifted mathematical programs with conventional equality and inequality constraints. Special versions of the sequential quadratic programming method are proposed for solving lifted problems. Preliminary numerical results indicate the competitiveness of this approach.     

Adaptive constraint reduction for convex quadratic programming

We propose an adaptive, constraint-reduced, primal-dual interior-point algorithm for convex quadratic programming with many more inequality constraints than variables. We reduce the computational effort by assembling, instead of the exact normal-equation matrix, an approximate matrix from a well chosen index set which includes indices of constraints that seem to be most critical. Starting with a large portion of the constraints, our proposed scheme excludes more unnecessary constraints at later iterations. We provide proofs for the global convergence and the quadratic local convergence rate of an affine-scaling variant. Numerical experiments on random problems, on a data-fitting problem, and on a problem in array pattern synthesis show the effectiveness of the constraint reduction in decreasing the time per iteration without significantly affecting the number of iterations. We note that a similar constraint-reduction approach can be applied to algorithms of Mehrotra’s predictor-corrector type, although no convergence theory is supplied.     

Fast algorithm for singly linearly constrained quadratic programs with box-like constraints

This paper focuses on a singly linearly constrained class of convex quadratic programs with box-like constraints. We propose a new fast algorithm based on parametric approach and secant approximation method to solve this class of quadratic problems. We design efficient implementations for our proposed algorithm and compare its performance with two state-of-the-art standard solvers called Gurobi and Mosek. Numerical results on a variety of test problems demonstrate that our algorithm is able to efficiently solve the large-scale problems with the dimension up to fifty million and it substantially outperforms Gurobi and Mosek in terms of the running time.     

Network planning under uncertainty with an application to hydropower generation

Summary We present a general modeling framework for therobust optimization of linear network problems with uncertainty in the values of the right-hand side. In contrast to traditional approaches in mathematical programming, we use scenarios to characterize the uncertainty. Solutions are obtained for each scenario and these individual scenarios are aggregated to yield a nonanticipative or implementable policy that minimizes the regret of wrong decisions. A given solution is termed robust if it minimizes the sum over the scenarios of the weighted upper difference between the objective function value for the solution and the objective function value for the optimal solution for each scenario, while satisfying certain nonanticipativity constraints. This approach results in a huge model with a network submodel per scenario plus coupling constraints. Several decomposition approaches are considered, namely Dantzig-Wolfe decomposition, various types of Benders decomposition and different quadratic network approaches for approximating Augmented Lagrangian decomposition. We present computational results for these methods, including two implementation versions of the Lagrangian based method: a sequential implementation and a parallel implementation on a network of three workstations.     

A convergent decomposition algorithm for support vector machines

In this work we consider nonlinear minimization problems with a single linear equality constraint and box constraints. In particular we are interested in solving problems where the number of variables is so huge that traditional optimization methods cannot be directly applied. Many interesting real world problems lead to the solution of large scale constrained problems with this structure. For example, the special subclass of problems with convex quadratic objective function plays a fundamental role in the training of Support Vector Machine, which is a technique for machine learning problems. For this particular subclass of convex quadratic problem, some convergent decomposition methods, based on the solution of a sequence of smaller subproblems, have been proposed. In this paper we define a new globally convergent decomposition algorithm that differs from the previous methods in the rule for the choice of the subproblem variables and in the presence of a proximal point modification in the objective function of the subproblems. In particular, the new rule for sequentially selecting the subproblems appears to be suited to tackle large scale problems, while the introduction of the proximal point term allows us to ensure the global convergence of the algorithm for the general case of nonconvex objective function. Furthermore, we report some preliminary numerical results on support vector classification problems with up to 100 thousands variables.