Differentiable McCormick relaxations

McCormick’s classical relaxation technique constructs closed-form convex and concave relaxations of compositions of simple intrinsic functions. These relaxations have several properties which make them useful for lower bounding problems in global optimization: they can be evaluated automatically, accurately, and computationally inexpensively, and they converge rapidly to the relaxed function as the underlying domain is reduced in size. They may also be adapted to yield relaxations of certain implicit functions and differential equation solutions. However, McCormick’s relaxations may be nonsmooth, and this nonsmoothness can create theoretical and computational obstacles when relaxations are to be deployed. This article presents a continuously differentiable variant of McCormick’s original relaxations in the multivariate McCormick framework of Tsoukalas and Mitsos. Gradients of the new differentiable relaxations may be computed efficiently using the standard forward or reverse modes of automatic differentiation. Extensions to differentiable relaxations of implicit functions and solutions of parametric ordinary differential equations are discussed. A C++ implementation based on the library MC++ is described and applied to a case study in nonsmooth nonconvex optimization.     

Some results on the strength of relaxations of multilinear functions

We study approaches for obtaining convex relaxations of global optimization problems containing multilinear functions. Specifically, we compare the concave and convex envelopes of these functions with the relaxations that are obtained with a standard relaxation approach, due to McCormick. The standard approach reformulates the problem to contain only bilinear terms and then relaxes each term independently. We show that for a multilinear function having a single product term, this approach yields the convex and concave envelopes if the bounds on all variables are symmetric around zero. We then review and extend some results on conditions when the concave envelope of a multilinear function can be written as a sum of concave envelopes of its individual terms. Finally, for bilinear functions we prove that the difference between the concave upper bounding and convex lower bounding functions obtained from the McCormick relaxation approach is always within a constant of the difference between the concave and convex envelopes. These results, along with numerical examples we provide, give insight into how to construct strong relaxations of multilinear functions.     

Global Optimization with Nonlinear Ordinary Differential Equations

This paper examines global optimization of an integral objective function subject to nonlinear ordinary differential equations. Theory is developed for deriving a convex relaxation for an integral by utilizing the composition result defined by McCormick (Mathematical Programming 10, 147–175, 1976) in conjunction with a technique for constructing convex and concave relaxations for the solution of a system of nonquasimonotone ordinary differential equations defined by Singer and Barton (SIAM Journal on Scientific Computing, Submitted). A fully automated implementation of the theory is briefly discussed, and several literature case study problems are examined illustrating the utility of the branch-and-bound algorithm based on these relaxations.     

Exploiting vector space properties to strengthen the relaxation of bilinear programs arising in the global optimization of process networks

In this paper we present a methodology for finding tight convex relaxations for a special set of quadratic constraints given by bilinear and linear terms that frequently arise in the optimization of process networks. The basic idea lies on exploiting the interaction between the vector spaces where the different set of variables are defined in order to generate cuts that will tighten the relaxation of traditional approaches. These cuts are not dominated by the McCormick convex envelopes and can be effectively used in conjunction with them. The performance of the method is tested in several case studies by implementing the resulting relaxation within a spatial branch and bound framework.     

Multivariate McCormick relaxations

McCormick (Math Prog 10(1):147–175, 1976) provides the framework for convex/concave relaxations of factorable functions, via rules for the product of functions and compositions of the form \(F\circ f\) , where \(F\) is a univariate function. Herein, the composition theorem is generalized to allow multivariate outer functions \(F\) , and theory for the propagation of subgradients is presented. The generalization interprets the McCormick relaxation approach as a decomposition method for the auxiliary variable method. In addition to extending the framework, the new result provides a tool for the proof of relaxations of specific functions. Moreover, a direct consequence is an improved relaxation for the product of two functions, at least as tight as McCormick’s result, and often tighter. The result also allows the direct relaxation of multilinear products of functions. Furthermore, the composition result is applied to obtain improved convex underestimators for the minimum/maximum and the division of two functions for which current relaxations are often weak. These cases can be extended to allow composition of a variety of functions for which relaxations have been proposed.     

Semidefinite relaxations for semi-infinite polynomial programming

This paper studies how to solve semi-infinite polynomial programming (SIPP) problems by semidefinite relaxation methods. We first recall two SDP relaxation methods for solving polynomial optimization problems with finitely many constraints. Then we propose an exchange algorithm with SDP relaxations to solve SIPP problems with compact index set. At last, we extend the proposed method to SIPP problems with noncompact index set via homogenization. Numerical results show that the algorithm is efficient in practice.     

Approximating Pareto curves using semidefinite relaxations

We approximate as closely as desired the Pareto curve associated with bicriteria polynomial optimization problems. We use three formulations (including the weighted sum approach and the Chebyshev approximation) and each of them is viewed as a parametric polynomial optimization problem. For each case is associated a hierarchy of semidefinite relaxations and from an optimal solution of each relaxation one approximates the Pareto curve by solving an inverse problem (first two cases) or by building a polynomial underestimator (third case).     

A note on semidefinite programming relaxations for polynomial optimization over a single sphere

We study two instances of polynomial optimization problem over a single sphere. The first problem is to compute the best rank-1 tensor approximation. We show the equivalence between two recent semidefinite relaxations methods. The other one arises from Bose-Einstein condensates (BEC), whose objective function is a summation of a probably nonconvex quadratic function and a quartic term. These two polynomial optimization problems are closely connected since the BEC problem can be viewed as a structured fourth-order best rank-1 tensor approximation. We show that the BEC problem is NP-hard and propose a semidefinite relaxation with both deterministic and randomized rounding procedures. Explicit approximation ratios for these rounding procedures are presented. The performance of these semidefinite relaxations are illustrated on a few preliminary numerical experiments.     

Global optimization of bounded factorable functions with discontinuities

A deterministic global optimization method is developed for a class of discontinuous functions. McCormick’s method to obtain relaxations of nonconvex functions is extended to discontinuous factorable functions by representing a discontinuity with a step function. The properties of the relaxations are analyzed in detail; in particular, convergence of the relaxations to the function is established given some assumptions on the bounds derived from interval arithmetic. The obtained convex relaxations are used in a branch-and-bound scheme to formulate lower bounding problems. Furthermore, convergence of the branch-and-bound algorithm for discontinuous functions is analyzed and assumptions are derived to guarantee convergence. A key advantage of the proposed method over reformulating the discontinuous problem as a MINLP or MPEC is avoiding the increase in problem size that slows global optimization. Several numerical examples for the global optimization of functions with discontinuities are presented, including ones taken from process design and equipment sizing as well as discrete-time hybrid systems.     

Semidefinite programming for min–max problems and games

We consider two min–max problems (1) minimizing the supremum of finitely many rational functions over a compact basic semi-algebraic set and (2) solving a 2-player zero-sum polynomial game in randomized strategies with compact basic semi-algebraic sets of pure strategies. In both problems the optimal value can be approximated by solving a hierarchy of semidefinite relaxations, in the spirit of the moment approach developed in Lasserre (SIAM J Optim 11:796–817, 2001; Math Program B 112:65–92, 2008). This provides a unified approach and a class of algorithms to compute Nash equilibria and min–max strategies of several static and dynamic games. Each semidefinite relaxation can be solved in time which is polynomial in its input size and practice on a sample of experiments reveals that few relaxations are needed for a good approximation (and sometimes even for finite convergence), a behavior similar to what was observed in polynomial optimization.     

Generalized McCormick relaxations

Convex and concave relaxations are used extensively in global optimization algorithms. Among the various techniques available for generating relaxations of a given function, McCormick’s relaxations are attractive due to the recursive nature of their definition, which affords wide applicability and easy implementation computationally. Furthermore, these relaxations are typically stronger than those resulting from convexification or linearization procedures. This article leverages the recursive nature of McCormick’s relaxations to define a generalized form which both affords a new framework within which to analyze the properties of McCormick’s relaxations, and extends the applicability of McCormick’s technique to challenging open problems in global optimization. Specifically, relaxations of the parametric solutions of ordinary differential equations are considered in detail, and prospects for relaxations of the parametric solutions of nonlinear algebraic equations are discussed. For the case of ODEs, a complete computational procedure for evaluating convex and concave relaxations of the parametric solutions is described. Through McCormick’s composition rule, these relaxations may be used to construct relaxations for very general optimal control problems.     

Completely positive and completely positive semidefinite tensor relaxations for polynomial optimization

Completely positive (CP) tensors, which correspond to a generalization of CP matrices, allow to reformulate or approximate a general polynomial optimization problem (POP) with a conic optimization problem over the cone of CP tensors. Similarly, completely positive semidefinite (CPSD) tensors, which correspond to a generalization of positive semidefinite (PSD) matrices, can be used to approximate general POPs with a conic optimization problem over the cone of CPSD tensors. In this paper, we study CP and CPSD tensor relaxations for general POPs and compare them with the bounds obtained via a Lagrangian relaxation of the POPs. This shows that existing results in this direction for quadratic POPs extend to general POPs. Also, we provide some tractable approximation strategies for CP and CPSD tensor relaxations. These approximation strategies show that, with a similar computational effort, bounds obtained from them for general POPs can be tighter than bounds for these problems obtained by reformulating the POP as a quadratic POP, which subsequently can be approximated using CP and PSD matrices. To illustrate our results, we numerically compare the bounds obtained from these relaxation approaches on small scale fourth-order degree POPs.     

0-1多项式规划问题的SDP松弛方法(英)

本文提出了一类新的构造0-1多项式规划的半定规划(SDP)松弛方法. 我们首先利用矩阵分解和分片线性逼近给出一种新的SDP松弛, 该 松弛产生的界比标准线性松弛产生的界更紧. 我们还利用 拉格朗日松弛和平方和(SOS)松弛方法给出了一种构造Lasserre的SDP 松弛的新方法.     

Semidefinite relaxations for quadratically constrained quadratic programming: A review and comparisons

At the intersection of nonlinear and combinatorial optimization, quadratic programming has attracted significant interest over the past several decades. A variety of relaxations for quadratically constrained quadratic programming (QCQP) can be formulated as semidefinite programs (SDPs). The primary purpose of this paper is to present a systematic comparison of SDP relaxations for QCQP. Using theoretical analysis, it is shown that the recently developed doubly nonnegative relaxation is equivalent to the Shor relaxation, when the latter is enhanced with a partial first-order relaxation-linearization technique. These two relaxations are shown to theoretically dominate six other SDP relaxations. A computational comparison reveals that the two dominant relaxations require three orders of magnitude more computational time than the weaker relaxations, while providing relaxation gaps averaging 3% as opposed to gaps of up to 19% for weaker relaxations, on 700 randomly generated problems with up to 60 variables. An SDP relaxation derived from Lagrangian relaxation, after the addition of redundant nonlinear constraints to the primal, achieves gaps averaging 13% in a few CPU seconds.     

Convergence analysis of Taylor models and McCormick-Taylor models

This article presents an analysis of the convergence order of Taylor models and McCormick-Taylor models, namely Taylor models with McCormick relaxations as the remainder bounder, for factorable functions. Building upon the analysis of McCormick relaxations by Bompadre and Mitsos (J Glob Optim 52(1):1–28, 2012), convergence bounds are established for the addition, multiplication and composition operations. It is proved that the convergence orders of both qth-order Taylor models and qth-order McCormick-Taylor models are at least q + 1, under relatively mild assumptions. Moreover, it is verified through simple numerical examples that these bounds are sharp. A consequence of this analysis is that, unlike McCormick relaxations over natural interval extensions, McCormick-Taylor models do not result in increased order of convergence over Taylor models in general. As demonstrated by the numerical case studies however, McCormick-Taylor models can provide tighter bounds or even result in a higher convergence rate.     

Monomial-wise optimal separable underestimators for mixed-integer polynomial optimization

We introduce a new method for solving box-constrained mixed-integer polynomial problems to global optimality. The approach, a specialized branch-and-bound algorithm, is based on the computation of lower bounds provided by the minimization of separable underestimators of the polynomial objective function. The underestimators are the novelty of the approach because the standard approaches in global optimization are based on convex relaxations. Thanks to the fact that only simple bound constraints are present, minimizing the separable underestimator can be done very quickly. The underestimators are computed monomial-wise after the original polynomial has been shifted. We show that such valid underestimators exist and their degree can be bounded when the degree of the polynomial objective function is bounded, too. For the quartic case, all optimal monomial underestimators are determined analytically. We implemented and tested the branch-and-bound algorithm where these underestimators are hardcoded. The comparison against standard global optimization and polynomial optimization solvers clearly shows that our method outperforms the others, the only exception being the binary case where, though, it is still competitive. Moreover, our branch-and-bound approach suffers less in case of dense polynomial objective function, i.e., in case of polynomials having a large number of monomials. This paper is an extended and revised version of the preliminary paper [4].     

Finding Robust Global Optimal Values of Bilevel Polynomial Programs with Uncertain Linear Constraints

This paper studies a bilevel polynomial program involving box data uncertainties in both its linear constraint set and its lower-level optimization problem. We show that the robust global optimal value of the uncertain bilevel polynomial program is the limit of a sequence of values of Lasserre-type hierarchy of semidefinite linear programming relaxations. This is done by first transforming the uncertain bilevel polynomial program into a single-level non-convex polynomial program using a dual characterization of the solution of the lower-level program and then employing the powerful Putinar’s Positivstellensatz of semi-algebraic geometry. We provide a numerical example to show how the robust global optimal value of the uncertain bilevel polynomial program can be calculated by solving a semidefinite programming problem using the MATLAB toolbox YALMIP.     

A conic relaxation model for searching for the global optimum of network data envelopment analysis

Network data envelopment analysis (DEA) models the internal structures of decision-making units (DMUs). Unlike the standard DEA model, multiplier-based network DEA models are often highly non-linear and cannot be converted into linear programs. As such, obtaining a non-linear network DEA's global optimal solution is a challenge because it corresponds to a nonconvex optimization problem. In this paper, we introduce a conic relaxation model that searches for the global optimum to the general multiplier-based network DEA model. We reformulate the general network DEA models and relax the new models into second order cone programming (SOCP) problems. In comparison with linear relaxation models, which is potentially applicable to general network DEA structures, the conic relaxation model guarantees applicability in general network DEA, since McCormick envelopes involved are ensured to be finite. Furthermore, the conic relaxation model avoids unnecessary linear relaxations of some nonlinear constraints. It generates, in a more convenient manner, feasible approximations and tighter upper bounds on the global optimal overall efficiency. Compared with a line-parameter search method that has been applied to solve non-linear network DEA models, the conic relaxation model keeps track of the distances between the optimal overall efficiency and its approximations. As a result, it is able to determine whether a qualified approximation has been achieved or not, with the help of a branch and bound algorithm. Hence, our proposed approach can substantially reduce the computations involved.     

Equivalences and differences in conic relaxations of combinatorial quadratic optimization problems

Various conic relaxations of quadratic optimization problems in nonnegative variables for combinatorial optimization problems, such as the binary integer quadratic problem, quadratic assignment problem (QAP), and maximum stable set problem have been proposed over the years. The binary and complementarity conditions of the combinatorial optimization problems can be expressed in several ways, each of which results in different conic relaxations. For the completely positive, doubly nonnegative and semidefinite relaxations of the combinatorial optimization problems, we discuss the equivalences and differences among the relaxations by investigating the feasible regions obtained from different representations of the combinatorial condition which we propose as a generalization of the binary and complementarity condition. We also study theoretically the issue of the primal and dual nondegeneracy, the existence of an interior solution and the size of the relaxations, as a result of different representations of the combinatorial condition. These characteristics of the conic relaxations affect the numerical efficiency and stability of the solver used to solve them. We illustrate the theoretical results with numerical experiments on QAP instances solved by SDPT3, SDPNAL+ and the bisection and projection method.     

Convergence rate of McCormick relaxations

Theory for the convergence order of the convex relaxations by McCormick (Math Program 10(1):147–175, 1976) for factorable functions is developed. Convergence rules are established for the addition, multiplication and composition operations. The convergence order is considered both in terms of pointwise convergence and of convergence in the Hausdorff metric. The convergence order of the composite function depends on the convergence order of the relaxations of the factors. No improvement in the order of convergence compared to that of the underlying bound calculation, e.g., via interval extensions, can be guaranteed unless the relaxations of the factors have pointwise convergence of high order. The McCormick relaxations are compared with the αBB relaxations by Floudas and coworkers (J Chem Phys, 1992, J Glob Optim, 1995, 1996), which guarantee quadratic convergence. Illustrative and numerical examples are given.