Warmstarting for interior point methods applied to the long-term power planning problem

The long-term planning of electricity generation in a liberalised market using the Bloom and Gallant model can be posed as a quadratic programming (QP) problem with an exponential number of linear inequality constraints called load-matching constraints (LMCs) and several other linear non-LMCs. Direct solution methods are inefficient at handling such problems and a heuristic procedure has been devised to generate only those LMCs that are likely to be active at the optimiser. The problem is then solved as a finite succession of QP problems with an increasing, though still limited, number of LMCs, which can be solved efficiently using a direct method, as would be the case with a QP interior-point algorithm. Warm starting between successive QP solutions helps then in reducing the number of iterations necessary to reach the optimiser.     

An LPCC approach to nonconvex quadratic programs

Filling a gap in nonconvex quadratic programming, this paper shows that the global resolution of a feasible quadratic program (QP), which is not known a priori to be bounded or unbounded below, can be accomplished in finite time by solving two linear programs with linear complementarity constraints, i.e., LPCCs. Specifically, this task can be divided into two LPCCs: the first confirms whether the QP is bounded below on the feasible set and, if not, computes a feasible ray on which the QP is unbounded; the second LPCC computes a globally optimal solution if it exists, by identifying a stationary point that yields the best quadratic objective value. In turn, the global resolution of these LPCCs can be accomplished by a parameter-free, mixed integer-programming based, finitely terminating algorithm developed recently by the authors, which can be enhanced in this context by a new kind of valid cut derived from the second-order conditions of the QP and by exploiting the special structure of the LPCCs. Throughout, our treatment makes no boundedness assumption of the QP; this is a significant departure from much of the existing literature which consistently employs the boundedness of the feasible set as a blanket assumption. The general theory is illustrated by 3 classes of indefinite problems: QPs with simple upper and lower bounds (existence of optimal solutions is guaranteed); same QPs with an additional inequality constraint (extending the case of simple bound constraints); and nonnegatively constrained copositive QPs (no guarantee of the existence of an optimal solution). We also present numerical results to support the special cuts obtained due to the QP connection.     

Identifying the optimal partition in convex quadratic programming

Given an optimal solution for a convex quadratic programming (QP) problem, the optimal partition of the QP can be computed by solving a pair of linear or QP problems for which nearly optimal solutions are known.     

一类全局优化问题的线性化方法

本文对带有不定二次约束且目标函数为非凸二次函数的最优化问题提出了一类新的确定型全局优化算法,通过对目标函数和约束函数的线性下界估计,建立了原规划的松弛线性规划,通过对松弛线性规划可行域的细分以及一系列松弛线性规划的求解过程,得到原问题的全局最优解.我们从理论上证明了算法能收敛到原问题的全局最优解.     

Reducing quadratic programming problem to regression problem: Stepwise algorithm

Quadratic programming is concerned with minimizing a convex quadratic function subject to linear inequality constraints. The variables are assumed to be nonnegative. The unique solution of quadratic programming (QP) problem (QPP) exists provided that a feasible region is non-empty (the QP has a feasible space).A method for searching for the solution to a QP is provided on the basis of statistical theory. It is shown that QPP can be reduced to an appropriately formulated least squares (LS) problem (LSP) with equality constraints and nonnegative variables. This approach allows us to obtain a simple algorithm to solve QPP. The applicability of the suggested method is illustrated with numerical examples.     

A note on the strong polynomiality of convex quadratic programming

We prove that a general convex quadratic program (QP) can be reduced to the problem of finding the nearest point on a simplicial cone inO(n ³ +n logL) steps, wheren andL are, respectively, the dimension and the encoding length of QP. The proof is quite simple and uses duality and repeated perturbation. The implication, however, is nontrivial since the problem of finding the nearest point on a simplicial cone has been considered a simpler problem to solve in the practical sense due to its special structure. Also we show that, theoretically, this reduction implies that (i) if an algorithm solves QP in a polynomial number of elementary arithmetic operations that is independent of the encoding length of data in the objective function then it can be used to solve QP in strongly polynomial time, and (ii) ifL is bounded by a first order exponential function ofn then (i) can be stated even in stronger terms: to solve QP in strongly polynomial time, it suffices to find an algorithm running in polynomial time that is independent of the encoding length of the quadratic term matrix or constraint matrix. Finally, based on these results, we propose a conjecture.corresponding author. The research was done when the author was at the Department of IE & OR, University of California at Berkeley, and partially supported by ONR grant N00014-91-j-1241.     

A coordinate gradient descent method for linearly constrained smooth optimization and support vector machines training

Support vector machines (SVMs) training may be posed as a large quadratic program (QP) with bound constraints and a single linear equality constraint. We propose a (block) coordinate gradient descent method for solving this problem and, more generally, linearly constrained smooth optimization. Our method is closely related to decomposition methods currently popular for SVM training. We establish global convergence and, under a local error bound assumption (which is satisfied by the SVM QP), linear rate of convergence for our method when the coordinate block is chosen by a Gauss-Southwell-type rule to ensure sufficient descent. We show that, for the SVM QP with n variables, this rule can be implemented in O(n) operations using Rockafellar’s notion of conformal realization. Thus, for SVM training, our method requires only O(n) operations per iteration and, in contrast to existing decomposition methods, achieves linear convergence without additional assumptions. We report our numerical experience with the method on some large SVM QP arising from two-class data classification. Our experience suggests that the method can be efficient for SVM training with nonlinear kernel.     

一种基于LVI求解二次规划问题的数值算法

给出并研究了一种数值算法(简称94LVI算法),用于求解带等式和双端约束的二次规划问题. 这类带约束的二次规划问题首先被转换为线性变分不等式问题,该问题等价于分段线性投影等式.接着使用94LVI算法求解上述分段线性投影等式,从而得到QP问题的最优解. 进一步给出了94LVI算法的全局收敛性证明. 94LVI算法与经典有效集算法的对比实验结果证实了给出的94LVI算法在求解二次规划问题上的高效性与优越性.     

解带有二次约束二次规划的一个整体优化方法

在本文中，我们提出了一种解带有二次约束二次规划问题（QP）的新算法，这种方法是基于单纯形分枝定界技术，其中包括极小极大问题和线性规划问题作为子问题，利用拉格朗日松弛和投影次梯度方法来确定问题（QP）最优值的下界，在问题（QP）的可行域是n维的条件下，如果这个算法有限步后终止，得到的点必是问题（QP）的整体最优解；否则，该算法产生的点的序列{v^k}的每一个聚点也必是问题（QP）的整体最优解。     

Optimal portfolios using linear programming models

The classical quadratic programming (QP) formulation of the well-known portfolio selection problem has traditionally been regarded as cumbersome and time consuming. This paper formulates two additional models: (i) maximin, and (ii) minimization of mean absolute deviation. Data from 67 securities over 48 months are used to examine to what extent all three formulations provide similar portfolios. As expected, the maximin formulation yields the highest return and risk, while the QP formulation provides the lowest risk and return, which also creates the efficient frontier. The minimization of mean absolute deviation is close to the QP formulation. When the expected returns are confronted with the true ones at the end of a 6-month period, the maximin portfolios seem to be the most robust of all.     

The projection method of optimization applied to engineering plasticity problems

In engineering plasticity, the behavior of a structure (e.g., a frame or truss) under a variety of loading conditions is studied. Two primary types of analysis are generally conducted. Limit analysis determines the rigid plastic collapse load for a structure and can be formulated as a linear program (LP). Deformation analysis at plastic collapse can be formulated as a quadratic program (QP). The constraints of the two optimization problems are closely related. This paper presents a specialization of the projection method for linear programming for the limit-load analysis problem. The algorithm takes advantage of the relationship between the LP constraints and QP constraints to provide advantageous starting data for the projection method applied to the QP problem. An important feature of the method is that it avoids problems of apparent infeasibility due to roundoff errors. Experimental results are given for two medium-sized problems.This work was supported by the National Research Council of Canada under Research Grant No. A8189.     

极大熵方法与二次规划子问题的显式解

本文对混合约束极大极小问题的目标函数与约束分别用熵函数来逼近,讨论了逼近问题的二次规划子问题的搜索方向的显式形式,并给出了极大极小问题和多目标规划的二次规划予问题的显式解。将所得结果用于相应的算法中,可提高算法的有效性。     

A more efficient algorithm for Convex Nonparametric Least Squares

Convex Nonparametric Least Squares (CNLSs) is a nonparametric regression method that does not require a priori specification of the functional form. The CNLS problem is solved by mathematical programming techniques; however, since the CNLS problem size grows quadratically as a function of the number of observations, standard quadratic programming (QP) and Nonlinear Programming (NLP) algorithms are inadequate for handling large samples, and the computational burdens become significant even for relatively small samples. This study proposes a generic algorithm that improves the computational performance in small samples and is able to solve problems that are currently unattainable. A Monte Carlo simulation is performed to evaluate the performance of six variants of the proposed algorithm. These experimental results indicate that the most effective variant can be identified given the sample size and the dimensionality. The computational benefits of the new algorithm are demonstrated by an empirical application that proved insurmountable for the standard QP and NLP algorithms.     

On reduced convex QP formulations of monotone LCPs

Techniques for transforming convex quadratic programs (QPs) into monotone linear complementarity problems (LCPs) and vice versa are well known. We describe a class of LCPs for which a reduced QP formulation – one that has fewer constraints than the “standard” QP formulation – is available. We mention several instances of this class, including the known case in which the coefficient matrix in the LCP is symmetric. Received: May 2000 / Accepted: February 22, 2001?Published online April 12, 2001     

Tightening a copositive relaxation for standard quadratic optimization problems

We focus in this paper the problem of improving the semidefinite programming (SDP) relaxations for the standard quadratic optimization problem (standard QP in short) that concerns with minimizing a quadratic form over a simplex. We first analyze the duality gap between the standard QP and one of its SDP relaxations known as “strengthened Shor’s relaxation”. To estimate the duality gap, we utilize the duality information of the SDP relaxation to construct a graph G ^?. The estimation can be then reduced to a two-phase problem of enumerating first all the minimal vertex covers of G ^? and solving next a family of second-order cone programming problems. When there is a nonzero duality gap, this duality gap estimation can lead to a strictly tighter lower bound than the strengthened Shor’s SDP bound. With the duality gap estimation improving scheme, we develop further a heuristic algorithm for obtaining a good approximate solution for standard QP.     

A kind of nonmonotone filter method for nonlinear complementarity problem

The nonlinear complementarity problem can be reformulated as a nonlinear programming. For solving nonlinear programming, sequential quadratic programming (SQP) type method is very effective. But the QP subproblem may be inconsistent. In this paper, we propose a kind nonmonotone filter method in which the QP subproblem is consistent. By means of nonmonotone filter, this method has no demand on the penalty parameter which is difficult to obtain. Moreover, the restoration phase is not needed any more. Under reasonable conditions, we obtain the global convergence of the algorithm. Some numerical results are presented.     

Computing upper and lower bounds in interval decision trees

This article presents algorithms for computing optima in decision trees with imprecise probabilities and utilities. In tree models involving uncertainty expressed as intervals and/or relations, it is necessary for the evaluation to compute the upper and lower bounds of the expected values. Already in its simplest form, computing a maximum of expectancies leads to quadratic programming (QP) problems. Unfortunately, standard optimization methods based on QP (and BLP – bilinear programming) are too slow for the evaluation of decision trees in computer tools with interactive response times. Needless to say, the problems with computational complexity are even more emphasized in multi-linear programming (MLP) problems arising from multi-level decision trees. Since standard techniques are not particularly useful for these purposes, other, non-standard algorithms must be used. The algorithms presented here enable user interaction in decision tools and are equally applicable to all multi-linear programming problems sharing the same structure as a decision tree.     

Global and Local Quadratic Minimization

We present a method which when applied to certain non-convex QP will locatethe globalminimum, all isolated local minima and some of the non-isolated localminima. The method proceeds by formulating a (multi) parametric convex QP interms ofthe data of the given non-convex QP. Based on the solution of the parametricQP,an unconstrained minimization problem is formulated. This problem ispiece-wisequadratic. A key result is that the isolated local minimizers (including theglobalminimizer) of the original non-convex problem are in one-to-one correspondencewiththose of the derived unconstrained problem.The theory is illustrated with several numerical examples. A numericalprocedure isdeveloped for a special class of non-convex QP's. It is applied to a problemfrom theliterature and verifies a known global optimum and in addition, locates apreviously unknown local minimum.     

Multicategory Classification by Support Vector Machines

We examine the problem of how to discriminate between objects of three or more classes. Specifically, we investigate how two-class discrimination methods can be extended to the multiclass case. We show how the linear programming (LP) approaches based on the work of Mangasarian and quadratic programming (QP) approaches based on Vapnik's Support Vector Machine (SVM) can be combined to yield two new approaches to the multiclass problem. In LP multiclass discrimination, a single linear program is used to construct a piecewise-linear classification function. In our proposed multiclass SVM method, a single quadratic program is used to construct a piecewise-nonlinear classification function. Each piece of this function can take the form of a polynomial, a radial basis function, or even a neural network. For the k > 2-class problems, the SVM method as originally proposed required the construction of a two-class SVM to separate each class from the remaining classes. Similarily, k two-class linear programs can be used for the multiclass problem. We performed an empirical study of the original LP method, the proposed k LP method, the proposed single QP method and the original k QP methods. We discuss the advantages and disadvantages of each approach.     

A Fast MPC Algorithm Using Nonfeasible Active Set Methods

Model predictive control (MPC) is an optimization-based control framework which is attractive to industry both because it can be practically implemented and it can deal with constraints directly. One of the main drawbacks of MPC is that large MPC horizon times can cause requirements of excessive computational time to solve the quadratic programming (QP) minimization which occurs in the calculation of the controller at each sampling interval. This motivates the study of finding faster ways for computing the QP problem associated with MPC. In this paper, a new nonfeasible active set method is proposed for solving the QP optimization problem that occurs in MPC. This method has the feature that it is typically an order of magnitude faster than traditional methods. This work has been supported by the Canadian NSERC under Grant A4396.