Multicategory Classification by Support Vector Machines

We examine the problem of how to discriminate between objects of three or more classes. Specifically, we investigate how two-class discrimination methods can be extended to the multiclass case. We show how the linear programming (LP) approaches based on the work of Mangasarian and quadratic programming (QP) approaches based on Vapnik's Support Vector Machine (SVM) can be combined to yield two new approaches to the multiclass problem. In LP multiclass discrimination, a single linear program is used to construct a piecewise-linear classification function. In our proposed multiclass SVM method, a single quadratic program is used to construct a piecewise-nonlinear classification function. Each piece of this function can take the form of a polynomial, a radial basis function, or even a neural network. For the k > 2-class problems, the SVM method as originally proposed required the construction of a two-class SVM to separate each class from the remaining classes. Similarily, k two-class linear programs can be used for the multiclass problem. We performed an empirical study of the original LP method, the proposed k LP method, the proposed single QP method and the original k QP methods. We discuss the advantages and disadvantages of each approach.     

Minimizing a Sum of Norms Subject to Linear Equality Constraints

Numerical analysis of a class of nonlinear duality problems is presented. One side of the duality is to minimize a sum of Euclidean norms subject to linear equality constraints (the constrained MSN problem). The other side is to maximize a linear objective function subject to homogeneous linear equality constraints and quadratic inequalities. Large sparse problems of this form result from the discretization of infinite dimensional duality problems in plastic collapse analysis.The solution method is based on the l ₁ penalty function approach to the constrained MSN problem. This can be formulated as an unconstrained MSN problem for which the first author has recently published an efficient Newton barrier method, and for which new methods are still being developed.Numerical results are presented for plastic collapse problems with up to 180000 variables, 90000 terms in the sum of norms and 90000 linear constraints. The obtained accuracy is of order 10^-8 measured in feasibility and duality gap.     

Warmstarting for interior point methods applied to the long-term power planning problem

The long-term planning of electricity generation in a liberalised market using the Bloom and Gallant model can be posed as a quadratic programming (QP) problem with an exponential number of linear inequality constraints called load-matching constraints (LMCs) and several other linear non-LMCs. Direct solution methods are inefficient at handling such problems and a heuristic procedure has been devised to generate only those LMCs that are likely to be active at the optimiser. The problem is then solved as a finite succession of QP problems with an increasing, though still limited, number of LMCs, which can be solved efficiently using a direct method, as would be the case with a QP interior-point algorithm. Warm starting between successive QP solutions helps then in reducing the number of iterations necessary to reach the optimiser.     

Reducing quadratic programming problem to regression problem: Stepwise algorithm

Quadratic programming is concerned with minimizing a convex quadratic function subject to linear inequality constraints. The variables are assumed to be nonnegative. The unique solution of quadratic programming (QP) problem (QPP) exists provided that a feasible region is non-empty (the QP has a feasible space).A method for searching for the solution to a QP is provided on the basis of statistical theory. It is shown that QPP can be reduced to an appropriately formulated least squares (LS) problem (LSP) with equality constraints and nonnegative variables. This approach allows us to obtain a simple algorithm to solve QPP. The applicability of the suggested method is illustrated with numerical examples.     

A Decomposition Method for Global and Local Quadratic Minimization

We present a decomposition method for indefinite quadratic programming problems having n variables and m linear constraints. The given problem is decomposed into at most m QP subproblems each having m linear constraints and n-1 variables. All global minima, all isolated local minima and some of the non-isolated local minima for the given problem are obtained from those of the lower dimensional subproblems. One way to continue solving the given problem is to apply the decomposition method again to the subproblems and repeatedly doing so until subproblems of dimension 1 are produced and these can be solved directly. A technique to reduce the potentially large number of subproblems is formulated.     

Optimal Prioritized Infeasibility Handling in Model Predictive Control: Parametric Preemptive Multiobjective Linear Programming Approach

All practical implementations of model-based predictive control (MPC) require a means to recover from infeasibility. We propose a strategy designed for linear state-space MPC with prioritized constraints. It relaxes optimally an infeasible MPC optimization problem into a feasible one by solving a single-objective linear program (LP) online in addition to the standard online MPC optimization problem at each sample. By optimal, it is meant that the violation of a lower prioritized constraint cannot be made less without increasing the violation of a higher prioritized constraint. The problem of computing optimal constraint violations is naturally formulated as a parametric preemptive multiobjective LP. By extending well-known results from parametric LP, the preemptive multiobjective LP is reformulated into an equivalent standard single-objective LP. An efficient algorithm for offline design of this LP is given, and the algorithm is illustrated on an example.     

塑性极限分析的不可微模型及其光滑化算法

藉助于凸规划的Lagrange对偶理论,建立了Mises屈服条件下理想刚塑性材料Hill最大塑性功原理的对偶问题,并据此建立了极限分析的一个不可微凸规划模型．该模型不仅避免了对屈服条件的线性化,而且其离散化形式为线性约束下Euclid模之和的极小化问题．针对Euclid模的不可微性,提出理想刚塑性体极限分析的一种光滑化算法．通过计算平面应力和平面应变问题的极限荷载因子和相应的坍塌机构,验证了算法的有效性．     

Balance constraints reduction of large-scale linear programming problems

This paper describes a method and the corresponding algorithms for simplification of large-scale linear programming models. It consists of the elimination of the balance constraints (i.e. constraints with zero RHS term). The idea is to apply some linear transformations to the original problem in order to nullify the balance constraints. These transformations are able to simultaneously eliminate more balance rows. The core of this contribution is the introduction of the reduction matrix and the associated theorems on the equivalent linear programs (original and reduced). The numerical experiments with this method of simplification proved this approach to be beneficial for a large class of LP problems.This research work was done while the first author was at Duisburg University, Mess-, Steuer und Regelungstechnik, Germany, under the greatly appreciated financial assistance given by the Alexander-von-Humboldt Foundation.     

Multi-product production planning: A goal programming approach

Cost minimization multi-product production problems with static production resource usage and internal product flow requirements have been solved by linear programming (LP) with input/output analysis. If the problem is complicated by interval resource estimates, interval linear programming (ILP) can be used. The solution of realistic problems by the above method is cumbersome. This paper suggests that linear goal programming (LGP) can be used to model a multi-product production system. LGP's unique modeling capabilities are used to solve a production planning problem with variable resource parameters. Input/output analysis is used to determine the technological coefficients for the goal constraints and is also used to derive an information sub-model that is used to reduce the number of variable resource goal constraints. Preliminary findings suggest that the LGP approach is more cost-efficient (in terms of CPU time) and in addition provides valuable information for aggregate planning.     

A compact formulation of an elastoplastic analysis problem

Two important problems in the area of engineering plasticity are limit load analysis and elastoplastic analysis. It is well known that these two problems can be formulated as linear and quadratic programming problems, respectively (Refs. 1–2). In applications, the number of variables in each of these mathematical programming problems tends to be large. Consequently, it is important to have efficient numerical methods for their solution. The purpose of this paper is to present a method which allows the quadratic programming formulation of the elastoplastic analysis to be reformulated as an equivalent quadratic programming problem which has significantly fewer variables than the original formulation. Indeed, in Section 4, we will present details of an example for which the original quadratic programming formulation required 297 variables and for which the equivalent formulation presented here required only two variables. The method is based on a characterization of the entire family of optimal solutions for a linear programming problem.This research was supported by the Natural Science and Engineering Council of Canada under Grant No. A8189 and by a Leave Fellowship from the Social Sciences and Humanities Research Council of Canada. The author takes pleasure in acknowledging many stimulating discussions with Professor D. E. Grierson.     

A New Semidefinite Programming Bound for Indefinite Quadratic Forms Over a Simplex

The paper describes a method for computing a lower bound of the global minimum of an indefinite quadratic form over a simplex. The bound is derived by computing an underestimator of the convex envelope by solving a semidefinite program (SDP). This results in a convex quadratic program (QP). It is shown that the optimal value of the QP is a lower bound of the optimal value of the original problem. Since there exist fast (polynomial time) algorithms for solving SDP's and QP's the bound can be computed in reasonable time. Numerical experiments indicate that the relative error of the bound is about 10 percent for problems up to 20 variables, which is much better than a known SDP bound.     

一种基于LVI求解二次规划问题的数值算法

给出并研究了一种数值算法(简称94LVI算法),用于求解带等式和双端约束的二次规划问题. 这类带约束的二次规划问题首先被转换为线性变分不等式问题,该问题等价于分段线性投影等式.接着使用94LVI算法求解上述分段线性投影等式,从而得到QP问题的最优解. 进一步给出了94LVI算法的全局收敛性证明. 94LVI算法与经典有效集算法的对比实验结果证实了给出的94LVI算法在求解二次规划问题上的高效性与优越性.     

Optimization Techniques for State-Constrained Control and Obstacle Problems

The design of control laws for systems subject to complex state constraints still presents a significant challenge. This paper explores a dynamic programming approach to a specific class of such problems, that of reachability under state constraints. The problems are formulated in terms of nonstandard minmax and maxmin cost functionals, and the corresponding value functions are given in terms of Hamilton-Jacobi-Bellman (HJB) equations or variational inequalities. The solution of these relations is complicated in general; however, for linear systems, the value functions may be described also in terms of duality relations of convex analysis and minmax theory. Consequently, solution techniques specific to systems with a linear structure may be designed independently of HJB theory. These techniques are illustrated through two examples.The first author was supported by the Russian Foundation for Basic Research, Grant 03-01-00663, by the program Universities of Russia, Grant 03.03.007, and by the program of the Russian Federation President for the support of scientific research in leading scientific schools, Grant NSh-1889.2003.1.The second author was supported by the National Science and Engineering Research Council of Canada and by ONR MURI Contract 79846-23800-44-NDSAS.The third and first authors were supported by NSF Grants ECS-0099824 and ECS-0424445.Communicated by G. Leitmann     

An improved sequential quadratic programming algorithm for solving general nonlinear programming problems

In this paper, a class of general nonlinear programming problems with inequality and equality constraints is discussed. Firstly, the original problem is transformed into an associated simpler equivalent problem with only inequality constraints. Then, inspired by the ideals of the sequential quadratic programming (SQP) method and the method of system of linear equations (SLE), a new type of SQP algorithm for solving the original problem is proposed. At each iteration, the search direction is generated by the combination of two directions, which are obtained by solving an always feasible quadratic programming (QP) subproblem and a SLE, respectively. Moreover, in order to overcome the Maratos effect, the higher-order correction direction is obtained by solving another SLE. The two SLEs have the same coefficient matrices, and we only need to solve the one of them after a finite number of iterations. By a new line search technique, the proposed algorithm possesses global and superlinear convergence under some suitable assumptions without the strict complementarity. Finally, some comparative numerical results are reported to show that the proposed algorithm is effective and promising.     

Construction of Test Problems for Concave Minimization Under Linear and Nonlinear Constraints

This paper presents a method for constructing test problems with known global solutions for concave minimization under linear constraints with an additional convex constraint and for reverse convex programs with an additional convex constraint. The importance of such a construction can be realized from the fact that the well known d.c. programming problem can be formulated in this form. Then, the method is further extended to generate test problems with more than one convex constraint, tight or untight at the global solution.     

Optimal design and sensitivity of large scale plane frames for the case of uncertainty

This paper deals with the optimal design and the robustness of large scale plane frames in dependence of their height. Using the first collapse theorem, the necessary and sufficient survival conditions of an elasto-plastic structure consist of the yield condition and the equilibrium condition. The basis for our consideration is provided by a plane n-storey frame which will be increased successively, and which is affected by applied random forces and moments. Taking into account these random applied loads, we get a stochastic structural optimization problem which cannot be solved using the traditional methods. Instead of that, an appropriate (deterministic) substitute problem is formulated. First, the recourse problem will be formulated in general and in the standard form of stochastic linear programming (SLP), and after the formulation of the stochastic optimization problem, the Recourse Problem based on Discretization (RPD) is introduced as a representative of substitute problems. The resulting (large scale) linear program (LP) can be solved efficiently by means of usual LP-solvers. (© 2010 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Optimal Security Liquidation Algorithms

This paper develops trading strategies for liquidation of a financial security, which maximize the expected return. The problem is formulated as a stochastic programming problem that utilizes the scenario representation of possible returns. Two cases are considered, a case with no constraint on risk and a case when the risk of losses associated with trading strategy is constrained by Conditional Value-at-Risk (CVaR) measure. In the first case, two algorithms are proposed; one is based on linear programming techniques, and the other uses dynamic programming to solve the formulated stochastic program. The third proposed algorithm is obtained by adding the risk constraints to the linear program. The algorithms provide path-dependent strategies, i.e., the fraction of security sold depends upon price sample-path of the security up to the current moment. The performance of the considered approaches is tested using a set of historical sample-paths of prices.     

A Fast MPC Algorithm Using Nonfeasible Active Set Methods

Model predictive control (MPC) is an optimization-based control framework which is attractive to industry both because it can be practically implemented and it can deal with constraints directly. One of the main drawbacks of MPC is that large MPC horizon times can cause requirements of excessive computational time to solve the quadratic programming (QP) minimization which occurs in the calculation of the controller at each sampling interval. This motivates the study of finding faster ways for computing the QP problem associated with MPC. In this paper, a new nonfeasible active set method is proposed for solving the QP optimization problem that occurs in MPC. This method has the feature that it is typically an order of magnitude faster than traditional methods. This work has been supported by the Canadian NSERC under Grant A4396.     

使用自正则度量的凸二次规划的原始对偶内点法的多项式复杂性

最近Peng等人使用新的搜索方向和自正则度量为求解线性规划问题提出了一个原始对偶内点法.本文将这个长步法延伸到凸二次规划.在线性规划情形时,原始空间和对偶空间中的尺度Newton方向是正交的,而在二次规划情形时这是不成立的.本文将处理这个问题并且证明多项式复杂性,并且得到复杂性的上界为O(n√log n log (n/ε)).     

Identifying the optimal partition in convex quadratic programming

Given an optimal solution for a convex quadratic programming (QP) problem, the optimal partition of the QP can be computed by solving a pair of linear or QP problems for which nearly optimal solutions are known.