Large-scale optimization with the primal-dual column generation method

The primal-dual column generation method (PDCGM) is a general-purpose column generation technique that relies on the primal-dual interior point method to solve the restricted master problems. The use of this interior point method variant allows to obtain suboptimal and well-centered dual solutions which naturally stabilizes the column generation process. As recently presented in the literature, reductions in the number of calls to the oracle and in the CPU times are typically observed when compared to the standard column generation, which relies on extreme optimal dual solutions. However, these results are based on relatively small problems obtained from linear relaxations of combinatorial applications. In this paper, we investigate the behaviour of the PDCGM in a broader context, namely when solving large-scale convex optimization problems. We have selected applications that arise in important real-life contexts such as data analysis (multiple kernel learning problem), decision-making under uncertainty (two-stage stochastic programming problems) and telecommunication and transportation networks (multicommodity network flow problem). In the numerical experiments, we use publicly available benchmark instances to compare the performance of the PDCGM against recent results for different methods presented in the literature, which were the best available results to date. The analysis of these results suggests that the PDCGM offers an attractive alternative over specialized methods since it remains competitive in terms of number of iterations and CPU times even for large-scale optimization problems.     

New developments in the primal–dual column generation technique

The optimal solutions of the restricted master problems typically leads to an unstable behavior of the standard column generation technique and, consequently, originates an unnecessarily large number of iterations of the method. To overcome this drawback, variations of the standard approach use interior points of the dual feasible set instead of optimal solutions. In this paper, we focus on a variation known as the primal–dual column generation technique which uses a primal–dual interior point method to obtain well-centered non-optimal solutions of the restricted master problems. We show that the method converges to an optimal solution of the master problem even though non-optimal solutions are used in the course of the procedure. Also, computational experiments are presented using linear-relaxed reformulations of three classical integer programming problems: the cutting stock problem, the vehicle routing problem with time windows, and the capacitated lot sizing problem with setup times. The numerical results indicate that the appropriate use of a primal–dual interior point method within the column generation technique contributes to a reduction of the number of iterations as well as the running times, on average. Furthermore, the results show that the larger the instance, the better the relative performance of the primal–dual column generation technique.     

A specialized primal-dual interior point method for the plastic truss layout optimization

We are concerned with solving linear programming problems arising in the plastic truss layout optimization. We follow the ground structure approach with all possible connections between the nodal points. For very dense ground structures, the solutions of such problems converge to the so-called generalized Michell trusses. Clearly, solving the problems for large nodal densities can be computationally prohibitive due to the resulting huge size of the optimization problems. A technique called member adding that has correspondence to column generation is used to produce a sequence of smaller sub-problems that ultimately approximate the original problem. Although these sub-problems are significantly smaller than the full formulation, they still remain large and require computationally efficient solution techniques. In this article, we present a special purpose primal-dual interior point method tuned to such problems. It exploits the algebraic structure of the problems to reduce the normal equations originating from the algorithm to much smaller linear equation systems. Moreover, these systems are solved using iterative methods. Finally, due to high degree of similarity among the sub-problems after preforming few member adding iterations, the method uses a warm-start strategy and achieves convergence within fewer interior point iterations. The efficiency and robustness of the method are demonstrated with several numerical experiments.     

Solving Stochastic Linear Programs with Restricted Recourse Using Interior Point Methods

In this paper we present a specialized matrix factorization procedure for computing the dual step in a primal-dual path-following interior point algorithm for solving two-stage stochastic linear programs with restricted recourse. The algorithm, based on the Birge-Qi factorization technique, takes advantage of both the dual block-angular structure of the constraint matrix and of the special structure of the second-stage matrices involved in the model. Extensive computational experiments on a set of test problems have been conducted in order to evaluate the performance of the developed code. The results are very promising, showing that the code is competitive with state-of-the-art optimizers.     

A note on the primal-dual column generation method for combinatorial optimization

In this paper, we address the primal-dual column generation technique, which relies on well-centred suboptimal solutions of the restricted master problems. We summarize new theoretical developments and present computational results for two classical combinatorial optimization problems, in which this technique has not been tested before. The results show that the primal-dual column generation technique usually leads to substantial reductions in the number of iterations and CPU time when compared to two other well-established approaches: the classical column generation technique and the analytic centre cutting plane method.     

An implementation of a parallel primal-dual interior point method for block-structured linear programs

An implementation of the primal-dual predictor-corrector interior point method is specialized to solve block-structured linear programs with side constraints. The block structure of the constraint matrix is exploited via parallel computation. The side constraints require the Cholesky factorization of a dense matrix, where a method that exploits parallelism for the dense Cholesky factorization is used. For testing, multicommodity flow problems were used. The resulting implementation is 65%–90% efficient, depending on the problem instance. For a problem with K commodities, an approximate speedup for the interior point method of 0.8K is realized.     

一类线性与框式约束凸规划问题的原始-对偶内点算法

本文对一类具有线性和框式约束的凸规划问题给出了一个原始-对偶内点算法, 该算法可在任一原始-对偶可行内点启动, 并且全局收敛,当初始点靠近中心路径时, 算法成为中心路径跟踪算法。 数值实验表明, 算法对求解大型的这类问题是有效的。     

A Center Cutting Plane Algorithm for a Likelihood Estimate Problem

We describe a method for solving the maximum likelihood estimate problem of a mixing distribution, based on an interior cutting plane algorithm with cuts through analytic centers. From increasingly refined discretized statistical problem models we construct a sequence of inner non-linear problems and solve them approximately applying a primal-dual algorithm to the dual formulation. Refining the statistical problem is equivalent to adding cuts to the inner problems.     

An infeasible interior-point algorithm for solving primal and dual geometric programs

In this paper an algorithm is presented for solving the classical posynomial geometric programming dual pair of problems simultaneously. The approach is by means of a primal-dual infeasible algorithm developed simultaneously for (i) the dual geometric program after logarithmic transformation of its objective function, and (ii) its Lagrangian dual program. Under rather general assumptions, the mechanism defines a primal-dual infeasible path from a specially constructed, perturbed Karush-Kuhn-Tucker system.Subfeasible solutions, as described by Duffin in 1956, are generated for each program whose primal and dual objective function values converge to the respective primal and dual program values. The basic technique is one of a predictor-corrector type involving Newton’s method applied to the perturbed KKT system, coupled with effective techniques for choosing iterate directions and step lengths. We also discuss implementation issues and some sparse matrix factorizations that take advantage of the very special structure of the Hessian matrix of the logarithmically transformed dual objective function. Our computational results on 19 of the most challenging GP problems found in the literature are encouraging. The performance indicates that the algorithm is effective regardless of thedegree of difficulty, which is a generally accepted measure in geometric programming. Research supported in part by the University of Iowa Obermann Fellowship and by NSF Grant DDM-9207347.     

竞争市场均衡问题的内点算法

本文应用最优化方法求解经济学中的经典问题-竞争市场均衡问题.本文对Ye的算法(Ye首先提出了解Fisher问题的原始-对偶路径跟踪算法)做了改进,分别给出了步长调整和迭代方向分解后的原始-对偶路径跟踪算法,并对算法做了理论证明和复杂性分析.最后分析了初始点的求法,做了初步的数值计算.计算结果表明算法能在有效时间内求得问题的解.     

A column generation heuristic for congested facility location problem with clearing functions

Nonlinear clearing functions, an idea initially suggested to reflect congestion effects in production planning, are used to express throughput of facilities prone to congestion in a facility location problem where each demand site is served by exactly one facility. The traditional constant capacity constraint for a facility is replaced with the nonlinear clearing function. The resulting nonlinear integer problem is solved by a column generation heuristic in which initial columns for the restricted master problem are generated by known existing algorithms and additional columns by a previously developed dynamic programming algorithm. Computational experimentation in terms of dual gap and CPU time based on both randomly generated and published data sets show not only clear dominance of the column generation over a Lagrangian heuristic previously developed, but also the high quality of results from the suggested heuristic for large problems.     

Numerical Experiments with an Interior-Exterior Point Method for Nonlinear Programming

The paper presents an algorithm for solving nonlinear programming problems. The algorithm is based on the combination of interior and exterior point methods. The latter is also known as the primal-dual nonlinear rescaling method. The paper shows that in certain cases when the interior point method (IPM) fails to achieve the solution with the high level of accuracy, the use of the exterior point method (EPM) can remedy this situation. The result is demonstrated by solving problems from COPS and CUTE problem sets using nonlinear programming solver LOQO that is modified to include the exterior point method subroutine.     

An Interior Multiple Objective Primal-dual Linear Programming Algorithm Using Efficient Anchoring Points

We present an interior Multiple Objective Linear Programming (MOLP) algorithm based on the path-following primal-dual algorithm. In contrast to the simplex algorithm, which generates a solution path on the exterior of the constraints polytope by following its vertices, the path-following primal-dual algorithm moves through the interior of the polytope. Interior algorithms lend themselves to modifications capable of addressing MOLP problems in a way that is quite different from current solution approaches. In addition, moving through the interior of the polytope results in a solution approach that is less sensitive to problem size than simplex-based MOLP algorithms. The modification of the interior single-objective algorithm to MOLP problems, as presented here, is accomplished by combining the step direction vectors generated by applying the single-objective algorithm to each of the cost vectors into a combined direction vector along which we step from the current iterate to the next iterate.     

Column Generation Algorithms for Nonlinear Optimization,I: Convergence Analysis

Column generation is an increasingly popular basic tool for the solution of large-scale mathematical programming problems. As problems being solved grow bigger, column generation may however become less efficient in its present form, where columns typically are not optimizing, and finding an optimal solution instead entails finding an optimal convex combination of a huge number of them. We present a class of column generation algorithms in which the columns defining the restricted master problem may be chosen to be optimizing in the limit, thereby reducing the total number of columns needed. This first article is devoted to the convergence properties of the algorithm class, and includes global (asymptotic) convergence results for differentiable minimization, finite convergence results with respect to the optimal face and the optimal solution, and extensions of these results to variational inequality problems. An illustration of its possibilities is made on a nonlinear network flow model, contrasting its convergence characteristics to that of the restricted simplicial decomposition (RSD) algorithm.     

一种基于邻近点算法的变步长原始-对偶算法

本文考虑求解一种源于信号及图像处理问题的鞍点问题.基于邻近点算法的思想,我们对原始-对偶算法进行改进,构造一种对称正定且可变的邻近项矩阵,得到一种新的原始-对偶算法.新算法可以看成一种邻近点算法,因此它的收敛性易于分析,且无需较强的假设条件.初步实验结果表明,当新算法被应用于求解图像去模糊问题时,和其他几种主流的高效算法相比,新算法能得到较高质量的结果,且计算时间也是有竞争力的.     

Interior-point methods applied to the AC active-reactive optimal power-flow problem using cartesian coordinates

The primal dual interior point methods are developed to the AC active and reactive optimal power flow problem. The representation of the tensions through cartesian coordinates is adopted, once the Hessian is constant and the Taylor expansion is accurate for the second order term. The advantage of working with polar coordinates, that easily model the tension magnitudes, lose importance due to the efficient treatment of inequalities proportionated by the interior point methods. Before the application of the method, the number of variables of the problem is reduced through the elimination of free dual variables. This elimination does not modify the sparse pattern of the problem. The linear system obtained can be further reduced to the dimension of twice the number of buses also with minor changes in the sparse structure of the matrices involved. Moreover, the final matrix is symmetric in structure. This feature can be exploited reducing the computational effort per iteration. Computational experiments for IEEE system problems are presented for several starting point strategies showing the advantages of the proposed approach. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Superlinear and quadratic convergence of some primal-dual interior point methods for constrained optimization

This paper proves local convergence rates of primal-dual interior point methods for general nonlinearly constrained optimization problems. Conditions to be satisfied at a solution are those given by the usual Jacobian uniqueness conditions. Proofs about convergence rates are given for three kinds of step size rules. They are: (i) the step size rule adopted by Zhang et al. in their convergence analysis of a primal-dual interior point method for linear programs, in which they used single step size for primal and dual variables; (ii) the step size rule used in the software package OB1, which uses different step sizes for primal and dual variables; and (iii) the step size rule used by Yamashita for his globally convergent primal-dual interior point method for general constrained optimization problems, which also uses different step sizes for primal and dual variables. Conditions to the barrier parameter and parameters in step size rules are given for each case. For these step size rules, local and quadratic convergence of the Newton method and local and superlinear convergence of the quasi-Newton method are proved. A preliminary version of this paper was presented at the conference “Optimization-Models and Algorithms” held at the Institute of Statistical Mathematics, Tokyo, March 1993.     

A Semidefinite Programming Based Polyhedral Cut and Price Approach for the Maxcut Problem

We investigate solution of the maximum cut problem using a polyhedral cut and price approach. The dual of the well-known SDP relaxation of maxcut is formulated as a semi-infinite linear programming problem, which is solved within an interior point cutting plane algorithm in a dual setting; this constitutes the pricing (column generation) phase of the algorithm. Cutting planes based on the polyhedral theory of the maxcut problem are then added to the primal problem in order to improve the SDP relaxation; this is the cutting phase of the algorithm. We provide computational results, and compare these results with a standard SDP cutting plane scheme. Research supported in part by NSF grant numbers CCR–9901822 and DMS–0317323. Work done as part of the first authors Ph.D. dissertation at RPI.     

An Efficient Parameterized Logarithmic Kernel Function for Semidefinite Optimization

Acta Mathematicae Applicatae Sinica, English Series - In this paper, we present a primal-dual interior point algorithm for semidefinite optimization problems based on a new class of kernel...     

Primal-dual proximal point algorithm for linearly constrained convex programming problems

A primal-dual version of the proximal point algorithm is developed for linearly constrained convex programming problems. The algorithm is an iterative method to find a saddle point of the Lagrangian of the problem. At each iteration of the algorithm, we compute an approximate saddle point of the Lagrangian function augmented by quadratic proximal terms of both primal and dual variables. Specifically, we first minimize the function with respect to the primal variables and then approximately maximize the resulting function of the dual variables. The merit of this approach exists in the fact that the latter function is differentiable and the maximization of this function is subject to no constraints. We discuss convergence properties of the algorithm and report some numerical results for network flow problems with separable quadratic costs.