A computational study of a gradient-based log-barrier algorithm for a class of large-scale SDPs

 The authors of this paper recently introduced a transformation [4] that converts a class of semidefinite programs (SDPs) into nonlinear optimization problems free of matrix-valued constraints and variables. This transformation enables the application of nonlinear optimization techniques to the solution of certain SDPs that are too large for conventional interior-point methods to handle efficiently. Based on the transformation, we proposed a globally convergent, first-order (i.e., gradient-based) log-barrier algorithm for solving a class of linear SDPs. In this paper, we discuss an efficient implementation of the proposed algorithm and report computational results on semidefinite relaxations of three types of combinatorial optimization problems. Our results demonstrate that the proposed algorithm is indeed capable of solving large-scale SDPs and is particularly effective for problems with a large number of constraints. Received: June 22, 2001 / Accepted: January 20, 2002 Published online: December 9, 2002 RID="†" ID="†"Computational results reported in this paper were obtained on an SGI Origin2000 computer at Rice University acquired in part with support from NSF Grant DMS-9872009. RID="⋆" ID="⋆"This author was supported in part by NSF Grants CCR-9902010, INT-9910084 and CCR-0203426 RID="⋆⋆" ID="⋆⋆"This author was supported in part by NSF Grants CCR-9902010, INT-9910084 and CCR-0203113 RID="⋆⋆⋆" ID="⋆⋆⋆"This author was supported in part by DOE Grant DE-FG03-97ER25331, DOE/LANL Contract 03891-99-23 and NSF Grant DMS-9973339. Key Words. semidefinite program – semidefinite relaxation – nonlinear programming – interior-point methods – limited memory quasi-Newton methods. Mathematics Subject Classification (1991): 90C06, 90C27, 90C30.     

On translation invariant operators which preserve the B-spline recurrence

It was observed in [4] that the Hilbert transform of the univariate B-spline preserves the B-spline recurrence. Motivated by this observation, we characterize translation invariant operators that preserve the multivariate B-spline recurrence and analogous results are also provided for the multivariate cube spline. Charles A. Micchelli was supported in part by the US National Science of Foundation under grant CCR-0407476. Yuesheng Xu was supported in part by the US National Science Foundation under grant CCR-0407476, by the Natural Science Foundation of China under grant 10371122, by the Chinese Academy of Sciences under the program “One Hundred Distinguished Chinese Scientists” and by Ministry of Education, People’s Republic of China, under the Changjian Scholarship through Zhongshan University.     

Efficient piecewise-linear function approximation using the uniform metric

We given anO(n logn)-time method for finding a bestk-link piecewise-linear function approximating ann-point planar point set using the well-known uniform metric to measure the error, ε≥0, of the approximation. Our methods is based upon new characterizations of such functions, which we exploit to design an efficient algorithm using a plane sweep in “ε space” followed by several applications of the parametric-searching technique. The previous best running time for this problems wasO(n ²). This research was announced in preliminary form at the 10th ACM Symposium on Computational Geometry. The author was partially supported by the NSF and DARPA under Grant CCR-8908092, and by the NSF under Grants IRI-9116843 and CCR-9300079.     

A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization

 In this paper, we present a nonlinear programming algorithm for solving semidefinite programs (SDPs) in standard form. The algorithm's distinguishing feature is a change of variables that replaces the symmetric, positive semidefinite variable X of the SDP with a rectangular variable R according to the factorization X=RR ^T . The rank of the factorization, i.e., the number of columns of R, is chosen minimally so as to enhance computational speed while maintaining equivalence with the SDP. Fundamental results concerning the convergence of the algorithm are derived, and encouraging computational results on some large-scale test problems are also presented. Received: March 22, 2001 / Accepted: August 30, 2002 Published online: December 9, 2002 Key Words. semidefinite programming – low-rank factorization – nonlinear programming – augmented Lagrangian – limited memory BFGS This research was supported in part by the National Science Foundation under grants CCR-9902010, INT-9910084, CCR-0203426 and CCR-0203113     

First- and second-order methods for semidefinite programming

 In this paper, we survey the most recent methods that have been developed for the solution of semidefinite programs. We first concentrate on the methods that have been primarily motivated by the interior point (IP) algorithms for linear programming, putting special emphasis in the class of primal-dual path-following algorithms. We also survey methods that have been developed for solving large-scale SDP problems. These include first-order nonlinear programming (NLP) methods and more specialized path-following IP methods which use the (preconditioned) conjugate gradient or residual scheme to compute the Newton direction and the notion of matrix completion to exploit data sparsity. Received: December 16, 2002 / Accepted: May 5, 2003 Published online: May 28, 2003 Key words. semidefinite programming – interior-point methods – polynomial complexity – path-following methods – primal-dual methods – nonlinear programming – Newton method – first-order methods – bundle method – matrix completion The author's research presented in this survey article has been supported in part by NSF through grants INT-9600343, INT-9910084, CCR-9700448, CCR-9902010, CCR-0203113 and ONR through grants N00014-93-1-0234, N00014-94-1-0340 and N00014-03-1-0401. Mathematics Subject Classification (2000): 65K05, 90C06, 90C22, 90C25, 90C30, 90C51     

Finding an interior point in the optimal face of linear programs

We study the problem of finding a point in the relative interior of the optimal face of a linear program. We prove that in the worst case such a point can be obtained in O(n ³ L) arithmetic operations. This complexity is the same as the complexity for solving a linear program. We also show how to find such a point in practice. We report and discuss computational results obtained for the linear programming problems in the NETLIB test set.Research supported in part by NSF Grant CCR-8810107, CCR-9019469 and a grant from GTE Laboratories.Research supported in part by NSF Grant DDM-8922636 and NSF Coop. Agr. No. CCR-8809615 through Rice University.     

On the matrix completion problem for multivariate filter bank construction

We survey the main techniques for the construction of multivariate filter banks and present new results about special matrices of order four and eight suitable for their construction. Qiuhui Chen: Supported in part by NSFC under grant 10201034 and project-sponsored by SRF for ROCS, SEM. Charles A. Micchelli: Supported in part by the US National Science Foundation under grant CCR-0407476. Yuesheng Xu: All correspondence to this author. Supported in part by the US National Science Foundation under grant CCR-0407476, by the Natural Science Foundation of China under grant 10371122 and by the Chinese Academy of Sciences under the program “One Hundred Distinguished Chinese Young Scientists”.     

An elementary approach to lower bounds in geometric discrepancy

For eachd≥2, it is possible to placen points ind-space so that, given any two-coloring of the points, a half-space exists within which one color outnumbers the other by as much ascn ^1/2−1/2d , for some constantc>0 depending ond. This result was proven in a slightly weaker form by Beck and the bound was later tightened by Alexander. It was recently shown to be asymptotically optimal by Matoušek. We present a proof of the lower bound, which is based on Alexander's technique but is technically simpler and more accessible. We present three variants of the proof, for three diffrent cases, to provide more intuitive insight into the “large-discrepancy” phenomenon. We also give geometric and probabilistic interpretations of the technique. Work by Bernard Chazelle has been supported in part by NSF Grant CCR-90-02352 and The Geometry Center, University of Minnesota, an STC funded by NSF, DOE, and Minnesota Technology, Inc. Work by Jiří Matoušek has been supported by Charles University Grant No. 351, by Czech Republic Grant GAČR 201/93/2167 and in part by DIMACS. Work by Micha Sharir has been supported by NSF Grant CCR-91-22103, by a Max-Planck Research Award, and by grants from the U.S.-Israeli Binational Science Foundation, the Fund for Basic Research administered by the Israeli Academy of Sciences, and the G.I.F., the German-Israeli Foundation for Scientific Research and Development.     

A strong bound on the integral of the central path curvature and its relationship with the iteration-complexity of primal-dual path-following LP algorithms

The main goals of this paper are to: i) relate two iteration-complexity bounds derived for the Mizuno-Todd-Ye predictor-corrector (MTY P-C) algorithm for linear programming (LP), and; ii) study the geometrical structure of the LP central path. The first iteration-complexity bound for the MTY P-C algorithm considered in this paper is expressed in terms of the integral of a certain curvature function over the traversed portion of the central path. The second iteration-complexity bound, derived recently by the authors using the notion of crossover events introduced by Vavasis and Ye, is expressed in terms of a scale-invariant condition number associated with m × n constraint matrix of the LP. In this paper, we establish a relationship between these bounds by showing that the first one can be majorized by the second one. We also establish a geometric result about the central path which gives a rigorous justification based on the curvature of the central path of a claim made by Vavasis and Ye, in view of the behavior of their layered least squares path following LP method, that the central path consists of long but straight continuous parts while the remaining curved part is relatively “short”. R. D. C. Monteiro was supported in part by NSF Grants CCR-0203113 and CCF-0430644 and ONR grant N00014-05-1-0183. T. Tsuchiya was supported in part by Japan-US Joint Research Projects of Japan Society for the Promotion of Science “Algorithms for linear programs over symmetric cones” and the Grants-in-Aid for Scientific Research (C) 15510144 of Japan Society for the Promotion of Science.     

An analysis of discontinuous Galerkin methods for elliptic problems

We study global and local behaviors for three kinds of discontinuous Galerkin schemes for elliptic equations of second order. We particularly investigate several a posteriori error estimations for the discontinuous Galerkin schemes. These theoretical results are applied to develop local/parallel and adaptive finite element methods, based on the discontinuous Galerkin methods. Dedicated to Dr. Charles A. Micchelli on the occasion of his 60th birthday with friendship and esteem Mathematics subject classifications (2000) 65N12, 65N15, 65N30. Aihui Zhou: Subsidized by the Special Funds for Major State Basic Research Projects, and also partially supported by National Science Foundation of China. Reinhold Schneider: Supported in part by DFG Sonderforschungsbereich SFB 393. Yuesheng Xu: Correspondence author. Supported in part by the US National Science Foundation under grants DMS-9973427 and CCR-0312113, by Natural Science Foundation of China under grant 10371122 and by the Chinese Academy of Sciences under program “Hundreds Distinguished Young Chinese Scientists”.     

Detecting “dense” columns in interior point methods for linear programs

During the iterations of interior point methods symmetric indefinite systems are decomposed by LD̂L ^T factorization. This step can be performed in a special way where the symmetric indefinite system is transformed to a positive definite one, called the normal equations system. This approach proved to be efficient in most of the cases and numerically reliable, due to the positive definite property. It has been recognized, however, that in case the linear program contains “dense” columns, this approach results in an undesirable fill–in during the computations and the direct factorization of the symmetric indefinite system is more advantageous. The paper describes a new approach to detect cases where the system of normal equations is not preferable for interior point methods and presents a new algorithm for detecting the set of columns which is responsible for the excessive fill–in in the matrix AA ^T . By solving large–scale linear programming problems we demonstrate that our heuristic is reliable in practice. This work was supported in part by the Hungarian Scientific Research Fund OTKA K60480.     

Fibrewise completion and unstable Adams spectral sequences

We describe a tower of spaces whose inverse limit is a “fiberwise completion” of a fibrationE →B, and study the resulting spectral sequence converging to the homotopy groups of the space of lifts of a mapX →B. This is used to give a proof of the “generalized Sullivan conjecture”. All three authors were supported in part by the National Science Foundation.     

Multi-Server Queueing Systems with Multiple Priority Classes

We present the first near-exact analysis of an M/PH/k queue with m > 2 preemptive-resume priority classes. Our analysis introduces a new technique, which we refer to as Recursive Dimensionality Reduction (RDR). The key idea in RDR is that the m-dimensionally infinite Markov chain, representing the m class state space, is recursively reduced to a 1-dimensionally infinite Markov chain, that is easily and quickly solved. RDR involves no truncation and results in only small inaccuracy when compared with simulation, for a wide range of loads and variability in the job size distribution. Our analytic methods are then used to derive insights on how multi-server systems with prioritization compare with their single server counterparts with respect to response time. Multi-server systems are also compared with single server systems with respect to the effect of different prioritization schemes—“smart” prioritization (giving priority to the smaller jobs) versus “stupid” prioritization (giving priority to the larger jobs). We also study the effect of approximating m class performance by collapsing the m classes into just two classes. Supported by NSF Career Grant CCR-0133077, NSF Theory CCR-0311383, NSF ITR CCR-0313148, and IBM Corporation via Pittsburgh Digital Greenhouse Grant 2003. AMS subject classification: 60K25, 68M20, 90B22, 90B36     

How to Play M 13?

A puzzle called “M ₁₃” J. H. Conway has described recently is explained. We report an implementation of the puzzle in the programming language Java. The program allows the human user to “play M ₁₃” interactively (and to cheat by solving it automatically). The program is an example on how to bring to life a nice piece of discrete mathematics. In this sense it presents not only a didactical way of seeing “mathematics at work”, but also displays the stabilizer chain method developed by C. Sims to solve group theoretic puzzles, the most famous of which being Rubik's cube.     

Geometric Aspects of Force Controllability for a Swimming Model

We study controllability properties (swimming capabilities) of a mathematical model of an abstract object which “swims” in the 2-D Stokes fluid. Our goal is to investigate how the geometric shape of this object affects the forces acting upon it. Such problems are of interest in biology and engineering applications dealing with propulsion systems in fluids. This work was supported in part by NSF Grant DMS-0504093.     

Efficient tree traversal to reduce code growth in tree-based genetic programming

Genetic programming is an evolutionary optimization method following the principle of program induction. Genetic programming often uses variable-length tree structures for representing candidate solutions. A serious problem with variable-length representations is code growth: during evolution these tree structures tend to grow in size without a corresponding increase in fitness. Many anti-bloat methods focus solely on size reduction and forget about fitness improvement, which is rather strange when using an “optimization” method. This paper reports the application of a semantically driven local search operator to control code growth and improve best fitness. Five examples, two theoretical benchmark applications and three real-life test problems are used to illustrate the obtained size reduction and fitness improvement. Performance of the local search operator is also compared with various other anti-bloat methods such as size and depth delimiters, an expression simplifier, linear and adaptive parsimony pressure, automatically defined functions and Tarpeian bloat control.     

Some remarks on solvability and various indices for implicit differential equations

It has been shown [17,18,21] that the notion of index for DAEs (Differential Algebraic Equations), or more generally implicit differential equations, could be interpreted in the framework of the formal theory of PDEs. Such an approach has at least two decisive advantages: on the one hand, its definition is not restricted to a “state-space” formulation (order one systems), so that it may be computed on “natural” model equations coming from physics (which can be, for example, second or fourth order in mechanics, second order in electricity, etc.) and there is no need to destroy this natural way through a first order rewriting. On the other hand, this formal framework allows for a straightforward generalization of the index to the case of PDEs (either “ordinary” or “algebraic”). In the present work, we analyze several notions of index that appeared in the literature and give a simple interpretation of each of them in the same general framework and exhibit the links they have with each other, from the formal point of view. Namely, we shall revisit the notions of differential, perturbation, local, global indices and try to give some clarification on the solvability of DAEs, with examples on time-varying implicit linear DAEs. No algorithmic results will be given here (see [34,35] for computational issues) but it has to be said that the complexity of computing the index, whatever approach is taken, is that of differential elimination, which makes it a difficult problem. We show that in fact one essential concept for our approach is that of formal integrability for usual DAEs and that of involution for PDEs. We concentrate here on the first, for the sake of simplicity. Last, because of the huge amount of work on DAEs in the past two decades, we shall mainly mention the most recent results. This revised version was published online in June 2006 with corrections to the Cover Date.     

Newton-KKT interior-point methods for indefinite quadratic programming

Two interior-point algorithms are proposed and analyzed, for the (local) solution of (possibly) indefinite quadratic programming problems. They are of the Newton-KKT variety in that (much like in the case of primal-dual algorithms for linear programming) search directions for the “primal” variables and the Karush-Kuhn-Tucker (KKT) multiplier estimates are components of the Newton (or quasi-Newton) direction for the solution of the equalities in the first-order KKT conditions of optimality or a perturbed version of these conditions. Our algorithms are adapted from previously proposed algorithms for convex quadratic programming and general nonlinear programming. First, inspired by recent work by P. Tseng based on a “primal” affine-scaling algorithm (à la Dikin) [J. of Global Optimization, 30 (2004), no. 2, 285–300], we consider a simple Newton-KKT affine-scaling algorithm. Then, a “barrier” version of the same algorithm is considered, which reduces to the affine-scaling version when the barrier parameter is set to zero at every iteration, rather than to the prescribed value. Global and local quadratic convergence are proved under nondegeneracy assumptions for both algorithms. Numerical results on randomly generated problems suggest that the proposed algorithms may be of great practical interest. The work of the first author was supported in part by the School of Computational Science of Florida State University through a postdoctoral fellowship. Part of this work was done while this author was a Research Fellow with the Belgian National Fund for Scientific Research (Aspirant du F.N.R.S.) at the University of Liège. The work of the second author was supported in part by the National Science Foundation under Grants DMI9813057 and DMI-0422931 and by the US Department of Energy under Grant DEFG0204ER25655. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation or those of the US Department of Energy.     

An Adaptive Finite Element Method for the H- ψ Formulation of Time-dependent Eddy Current Problems

In this paper, we develop an adaptive finite element method based on reliable and efficient a posteriori error estimates for the H − ψ formulation of eddy current problems with multiply connected conductors. Multiply connected domains are considered by making “cuts”. The competitive performance of the method is demonstrated by an engineering benchmark problem, Team Workshop Problem 7, and a singular problem with analytic solution.W. Zheng was supported in part by China NSF under the grant 10401040.Z. Chen was supported in part by China NSF under the grant 10025102 and 10428105, and by the National Basic Research Project under the grant 2005CB321701.     

Two-letter group codes that preserve aperiodicity of inverse finite automata

We construct group codes over two letters (i.e., bases of subgroups of a two-generated free group) with special properties. Such group codes can be used for reducing algorithmic problems over large alphabets to algorithmic problems over a two-letter alphabet. Our group codes preserve aperiodicity of inverse finite automata. As an application we show that the following problems are PSpace-complete for two-letter alphabets (this was previously known for large enough finite alphabets): The intersection-emptiness problem for inverse finite automata, the aperiodicity problem for inverse finite automata, and the closure-under-radical problem for finitely generated subgroups of a free group. The membership problem for 3-generated inverse monoids is PSpace-complete. Both authors were supported in part by NSF grant DMS-9970471. The first author was also supported in part by NSF grant CCR-0310793. The second author acknowledges the support of the Excellency Center, “Group Theoretic Methods for the Study of Algebraic Varieties” of the Israeli Science Foundation.