Estimating the complexity of a class of path-following methods for solving linear programs by curvature integrals

In this paper we study a particular class of primal-dual path-following methods which try to follow a trajectory of interior feasible solutions in primal-dual space toward an optimal solution to the primal and dual problem. The methods investigated are so-called first-order methods: each iteration consists of a long step along the tangent of the trajectory, followed by explicit recentering steps to get close to the trajectory again. It is shown that the complexity of these methods, which can be measured by the number of points close to the trajectory which have to be computed in order to achieve a desired gain in accuracy, is bounded by an integral along the trajectory. The integrand is a suitably weighted measure of the second derivative of the trajectory with respect to a distinguished path parameter, so the integral may be loosely called a curvature integral.     

Primal Central Paths and Riemannian Distances for Convex Sets

In this paper, we study the Riemannian length of the primal central path in a convex set computed with respect to the local metric defined by a self-concordant function. Despite some negative examples, in many important situations, the length of this path is quite close to the length of a geodesic curve. We show that in the case of a bounded convex set endowed with a ν-self-concordant barrier, the length of the central path is within a factor O(ν ^1/4) of the length of the shortest geodesic curve. This paper presents research results of the Belgian Program on Interuniversity Poles of Attraction initiated by the Belgian State, Prime Minister’s Office, Science Policy Programming.     

Existence, uniqueness, and convergence of the regularized primal-dual central path

In a recent work [J. Castro, J. Cuesta, Quadratic regularizations in an interior-point method for primal block-angular problems, Mathematical Programming, in press (doi:10.1007/s10107-010-0341-2)] the authors improved one of the most efficient interior-point approaches for some classes of block-angular problems. This was achieved by adding a quadratic regularization to the logarithmic barrier. This regularized barrier was shown to be self-concordant, thus fitting the general structural optimization interior-point framework. In practice, however, most codes implement primal-dual path-following algorithms. This short paper shows that the primal-dual regularized central path is well defined, i.e., it exists, it is unique, and it converges to a strictly complementary primal-dual solution.     

A strong bound on the integral of the central path curvature and its relationship with the iteration-complexity of primal-dual path-following LP algorithms

The main goals of this paper are to: i) relate two iteration-complexity bounds derived for the Mizuno-Todd-Ye predictor-corrector (MTY P-C) algorithm for linear programming (LP), and; ii) study the geometrical structure of the LP central path. The first iteration-complexity bound for the MTY P-C algorithm considered in this paper is expressed in terms of the integral of a certain curvature function over the traversed portion of the central path. The second iteration-complexity bound, derived recently by the authors using the notion of crossover events introduced by Vavasis and Ye, is expressed in terms of a scale-invariant condition number associated with m × n constraint matrix of the LP. In this paper, we establish a relationship between these bounds by showing that the first one can be majorized by the second one. We also establish a geometric result about the central path which gives a rigorous justification based on the curvature of the central path of a claim made by Vavasis and Ye, in view of the behavior of their layered least squares path following LP method, that the central path consists of long but straight continuous parts while the remaining curved part is relatively “short”. R. D. C. Monteiro was supported in part by NSF Grants CCR-0203113 and CCF-0430644 and ONR grant N00014-05-1-0183. T. Tsuchiya was supported in part by Japan-US Joint Research Projects of Japan Society for the Promotion of Science “Algorithms for linear programs over symmetric cones” and the Grants-in-Aid for Scientific Research (C) 15510144 of Japan Society for the Promotion of Science.     

Infinite-dimensional quadratic optimization: Interior-point methods and control applications

An infinite-dimensional convex optimization problem with the linear-quadratic cost function and linear-quadratic constraints is considered. We generalize the interior-point techniques of Nesterov-Nemirovsky to this infinite-dimensional situation. The complexity estimates obtained are similar to finite-dimensional ones. We apply our results to the linear-quadratic control problem with quadratic constraints. It is shown that for this problem the Newton step is basically reduced to the standard LQ problem. This research was supported in part by the Cooperative Research Centre for Robust and Adaptive Systems while the first author visited the Australian National University and by NSF Grant DMS 94-23279.     

The Kantorovich Theorem and interior point methods

The Kantorovich Theorem is a fundamental tool in nonlinear analysis which has been extensively used in classical numerical analysis. In this paper we show that it can also be used in analyzing interior point methods. We obtain optimal bounds for Newtons method when relied upon in a path following algorithm for linear complementarity problems.Given a point z that approximates a point z() on the central path with complementarity gap , a parameter (0,1) is determined for which the point z satisfies the hypothesis of the affine invariant form of the Kantorovich Theorem, with regards to the equation defining z((1–)). The resulting iterative algorithm produces a point with complementarity gap less than in at most Newton steps, or simplified Newton steps, where ₀ is the complementarity gap of the starting point and n is the dimension of the problem. Thus we recover the best complexity results known to date and, in addition, we obtain the best bounds for Newtons method in this context.Mathematics Subject Classification (2000): 49M15, 65H10, 65K05, 90C33Work supported by the National Science Foundation under Grants No. 9996154, 0139701Acknowledgement The original version of this paper [38] was written when the author was visiting the Konrad-Zuse-Zentrum für Informationstechnik Berlin (ZIB). The author would like to thank Peter Deuflhard, President of ZIB, for the excellent working conditions at ZIB, and for very interesting discussions on various mathematical problems related to the subject of the present paper. The author would also like to thank two anonymous referees whose comments and suggestions led to a better presentation of our results.     

Implementation of a continuation method for normal maps

This paper presents an implementation of a nonsmooth continuation method of which the idea was originally put forward by Alexander et al. We show how the method can be computationally implemented and present numerical results for variational inequality problems in up to 96 variables. The research reported here was sponsored by the Air Force Office of Scientific Research, Air Force Materiel Command, USAF, under grant numbers F49620-93-1-0068 and F49620-95-1-0222, by the U.S. Army Research Office under grant number DAAH04-95-1-0149, and by the National Science Foundation under grant number CCR-9109345. The U.S. Government has certain rights in this material, and is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the sponsoring agencies or the U.S. Government.     

Combining a path method and parametric linear programming for the computation of competitive equilibria

A path-following philosophy (continuation method, global Newton method) is used to compute equilibria for piecewise linear economies while taking advantage of the linear structure of the model. The existence of a path leading through certain faces of a polyhedral set to an equilibrium point is demonstrated. Computational experience is reported which indicates that this method is promising for models dealing with many commodities and relatively few consumers.Most of this paper has been extracted from the author's doctoral dissertation for the Department of Operations Research at Stanford University; the author would like to express indebtedness to his advisor, R. Wilson. Major revisions were made while the author was at Bell Laboratories in Whippany, New Jersey.     

Accelerated Path-Following Iterative Shrinkage Thresholding Algorithm With Application to Semiparametric Graph Estimation

We propose an accelerated path-following iterative shrinkage thresholding algorithm (APISTA) for solving high-dimensional sparse nonconvex learning problems. The main difference between APISTA and the path-following iterative shrinkage thresholding algorithm (PISTA) is that APISTA exploits an additional coordinate descent subroutine to boost the computational performance. Such a modification, though simple, has profound impact: APISTA not only enjoys the same theoretical guarantee as that of PISTA, that is, APISTA attains a linear rate of convergence to a unique sparse local optimum with good statistical properties, but also significantly outperforms PISTA in empirical benchmarks. As an application, we apply APISTA to solve a family of nonconvex optimization problems motivated by estimating sparse semiparametric graphical models. APISTA allows us to obtain new statistical recovery results that do not exist in the existing literature. Thorough numerical results are provided to back up our theory.     

General primal-dual penalty/barrier path-following Newton methods for nonlinear programming

In the present article rather general penalty/barrier-methods are considered, that define a local continuously differentiable primal-dual path. The class of penalty/barrier terms includes most of the usual techniques like logarithmic barriers, SUMT, quadratic loss functions as well as exponential penalties, and the optimization problem which may contain inequality as well as equality constraints. The convergence of the corresponding general primal-dual path-following method is shown for local minima that satisfy strong second-order sufficiency conditions with linear independence constraint qualification (LICQ) and strict complementarity. A basic tool in the analysis of these methods is to estimate the radius of convergence of Newton's method depending on the penalty/barrier-parameter. Without using self-concordance properties convergence bounds are derived by direct estimations of the solutions of the Newton equations. Parameter selection rules are proposed which guarantee the local convergence of the considered penalty/barrier-techniques with only a finite number of Newton steps at each parameter level. Numerical examples illustrate the practical behavior of the proposed class of methods.