Numerical comparison of several variable-metric algorithms

The paper compares the numerical performances of the LDL decomposition of the BFGS variable-metric algorithm, the Dennis-Mei dogleg algorithm on the BFGS update, and Davidon's projections with the BFGS update with the straight BFGS update on a number of standard test problems. Numerical results indicate that the standard BFGS algorithm is superior to all of the more complex strategies.This research was supported by the National Science Foundation under Research Grant No. MCS77-07327.     

Interior-point methods for nonconvex nonlinear programming: cubic regularization

In this paper, we present a primal-dual interior-point method for solving nonlinear programming problems. It employs a Levenberg-Marquardt (LM) perturbation to the Karush-Kuhn-Tucker (KKT) matrix to handle indefinite Hessians and a line search to obtain sufficient descent at each iteration. We show that the LM perturbation is equivalent to replacing the Newton step by a cubic regularization step with an appropriately chosen regularization parameter. This equivalence allows us to use the favorable theoretical results of Griewank (The modification of Newton’s method for unconstrained optimization by bounding cubic terms, 1981), Nesterov and Polyak (Math. Program., Ser. A 108:177–205, 2006), Cartis et al. (Math. Program., Ser. A 127:245–295, 2011; Math. Program., Ser. A 130:295–319, 2011), but its application at every iteration of the algorithm, as proposed by these papers, is computationally expensive. We propose a hybrid method: use a Newton direction with a line search on iterations with positive definite Hessians and a cubic step, found using a sufficiently large LM perturbation to guarantee a steplength of 1, otherwise. Numerical results are provided on a large library of problems to illustrate the robustness and efficiency of the proposed approach on both unconstrained and constrained problems.     

A unified view of interior point methods for linear programming

The paper shows how various interior point methods for linear programming may all be derived from logarithmic barrier methods. These methods include primal and dual projective methods, affine methods, and methods based on the method of centers. In particular, the paper demonstrates that Karmarkar's algorithm is equivalent to a classical logarithmic barrier method applied to a problem in standard form.Invited paper presented at the Workshop on Supercomputers in Optimization, Minneapolis, Minn., May 1988.The work of this author was supported by the Air Force Office of Scientific Research, Air Force Systems Command, USA, under Grants AFOSR-87-0215 and AFOSR-85-0271. The US Government is authorized to reproduce and distribute reprints for Governmental purposes not withstanding any copyright notation thereon.     

An exact primal–dual penalty method approach to warmstarting interior-point methods for linear programming

One perceived deficiency of interior-point methods in comparison to active set methods is their inability to efficiently re-optimize by solving closely related problems after a warmstart. In this paper, we investigate the use of a primal–dual penalty approach to overcome this problem. We prove exactness and convergence and show encouraging numerical results on a set of linear and mixed integer programming problems. Research of the first author is sponsored by ONR grant N00014-04-1-0145. Research of the second author is supported by NSF grant DMS-0107450.     

A note on solving nonlinear minimax problems via a differentiable penalty function

The note demonstrates that modeling a nonlinear minimax problem as a nonlinear programming problem and applying a classical differentiable penalty function to attempt to solve the problem can lead to convergence to a stationary point of the penalty function which is not a feasible point of the nonlinear programming problem. This occurred naturally in an application from statistical reliability theory. The note resolves the problem through modification of both the problem formulation and the iterative penalty function method.     

A reduced-gradient variant of Karmarkar's algorithm and null-space projections

This paper discusses the relationship between Karmarkar's new method for linear programming and the traditional simplex method. It is shown how null-space Karmarkar projections can be done using a basis matrix to compute the projections in the null space. Preliminary computational evidence shows that problems exist in the choice of a basis matrix, but that, given a basis, very inexact and computationally efficient projections are computationally sound.     

Globally convergent conjugate gradient algorithms

The paper examines a method of implementing an angle test for determining when to restart conjugate gradient methods in a steepest descent direction. The test is based on guaranteeing that the cosine of the angle between the search direction and the negative gradient is within a constant multiple of the cosine of the angle between the Fletcher-Reeves search direction and the negative gradient. This guarantees convergence, for the Fletcher-Reeves method is known to converge. Numerical results indicate little, if anything, is lost in efficiency, and indicate gains may well be possible for large problems.     

Cubic regularization in symmetric rank-1 quasi-Newton methods

Quasi-Newton methods based on the symmetric rank-one (SR1) update have been known to be fast and provide better approximations of the true Hessian than popular rank-two approaches, but these properties are guaranteed under certain conditions which frequently do not hold. Additionally, SR1 is plagued by the lack of guarantee of positive definiteness for the Hessian estimate. In this paper, we propose cubic regularization as a remedy to relax the conditions on the proofs of convergence for both speed and accuracy and to provide a positive definite approximation at each step. We show that the n-step convergence property for strictly convex quadratic programs is retained by the proposed approach. Extensive numerical results on unconstrained problems from the CUTEr test set are provided to demonstrate the computational efficiency and robustness of the approach.     

Computational experience with a globally convergent primal—dual predictor—corrector algorithm for linear programming

Kojima, Megiddo, and Mizuno proved global convergence of a primal—dual algorithm that corresponds to methods used in practice. Here, the numerical efficiency of a predictor—corrector extension of that algorithm is tested. Numerical results are extremely positive, indicating that the safety of a globally convergent algorithm can be obtained at little computational cost. The algorithm is tested on infeasible problems with less success. Finally, the algorithm is applied to a warm started problem, with very encouraging preliminary results.Corresponding author. The research of this author is sponsored by the Air Force Office of Scientific Research, Air Force System Command under Grant AFOSR-92-J0046. The United States Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notations thereon.