The method of successive affine reduction for nonlinear minimization

The traditional development of conjugate gradient (CG) methods emphasizes notions of conjugacy and the minimization of quadratic functions. The associated theory of conjugate direction methods, strictly a branch of numerical linear algebra, is both elegant and useful for obtaining insight into algorithms for nonlinear minimization. Nevertheless, it is preferable that favorable behavior on a quadratic be a consquence of a more general approach, one which fits in more naturally with Newton and variable metric methods. We give new CG algorithms along these lines and discuss some of their properties, along with some numerical supporting evidence.I acknowledge partial support by ONR Contract #N00014-76-C-0013 at the Center for Pure & Applied Mathematics, University of California, Berkeley in the dissemination of this article.