A model trust-region modification of Newton's method for nonlinear two-point boundary-value problems

The method of quasilinearization for nonlinear two-point boundary-value problems is Newton's method for a nonlinear differential operator equation. A model trust-region approach to globalizing the quasilinearization algorithm is presented. A double-dogleg implementation yields a globally convergent algorithm that is robust in solving difficult problems.This work was supported in part by DOE Contract DE-AS05-82-ER13016 and NSF Grant RII-89-17691 and was part of the author's doctoral thesis at Rice University. It is a pleasure to thank the author's thesis advisors, Professor J. E. Dennis, Jr., and Professor R. A. Tapia.     

An inexact Newton method for nonlinear two-point boundary-value problems

The method of quasilinearization for nonlinear two-point boundary-value problems is an application of Newton's method to a nonlinear differential operator equation. Since the linear boundary-value problem to be solved at each iteration must be discretized, it is natural to consider quasilinearization in the framework of an inexact Newton method. More importantly, each linear problem is only a local model of the nonlinear problem, and so it is inefficient to try to solve the linear problems to full accuracy. Conditions on size of the relative residual of the linear differential equation can then be specified to guarantee rapid local convergence to the solution of the nonlinear continuous problem. If initial-value techniques are used to solve the linear boundary-value problems, then an integration step selection scheme is proposed so that the residual criteria are satisfied by the approximate solutions. Numerical results are presented that demonstrate substantial computational savings by this type of economizing on the intermediate problems.This work was supported in part by DOE Contract DE-AS05-82-ER13016 and NSF Grant RII-89-17691 and was part of the author's doctoral thesis at Rice University. It is a pleasure to thank the author's thesis advisors, Professor R. A. Tapia and Professor J. E. Dennis, Jr.     

Superlinear and quadratic convergence of primal-dual interior-point methods for linear programming revisited

Recently, Zhang, Tapia, and Dennis (Ref. 1) produced a superlinear and quadratic convergence theory for the duality gap sequence in primal-dual interior-point methods for linear programming. In this theory, a basic assumption for superlinear convergence is the convergence of the iteration sequence; and a basic assumption for quadratic convergence is nondegeneracy. Several recent research projects have either used or built on this theory under one or both of the above-mentioned assumptions. In this paper, we remove both assumptions from the Zhang-Tapia-Dennis theory.Dedicated to the Memory of Magnus R. Hestenes, 1906–1991This research was supported in part by NSF Cooperative Agreement CCR-88-09615 and was initiated while the first author was at Rice University as a Visiting Member of the Center for Research in Parallel Computation.The authors thank Yinyu Ye for constructive comments and discussions concerning this material.This author was supported in part by NSF Grant DMS-91-02761 and DOE Grant DE-FG05-91-ER25100.This author was supported in part by AFOSR Grant 89-0363, DOE Grant DE-FG05-86-ER25017, and ARO Grant 9DAAL03-90-G-0093.     

A quadratically convergent O(\sqrt n L)-iteration algorithm for linear programming

Recently, Ye, Tapia and Zhang (1991) demonstrated that Mizuno—Todd—Ye's predictor—corrector interior-point algorithm for linear programming maintains the O( L)-iteration complexity while exhibiting superlinear convergence of the duality gap to zero under the assumption that the iteration sequence converges, and quadratic convergence of the duality gap to zero under the assumption of nondegeneracy. In this paper we establish the quadratic convergence result without any assumption concerning the convergence of the iteration sequence or nondegeneracy. This surprising result, to our knowledge, is the first instance of a demonstration of polynomiality and superlinear (or quadratic) convergence for an interior-point algorithm which does not assume the convergence of the iteration sequence or nondegeneracy.Supported in part by NSF Grant DDM-8922636 and NSF Coop. Agr. No. CCR-8809615, the Iowa Business School Summer Grant, and the Interdisciplinary Research Grant of the University of Iowa Center for Advanced Studies.Supported in part by NSF Coop. Agr. No. CCR-8809615, AFOSR 89-0363, DOE DEFG05-86ER25017 and ARO 9DAAL03-90-G-0093.Supported in part by NSF Grant DMS-9102761 and DOE Grant DE-FG05-91ER25100.     

An interior semi-infinite programming method

In this paper, a new method for semi-infinite programming problems with convex constraints is presented. The method generates a sequence of feasible points whose cluster points are solutions of the original problem. No maximization over the index set is required. Some computational results are also presented.This work was partly supported by Republicka Zajednica za Nauku Socijalisticke Republike Srbije. The authors are indebted to Professor R. A. Tapia for encouraging the approach taken in this research.     

A note on the method of multipliers

Implementation of the penalty function method for constrained optimization poses numerical difficulties as the penalty parameter increases. To offset this problem, one often resorts to Newton's method. In this note, working in the context of the penalty function method, we establish an intimate connection between the second-order updating formulas which result from Newton's method on the primal problem and Newton's method on the dual problem.The author wishes to thank Professor R. A. Tapia for his careful review of this note. He has contributed significantly to its content through several crucial observations.     

Exact order of convergence of the secant method

We study the exact order of convergence of the secant method when applied to the problem of finding a zero of a nonlinear function defined from into . Under the standard assumptions for which Newton's method has the exact Q-order of convergencep, wherep is some positive integer, we establish that the secant method has the Q-order and the exact R-order of convergence . We prove also that, forp=2 andp=3, the secant method has the exact Q-order of convergenceS(p). Moreover, we present a counterexample to show that, forp4, it may not have an exact Q-order of convergence.The author wishes to thank Florian Potra, Richard Tapia, and the referees for helpful comments and suggestions.This paper was prepared while the author was Visiting Professor, Department of Mathematics, University of Kentucky, Lexington, Kentucky.     

Hybrid method for nonlinear least-square problems without calculating derivatives

This paper presents a no-derivative modification of the hybrid Gauss-Newton-BFGS method for nonlinear least-square problems suggested initially by Al-Baali and Fletcher and modified later by Fletcher and Xu. The modification is made in such a way that, in a Gauss-Newton step, the Broyden's rank-one updating formula is used to obtain an approximate Jacobian and, in a BFGS step, the Jacobian is estimated using difference formulas. A set of numerical comparisons among the new hybrid method, the Gauss-Newton-Broyden method, and the finite-difference BFGS method is made and shows that the new hybrid method combines the better features of the Gauss-Newton-Broyden method and the finite-difference BFGS method. This paper also extends to the least-square problem the finite-termination property of the Broyden method, proved for a nonsingular system of equations by Gay and for the full-rank rectangular system of equations by Gerber and Luk.The author would like to acknowledge the support of Xian Jiaotong University, Xian, China, and the award of a United Kingdom ORS studentship. The author wishes to express his gratitute to Professor R. Fletcher for his encouragement and to thank Dr. G. A. Watson and Dr. M. C. Bartholomew-Biggs for their useful comments during the preparation of this paper. The author also wishes to acknowledge Professor R. A. Tapia for his valuable suggestions.     

OnQ-order andR-order of convergence

We give sufficient conditions for a sequence to have theQ-order and/or theR-order of convergence greater than one. If an additional condition is satisfied, then the sequence has an exactQ-order of convergence. We show that our results are sharp and we compare them with older results.This work was supported in part by the National Science Foundation under Grant No. DMS-85-03365. The author wishes to thank J. E. Dennis and R. A. Tapia for helpful comments, and the referee for pointing out a number of typographical and mathematical errors in the original version of this paper.     

Local convergence analysis of a grouped variable version of coordinate descent

LetF(x,y) be a function of the vector variablesxR ⁿ andyR ^m . One possible scheme for minimizingF(x,y) is to successively alternate minimizations in one vector variable while holding the other fixed. Local convergence analysis is done for this vector (grouped variable) version of coordinate descent, and assuming certain regularity conditions, it is shown that such an approach is locally convergent to a minimizer and that the rate of convergence in each vector variable is linear. Examples where the algorithm is useful in clustering and mixture density decomposition are given, and global convergence properties are briefly discussed.This research was supported in part by NSF Grant No. IST-84-07860. The authors are indebted to Professor R. A. Tapia for his help in improving this paper.     

Globalizing a nonsmooth Newton method via nonmonotone path search

We give a framework for the globalization of a nonsmooth Newton method. In part one we start with recalling B. Kummer’s approach to convergence analysis of a nonsmooth Newton method and state his results for local convergence. In part two we give a globalized version of this method. Our approach uses a path search idea to control the descent. After elaborating the single steps, we analyze and prove the global convergence resp. the local superlinear or quadratic convergence of the algorithm. In the third part we illustrate the method for nonlinear complementarity problems.     

Numerical Comparisons of Path-Following Strategies for a Primal-Dual Interior-Point Method for Nonlinear Programming

An important research activity in primal-dual interior-point methods for general nonlinear programming is to determine effective path-following strategies and their implementations. The objective of this work is to present numerical comparisons of several path-following strategies for the local interior-point Newton method given by El-Bakry, Tapia, Tsuchiya, and Zhang. We conduct numerical experimentation of nine strategies using two central regions, three notions of proximity measures, and three merit functions to obtain an optimal solution. Six of these strategies are implemented for the first time. The numerical results show that the best path-following strategy is that given by Argáez and Tapia.     

Superlinear convergence of a trust region-type successive linear programming method

The convergence rate of the SLP method suggested in Ref. 1 is discussed for composite nondifferentiable optimization problems. A superlinear rate is assured under a growth condition, and it is further strengthened to a quadratic rate if the inside function is twice differentiable. Several sufficient conditions are given which make the growth condition true. These conditions can be relaxed considerably in practical use.This research was supported in part by the National Natural Science Foundation of China.The author is grateful to Professor E. Sachs, Universität Trier, Trier, West Germany, for his helpful suggestions.     

A superlinearly and quadratically convergent SQP type feasible method for constrained optimization

A new SQP type feasible method for inequality constrained optimization is presented, it is a combination of a master algorithm and an auxiliary algorithm which is taken only in finite iterations. The directions of the master algorithm are generated by only one quadratic programming, and its step-size is always one, the directions of the auxiliary algorithm are new “secondorder“ feasible descent. Under suitable assumptions, the algorithm is proved to possess global and strong convergence, superlinear and quadratic convergence.     

An algebraic multigrid method for finite element systems on criss-cross grids

In this paper, we design and analyze an algebraic multigrid method for a condensed finite element system on criss-cross grids and then provide a convergence analysis. Criss-cross grid finite element systems represent a large class of finite element systems that can be reduced to a smaller system by first eliminating certain degrees of freedoms. The algebraic multigrid method that we construct is analogous to many other algebraic multigrid methods for more complicated problems such as unstructured grids, but, because of the specialty of our problem, we are able to provide a rigorous convergence analysis to our algebraic multigrid method. Dedicated to Professor Charles A. Micchelli on the occasion of his 60th birthday The work was supported in part by NSAF(10376031) and National Major Key Project for basic researches and by National High-Tech ICF Committee in China.     

On the convergence rate of Newton interior-point methods in the absence of strict complementarity

In the absence of strict complementarity, Monteiro and Wright [7] proved that the convergence rate for a class of Newton interior-point methods for linear complementarity problems is at best linear. They also established an upper bound of 1/4 for the Q ₁-factor of the duality gap sequence when the steplengths converge to one. In the current paper, we prove that the Q ₁ factor of the duality gap sequence is exactly 1/4. In addition, the convergence of the Tapia indicators is also discussed.This author was supported in part by NSF Coop. Agr. No. CCR-8809615 and AFOSR 89-0363 and the REDI Foundation.This author was supported in part by NSF Coop. Agr. No. CCR-8809615, AFOSR 89-0363, DOE DEFG05-86ER25017 and ARO 9DAAL03-90-G-0093.Visiting member of the Center for Research on Parallel Computations, Rice University, Houston, Texas, 77251-1892. This author was supported in part by DOE DE-FG02-93ER25171.     

Modified Newton’s method for systems of nonlinear equations with singular Jacobian

It is well known that Newton’s method for a nonlinear system has quadratic convergence when the Jacobian is a nonsingular matrix in a neighborhood of the solution. Here we present a modification of this method for nonlinear systems whose Jacobian matrix is singular. We prove, under certain conditions, that this modified Newton’s method has quadratic convergence. Moreover, different numerical tests confirm the theoretical results and allow us to compare this variant with the classical Newton’s method.     

Two-step and three-step Q-superlinear convergence of SQP methods

This paper investigates the convergence rates of the variable-multiplier pair (x, ) in sequential quadratic programming methods for equality constrained optimization. The two main results of the paper are that the Q-superlinear convergence of {x _k } implies two-step Q-superlinear convergence for {(x _k , _k )} and that the two-step Q-superlinear convergence of {x _k } implies three-step Q-superlinear convergence for {(x _k , _k )}.The author is indebted to Professor Richard Tapia for many helpful comments and suggestions on the paper. The comments by Professors A. R. Conn and N. I. M. Gould on an earlier version are also acknowledged. This research was funded by SERC and ESRC research contracts.     

Grouped coordinate minimization using Newton's method for inexact minimization in one vector coordinate

Letf(x,y) be a function of the vector variablesx R ⁿ andy R ^m. The grouped (variable) coordinate minimization (GCM) method for minimizingf consists of alternating exact minimizations in either of the two vector variables, while holding the other fixed at the most recent value. This scheme is known to be locally,q-linearly convergent, and is most useful in certain types of statistical and pattern recognition problems where the necessary coordinate minimizers are available explicitly. In some important cases, the exact minimizer in one of the vector variables is not explicitly available, so that an iterative technique such as Newton's method must be employed. The main result proved here shows that a single iteration of Newton's method solves the coordinate minimization problem sufficiently well to preserve the overall rate of convergence of the GCM sequence.The authors are indebted to Professor R. A. Tapia for his help in improving this paper.     

非线性特征值问题的二次近似方法

本文研究求解非线性特征值问题的数值方法.基于矩阵值函数的二次近似,将非线性特征值问题转化为二次特征值问题,提出了求解非线性特征值问题的逐次二次近似方法,分析了该方法的收敛性.结合求解二次特征值问题的Arnoldi方法和Jacobi-Davidson方法,给出求解非线性特征值问题的一些二次近似方法.数值结果表明本文所给算法是有效的.