Semilocal and global convergence of the Newton‐HSS method for systems of nonlinear equations

Newton‐HSS methods, which are variants of inexact Newton methods different from the Newton–Krylov methods, have been shown to be competitive methods for solving large sparse systems of nonlinear equations with positive‐definite Jacobian matrices (J. Comp. Math. 2010; 28 :235–260). In that paper, only local convergence was proved. In this paper, we prove a Kantorovich‐type semilocal convergence. Then we introduce Newton‐HSS methods with a backtracking strategy and analyse their global convergence. Finally, these globally convergent Newton‐HSS methods are shown to work well on several typical examples using different forcing terms to stop the inner iterations. Copyright © 2010 John Wiley & Sons, Ltd.     

Hybrid Newton-Type Method for a Class of Semismooth Equations

In this paper, we present a hybrid method for the solution of a class of composite semismooth equations encountered frequently in applications. The method is obtained by combining a generalized finite-difference Newton method to an inexpensive direct search method. We prove that, under standard assumptions, the method is globally convergent with a local rate of convergence which is superlinear or quadratic. We report also several numerical results obtained applying the method to suitable reformulations of well-known nonlinear complementarity problems.     

Truncated Trust Region Methods Based on Preconditioned Iterative Subalgorithms for Large Sparse Systems of Nonlinear Equations

This paper is devoted to globally convergent methods for solving large sparse systems of nonlinear equations with an inexact approximation of the Jacobian matrix. These methods include difference versions of the Newton method and various quasi-Newton methods. We propose a class of trust region methods together with a proof of their global convergence and describe an implementable globally convergent algorithm which can be used as a realization of these methods. Considerable attention is concentrated on the application of conjugate gradient-type iterative methods to the solution of linear subproblems. We prove that both the GMRES and the smoothed COS well-preconditioned methods can be used for the construction of globally convergent trust region methods. The efficiency of our algorithm is demonstrated computationally by using a large collection of sparse test problems.     

A Convergent Variant of the Nelder–Mead Algorithm

The Nelder–Mead algorithm (1965) for unconstrained optimization has been used extensively to solve parameter estimation and other problems. Despite its age, it is still the method of choice for many practitioners in the fields of statistics, engineering, and the physical and medical sciences because it is easy to code and very easy to use. It belongs to a class of methods which do not require derivatives and which are often claimed to be robust for problems with discontinuities or where the function values are noisy. Recently (1998), it has been shown that the method can fail to converge or converge to nonsolutions on certain classes of problems. Only very limited convergence results exist for a restricted class of problems in one or two dimensions. In this paper, a provably convergent variant of the Nelder–Mead simplex method is presented and analyzed. Numerical results are included to show that the modified algorithm is effective in practice.     

一类多值多导数方法的收敛性与稳定性

1 引言 对于多值多导数方法,由于其多值多导的结构特点有利于提高解的精度,以及其包容性大,它包含了当今常用的多种常微数值方法,诸如:线性多步法,单支方法,多步多导方法,多(单)步Runge—Kutta方法,多导Runge-Kutta方法以及混合方法等.因此收敛性与稳定性的研究具有重要的实践意义和广泛的理论指导意义,也正因如此,这方面的研究工作引起了众多数值工作者们的兴趣,近年来,多值多导法求解刚性问题的B—收敛及其非线性稳定性的研究工作巳获得较大进展,其相应成果可参见文献[1—3],在文献[4,5]中笔者则针对Banach空间中一类非刚性问题-K~((p))类问题,分别探讨了多步多导法及单支方法的收敛性     

一种新的求解线性方程组的外推加速方法

本文提出两种优化模型, 通过在子空间{x^(k),…, x^(k-m)}上寻找最优解, 建立了一种新的外推加速方法. 讨论了该方法的收敛性和收敛速度. 最后, 通过三个数值实例展示了算法是可行的和有效的.     

On the uniform nonsingularity of matrices of search directions and the rate of convergence in minimization algorithms

A convergent minimization algorithm made up of repetitive line searches is considered in ⁿ . It is shown that the uniform nonsingularity of the matrices consisting ofn successive normalized search directions guarantees a speed of convergence which is at leastn-step Q-linear. Consequences are given for multistep methods, including Powell's 1964 procedure for function minimization without calculating derivatives as well as Zangwill's modifications of this procedure.The authors wish to thank the Namur Department of Mathematics, especially its optimization group, for many discussions and encouragement. They also thank the reviewers for many helpful suggestions.     

修正乘子交替方向法求解三个可分离算子的凸优化

指出直接推广的经典乘子交替方向法对三个算子的问题不能保证收敛的原因, 并且给出将其改造成收敛算法的相应策略. 同时, 在一个统一框架下, 证明了修正的乘子交替方向法的收敛性和遍历意义下具有O(1/t)~收敛速率.     

Numerical solution of nonlinear equations by an optimal eighth-order class of iterative methods

Solving nonlinear equations by using iterative methods is discussed in this paper. An optimally convergent class of efficient three-point three-step methods without memory is suggested. Analytical proof for the class of methods is given to show the eighth-order convergence and also reveal its consistency with the conjecture of Kung and Traub. The beauty in the proposed methods from the class can be seen because of the optimization in important effecting factors, i.e. optimality order, lesser number of functional evaluations; as well as in viewpoint of efficiency index. The accuracy of some iterative methods from the proposed derivative-involved scheme is illustrated by solving numerical test problems and comparing with the available methods in the literatures.     

Globally Convergent Optimization Algorithms on Riemannian Manifolds: Uniform Framework for Unconstrained and Constrained Optimization

This paper proposes several globally convergent geometric optimization algorithms on Riemannian manifolds, which extend some existing geometric optimization techniques. Since any set of smooth constraints in the Euclidean space R ⁿ (corresponding to constrained optimization) and the R ⁿ space itself (corresponding to unconstrained optimization) are both special Riemannian manifolds, and since these algorithms are developed on general Riemannian manifolds, the techniques discussed in this paper provide a uniform framework for constrained and unconstrained optimization problems. Unlike some earlier works, the new algorithms have less restrictions in both convergence results and in practice. For example, global minimization in the one-dimensional search is not required. All the algorithms addressed in this paper are globally convergent. For some special Riemannian manifold other than R ⁿ , the new algorithms are very efficient. Convergence rates are obtained. Applications are discussed. This paper is based on part of the Ph.D Thesis of the author under the supervision of Professor Tits, University of Maryland, College Park, Maryland. The author is in debt to him for invaluable suggestions on earlier versions of this paper. The author is grateful to the Associate Editor and anonymous reviewers, who pointed out a number of papers that have been included in the references; they made also detailed suggestions that lead to significant improvements of the paper. Finally, the author thanks Dr. S.T. Smith for making available his Ph.D Thesis.     

Global convergence and the Powell singular function

The Powell singular function was introduced 1962 by M.J.D. Powell as an unconstrained optimization problem. The function is also used as nonlinear least squares problem and system of nonlinear equations. The function is a classic test function included in collections of test problems in optimization as well as an example problem in text books. In the global optimization literature the function is stated as a difficult test case. The function is convex and the Hessian has a double singularity at the solution. In this paper we consider Newton’s method and methods in Halley class and we discuss the relationship between these methods on the Powell Singular Function. We show that these methods have global but linear rate of convergence. The function is in a subclass of unary functions and results for Newton’s method and methods in the Halley class can be extended to this class. Newton’s method is often made globally convergent by introducing a line search. We show that a full Newton step will satisfy many of standard step length rules and that exact line searches will yield slightly faster linear rate of convergence than Newton’s method. We illustrate some of these properties with numerical experiments.     

A hybrid algorithm for nonlinear minimax problems

In this paper, a hybrid algorithm for solving finite minimax problem is presented. In the algorithm, we combine the trust-region methods with the line-search methods and curve-search methods. By means of this hybrid technique, the algorithm, according to the specific situation at each iteration, can adaptively performs the trust-region step, line-search step or curve-search step, so as to avoid possibly solving the trust-region subproblems many times, and make better use of the advantages of different methods. Moreover, we use second-order correction step to circumvent the difficulties of the Maratos effect occurred in the nonsmooth optimization. Under mild conditions, we prove that the new algorithm is of global convergence and locally superlinear convergence. The preliminary experiments show that the new algorithm performs efficiently.     

无记忆非拟 Newton 算法的导出和全局收敛性

In this paper, a new class of memoryless non-quasi-Newton method for solving unconstrained optimization problems is proposed, and the global convergence of this method with inexact line search is proved. Furthermore, we propose a hybrid method that mixes both the memoryless non-quasi-Newton method and the memoryless Perry-Shanno quasi-Newton method. The global convergence of this hybrid memoryless method is proved under mild assumptions. The initial results show that these new methods are efficient for the given test problems. Especially the memoryless non-quasi-Newton method requires little storage and computation, so it is able to efficiently solve large scale optimization problems.     

General primal-dual penalty/barrier path-following Newton methods for nonlinear programming

In the present article rather general penalty/barrier-methods are considered, that define a local continuously differentiable primal-dual path. The class of penalty/barrier terms includes most of the usual techniques like logarithmic barriers, SUMT, quadratic loss functions as well as exponential penalties, and the optimization problem which may contain inequality as well as equality constraints. The convergence of the corresponding general primal-dual path-following method is shown for local minima that satisfy strong second-order sufficiency conditions with linear independence constraint qualification (LICQ) and strict complementarity. A basic tool in the analysis of these methods is to estimate the radius of convergence of Newton's method depending on the penalty/barrier-parameter. Without using self-concordance properties convergence bounds are derived by direct estimations of the solutions of the Newton equations. Parameter selection rules are proposed which guarantee the local convergence of the considered penalty/barrier-techniques with only a finite number of Newton steps at each parameter level. Numerical examples illustrate the practical behavior of the proposed class of methods.     

Modified parallel multisplitting iterative methods for non-Hermitian positive definite systems

In this paper we present three modified parallel multisplitting iterative methods for solving non-Hermitian positive definite systems Ax?=?b. The first is a direct generalization of the standard parallel multisplitting iterative method for solving this class of systems. The other two are the iterative methods obtained by optimizing the weighting matrices based on the sparsity of the coefficient matrix A. In our multisplitting there is only one that is required to be convergent (in a standard method all the splittings must be convergent), which not only decreases the difficulty of constructing the multisplitting of the coefficient matrix A, but also releases the constraints to the weighting matrices (unlike the standard methods, they are not necessarily be known or given in advance). We then prove the convergence and derive the convergent rates of the algorithms by making use of the standard quadratic optimization technique. Finally, our numerical computations indicate that the methods derived are feasible and efficient.     

A parameterized proximal point algorithm for separable convex optimization

In this paper, we develop a parameterized proximal point algorithm (P-PPA) for solving a class of separable convex programming problems subject to linear and convex constraints. The proposed algorithm is provable to be globally convergent with a worst-case O(1 / t) convergence rate, where t denotes the iteration number. By properly choosing the algorithm parameters, numerical experiments on solving a sparse optimization problem arising from statistical learning show that our P-PPA could perform significantly better than other state-of-the-art methods, such as the alternating direction method of multipliers and the relaxed proximal point algorithm.     

Jacobian smoothing Brown’s method for NCP

Jacobian smoothing Brown’s method for nonlinear complementarity problems (NCP) is studied in this paper. This method is a generalization of classical Brown’s method. It belongs to the class of Jacobian smoothing methods for solving semismooth equations. Local convergence of the proposed method is proved in the case of a strictly complementary solution of NCP. Furthermore, a locally convergent hybrid method for general NCP is introduced. Some numerical experiments are also presented.     

非线性不等式约束最优化一个超线性与二次收敛的强次可行方法

本文讨论非线性不等式约束最优化问题，借助于序列线性方程组技术和强次可行方法思想，建立了问题的一个初始点任意的快速收敛新算法．在每次迭代中，算法只需解一个结构简单的线性方程组．算法的初始迭代点不仅可以是任意的，而且不使用罚函数和罚参数，在迭代过程中，迭代点列的可行性单调不减．在相对弱的假设下，算法具有较好的收敛性和收敛速度，即具有整体与强收敛性，超线性与二次收敛性．文中最后给出一些数值试验结果．