Convergence of quasi-Newton method with new inexact line search

Quasi-Newton method is a well-known effective method for solving optimization problems. Since it is a line search method, which needs a line search procedure after determining a search direction at each iteration, we must decide a line search rule to choose a step size along a search direction. In this paper, we propose a new inexact line search rule for quasi-Newton method and establish some global convergent results of this method. These results are useful in designing new quasi-Newton methods. Moreover, we analyze the convergence rate of quasi-Newton method with the new line search rule.     

在放宽的强Wolfe搜索下一般三项共轭梯度法的全局收敛性

Abstract. The global convergence of the general three-term conjugate gradient methods withthe relaxed strong Wolfe line-search is proved.     

A gradient-related algorithm with inexact line searches

In this paper, a new gradient-related algorithm for solving large-scale unconstrained optimization problems is proposed. The new algorithm is a kind of line search method. The basic idea is to choose a combination of the current gradient and some previous search directions as a new search direction and to find a step-size by using various inexact line searches. Using more information at the current iterative step may improve the performance of the algorithm. This motivates us to find some new gradient algorithms which may be more effective than standard conjugate gradient methods. Uniformly gradient-related conception is useful and it can be used to analyze global convergence of the new algorithm. The global convergence and linear convergence rate of the new algorithm are investigated under diverse weak conditions. Numerical experiments show that the new algorithm seems to converge more stably and is superior to other similar methods in many situations.     

GLOBAL COVERGENCE OF THE NON-QUASI-NEWTON METHOD FOR UNCONSTRAINED OPTIMIZATION PROBLEMS

In this paper, the non-quasi-Newton＇s family with inexact line search applied to unconstrained optimization problems is studied. A new update formula for non-quasi-Newton＇s family is proposed. It is proved that the constituted algorithm with either Wolfe-type or Armijotype line search converges globally and Q-superlinearly if the function to be minimized has Lipschitz continuous gradient.     

A descent method with inexact linear search for nonsmooth equilibrium problems

A descent method with respect to the gap function is formulated and justified for the nonsmooth equilibrium problem. It uses the procedure of inexact linear search of the Armijo type. The proposed method converges under the same assumptions as the methods with exact linear search.     

A smoothing-type algorithm for solving nonlinear complementarity problems with a non-monotone line search

The smoothing-type algorithm has been successfully applied to solve various optimization problems. In general, the smoothing-type algorithm is designed based on some monotone line search. However, in order to achieve better numerical results, the non-monotone line search technique has been used in the numerical computations of some smoothing-type algorithms. In this paper, we propose a smoothing-type algorithm for solving the nonlinear complementarity problem with a non-monotone line search. We show that the proposed algorithm is globally and locally superlinearly convergent under suitable assumptions. The preliminary numerical results are also reported.     

Minimization of extended quadratic functions with inexact line searches

An algorithm for unconstrained minimization is proposed which is invariant to the nonlinear scaling of a strictly convex quadratic function and which generates mutually conjugate directions for extended quadratic functions. It is derived for inexact line searches and is designed for general use. It compares favorably in numerical tests (eight test functions, dimensionality up to 1000) with the 1975 Dixon algorithm on which this new algorithm is based.     

On the convergence of the LJ search method

The convergence of the Luus-Jaakola search method for unconstrained optimization problems is established.Notation E _n Euclideann-space - f Gradient off(x) - ² f Hessian matrix - (·) ^T Transpose of (·) - I Index set {1, 2, ...,n} - [x _i1 ^*(j) ] Point around which search is made in the (j + 1)th iteration, i.e., [x _1l ^*(j) ,x _2l ^*(j) ,...,x _n1 ^*(j) ] - r _i ⁽ⁱ⁾ Range ofx _il ^*(i) in the (j + 1)th iteration - l ₁ min_i {r _i ⁽⁰⁾ } - l ₂ min_i {r _i ⁽⁰⁾ } - A _j Region of search in thejth iteration, i.e., {x E _n:x _il ^*(j-1) –0.5r _i ^(j-1) x _ix _il ^*(j-1) +0.5r _i ^(j-1) ,i I} - S _j Closed sphere with center origin and radius _j - Reduction factor in each iteration - 1– - (·) Gamma function Many discussions with Dr. S. N. Iyer, Professor of Electrical Engineering, College of Engineering, Trivandrum, India, are gratefully acknowledged. The author has great pleasure to thank Dr. K. Surendran, Professor, Department of Electrical Engineering, P.S.G. College of Technology, Coimbatore, India, for suggesting this work.     

Generalized monotone line search SQP algorithm for constrained minimax problems

In this article, non-linear minimax problems with general constraints are discussed. By means of solving one quadratic programming an improved direction is yielded and a second-order correction direction can also be at hand via one system of linear equations. So a new algorithm for solving the discussed problems is presented. In connection with a special merit function, the generalized monotone line search is used to yield the step size at each iteration. Under mild conditions, we can ensure global and superlinear convergence. Finally, some numerical experiments are operated to test our algorithm, and the results demonstrate that it is promising.     

推广线搜索下一类共轭梯度法的全局收敛性

在推广线搜索下给出了一类共轭梯度法的全局收敛结果     

On the convergence of a basic iterative method for the implicit complementarity problem

In Part 1 of this study (Ref. 1), we have defined the implicit complementarity problem and investigated its existence and uniqueness of solution. In the present paper, we establish a convergence theory for a certain iterative algorithm to solve the implicit complementarity problem. We also demonstrate how the algorithm includes as special cases many existing iterative methods for solving a linear complementarity problem.This research was prepared as part of the activities of the Management Sciences Research Group, Carnegie-Mellon University, under Contract No. N00014-75-C-0621-NR-047-048 with the Office of Naval Research.     

On the convergence of basic iterative methods for convection–diffusion equations

In this paper we analyze convergence of basic iterative Jacobi and Gauss–Seidel type methods for solving linear systems which result from finite element or finite volume discretization of convection–diffusion equations on unstructured meshes. In general the resulting stiffness matrices are neither M‐matrices nor satisfy a diagonal dominance criterion. We introduce two newmatrix classes and analyse the convergence of the Jacobi and Gauss–Seidel methods for matrices from these classes. A new convergence result for the Jacobi method is proved and negative results for the Gauss–Seidel method are obtained. For a few well‐known discretization methods it is shown that the resulting stiffness matrices fall into the new matrix classes. Copyright © 1999 John Wiley & Sons, Ltd.     

On geometric convergence rate of Markov search towards the fat target

Let $f:A\to \mathbb{R}$ be a continuous function with the minimal value $f^{?}$, where $A$ is the compact metric space. Let ${\left\{X_t\right\}}_{t\in \mathbb{N}}$ be a Markov chain which represents the global optimization process on $A$. We present sufficient conditions for very strong, geometric convergence mode of the form $Ef\left(X_t\right)?f^1\le c^t?(Ef\left(X_0\right)?f^1)$, where $c\in (0,1)$ is some constant. This convergence mode is natural if the set of global minima is fat.     

On the convergence of an inexact Newton-type method

In this paper we give local convergence results of an inexact Newton-type method for monotone equations under a local error bound condition. This condition may hold even for problems with non-isolated solutions, and it therefore is weaker than the standard non-singularity condition.     

Use of a nonquadratic model in a conjugate-gradient method of optimization with inexact line searches

An algorithm for unconstrained minimization is developed which uses a rational function model, rather than a quadratic, as a basis for conjugate directions. The algorithm is similar to one previously proposed by the authors, but inexact linear searches are investigated in the present paper.     

A nonmonotone line search based algorithm for distribution center location selected

The minimax optimization model introduced in this paper is an important model which has received some attention over the past years. In this paper, the application of minimax model on how to select the distribution center location is first introduced. Then a new algorithm with nonmonotone line search to solve the non-decomposable minimax optimization is proposed. We prove that the new algorithm is global Convergent. Numerical results show the proposed algorithm is effective.     

A smoothing-type algorithm for the second-order cone complementarity problem with a new nonmonotone line search

The second-order cone complementarity problem (denoted by SOCCP) can be effectively solved by smoothing-type algorithms, which in general are designed based on some monotone line search. In this paper, based on a new smoothing function of the Fischer–Burmeister function, we propose a smoothing-type algorithm for solving the SOCCP. The proposed algorithm uses a new nonmonotone line search scheme, which contains the usual monotone line search as a special case. Under suitable assumptions, we show that the proposed algorithm is globally and locally quadratically convergent. Some numerical results are reported which indicate the effectiveness of the proposed algorithm.     

The convergence of conjugate gradient method with nonmonotone line search

The conjugate gradient method is a useful and powerful approach for solving large-scale minimization problems. Liu and Storey developed a conjugate gradient method, which has good numerical performance but no global convergence under traditional line searches such as Armijo line search, Wolfe line search, and Goldstein line search. In this paper we propose a new nonmonotone line search for Liu-Storey conjugate gradient method (LS in short). The new nonmonotone line search can guarantee the global convergence of LS method and has a good numerical performance. By estimating the Lipschitz constant of the derivative of objective functions in the new nonmonotone line search, we can find an adequate step size and substantially decrease the number of functional evaluations at each iteration. Numerical results show that the new approach is effective in practical computation.     

一类新的共轭梯度法

本文提出了一类新的共轭梯度法，在算法的迭代过程中，迭代方向保持下降性，并在一类非精确性搜索条件下证明了其全局收敛性。