A new structured quasi-Newton algorithm using partial information on Hessian

This paper presents a modified quasi-Newton method for structured unconstrained optimization. The usual SQN equation employs only the gradients, but ignores the available function value information. Several researchers paid attention to other secant conditions to get a better approximation of the Hessian matrix of the objective function. Recently Yabe et al. (2007) [6] proposed the modified secant condition which uses both gradient and function value information in order to get a higher-order accuracy in approximating the second curvature of the objective function. In this paper, we derive a new progressive modified SQN equation, with a vector parameter which use both available gradient and function value information, that maintains most properties of the usual and modified structured quasi-Newton methods. Furthermore, local and superlinear convergence of the algorithm is obtained under some reasonable conditions.     

大规模无约束优化的一族LBFGS类算法

尝试在有限存储类算法中利用目标函数值所提供的信息.首先利用插值条件构造了一个新的二次函数逼近目标函数,得到了一个新的弱割线方程,然后将此弱割线方程与袁[1]的弱割线方程相结合,给出了一族包括标准LBFGS的有限存储BFGS类算法,证明了这族算法的收敛性.从标准试验函数库CUTE中选择试验函数进行了数值试验,试验结果表明这族算法的数值表现都与标准LBFGS类似.     

A three-term conjugate gradient algorithm for large-scale unconstrained optimization problems

In this paper, a three-term conjugate gradient algorithm is developed for solving large-scale unconstrained optimization problems. The search direction at each iteration of the algorithm is determined by rectifying the steepest descent direction with the difference between the current iterative points and that between the gradients. It is proved that such a direction satisfies the approximate secant condition as well as the conjugacy condition. The strategies of acceleration and restart are incorporated into designing the algorithm to improve its numerical performance. Global convergence of the proposed algorithm is established under two mild assumptions. By implementing the algorithm to solve 75 benchmark test problems available in the literature, the obtained results indicate that the algorithm developed in this paper outperforms the existent similar state-of-the-art algorithms.     

大规模无约束优化的一族LBFGS类算法

尝试在有限存储类算法中利用目标函数值所提供的信息.首先利用插值条件构造了一个新的二次函数逼近目标函数,得到了一个新的弱割线方程,然后将此弱割线方程与袁[1]的弱割线方程相结合,给出了一族包括标准LBFGS的有限存储BFGS类算法,证明了这族算法的收敛性.从标准试验函数库CUTE中选择试验函数进行了数值试验,试验结果表明...     

A derivative-free three-term projection algorithm involving spectral quotient for solving nonlinear monotone equations

ABSTRACT

In this paper, we develop a three-term conjugate gradient method involving spectral quotient, which always satisfies the famous Dai-Liao conjugacy condition and quasi-Newton secant equation, independently of any line search. This new three-term conjugate gradient method can be regarded as a variant of the memoryless Broyden-Fletcher-Goldfarb-Shanno quasi-Newton method with regard to spectral quotient. By combining this method with the projection technique proposed by Solodov and Svaiter in 1998, we establish a derivative-free three-term projection algorithm for dealing with large-scale nonlinear monotone system of equations. We prove the global convergence of the algorithm and obtain the R-linear convergence rate under some mild conditions. Numerical results show that our projection algorithm is effective and robust, and is more competitive with the TTDFP algorithm proposed Liu and Li [A three-term derivative-free projection method for nonlinear monotone system of equations. Calcolo. 2016;53:427–450].     

An effective adaptive trust region algorithm for nonsmooth minimization

In this paper, an adaptive trust region algorithm that uses Moreau–Yosida regularization is proposed for solving nonsmooth unconstrained optimization problems. The proposed algorithm combines a modified secant equation with the BFGS update formula and an adaptive trust region radius, and the new trust region radius utilizes not only the function information but also the gradient information. The global convergence and the local superlinear convergence of the proposed algorithm are proven under suitable conditions. Finally, the preliminary results from comparing the proposed algorithm with some existing algorithms using numerical experiments reveal that the proposed algorithm is quite promising for solving nonsmooth unconstrained optimization problems.     

牛顿迭代法与几种改进格式的效率指数

研究牛顿迭代、牛顿弦截法以及它们的六种改进格式的计算效率,计算了它们的效率指数,得到牛顿迭代、改进牛顿法、弦截法和改进弦截法(即所谓牛顿迭代的P.C格式)、二次插值迭代格式、推广的牛顿迭代法、调和平均牛顿法和中点牛顿法的效率指数分别为0.347/n、0.3662/n、0.4812/n、0.4812/n、0.347/n、0.3662/n、0.3662/n、0.3662/n.我们的结果显示,利用抛物插值多项式推出的迭代格式和改进弦截法并没有真正提高迭代的计算效率.此外,我们还证明了改进弦截法与牛顿弦截法等价,并利用这一结论给出了改进弦截法收敛阶为2.618的一个简化证明.     

A new adaptive trust region algorithm for optimization problems

It is well known that trust region methods are very effective for optimization problems. In this article, a new adaptive trust region method is presented for solving unconstrained optimization problems. The proposed method combines a modified secant equation with the BFGS updated formula and an adaptive trust region radius, where the new trust region radius makes use of not only the function information but also the gradient information. Under suitable conditions, global convergence is proved, and we demonstrate the local superlinear convergence of the proposed method. The numerical results indicate that the proposed method is very efficient.     

A secant algorithm with line search filter method for nonlinear optimization

Filter methods were initially designed for nonlinear programming problems by Fletcher and Leyffer. In this paper we propose a secant algorithm with line search filter method for nonlinear equality constrained optimization. The algorithm yields the global convergence under some reasonable conditions. By using the Lagrangian function value in the filter we establish that the proposed algorithm can overcome the Maratos effect without using second order correction step, so that fast local superlinear convergence to second order sufficient local solution is achieved. The primary numerical results are presented to confirm the robustness and efficiency of our approach.     

求解单变量无约束优化问题的一类新割线法

本文基于分式逼近提出了一类求解单变量无约束优化问题的新割线法,给出并证明了该方法的收敛阶是(√2+1).并进一步对新方法的性能进行了分析,给出了新方法、经典的牛顿法和其他修正的割线类方法解单变量无约束优化问题的数值实验.理论和数值结果均表明新的割线法是有效的.     

An improved three-term conjugate gradient algorithm for solving unconstrained optimization problems

In this article, we present an improved three-term conjugate gradient algorithm for large-scale unconstrained optimization. The search directions in the developed algorithm are proved to satisfy an approximate secant equation as well as the Dai-Liao’s conjugacy condition. With the standard Wolfe line search and the restart strategy, global convergence of the algorithm is established under mild conditions. By implementing the algorithm to solve 75 benchmark test problems with dimensions from 1000 to 10,000, the obtained numerical results indicate that the algorithm outperforms the state-of-the-art algorithms available in the literature. It costs less CPU time and smaller number of iterations in solving the large-scale unconstrained optimization.     

MODIFIED NEWTON AND SECANT METHODS FOR SOLVING AN ORDER O(h~4) FINITE-DIFFERENCE PROBLEM

1IntroductionSolution0fn0nlineartwo-pointb0undaryvaIuepr0blems(NBVP)canoftenbefoundbythefinite-differenceappr0ach,wheref(t,y)isaconti-nuousfunction.Collatz[1]firstpresentedanapproximation0ffourthorderfwherey=(y1,''tyN)',g=(g1,'-,gN)'andtherelativepaperscanals0beseenin[2].Toestablishthesolutionof(1.l),thef0llowingmethodscanbeusedfnonlinearsuccessiverelaxati0n(NSOR)method[3],thedifferenceNewt0nmethod(0rNewtonmethod)[4],therelativesparsenonlinearequationpr0blemscanals0beseenin[5-8]-lnthisp…     

An inexact secant algorithm for large scale nonlinear systems of equalities and inequalities

In this paper, an inexact secant algorithm in association with nonmonotone technique and filter is proposed for solving the large scale nonlinear systems of equalities and inequalities. The systems are transformed into a continuous constrained optimization solved by inexact secant algorithm. Global convergence of the proposed algorithm is established under the reasonable conditions. Numerical results validate the effectiveness of our approach.     

Two new conjugate gradient methods based on modified secant equations

Following the approach proposed by Dai and Liao, we introduce two nonlinear conjugate gradient methods for unconstrained optimization problems. One of our proposed methods is based on a modified version of the secant equation proposed by Zhang, Deng and Chen, and Zhang and Xu, and the other is based on the modified BFGS update proposed by Yuan. An interesting feature of our methods is their account of both the gradient and function values. Under proper conditions, we show that one of the proposed methods is globally convergent for general functions and that the other is globally convergent for uniformly convex functions. To enhance the performance of the line search procedure, we also propose a new approach for computing the initial steplength to be used for initiating the procedure. We provide a comparison of implementations of our methods with the efficient conjugate gradient methods proposed by Dai and Liao, and Hestenes and Stiefel. Numerical test results show the efficiency of our proposed methods.     

Multi-step nonlinear conjugate gradient methods for unconstrained minimization

Conjugate gradient methods are appealing for large scale nonlinear optimization problems, because they avoid the storage of matrices. Recently, seeking fast convergence of these methods, Dai and Liao (Appl. Math. Optim. 43:87–101, 2001) proposed a conjugate gradient method based on the secant condition of quasi-Newton methods, and later Yabe and Takano (Comput. Optim. Appl. 28:203–225, 2004) proposed another conjugate gradient method based on the modified secant condition. In this paper, we make use of a multi-step secant condition given by Ford and Moghrabi (Optim. Methods Softw. 2:357–370, 1993; J. Comput. Appl. Math. 50:305–323, 1994) and propose two new conjugate gradient methods based on this condition. The methods are shown to be globally convergent under certain assumptions. Numerical results are reported.     

Solving non-differentiable equations by a new one-point iterative method with memory

We construct a new iterative method for approximating the solutions of nonlinear operator equations, where the operator involved is not differentiable. The algorithm proposed does not need to evaluate derivatives and is more efficient than the secant method. For this, we extend a result of Traub for one-point iterative methods to one-point iterative methods with memory.     

A TRUST REGION-TYPE METHOD FOR SOLVINGMONOTONE VARIATIONAL INEQUALITY

1.IntroductionLetSbeanonemptyclosedconvexsubsetofR"andletF:R"-R"beacontinuousmapping.ThevariatiollalillequalityproblemFindx*6Ssuchthat(F(x*),x--x*)20forallxeS(VIP)iswidelyusedtostudyvariousequilibriummodelsarisingilleconomic,operatiollsresearch,transportatiollandregionalsciellces[2'3I?where(.,.)dellotestheinnerproductinR".Manyiterativemethodsfor(VIP)havebeendeveloped,forexample,projectionmethods[7ts],thenonlinearJacobimethod[5],thesuccessiveoverrelaxation.ethod[9]andgeneralizedgradient.…     

Local and superlinear convergence of quasi-Newton methods based on modified secant conditions

For solving unconstrained minimization problems, quasi-Newton methods are popular iterative methods. The secant condition which employs only the gradient information is imposed on these methods. Several researchers paid attention to other secant conditions to get a better approximation of the Hessian matrix of the objective function. Recently, Zhang et al. [New quasi-Newton equation and related methods for unconstrained optimization, J. Optim. Theory Appl. 102 (1999) 147–167] and Zhang and Xu [Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations, J. Comput. Appl. Math. 137 (2001) 269–278] proposed the modified secant condition which uses both gradient and function value information in order to get a higher order accuracy in approximating the second curvature of the objective function. They showed the local and q-superlinear convergence property of the BFGS-like and DFP-like updates based on their proposed secant condition. In this paper, we incorporate one parameter into this secant condition to smoothly switch the standard secant condition and the secant condition of Zhang et al. We consider a modified Broyden family which includes the BFGS-like and the DFP-like updates proposed by Zhang et al. We prove the local and q-superlinear convergence of our method.     

A weak condition for secant method to solve systems of nonlinear equations

In this paper, a new weak condition for the convergence of secant method to solve the systems of nonlinear equations is proposed. A convergence ball with the center x0 is replaced by that with xl, the first approximation generated by the secant method with the initial data x-1 and x0. Under the bounded conditions of the divided difference, a convergence theorem is obtained and two examples to illustrate the weakness of convergence conditions are provided. Moreover, the secant method is applied to a system of nonlinear equations to demonstrate the viability and effectiveness of the results in the paper.     

Scaling on the Spectral Gradient Method

This paper presents a new method for steplength selection in the frame of spectral gradient methods. The steplength formula is based on the interpolation scheme as well as some modified secant equations. The corresponding algorithm selects the initial positive steplength per iteration according to the satisfaction of the secant condition, and then a backtracking procedure along the negative gradient is performed. The numerical experience shows that this algorithm improves favorably the efficiency property of the standard Barzilai–Borwein method as well as some other recently modified Barzilai–Borwein approaches.