Stochastic Trust-Region Methods with Trust-Region Radius Depending on Probabilistic Models

We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models. Especially, we propose a specific algorithm termed STRME, in which the trust-region radius depends linearly on the gradient used to define the latest model. The complexity results of the STRME method in nonconvex, convex and strongly convex settings are presented, which match those of the existing algorithms based on probabilistic properties. In addition, several numerical experiments are carried out to reveal the benefits of the proposed methods compared to the existing stochastic trust-region methods and other relevant stochastic gradient methods.     

Superlinearly Convergent Trust-Region Method without the Assumption of Positive-Definite Hessian

In this paper, we reinvestigate the trust-region method by reformulating its subproblem: the trust-region radius is guided by gradient information at the current iteration and is self-adaptively adjusted. A trust-region algorithm based on the proposed subproblem is proved to be globally convergent. Moreover, the superlinear convergence of the new algorithm is shown without the condition that the Hessian of the objective function at the solution be positive definite. Preliminary numerical results indicate that the performance of the new method is notable. The authors thank the Associate Editor and two anonymous referees for valuable comments and suggestions. This work was supported by the National Science Foundation of China, Grant 70302003. Communicated by T. Rapcsak     

Modified Partial-Update Newton-Type Algorithms for Unary Optimization

In this paper, we propose two modified partial-update algorithms for solving unconstrained unary optimization problems based on trust-region stabilization via indefinite dogleg curves. The two algorithms partially update an approximation to the Hessian matrix in each iteration by utilizing a number of times the rank-one updating of the Bunch–Parlett factorization. In contrast with the original algorithms in Ref. 1, the two algorithms not only converge globally, but possess also a locally quadratic or superlinear convergence rate. Furthermore, our numerical experiments show that the new algorithms outperform the trust-region method which uses the partial update criteria suggested in Ref. 1.     

STRSCNE: A Scaled Trust-Region Solver for Constrained Nonlinear Equations

In this paper a Matlab solver for constrained nonlinear equations is presented. The code, called STRSCNE, is based on the affine scaling trust-region method STRN, recently proposed by the authors. The approach taken in implementing the key steps of the method is discussed. The structure and the usage of STRSCNE are described and its features and capabilities are illustrated by numerical experiments. The results of a comparison with high quality codes for nonlinear optimization are shown.     

Nonmonotone Trust-Region Method for Nonlinear Programming with General Constraints and Simple Bounds

In this paper, we propose a nonmonotone trust-region algorithm for the solution of optimization problems with general nonlinear equality constraints and simple bounds. Under a constant rank assumption on the gradients of the active constraints, we analyze the global convergence of the proposed algorithm.     

Active-Set Projected Trust-Region Algorithm for Box-Constrained Nonsmooth Equations

In this paper, by means of an active-set strategy, we present a trust-region method for solving box-constrained nonsmooth equations. Nice properties of the proposed method include: (a) all iterates remain feasible; (b) the search direction, as adequate combination of the projected gradient direction and the trust-region direction, is an asymptotic Newton direction under mild conditions; (c) the subproblem of the proposed method, possessing the form of an unconstrained trust-region subproblem, can be solved by existing methods; (d) the subproblem of the proposed method is of reduced dimension, which is potentially cheaper when applied to solve large-scale problems. Under appropriate conditions, we establish global and local superlinear/quadratic convergence of the method. Preliminary numerical results are given.     

一类优化问题的非单调信赖域算法

本文提出了一类带不等式约束和简单边界的非线性优化问题的非单调信赖域算法,在一定的条件下,证明了算法的全局收敛性,并通过数值实验验证了算法的合理性。     

一类广义拟牛顿算法的收敛性

本文提出一类广义拟牛顿算法,新类算法降低了关于目标函数的假设条件,将线搜索扩展 到一般形式,它概括了若干种常用的非精确线搜索技术.此外,算法对迭代校正公式中的参数Φk的 选取范围做了较大扩展(可以取负值).     

无约束最优化中的一阶曲线法

本文对无约束最优化问题提出一阶曲线法，给出其全局收敛结果．对由微分方程定义的一阶曲线，提出渐近收敛指数概念，并计算出几个常用曲线模型的渐近收敛指数．     

一般无约束优化问题的广义拟牛顿法

对一般目标函数极小化问题的拟牛顿法及其全局收敛性的研究,已经成为拟牛顿法理论中最基本的开问题之一．本文对这个问题做了进一步的研究,对无约束优化问题提出一类新的广义拟牛顿算法,并结合Goldstein线搜索证明了算法对一般非凸目标函数极小化问题的全局收敛性．     

一类带线搜索的非单调信赖域新算法

结合非单调信赖域方法,和非单调线搜索技术,提出了一种新的无约束优化算法.信赖域方法的每一步采用线搜索,使得迭代每一步都充分下降加快了迭代速度.在一定条件下,证明了算法具有全局收敛性和局部超线性.收敛速度.数值试验表明算法是十分有效的.     

类Broyden族非拟牛顿算法对一般目标函数的全局收敛性

应用双参数的类Broyden族校正公式,为研究求解无约束最优化问题的拟牛顿类算法对一般目标函数的收敛性这个开问题提供了一种新的方法．     

有奇异解的无约束最优化问题的一种混合法

本文研究求解含有奇异解的无约束最优化问题算法 .该类问题的一个重要特性是目标函数的Hessian阵可能处处奇异 .我们提出求解该类问题的一种梯度 -正则化牛顿型混合算法 .并在一定的条件下得到了算法的全局收敛性 .而且 ,经一定迭代步后 ,算法还原为正则化 Newton法 .因而 ,算法具有局部二次收敛性 .     

A New Diagonal Quasi-Newton Updating Method With Scaled Forward Finite Differences Directional Derivative for Unconstrained Optimization

A new diagonal quasi-Newton updating algorithm for unconstrained optimization is presented. The elements of the diagonal matrix approximating the Hessian are determined as scaled forward finite differences directional derivatives of the components of the gradient. Under mild classical assumptions, the convergence of the algorithm is proved to be linear. Numerical experiments with 80 unconstrained optimization test problems, of different structures and complexities, as well as five applications from MINPACK-2 collection, prove that the suggested algorithm is more efficient and more robust than the quasi-Newton diagonal algorithm retaining only the diagonal elements of the BFGS update, than the weak quasi-Newton diagonal algorithm, than the quasi-Cauchy diagonal algorithm, than the diagonal approximation of the Hessian by the least-change secant updating strategy and minimizing the trace of the matrix, than the Cauchy with Oren and Luenberger scaling algorithm in its complementary form (i.e. the Barzilai-Borwein algorithm), than the steepest descent algorithm, and than the classical BFGS algorithm. However, our algorithm is inferior to the limited memory BFGS algorithm (L-BFGS).     

Global Convergence Analysis of a New Nonmonotone BFGS Algorithm on Convex Objective Functions

In this paper, a new nonmonotone BFGS algorithmfor unconstrained optimization is introduced. Under mild conditions,the global convergence of this new algorithm on convex functions isproved. Some numerical experiments show that this new nonmonotoneBFGS algorithm is competitive to the BFGS algorithm.     

On the Convergence of a Trust-Region Method for Solving Constrained Nonlinear Equations with Degenerate Solutions

This paper presents a trust-region method for solving the constrained nonlinear equation F(x) = 0, x , where R ⁿ is a nonempty and closed convex set, F is defined on the open set containing and is continuously differentiable. The iterates generated by the method are feasible. The method is globally and quadratically convergent under local error bounded assumption on F. The results obtained are extensions of the work of Yamashita Fukushima (Ref. 1) and Fan Yuan (Ref. 2) for unconstrained nonlinear equations. Numerical results show that the new algorithm works quite well.     

A modified BFGS algorithm based on a hybrid secant equation

By making a convex combination of the modified secant equations proposed by Yuan and Wei et al.,a hybrid secant equation and also,a modified BFGS algorithm is proposed.The hybridization parameter is effectively computed using the available information of recent iterations.Under proper conditions,it is shown that the proposed algorithm is globally,locally and superlinearly convergent.By using the performance profile introduced by Dolan and Mor'e,a comparison between the implementations of the proposed algori...     

结合一种新搜索的Broyden算法类的全局收敛性

本文提出了一种与回追搜索（backtrackinglinesearch）有关的可行线性搜索．在通常的条件下，证明了结合这一新的搜索的Broyden算法类具有全局收敛性．     

A New Dai-Liao Conjugate Gradient Method with Optimal Parameter Choice

A new nonlinear conjugate gradient method is proposed to solve large-scale unconstrained optimization problems. The direction is given by a search direction matrix, which contains a positive parameter. The value of the parameter is calculated by minimizing the upper bound of spectral condition number of the matrix defining it in order to cluster all the singular values. The new search direction satisfies the sufficient descent condition. Under some mild assumptions, the global convergence of the proposed method is proved for uniformly convex functions and the general functions. Numerical experiments show that, for the CUTEr library and the test problem collection given by Andrei, the proposed method is superior to M1 proposed by Babaie-Kafaki and Ghanbari (Eur. J. Oper. Res. 234(3), 625–630, 2014), CG_DESCENT(5.3), and CGOPT.     

极大极小问题的广义投影算法

研究了无约束极大极小问题.通过引入一个可微的辅助函数,利用广义投影技术产生下降搜索方向,结合Armjio非精确线搜索建立了一个广义梯度投影算法.在初始点任意的条件下,证明了算法的全局收敛性.