A NEW STEPSIZE FOR THE STEEPEST DESCENT METHOD

The steepest descent method is the simplest gradient method for optimization. It is well known that exact line searches along each steepest descent direction may converge very slowly. An important result was given by Barzilar and Borwein, which is proved to be superlinearly convergent for convex quadratic in two dimensional space, and performs quite well for high dimensional problems. The BB method is not monotone, thus it is not easy to be generalized for general nonlinear functions unless certain non-monotone techniques being applied. Therefore, it is very desirable to find stepsize formulae which enable fast convergence and possess the monotone property. Such a stepsize αk for the steepest descent method is suggested in this paper. An algorithm with this new stepsize in even iterations and exact line search in odd iterations is proposed. Numerical results are presented, which confirm that the new method can find the exact solution within 3 iteration for two dimensional problems. The new method is very efficient for small scale problems. A modified version of the new method is also presented, where the new technique for selecting the stepsize is used after every two exact line searches. The modified algorithm is comparable to the Barzilar-Borwein method for large scale problems and better for small scale problems.     

求解图像分割CV模型的BB算法

给出图像分割的一种新算法——BB算法.该方法的优点在于利用迭代过程中当前点和前一点的信息确定搜索步长,从而更有效地搜索最优解.为此,首先通过变分水平集方法将CV模型转化为最优化问题;其次,将BB算法引入该优化问题进行求解;然后,对BB算法进行收敛性分析,为该算法应用在CV模型中提供了理论依据;最后将该方法与已有的最速下降法、共轭梯度法的分割结果进行比较.结果表明,跟其他两种方法相比,BB算法在保证较好分割效果的前提下,提高了算法的速度和性能.     

Gradient Methods with Adaptive Step-Sizes

Motivated by the superlinear behavior of the Barzilai-Borwein (BB) method for two-dimensional quadratics, we propose two gradient methods which adaptively choose a small step-size or a large step-size at each iteration. The small step-size is primarily used to induce a favorable descent direction for the next iteration, while the large step-size is primarily used to produce a sufficient reduction. Although the new algorithms are still linearly convergent in the quadratic case, numerical experiments on some typical test problems indicate that they compare favorably with the BB method and some other efficient gradient methods.     

An LQP-based descent method for structured monotone variational inequalities

This paper proposes a descent method to solve a class of structured monotone variational inequalities. The descent directions are constructed from the iterates generated by a prediction-correction method [B.S. He, Y. Xu, X.M. Yuan, A logarithmic-quadratic proximal prediction-correction method for structured monotone variational inequalities, Comput. Optim. Appl. 35 (2006) 19-46], which is based on the logarithmic-quadratic proximal method. In addition, the optimal step-sizes along these descent directions are identified to accelerate the convergence of the new method. Finally, some numerical results for solving traffic equilibrium problems are reported.     

Alternate step gradient method*

The Barzilai and Borwein (BB) gradient method does not guarantee a descent in the objective function at each iteration, but performs better than the classical steepest descent (SD) method in practice. So far, the BB method has found many successful applications and generalizations in linear systems, unconstrained optimization, convex-constrained optimization, stochastic optimization, etc. In this article, we propose a new gradient method that uses the SD and the BB steps alternately. Hence the name “alternate step (AS) gradient method.” Our theoretical and numerical analyses show that the AS method is a promising alternative to the BB method for linear systems. Unconstrained optimization algorithms related to the AS method are also discussed. Particularly, a more efficient gradient algorithm is provided by exploring the idea of the AS method in the GBB algorithm by Raydan (1997).

To establish a general R-linear convergence result for gradient methods, an important property of the stepsize is drawn in this article. Consequently, R-linear convergence result is established for a large collection of gradient methods, including the AS method. Some interesting insights into gradient methods and discussion about monotonicity and nonmonotonicity are also given.     

无约束最优化的信赖域BB法

梯度法是求解无约束最优化的一类重要方法.步长选取的好坏与梯度法的数值表现息息相关.注意到BB步长隐含了目标函数的二阶信息,本文将BB法与信赖域方法相结合,利用BB步长的倒数去近似目标函数的Hesse矩阵,同时利用信赖域子问题更加灵活地选取梯度法的步长,给出求解无约束最优化问题的单调和非单调信赖域BB法.在适当的假设条件下,证明了算法的全局收敛性.数值试验表明,与已有的求解无约束优化问题的BB类型的方法相比,非单调信赖域BB法中e_k=‖x_k-x~*‖的下降呈现更明显的阶梯状和单调性,因此收敛速度更快.     

一种求解单调变分不等式的部分并行分裂LQP交替方向法

LQP交替方向法是求解可分离结构型单调变分不等式问题的一种非常有效的方法.它不仅可以充分地利用目标函数的可分结构,将原问题分解为多个更易求解的子问题,还更适合求解大规模问题.对于带有三个可分离算子的单调变分不等式问题,结合增广拉格朗日算法和LQP交替方向法提出了一种部分并行分裂LQP交替方向法,构造了新算法的两个下降方向,结合这两个下降方向得到了一个新的下降方向,沿着这个新的下降方向给出了最优步长.并在较弱的假设条件下,证明了新算法的全局收敛性.     

An efficient iterative method for finding common fixed point and variational inequalities in Hilbert spaces

ABSTRACT

In this paper, we investigate the problem of finding a common solution to a fixed point problem involving demi-contractive operator and a variational inequality with monotone and Lipschitz continuous mapping in real Hilbert spaces. Inspired by the projection and contraction method and the hybrid descent approximation method, a new and efficient iterative method for solving the problem is introduced. Strong convergence theorem of the proposed method is established under standard and mild conditions. Our scheme generalizes and extends some of the existing results in the literature, and moreover, its computational effort is less per each iteration compared with related works.     

A modified alternating projection based prediction–correction method for structured variational inequalities

In this paper, we propose a novel alternating projection based prediction–correction method for solving the monotone variational inequalities with separable structures. At each iteration, we adopt the weak requirements for the step sizes to derive the predictors, which affords fewer trial and error steps to accomplish the prediction phase. Moreover, we design a new descent direction for the merit function in correction phase. Under some mild assumptions, we prove the global convergence of the modified method. Some preliminary computational results are reported to demonstrate the promising and attractive performance of the modified method compared to some state-of-the-art prediction–contraction methods.     

Weak convergence theorem for zero points of inverse strongly monotone mapping and fixed points of nonexpansive mapping in Hilbert space

We know that variational inequality problem is very important in the nonlinear analysis. For a variational inequality problem defined over a nonempty fixed point set of a nonexpansive mapping in Hilbert space, the strong convergence theorem has been proposed by I. Yamada. The algorithm in this theorem is named the hybrid steepest descent method. Based on this method, we propose a new weak convergence theorem for zero points of inverse strongly monotone mapping and fixed points of nonexpansive mapping in Hilbert space. Using this result, we obtain some new weak convergence theorems which are useful in nonlinear analysis and optimization problem.     

低秩稀疏矩阵恢复的快速非单调交替极小化方法

针对低秩稀疏矩阵恢复问题的一个非凸优化模型,本文提出了一种快速非单调交替极小化方法.主要思想是对低秩矩阵部分采用交替极小化方法,对稀疏矩阵部分采用非单调线搜索技术来分别进行迭代更新.非单调线搜索技术是将单步下降放宽为多步下降,从而提高了计算效率.文中还给出了新算法的收敛性分析.最后,通过数值实验的比较表明,矩阵恢复的非单调交替极小化方法比原单调类方法更有效.     

Some Methods Based on the D-Gap Function for Solving Monotone Variational Inequalities

The D-gap function has been useful in developing unconstrained descent methods for solving strongly monotone variational inequality problems. We show that the D-gap function has certain properties that are useful also for monotone variational inequality problems with bounded feasible set. Accordingly, we develop two unconstrained methods based on them that are similar in spirit to a feasible method of Zhu and Marcotte based on the regularized-gap function. We further discuss a third method based on applying the D-gap function to a regularized problem. Preliminary numerical experience is also reported.     

求单调变分不等式隐式方法的一个单调下降性质

1　引　　言变分不等式在数学规划中起着很重要的作用 ,许多研究者 [3 ]讨论了这一问题 .对于单调线性变分不等式问题 ,文 [4 -7]提出了几种投影收缩算法 ( PC) .最近文 [7]中研究了如下的一类变分不等式问题( VI)　　 u∈Ω ,　　 ( v -u) TF( u)≥ 0 ,　　 v∈Ω . ( 1 )其中Ω Rn 是一个闭凸集 ,F是 Rn到自身的连续单调映射 ,即F( u) -F( v) T( u -v)≥ 0 ,　　　 u,v∈ Rn. ( 2 )由 [1 ]知 ,对于任意的 β>0 ,变分不等式 ( 1 )等价于投影方程于是求解 ( 1 )即是寻求e( u,β)∶ =u -PΩ[u -βF( u) ]的零点 .本文中 Ω*表示 ( …     

A Linearly Convergent Derivative-Free Descent Method for Strongly Monotone Complementarity Problems

We establish the first rate of convergence result for the class of derivative-free descent methods for solving complementarity problems. The algorithm considered here is based on the implicit Lagrangian reformulation [26, 35] of the nonlinear complementarity problem, and makes use of the descent direction proposed in [42], but employs a different Armijo-type linesearch rule. We show that in the strongly monotone case, the iterates generated by the method converge globally at a linear rate to the solution of the problem.     

An Adaptive Nonmonotone Projected Barzilai-Borwein Gradient Method with Active Set Prediction for Nonnegative Matrix Factorization

In this paper, we first present an adaptive nonmonotone term to improve the efficiency of nonmonotone line search, and then an active set identification technique is suggested to get more efficient descent direction such that it improves the local convergence behavior of algorithm and decreases the computation cost. By means of the adaptive nonmonotone line search and the active set identification technique, we put forward a global convergent gradient-based method to solve the nonnegative matrix factorization (NMF) based on the alternating nonnegative least squares framework, in which we introduce a modified Barzilai-Borwein (BB) step size. The new modified BB step size and the larger step size strategy are exploited to accelerate convergence. Finally, the results of extensive numerical experiments using both synthetic and image datasets show that our proposed method is efficient in terms of computational speed.     

A globally convergent descent method for nonsmooth variational inequalities

We propose a descent method via gap functions for solving nonsmooth variational inequalities with a locally Lipschitz operator. Assuming monotone operator (not necessarily strongly monotone) and bounded domain, we show that the method with an Armijo-type line search is globally convergent. Finally, we report some numerical experiments. This work has been supported by the National Research Program PRIN/2005017083 “Innovative Problems and Methods in Nonlinear Optimization”.     

Inexact decomposition methods for solving variational inequalities with linear equality constraint

In this paper, we propose two inexact decomposition methods for solving variational inequalities(VI) with linear equality constraint, which improve the decomposition method proposed by Gabay (in Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian methods: applications to the solution of boundary-valued problems, pp.?299?C331, North-Holland, Amsterdam, 1983), Gabay and Mercier (Comput. Math. Appl. 2(1):17?C40, 1976) in the following two senses: in each iteration, both methods allow the involved strongly monotone sub-VI to be solved approximately; the temporal iterate generated by the sub-VI is utilized to generate descent direction, and the new iterate is generated along the descent direction. Under mild conditions, the global convergence of the inexact methods is proved. Some numerical experiments are carried out to validate the efficiency and practicality of the proposed methods.     

A variable metric Proximal-Descent Algorithm for monotone operators

In this paper, we discuss a variable metric Proximal-Descent Algorithm for finding a zero of any given maximal monotone operator. At each iteration, it first implements a proximal step and then a descent step to locate the new iterate. In the proximal step, we have replaced the regularization parameter by some positive definite matrix, which may vary from iteration to iteration. Under standard assumptions, we prove its global convergence without the matrix??s symmetry. Some key aspects of the algorithm??s applications are discussed. Preliminary numerical experiments show the efficiency in practical implementations.     

采用不定Dogleg路径的非单调自适应信赖域算法解无约束极小化(英文)

In this paper, we combine the nonmonotone and adaptive techniques with trust region method for unconstrained minimization problems. We set a new ratio of the actual descent and predicted descent. Then, instead of the monotone sequence, the nonmonotone sequence of function values are employed. With the adaptive technique, the radius of trust region △k can be adjusted automatically to improve the efficiency of trust region methods. By means of the Bunch-Parlett factorization, we construct a method with indefinite dogleg path for solving the trust region subproblem which can handle the indefinite approximate Hessian Bk. The convergence properties of the algorithm are established. Finally, detailed numerical results are reported to show that our algorithm is efficient.     

Three adaptive hybrid derivative-free projection methods for constrained monotone nonlinear equations and their applications

In this work, by considering the hyperplane projection and hybrid techniques, three scaled three-term conjugate gradient methods are extended to solve the system of constrained monotone nonlinear equations, and the developed methods have the advantages of low storage and only using function values. The new methods satisfy the sufficient descent condition independent of any line search criterion. It has been proved that three new methods converge globally under some mild conditions. The numerical experiments for constrained monotone nonlinear equations and image de-blurring problems illustrate that the proposed methods are numerically effective and efficient.