一类凸优化的加速混合下降算法

凸优化问题的混合下降算法利用近似条件的已知信息和随机数扩张预测校正步得到了一组下降方向.而前向加速收缩算法利用高斯赛德尔迭代算法的技术,结合邻近点算法和近似邻近点算法的思想,构造了富有扩张性的下降方向.本文借鉴混合下降算法和前向加速收缩算法的思想,利用已有近似规则信息改善了混合下降算法的下降方向,得到了一类凸优化问题的加速混合下降算法.随后利用Markov不等式、凸函数性质和投影的基本性质等,实现了算法的依概率收敛证明.一系列数值试验表明了加速混合下降算法的有效性和效率性.     

Weak convergence theorem for zero points of inverse strongly monotone mapping and fixed points of nonexpansive mapping in Hilbert space

We know that variational inequality problem is very important in the nonlinear analysis. For a variational inequality problem defined over a nonempty fixed point set of a nonexpansive mapping in Hilbert space, the strong convergence theorem has been proposed by I. Yamada. The algorithm in this theorem is named the hybrid steepest descent method. Based on this method, we propose a new weak convergence theorem for zero points of inverse strongly monotone mapping and fixed points of nonexpansive mapping in Hilbert space. Using this result, we obtain some new weak convergence theorems which are useful in nonlinear analysis and optimization problem.     

Nonlinear iteration method for proximal split feasibility problems

The purpose of this paper is to introduce iterative algorithm which is a combination of hybrid viscosity approximation method and the hybrid steepest‐descent method for solving proximal split feasibility problems and obtain the strong convergence of the sequences generated by the iterative scheme under certain weaker conditions in Hilbert spaces. Our results improve many recent results on the topic in the literature. Several numerical experiments are presented to illustrate the effectiveness of our proposed algorithm, and these numerical results show that our result is computationally easier and faster than previously known results on proximal split feasibility problem.     

A Hybrid Descent Method for Global Optimization

In this paper, a hybrid descent method, consisting of a simulated annealing algorithm and a gradient-based method, is proposed. The simulated annealing algorithm is used to locate descent points for previously converged local minima. The combined method has the descent property and the convergence is monotonic. To demonstrate the effectiveness of the proposed hybrid descent method, several multi-dimensional non-convex optimization problems are solved. Numerical examples show that global minimum can be sought via this hybrid descent method.     

Parallel algorithms for variational inequalities over the Cartesian product of the intersections of the fixed point sets of nonexpansive mappings

This paper presents a framework of iterative algorithms for the variational inequality problem over the Cartesian product of the intersections of the fixed point sets of nonexpansive mappings in real Hilbert spaces. Strong convergence theorems are established under a certain contraction assumption with respect to the weighted maximum norm. The proposed framework produces as a simplest example the hybrid steepest descent method, which has been developed for solving the monotone variational inequality problem over the intersection of the fixed point sets of nonexpansive mappings. An application to a generalized power control problem and numerical examples are demonstrated.     

New monotone hybrid algorithm for hemi-relatively nonexpansive mappings and maximal monotone operators

The purpose of this article is to prove the strong convergence theorems for hemi-relatively nonexpansive mappings in Banach spaces. In order to get the strong convergence theorems for hemi-relatively nonexpansive mappings, a new monotone hybrid iteration algorithm is presented and is used to approximate the fixed point of hemi-relatively nonexpansive mappings. Noting that, the general hybrid iteration algorithm can be used for relatively nonexpansive mappings but it can not be used for hemi-relatively nonexpansive mappings. However, this new monotone hybrid algorithm can be used for hemi-relatively nonexpansive mappings. In addition, a new method of proof has been used in this article. That is, by using this new monotone hybrid algorithm, we firstly claim that, the iterative sequence is a Cauchy sequence. The results of this paper modify and improve the results of Matsushita and Takahashi, and some others.     

Steepest‐descent proximal point algorithms for a class of variational inequalities in Banach spaces

In this paper, we present a new approach to the problem of finding a common zero for a system of m‐accretive mappings in a uniformly convex Banach space with a uniformly Gâteaux differentiable norm. We propose an implicit iteration method and two explicit ones, based on compositions of resolvents with the steepest‐descent method. We show that our results contain some iterative methods in literature as special cases. An extension of the Xu's regularization method for the proximal point algorithm from Hilbert spaces onto Banach ones under simple conditions of convergence and a new variant for the method of alternating resolvents are obtained. Numerical experiments are given to affirm efficiency of the methods.     

A combination of potential reduction steps and steepest descent steps for solving convex programming problems

This paper studies the possibility of combining interior point strategy with a steepest descent method when solving convex programming problems, in such a way that the convergence property of the interior point method remains valid but many iterations do not request the solution of a system of equations. Motivated by this general idea, we propose a hybrid algorithm which combines a primal–dual potential reduction algorithm with the use of the steepest descent direction of the potential function. The complexity of the potential reduction algorithm remains valid but the overall computational cost can be reduced. Our numerical experiments are also reported. Copyright © 2002 John Wiley & Sons, Ltd.     

Strong convergence theorem for nonexpansive semigroups in Hilbert space

The purpose of this paper is to prove the strong convergence of a method combining the descent method and the hybrid method in mathematical programming for finding a point in the common fixed point set of a semigroup of nonexpansive mappings in Hilbert space.     

Strong convergence theorems for a common fixed point of two hemi-relatively nonexpansive mappings

The purpose of this article is to prove strong convergence theorems for common fixed points of two closed hemi-relatively nonexpansive mappings in Banach spaces. In order to get the strong convergence theorems, the monotone hybrid algorithms are presented and are used to approximate the common fixed points. Finally, a new simplified hybrid algorithm has been proposed and relative convergence theorem has been proved by using the new method for proofs. The results of this article modify and improve the results of Matsushita, Takahashi [S. Matsushita, W. Takahashi, A strong convergence theorem for relatively nonexpansive mappings in a Banach space, J. Approx. Theory 134 (2005) 257–266] and the results of Plubtieng, Ungchittrakool [S. Plubtieng, K. Ungchittrakool, Strong convergence theorems for a common fixed point of two relatively nonexpansive mappings in a Banach space, J. Approx. Theory 149 (2007) 103–115], and many others.     

Steepest descent preconditioning for nonlinear GMRES optimization

Steepest descent preconditioning is considered for the recently proposed nonlinear generalized minimal residual (N‐GMRES) optimization algorithm for unconstrained nonlinear optimization. Two steepest descent preconditioning variants are proposed. The first employs a line search, whereas the second employs a predefined small step. A simple global convergence proof is provided for the N‐GMRES optimization algorithm with the first steepest descent preconditioner (with line search), under mild standard conditions on the objective function and the line search processes. Steepest descent preconditioning for N‐GMRES optimization is also motivated by relating it to standard non‐preconditioned GMRES for linear systems in the case of a standard quadratic optimization problem with symmetric positive definite operator. Numerical tests on a variety of model problems show that the N‐GMRES optimization algorithm is able to very significantly accelerate convergence of stand‐alone steepest descent optimization. Moreover, performance of steepest‐descent preconditioned N‐GMRES is shown to be competitive with standard nonlinear conjugate gradient and limited‐memory Broyden–Fletcher–Goldfarb–Shanno methods for the model problems considered. These results serve to theoretically and numerically establish steepest‐descent preconditioned N‐GMRES as a general optimization method for unconstrained nonlinear optimization, with performance that appears promising compared with established techniques. In addition, it is argued that the real potential of the N‐GMRES optimization framework lies in the fact that it can make use of problem‐dependent nonlinear preconditioners that are more powerful than steepest descent (or, equivalently, N‐GMRES can be used as a simple wrapper around any other iterative optimization process to seek acceleration of that process), and this potential is illustrated with a further application example. Copyright © 2012 John Wiley & Sons, Ltd.     

Strong convergence theorems for countable families of asymptotically relatively nonexpansive mappings with applications

The purpose of this paper is by using the hybrid iterative method to prove some strong convergence theorems for approximating a common element of the set of solutions to a system of generalized mixed equilibrium problems and the set of common fixed points for two countable families of closed and asymptotically relatively nonexpansive mappings in Banach space. The results presented in the paper improve and extend the corresponding results of Su et al. [Y.F. Su, H.K. Xu, X. Zhang, Strong convergence theorems for two countable families of weak relatively nonexpansive mappings and applications, Nonlinear Anal. 73 (2010) 3890-3906], Li and Su [H.Y. Li, Y.F. Su, Strong convergence theorems by a new hybrid for equilibrium problems and variational inequality problems, Nonlinear Anal. 72 (2) (2010) 847-855], Chang et al. [S.S. Chang, H.W. Joseph Lee, Chi Kin Chan, A new hybrid method for solving a generalized equilibrium problem solving a variational inequality problem and obtaining common fixed points in Banach spaces with applications, Nonlinear Anal. TMA 73 (2010) 2260-2270], Kang et al. [J. Kang, Y. Su, X. Zhang, Hybrid algorithm for fixed points of weak relatively nonexpansive mappings and applications, Nonlinear Anal. HS 4 (4) (2010) 755-765], Matsushita and Takahashi [S. Matsushita, W. Takahashi, A strong convergence theorem for relatively nonexpansive mappings in Banach spaces, J. Approx. Theory 134 (2005) 257-266], Tan et al. [J.F. Tan, S.S. Chang, M. Liu, J.I. Liu, Strong convergence theorems of a hybrid projection algorithm for a family of quasi-?-asymptotically nonexpansive mappings, Opuscula Math. 30 (3) (2010) 341-348], Takahashia and Zembayashi [W. Takahashi, K. Zembayashi, Strong and weak convergence theorems for equilibrium problems and relatively nonexpansive mappings in Banach spaces, Nonlinear Anal. 70 (2009) 45-57] and Wattanawitoon and Kumam [K. Wattanawitoon, P. Kumam, Strong convergence theorems by a new hybrid projection algorithm for fixed point problem and equilibrium problems of two relatively quasi-nonexpansive mappings, Nonlinear Anal. Hybrid Systems 3 (2009) 11-20] and others.     

Modified block iterative algorithm for Quasi-?-asymptotically nonexpansive mappings and equilibrium problem in banach spaces

The purpose of this paper is to propose a modified block iterative algorithm for find a common element of the set of common fixed points of an infinite family of quasi-?-asymptotically nonexpansive mappings and the set of an equilibrium problem. Under suitable conditions, some strong convergence theorems are established in a uniformly smooth and strictly convex Banach space with the Kadec-Klee property. As an application, at the end of the paper a numerical example is given. The results presented in the paper improve and extend the corresponding results in Qin et al. [Convergence theorems of common elements for equilibrium problems and fixed point problem in Banach spaces, J. Comput. Appl. Math., 225, 2009, 20-30], Zhou et al. [Convergence theorems of a modified hybrid algorithm for a family of quasi-?-asymptotically nonexpansive mappings, J. Appl. Math. Compt., 17 March, 2009, doi:10.1007/s12190-009-0263-4], Takahashi and Zembayshi [Strong and weak convergence theorems for equilibrium problems and relatively nonexpansive mappings in Banach spaces, Nonlinear Anal., 70, 2009, 45-57], Wattanawitoon and Kumam [Strong convergence theorems by a new hybrid projection algorithm for fixed point problem and equilibrium problems of two relatively quasi-nonexpansive mappings, Nonlinear Anal. Hybrid Syst., 3, 2009, 11-20] and Matsushita and Takahashi [A strong convergence theorem for relatively nonexpansive mappings in Banach spaces, J. Approx. Theory, 134, 2005, 257-266] and others.     

A hybrid extragradient method for general variational inequalities

In this paper, we introduce and study a hybrid extragradient method for finding solutions of a general variational inequality problem with inverse-strongly monotone mapping in a real Hilbert space. An iterative algorithm is proposed by virtue of the hybrid extragradient method. Under two sets of quite mild conditions, we prove the strong convergence of this iterative algorithm to the unique common element of the set of fixed points of a nonexpansive mapping and the set of solutions of the general variational inequality problem, respectively. L. C. Zeng’s research was partially supported by the National Science Foundation of China (10771141), Ph.D. Program Foundation of Ministry of Education of China (20070270004), and Science and Technology Commission of Shanghai Municipality grant (075105118). J. C. Yao’s research was partially supported by a grant from the National Science Council of Taiwan.     

A descent algorithm for nonsmooth convex optimization

This paper presents a new descent algorithm for minimizing a convex function which is not necessarily differentiable. The algorithm can be implemented and may be considered a modification of the ε-subgradient algorithm and Lemarechal's descent algorithm. Also our algorithm is seen to be closely related to the proximal point algorithm applied to convex minimization problems. A convergence theorem for the algorithm is established under the assumption that the objective function is bounded from below. Limited computational experience with the algorithm is also reported.     

A novel hybrid immune algorithm and its convergence based on the steepest descent algorithm

This paper proposes a novel hybrid immune algorithm (HIA) that can overcome the typical drawback of the artificial immune algorithm (AIA), which runs slowly and experiences slow convergence. The HIA combines the adaptive AIA based on the steepest descent algorithm. The HIA fully displays global search ability and the global convergence of the immune algorithm. At the same time, it inserts a quasi-descent operator to strengthen its local search ability. A good convergence of the HIA with the quasi-descent idea is shown as well. Numerical experiment results show that the HIA successfully improves running speed and convergence performance.     

An iterative approximation method for solving a general system of variational inequality problems and mixed equilibrium problems

In this paper, we introduce an iterative algorithm for finding a common element of the set of solutions of a mixed equilibrium problem, the set of fixed points of an infinite family of nonexpansive mappings and the set of solutions of a general system of variational inequalities for a cocoercive mapping in a real Hilbert space. Furthermore, we prove that the proposed iterative algorithm converges strongly to a common element of the above three sets. Our results extend and improve the corresponding results of Ceng, Wang, and Yao [L.C. Ceng, C.Y. Wang, J.C. Yao, Strong convergence theorems by a relaxed extragradient method for a general system of variational inequalities, Math. Methods Oper. Res. 67 (2008) 375–390], Ceng and Yao [L.C. Ceng, J.C. Yao, A hybrid iterative scheme for mixed equilibrium problems and fixed point problems, J. Comput. Appl. Math. doi:10.1016/j.cam.2007.02.022], Takahashi and Takahashi [S. Takahashi, W. Takahashi, Viscosity approximation methods for equilibrium problems and fixed point problems in Hilbert spaces, J. Math. Anal. Appl. 331 (2007) 506–515] and many others.     

Sufficient Descent Riemannian Conjugate Gradient Methods

This paper considers sufficient descent Riemannian conjugate gradient methods with line search algorithms. We propose two kinds of sufficient descent nonlinear conjugate gradient method and prove that these methods satisfy the sufficient descent condition on Riemannian manifolds. One is a hybrid method combining a Fletcher–Reeves-type method with a Polak–Ribière–Polyak-type method, and the other is a Hager–Zhang-type method, both of which are generalizations of those used in Euclidean space. Moreover, we prove that the hybrid method has a global convergence property under the strong Wolfe conditions and the Hager–Zhang-type method has the sufficient descent property regardless of whether a line search is used or not. Further, we review two kinds of line search algorithm on Riemannian manifolds and numerically compare our generalized methods by solving several Riemannian optimization problems. The results show that the performance of the proposed hybrid methods greatly depends on the type of line search used. Meanwhile, the Hager–Zhang-type method has the fast convergence property regardless of the type of line search used.

     

Extension of the Hybrid Steepest Descent Method to a Class of Variational Inequalities and Fixed Point Problems with Nonself-Mappings

In this paper, we discuss the strong convergence of the hybrid steepest descent method relative to the case when the involved operators belong to a wide class of possibly nonself-mappings. Our convergence results cover previous ones, and the techniques of analysis used are simple and can be adapted to many other fixed point methods.     

三项混合共轭梯度算法及其收敛性

本文对求解无约束优化问题提出一类三项混合共轭梯度算法,新算法将Hestenes- stiefel算法与Dai-Yuan方法相结合,并在不需给定下降条件的情况下,证明了算法在Wolfe线搜索原则下的收敛性,数值试验亦显示出这种混合共轭梯度算法较之HS和PRP的优势．