Metric subregularity and the proximal point method

We examine the linear convergence rates of variants of the proximal point method for finding zeros of maximal monotone operators. We begin by showing how metric subregularity is sufficient for local linear convergence to a zero of a maximal monotone operator. This result is then generalized to obtain convergence rates for the problem of finding a common zero of multiple monotone operators by considering randomized and averaged proximal methods.     

A First-Order Stochastic Primal-Dual Algorithm with Correction Step

In this article, we investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive operator and a composite monotone operator. The proposed method is the stochastic extension to monotone inclusions of a proximal method studied in the literature for saddle point problems. It consists in a forward step determined by the stochastic evaluation of the cocoercive operator, a backward step in the dual variables involving the resolvent of the monotone operator, and an additional forward step using the stochastic evaluation of the cocoercive operator introduced in the first step. We prove weak almost sure convergence of the iterates by showing that the primal-dual sequence generated by the method is stochastic quasi-Fejér-monotone with respect to the set of zeros of the considered primal and dual inclusions. Additional results on ergodic convergence in expectation are considered for the special case of saddle point models.     

Penalty schemes with inertial effects for monotone inclusion problems

We introduce a penalty term-based splitting algorithm with inertial effects designed for solving monotone inclusion problems involving the sum of maximally monotone operators and the convex normal cone to the (nonempty) set of zeros of a monotone and Lipschitz continuous operator. We show weak ergodic convergence of the generated sequence of iterates to a solution of the monotone inclusion problem, provided a condition expressed via the Fitzpatrick function of the operator describing the underlying set of the normal cone is verified. Under strong monotonicity assumptions we can even show strong nonergodic convergence of the iterates. This approach constitutes the starting point for investigating from a similar perspective monotone inclusion problems involving linear compositions of parallel-sum operators and, further, for the minimization of a complexly structured convex objective function subject to the set of minima of another convex and differentiable function.     

On the convergence analysis of inexact hybrid extragradient proximal point algorithms for maximal monotone operators

In this paper we introduce general iterative methods for finding zeros of a maximal monotone operator in a Hilbert space which unify two previously studied iterative methods: relaxed proximal point algorithm [H.K. Xu, Iterative algorithms for nonlinear operators, J. London Math Soc. 66 (2002) 240–256] and inexact hybrid extragradient proximal point algorithm [R.S. Burachik, S. Scheimberg, B.F. Svaiter, Robustness of the hybrid extragradient proximal-point algorithm, J. Optim. Theory Appl. 111 (2001) 117–136]. The paper establishes both weak convergence and strong convergence of the methods under suitable assumptions on the algorithm parameters.     

The Proximal Point Algorithm Revisited

In this paper, we consider the proximal point algorithm for the problem of finding zeros of any given maximal monotone operator in an infinite-dimensional Hilbert space. For the usual distance between the origin and the operator’s value at each iterate, we put forth a new idea to achieve a new result on the speed at which the distance sequence tends to zero globally, provided that the problem’s solution set is nonempty and the sequence of squares of the regularization parameters is nonsummable. We show that it is comparable to a classical result of Brézis and Lions in general and becomes better whenever the proximal point algorithm does converge strongly. Furthermore, we also reveal its similarity to Güler’s classical results in the context of convex minimization in the sense of strictly convex quadratic functions, and we discuss an application to an ?-approximation solution of the problem above.     

Error bounds for proximal point subproblems and associated inexact proximal point algorithms

We study various error measures for approximate solution of proximal point regularizations of the variational inequality problem, and of the closely related problem of finding a zero of a maximal monotone operator. A new merit function is proposed for proximal point subproblems associated with the latter. This merit function is based on Burachik-Iusem-Svaiter’s concept of ε-enlargement of a maximal monotone operator. For variational inequalities, we establish a precise relationship between the regularized gap function, which is a natural error measure in this context, and our new merit function. Some error bounds are derived using both merit functions for the corresponding formulations of the proximal subproblem. We further use the regularized gap function to devise a new inexact proximal point algorithm for solving monotone variational inequalities. This inexact proximal point method preserves all the desirable global and local convergence properties of the classical exact/inexact method, while providing a constructive error tolerance criterion, suitable for further practical applications. The use of other tolerance rules is also discussed. Received: April 28, 1999 / Accepted: March 24, 2000?Published online July 20, 2000     

A new iterative algorithm for equilibrium and fixed point problems of nonexpansive mapping

In this paper, a new iterative algorithm involving nonexpansive mapping in Hilbert space is proposed and proved to be strongly convergent to a point which is simultaneously a fixed point of a nonexpansive mapping and a solution of an equilibrium problem. The results of the paper extend previous results, see, for instance, Takahashi and Takahashi (J Math Anal Appl 331:506–515, 2007), and other results in this field. Moreover, this algorithm is applied to find zeros of a maximal monotone operator and solve an optimization problem.     

A contraction proximal point algorithm with two monotone operators

It is a known fact that the method of alternating projections introduced long ago by von Neumann fails to converge strongly for two arbitrary nonempty, closed and convex subsets of a real Hilbert space. In this paper, a new iterative process for finding common zeros of two maximal monotone operators is introduced and strong convergence results associated with it are proved. If the two operators are subdifferentials of indicator functions, this new algorithm coincides with the old method of alternating projections. Several other important algorithms, such as the contraction proximal point algorithm, occur as special cases of our algorithm. Hence our main results generalize and unify many results that occur in the literature.     

An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems

We introduce and investigate the convergence properties of an inertial forward-backward-forward splitting algorithm for approaching the set of zeros of the sum of a maximally monotone operator and a single-valued monotone and Lipschitzian operator. By making use of the product space approach, we expand it to the solving of inclusion problems involving mixtures of linearly composed and parallel-sum type monotone operators. We obtain in this way an inertial forward-backward-forward primal-dual splitting algorithm having as main characteristic the fact that in the iterative scheme all operators are accessed separately either via forward or via backward evaluations. We present also the variational case when one is interested in the solving of a primal-dual pair of convex optimization problems with complexly structured objectives, which we also illustrate by numerical experiments in image processing.     

Iterative methods for a class of variational inequalities in Hilbert spaces

In this paper, we introduce implicit and explicit iterative methods for solving a variational inequality problem over the set of zeros for a maximal monotone mapping in Hilbert spaces. As consequence, new modifications of the proximal point method are obtained.     

A general algorithm for multiple-sets split feasibility problem involving resolvents and Bregman mappings

In this paper, by using Bregman distance, we introduce a new iterative process involving products of resolvents of maximal monotone operators for approximating a common element of the set of common fixed points of a finite family of multi-valued Bregman relatively nonexpansive mappings and the solution set of the multiple-sets split feasibility problem and common zeros of maximal monotone operators. We derive a strong convergence theorem of the proposed iterative algorithm under appropriate situations. Finally, we mention several corollaries and two applications of our algorithm.     

On Rockafellar’s theorem using proximal point algorithm involving -maximal monotonicity framework

On the basis of the general framework of H-maximal monotonicity (also referred to as H-monotonicity in the literature), a generalization to Rockafellar’s theorem in the context of solving a general inclusion problem involving a set-valued maximal monotone operator using the proximal point algorithm in a Hilbert space setting is explored. As a matter of fact, this class of inclusion problems reduces to a class of variational inequalities as well as to a class of complementarity problems. This proximal point algorithm turns out to be of interest in the sense that it plays a significant role in certain computational methods of multipliers in nonlinear programming. The notion of H-maximal monotonicity generalizes the general theory of set-valued maximal monotone mappings to a new level. Furthermore, some results on general firm nonexpansiveness and resolvent mapping corresponding to H-monotonicity are also given.     

A generalized contraction proximal point algorithm with two monotone operators

Abstract

In this work, we introduce a generalized contraction proximal point algorithm and use it to approximate common zeros of maximal monotone operators A and B in a real Hilbert space setting. The algorithm is a two step procedure that alternates the resolvents of these operators and uses general assumptions on the parameters involved. For particular cases, these relaxed parameters improve the convergence rate of the algorithm. A strong convergence result associated with the algorithm is proved under mild conditions on the parameters. Our main result improves and extends several results in the literature.     

A simplified proof of weak convergence in Douglas–Rachford method

Douglas–Rachford method is a splitting algorithm for finding a zero of the sum of two maximal monotone operators. Weak convergence in this method to a solution of the underlying monotone inclusion problem in the general case remained an open problem for 30 years and was proved by the author 7 years ago. That proof was cluttered with technicalities because we considered the inexact version with summable errors. In this short communication we present a streamlined proof of this result.     

Hybrid iterative scheme by a relaxed extragradient method for solutions of equilibrium problems and a general system of variational inequalities with application to optimization

In this paper, we introduce a new iterative process for finding the common element of the set of fixed points of a nonexpansive mapping, the set of solutions of an equilibrium problem and the solutions of the variational inequality problem for two inverse-strongly monotone mappings. We introduce a new viscosity relaxed extragradient approximation method which is based on the so-called relaxed extragradient method and the viscosity approximation method. We show that the sequence converges strongly to a common element of the above three sets under some parametric controlling conditions. Moreover, using the above theorem, we can apply to finding solutions of a general system of variational inequality and a zero of a maximal monotone operator in a real Hilbert space. The results of this paper extended, improved and connected with the results of Ceng et al., [L.-C. Ceng, C.-Y. Wang, J.-C. Yao, Strong convergence theorems by a relaxed extragradient method for a general system of variational inequalities, Math. Meth. Oper. Res. 67 (2008), 375–390], Plubtieng and Punpaeng, [S. Plubtieng, R. Punpaeng, A new iterative method for equilibrium problems and fixed point problems of nonexpansive mappings and monotone mappings, Appl. Math. Comput. 197 (2) (2008) 548–558] Su et al., [Y. Su, et al., An iterative method of solution for equilibrium and optimization problems, Nonlinear Anal. 69 (8) (2008) 2709–2719], Li and Song [Liwei Li, W. Song, A hybrid of the extragradient method and proximal point algorithm for inverse strongly monotone operators and maximal monotone operators in Banach spaces, Nonlinear Anal.: Hybrid Syst. 1 (3) (2007), 398-413] and many others.     

Approximating Solutions of Maximal Monotone Operators in Hilbert Spaces

Let H be a real Hilbert space and let T: H→2^H be a maximal monotone operator. In this paper, we first introduce two algorithms of approximating solutions of maximal monotone operators. One of them is to generate a strongly convergent sequence with limit vT⁻¹0. The other is to discuss the weak convergence of the proximal point algorithm. Next, using these results, we consider the problem of finding a minimizer of a convex function. Our methods are motivated by Halpern's iteration and Mann's iteration.     

Convergence of over-relaxed contraction-proximal point algorithm in Hilbert spaces

The proximal point algorithm (PPA) is a classical method for finding zeros of maximal monotone operators. It is known that the algorithm only has weak convergence in a general Hilbert space. Recently, Wang, Wang and Xu proposed two modifications of the PPA and established strong convergence theorems on these two algorithms. However, these two convergence theorems exclude an important case, namely, the over-relaxed case. In this paper, we extend the above convergence theorems from under-relaxed case to the over-relaxed case, which in turn improve the performance of these two algorithms. Preliminary numerical experiments show that the algorithm with over-relaxed parameter performs better than that with under-relaxed parameter.     

Forward-Backward and Tseng’s Type Penalty Schemes for Monotone Inclusion Problems

We deal with monotone inclusion problems of the form 0 ∈ A x + D x + N _C (x) in real Hilbert spaces, where A is a maximally monotone operator, D a cocoercive operator and C the nonempty set of zeros of another cocoercive operator. We propose a forward-backward penalty algorithm for solving this problem which extends the one proposed by Attouch et al. (SIAM J. Optim. 21(4): 1251-1274, 2011). The condition which guarantees the weak ergodic convergence of the sequence of iterates generated by the proposed scheme is formulated by means of the Fitzpatrick function associated to the maximally monotone operator that describes the set C. In the second part we introduce a forward-backward-forward algorithm for monotone inclusion problems having the same structure, but this time by replacing the cocoercivity hypotheses with Lipschitz continuity conditions. The latter penalty type algorithm opens the gate to handle monotone inclusion problems with more complicated structures, for instance, involving compositions of maximally monotone operators with linear continuous ones.     

Visco-penalization of the sum of two monotone operators

A new type of approximating curve for finding a particular zero of the sum of two maximal monotone operators in a Hilbert space is investigated. This curve consists of the zeros of perturbed problems in which one operator is replaced with its Yosida approximation and a viscosity term is added. As the perturbation vanishes, the curve is shown to converge to the zero of the sum that solves a particular strictly monotone variational inequality. As an off-spring of this result, we obtain an approximating curve for finding a particular zero of the sum of several maximal monotone operators. Applications to convex optimization are discussed.     

Graph-distance convergence and uniform local boundedness of monotone mappings

In this article we study graph-distance convergence of monotone operators. First, we prove a property that has been an open problem up to now: the limit of a sequence of graph-distance convergent maximal monotone operators in a Hilbert space is a maximal monotone operator. Next, we show that a sequence of maximal monotone operators converging in the same sense in a reflexive Banach space is uniformly locally bounded around any point from the interior of the domain of the limit mapping. The result is an extension of a similar one from finite dimensions. As an application we give a simplified condition for the stability (under graph-distance convergence) of the sum of maximal monotone mappings in Hilbert spaces.