Forcing strong convergence of proximal point iterations in a Hilbert space

This paper concerns with convergence properties of the classical proximal point algorithm for finding zeroes of maximal monotone operators in an infinite-dimensional Hilbert space. It is well known that the proximal point algorithm converges weakly to a solution under very mild assumptions. However, it was shown by Güler [11] that the iterates may fail to converge strongly in the infinite-dimensional case. We propose a new proximal-type algorithm which does converge strongly, provided the problem has a solution. Moreover, our algorithm solves proximal point subproblems inexactly, with a constructive stopping criterion introduced in [31]. Strong convergence is forced by combining proximal point iterations with simple projection steps onto intersection of two halfspaces containing the solution set. Additional cost of this extra projection step is essentially negligible since it amounts, at most, to solving a linear system of two equations in two unknowns. Received January 6, 1998 / Revised version received August 9, 1999?Published online November 30, 1999     

Inexact implicit methods for monotone general variational inequalities

Solving a variational inequality problem is equivalent to finding a solution of a system of nonsmooth equations. Recently, we proposed an implicit method, which solves monotone variational inequality problem via solving a series of systems of nonlinear smooth (whenever the operator is smooth) equations. It can exploit the facilities of the classical Newton–like methods for smooth equations. In this paper, we extend the method to solve a class of general variational inequality problems Moreover, we improve the implicit method to allow inexact solutions of the systems of nonlinear equations at each iteration. The method is shown to preserve the same convergence properties as the original implicit method. Received July 31, 1995 / Revised version received January 15, 1999? Published online May 28, 1999     

The prox-Tikhonov regularization method for the proximal point algorithm in Banach spaces

It is known, by Rockafellar (SIAM J Control Optim 14:877–898, 1976), that the proximal point algorithm (PPA) converges weakly to a zero of a maximal monotone operator in a Hilbert space, but it fails to converge strongly. Lehdili and Moudafi (Optimization 37:239–252, 1996) introduced the new prox-Tikhonov regularization method for PPA to generate a strongly convergent sequence and established a convergence property for it by using the technique of variational distance in the same space setting. In this paper, the prox-Tikhonov regularization method for the proximal point algorithm of finding a zero for an accretive operator in the framework of Banach space is proposed. Conditions which guarantee the strong convergence of this algorithm to a particular element of the solution set is provided. An inexact variant of this method with error sequence is also discussed.     

A new inexact alternating directions method for monotone variational inequalities

The alternating directions method (ADM) is an effective method for solving a class of variational inequalities (VI) when the proximal and penalty parameters in sub-VI problems are properly selected. In this paper, we propose a new ADM method which needs to solve two strongly monotone sub-VI problems in each iteration approximately and allows the parameters to vary from iteration to iteration. The convergence of the proposed ADM method is proved under quite mild assumptions and flexible parameter conditions. Received: January 4, 2000 / Accepted: October 2001?Published online February 14, 2002     

On inexact generalized proximal methods with a weakened error tolerance criterion

Two inexact versions of a Bregman-function-based proximal method for finding a zero of a maximal monotone operator, suggested in [J. Eckstein (1998). Approximate iterations in Bregman-function-based proximal algorithms. Math. Programming, 83, 113–123; P. da Silva, J. Eckstein and C. Humes (2001). Rescaling and stepsize selection in proximal methods using separable generalized distances. SIAM J. Optim., 12, 238–261], are considered. For a wide class of Bregman functions, including the standard entropy kernel and all strongly convex Bregman functions, convergence of these methods is proved under an essentially weaker accuracy condition on the iterates than in the original papers.

Also the error criterion of a logarithmic–quadratic proximal method, developed in [A. Auslender, M. Teboulle and S. Ben-Tiba (1999). A logarithmic-quadratic proximal method for variational inequalities. Computational Optimization and Applications, 12, 31–40], is relaxed, and convergence results for the inexact version of the proximal method with entropy-like distance functions are described.

For the methods mentioned, like in [R.T. Rockafellar (1976). Monotone operators and the proximal point algorithm. SIAM J. Control Optim., 14, 877–898] for the classical proximal point algorithm, only summability of the sequence of error vector norms is required.     

On the superlinear convergence of the variable metric proximal point algorithm using Broyden and BFGS matrix secant updating

In previous work, the authors provided a foundation for the theory of variable metric proximal point algorithms in Hilbert space. In that work conditions are developed for global, linear, and super–linear convergence. This paper focuses attention on two matrix secant updating strategies for the finite dimensional case. These are the Broyden and BFGS updates. The BFGS update is considered for application in the symmetric case, e.g., convex programming applications, while the Broyden update can be applied to general monotone operators. Subject to the linear convergence of the iterates and a quadratic growth condition on the inverse of the operator at the solution, super–linear convergence of the iterates is established for both updates. These results are applied to show that the Chen–Fukushima variable metric proximal point algorithm is super–linearly convergent when implemented with the BFGS update. Received: September 12, 1996 / Accepted: January 7, 2000?Published online March 15, 2000     

On Rockafellar’s theorem using proximal point algorithm involving -maximal monotonicity framework

On the basis of the general framework of H-maximal monotonicity (also referred to as H-monotonicity in the literature), a generalization to Rockafellar’s theorem in the context of solving a general inclusion problem involving a set-valued maximal monotone operator using the proximal point algorithm in a Hilbert space setting is explored. As a matter of fact, this class of inclusion problems reduces to a class of variational inequalities as well as to a class of complementarity problems. This proximal point algorithm turns out to be of interest in the sense that it plays a significant role in certain computational methods of multipliers in nonlinear programming. The notion of H-maximal monotonicity generalizes the general theory of set-valued maximal monotone mappings to a new level. Furthermore, some results on general firm nonexpansiveness and resolvent mapping corresponding to H-monotonicity are also given.     

The perturbed proximal point algorithm and some of its applications

Following the works of R. T. Rockafellar, to search for a zero of a maximal monotone operator, and of B. Lemaire, to solve convex optimization problems, we present a perturbed version of the proximal point algorithm. We apply this new algorithm to convex optimization and to variational inclusions or, more particularly, to variational inequalities.     

A note on the regularized proximal point algorithm

Recently, Xu (J Glob Optim 36:115–125 (2006)) introduced a regularized proximal point algorithm for approximating a zero of a maximal monotone operator. In this note, we shall prove the strong convergence of this algorithm under some weaker conditions.     

Global s-type error bound for the extended linear complementarity problem and applications

For the extended linear complementarity problem over an affine subspace, we first study some characterizations of (strong) column/row monotonicity and (strong) R ₀-property. We then establish global s-type error bound for this problem with the column monotonicity or R ₀-property, especially for the one with the nondegeneracy and column monotonicity, and give several equivalent formulations of such error bound without the square root term for monotone affine variational inequality. Finally, we use this error bound to derive some properties of the iterative sequence produced by smoothing methods for solving such a problem under suitable assumptions. Received: May 2, 1999 / Accepted: February 21, 2000?Published online July 20, 2000     

On the convergence analysis of inexact hybrid extragradient proximal point algorithms for maximal monotone operators

In this paper we introduce general iterative methods for finding zeros of a maximal monotone operator in a Hilbert space which unify two previously studied iterative methods: relaxed proximal point algorithm [H.K. Xu, Iterative algorithms for nonlinear operators, J. London Math Soc. 66 (2002) 240–256] and inexact hybrid extragradient proximal point algorithm [R.S. Burachik, S. Scheimberg, B.F. Svaiter, Robustness of the hybrid extragradient proximal-point algorithm, J. Optim. Theory Appl. 111 (2001) 117–136]. The paper establishes both weak convergence and strong convergence of the methods under suitable assumptions on the algorithm parameters.     

An inexact proximal point algorithm for maximal monotone vector fields on Hadamard manifolds

In this paper, an inexact proximal point algorithm concerned with the singularity of maximal monotone vector fields is introduced and studied on Hadamard manifolds, in which a relative error tolerance with squared summable error factors is considered. It is proved that the sequence generated by the proposed method is convergent to a solution of the problem. Moreover, an application to the optimization problem on Hadamard manifolds is given. The main results presented in this paper generalize and improve some corresponding known results given in the literature.     

Inexact proximal point method for general variational inequalities

In this paper, we suggest and analyze a new inexact proximal point method for solving general variational inequalities, which can be considered as an implicit predictor-corrector method. An easily measurable error term is proposed with further relaxed error bound and an optimal step length is obtained by maximizing the profit-function and is dependent on the previous points. Our results include several known and new techniques for solving variational inequalities and related optimization problems. Results obtained in this paper can be viewed as an important improvement and refinement of the previously known results. Preliminary numerical experiments are included to illustrate the advantage and efficiency of the proposed method.     

A proximal point method for the sum of maximal monotone operators

In this paper, we concentrate on the maximal inclusion problem of locating the zeros of the sum of maximal monotone operators in the framework of proximal point method. Such problems arise widely in several applied mathematical fields such as signal and image processing. We define two new maximal monotone operators and characterize the solutions of the considered problem via the zeros of the new operators. The maximal monotonicity and resolvent of both of the defined operators are proved and calculated, respectively. The traditional proximal point algorithm can be therefore applied to the considered maximal inclusion problem, and the convergence is ensured. Furthermore, by exploring the relationship between the proposed method and the generalized forward‐backward splitting algorithm, we point out that this algorithm is essentially the proximal point algorithm when the operator corresponding to the forward step is the zero operator. Copyright © 2013 John Wiley & Sons, Ltd.     

求解单调变分不等式的近似邻近点算法的收敛性分析

为了求解单调变分不等式,建立了一个新的误差准则,并且在不需要增加诸如投影,外梯度等步骤的情况下证明了邻近点算法的收敛性.     

Metric subregularity and the proximal point method

We examine the linear convergence rates of variants of the proximal point method for finding zeros of maximal monotone operators. We begin by showing how metric subregularity is sufficient for local linear convergence to a zero of a maximal monotone operator. This result is then generalized to obtain convergence rates for the problem of finding a common zero of multiple monotone operators by considering randomized and averaged proximal methods.     

Epsilon-proximal decomposition method

We propose a modification of the proximal decomposition method investigated by Spingarn [30] and Mahey et al. [19] for minimizing a convex function on a subspace. For the method to be favorable from a computational point of view, particular importance is the introduction of approximations in the proximal step. First, we couple decomposition on the graph of the epsilon-subdifferential mapping and cutting plane approximations to get an algorithmic pattern that falls in the general framework of Rockafellar inexact proximal-point algorithms [26]. Recently, Solodov and Svaiter [27] proposed a new proximal point-like algorithm that uses improved error criteria and an enlargement of the maximal monotone operator defining the problem. We combine their idea with bundle mecanism to devise an inexact proximal decomposition method with error condition which is not hard to satisfy in practice. Then, we present some applications favorable to our development. First, we give a new regularized version of Benders decomposition method in convex programming called the proximal convex Benders decomposition algorithm. Second, we derive a new algorithm for nonlinear multicommodity flow problems among which the message routing problem in telecommunications data networks.     

Lagrange multipliers, (exact) regularization and error bounds for monotone variational inequalities

We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone mapping, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associated with the existence of Lagrange multipliers for the regularized variational inequality. A regularization is said to be exact if a solution to the regularized problem is a solution to the unregularized problem for all parameters beyond a certain value. The Lagrange multipliers corresponding to a particular regularization of a variational inequality, on the other hand, are defined via the dual gap function. Our analysis suggests various conceptual, iteratively regularized numerical schemes, for which we provide error bounds, and hence stopping criteria, under the additional assumption that the solution set to the unregularized problem is what we call weakly sharp of order greater than one.     

A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities

In this paper we take a new look at smoothing Newton methods for solving the nonlinear complementarity problem (NCP) and the box constrained variational inequalities (BVI). Instead of using an infinite sequence of smoothing approximation functions, we use a single smoothing approximation function and Robinson’s normal equation to reformulate NCP and BVI as an equivalent nonsmooth equation H(u,x)=0, where H:ℜ ²ⁿ →ℜ ²ⁿ , u∈ℜ ⁿ is a parameter variable and x∈ℜ ⁿ is the original variable. The central idea of our smoothing Newton methods is that we construct a sequence {z ^k =(u ^k ,x ^k )} such that the mapping H(·) is continuously differentiable at each z ^k and may be non-differentiable at the limiting point of {z ^k }. We prove that three most often used Gabriel-Moré smoothing functions can generate strongly semismooth functions, which play a fundamental role in establishing superlinear and quadratic convergence of our new smoothing Newton methods. We do not require any function value of F or its derivative value outside the feasible region while at each step we only solve a linear system of equations and if we choose a certain smoothing function only a reduced form needs to be solved. Preliminary numerical results show that the proposed methods for particularly chosen smoothing functions are very promising. Received June 23, 1997 / Revised version received July 29, 1999?Published online December 15, 1999     

On the proximal point method in Hadamard spaces

Proximal point method is one of the most influential procedure in solving nonlinear variational problems. It has recently been introduced in Hadamard spaces for solving convex optimization, and later for variational inequalities. In this paper, we study the general proximal point method for finding a zero point of a maximal monotone set-valued vector field defined on a Hadamard space and valued in its dual. We also give the relation between the maximality and Minty’s surjectivity condition, which is essential for the proximal point method to be well-defined. By exploring the properties of monotonicity and the surjectivity condition, we were able to show under mild assumptions that the proximal point method converges weakly to a zero point. Additionally, by taking into account the metric subregularity, we obtained the local strong convergence in linear and super-linear rates.