A note on a paper “A regularization method for the proximal point algorithm”

In this note, a small gap is corrected in the proof of H.K. Xu [Theorem 3.3, A regularization method for the proximal point algorithm, J. Glob. Optim. 36, 115–125 (2006)], and some strict restriction is removed also.      

The prox-Tikhonov regularization method for the proximal point algorithm in Banach spaces

It is known, by Rockafellar (SIAM J Control Optim 14:877–898, 1976), that the proximal point algorithm (PPA) converges weakly to a zero of a maximal monotone operator in a Hilbert space, but it fails to converge strongly. Lehdili and Moudafi (Optimization 37:239–252, 1996) introduced the new prox-Tikhonov regularization method for PPA to generate a strongly convergent sequence and established a convergence property for it by using the technique of variational distance in the same space setting. In this paper, the prox-Tikhonov regularization method for the proximal point algorithm of finding a zero for an accretive operator in the framework of Banach space is proposed. Conditions which guarantee the strong convergence of this algorithm to a particular element of the solution set is provided. An inexact variant of this method with error sequence is also discussed.     

Inertial proximal point regularization algorithm for unconstrained vector convex optimization problems

The purpose of this paper is to investigate an iterative regularization method of proximal point type for solving ill posed vector convex optimization problems in Hilbert spaces. Applications to the convex feasibility problems and the problem of common fixed points for nonexpansive potential mappings are also given. Published in Ukrains'kyi Matematychnyi Zhurnal, Vol. 60, No. 9, pp. 1275–1281, September, 2008.     

Iterative proximal regularization method for finding a saddle point in the semicoercive Signorini problem

An algorithm for seeking a saddle point for the semicoercive variational Signorini inequality is studied. The algorithm is based on an iterative proximal regularization of a modified Lagrangian functional.     

A proximal point algorithm for minimax problems

This paper presents a proximal point algorithm for solving discretel _∞ approximation problems of the form minimize ∥Ax−b∥_∞. Let ε_∞ be a preassigned positive constant and let ε _l ,l = 0,1,2,... be a sequence of positive real numbers such that 0 < ε _l < ε_∞. Then, starting from an arbitrary pointz ₀, the proposed method generates a sequence of points z _l ,l= 0,1,2,..., via the rule . One feature that characterizes this algorithm is its finite termination property. That is, a solution is reached within a finite number of iterations. The smaller are the numbers ε _l the smaller is the number of iterations. In fact, if ε ₀ is sufficiently small then z₁ solves the original minimax problem. The practical value of the proposed iteration depends on the availability of an efficient code for solving a regularized minimax problem of the form minimize where ∈ is a given positive constant. It is shown that the dual of this problem has the form maximize , and ify solves the dual thenx=A ^T y solves the primal. The simple structure of the dual enables us to apply a wide range of methods. In this paper we design and analyze a row relaxation method which is suitable for solving large sparse problems. Numerical experiments illustrate the feasibility of our ideas.     

A proximal point algorithm for control approximation problems

We consider the optimization problem $$f(x): = (c,x) + \sum\limits_{i = 1}^n {\alpha _i \parallel A^i x - a^i \parallel ^{\beta _i } \to \mathop {\min }\limits_{x \in D} ,} $$ which is an extension of a problem studied by Idrissi, Lefebvre and Michelot [7]. This class of problems contains many practically important special cases so as approximation, location and optimal control problems, perturbed linear programming problems and surrogate problems for linear programming. Necessary and sufficient optimality conditions are derived using the sub-differential calculus. A proximal point algorithm is modified by the method of partial inverse [13] in order to solve the optimality conditions.     

A proximal point algorithm for generalized fractional programs

In this paper, the problem of solving generalized fractional programs will be addressed. This problem has been extensively studied and several algorithms have been proposed. In this work, we propose an algorithm that combines the proximal point method with a continuous min–max formulation of discrete generalized fractional programs. The proposed method can handle non-differentiable convex problems with possibly unbounded feasible constraints set, and solves at each iteration a convex program with unique dual solution. It generates two sequences that approximate the optimal value of the considered problem from below and from above at each step. For a class of functions, including the linear case, the convergence rate is at least linear.     

A proximal point algorithm revisit on the alternating direction method of multipliers

The alternating direction method of multipliers(ADMM)is a benchmark for solving convex programming problems with separable objective functions and linear constraints.In the literature it has been illustrated as an application of the proximal point algorithm(PPA)to the dual problem of the model under consideration.This paper shows that ADMM can also be regarded as an application of PPA to the primal model with a customized choice of the proximal parameter.This primal illustration of ADMM is thus complemental to its dual illustration in the literature.This PPA revisit on ADMM from the primal perspective also enables us to recover the generalized ADMM proposed by Eckstein and Bertsekas easily.A worst-case O(1/t)convergence rate in ergodic sense is established for a slight extension of Eckstein and Bertsekas’s generalized ADMM.     

一个新的BUNDLE近似点算法及其收敛性质

提出了一类修正的近似点算法并讨论了算法的收敛性质及其Ｂｕｄｌｅ变形的收敛性质。     

近似邻近点算法收敛性的一个注记

近似邻近点算法在最优化理论与方法研究中具有重要作用.在不同误差准则下,近似邻近点算法具有不同的收敛性.利用极大单调算子等工具给出了一个具体的例子,解释了在一些误差准则下近似邻近点算法的收敛性.     

A note on the regularized proximal point algorithm

Recently, Xu (J Glob Optim 36:115–125 (2006)) introduced a regularized proximal point algorithm for approximating a zero of a maximal monotone operator. In this note, we shall prove the strong convergence of this algorithm under some weaker conditions.     

A proximal point algorithm converging strongly for general errors

In this paper a proximal point algorithm (PPA) for maximal monotone operators with appropriate regularization parameters is considered. A strong convergence result for PPA is stated and proved under the general condition that the error sequence tends to zero in norm. Note that Rockafellar (SIAM J Control Optim 14:877–898, 1976) assumed summability for the error sequence to derive weak convergence of PPA in its initial form, and this restrictive condition on errors has been extensively used so far for different versions of PPA. Thus this Note provides a solution to a long standing open problem and in particular offers new possibilities towards the approximation of the minimum points of convex functionals.     

A parameterized proximal point algorithm for separable convex optimization

In this paper, we develop a parameterized proximal point algorithm (P-PPA) for solving a class of separable convex programming problems subject to linear and convex constraints. The proposed algorithm is provable to be globally convergent with a worst-case O(1 / t) convergence rate, where t denotes the iteration number. By properly choosing the algorithm parameters, numerical experiments on solving a sparse optimization problem arising from statistical learning show that our P-PPA could perform significantly better than other state-of-the-art methods, such as the alternating direction method of multipliers and the relaxed proximal point algorithm.     

An inexact proximal regularization method for unconstrained optimization

We present a regularization algorithm to solve a smooth unconstrained minimization problem.This algorithm is suitable to solve a degenerate problem, when the Hessian is singular at a local optimal solution. The main feature of our algorithm is that it uses an outer/inner iteration scheme. We show that the algorithm has a strong global convergence property under mild assumptions. A local convergence analysis shows that the algorithm is superlinearly convergent under a local error bound condition. Some numerical experiments are reported.     

A proximal point algorithm for the monotone second-order cone complementarity problem

This paper is devoted to the study of the proximal point algorithm for solving monotone second-order cone complementarity problems. The proximal point algorithm is to generate a sequence by solving subproblems that are regularizations of the original problem. After given an appropriate criterion for approximate solutions of subproblems by adopting a merit function, the proximal point algorithm is verified to have global and superlinear convergence properties. For the purpose of solving the subproblems efficiently, we introduce a generalized Newton method and show that only one Newton step is eventually needed to obtain a desired approximate solution that approximately satisfies the appropriate criterion under mild conditions. Numerical comparisons are also made with the derivative-free descent method used by Pan and Chen (Optimization 59:1173–1197, 2010), which confirm the theoretical results and the effectiveness of the algorithm.     

Generalized proximal point algorithm for convex optimization

Ha (Ref. 1) recently introduced a generalized proximal point algorithm for solving a generalized equation. In this note, we present a generalized proximal point algorithm for convex optimization problems based on Ha's work. The idea behind this algorithm is that, instead of adding a quadratic term to all the variables, we add a quadratic term to a subset of the variables. We extend the criteria for approximate solutions given by Rockafellar (Ref. 2) and Auslender (Ref. 3) and present convergence results. Finally, we show how this algorithm can be applied to solve block-angular linear and quadratic programming problems.     

Interior proximal point algorithm for linear programs

An interior proximal point algorithm for finding a solution of a linear program is presented. The distinguishing feature of this algorithm is the addition of a quadratic proximal term to the linear objective function. This perturbation has allowed us to obtain solutions with better feasibility. Implementation of this algorithm shows that the algorithms. We also establish global convergence and local linear convergence of the algorithm.This research was supported by National Science Foundation Grants DCR-85-21228 and CCR-87-23091 and by Air Force Office of Scientific Research Grants AFOSR-86-0172 and AFOSR-89-0410. It was conducted while the author was a Graduate Student at the Computer Sciences Department, University of Wisconsin, Madison, Wisconsin.     

A new approximate proximal point algorithm for maximal monotone operator

The problem concerned in this paper is the set-valued equation 0 ∈T(z) where T is a maximal monotone operator. For given xk and βk > 0, some existing approximate proximal point algorithms take x~(k+1) = xk such thatwhere {ηk} is a non-negative summable sequence. Instead of xk+1 = xk , the new iterate of the proposing method is given bywhere Ω is the domain of T and PΩ(·) denotes the projection on Ω. The convergence is proved under a significantly relaxed restriction supk>0 ηk<1.