广义交替近似梯度算法的线性收敛分析

针对两个可分凸函数的和在线性约束下的极小化问题,在交替方向法的框架下,提出广义的交替近似梯度算法.在一定的条件下,该算法具有全局及线性收敛性.数值实验表明该算法有好的数值表现.     

一类自适应广义交替方向乘子法

广义交替方向乘子法是求解凸优化问题的有效算法.当实际问题中子问题难以求解时,可以采用在子问题中添加邻近项的方法处理,邻近矩阵正定时,算法收敛,然而这也会使迭代步长较小.最新研究表明,邻近矩阵可以有一定的不正定性.本文在基于不定邻近项的广义交替方向乘子法框架下,提出一种自适应的广义交替方向乘子法,动态地选择邻近矩阵,增大迭代步长.在一些较弱的假设下,证明了算法的全局收敛性.我们进行一些初等数值实验,验证了算法的有效性.     

Pointwise and Ergodic Convergence Rates of a Variable Metric Proximal Alternating Direction Method of Multipliers

In this paper, we obtain global pointwise and ergodic convergence rates for a variable metric proximal alternating direction method of multipliers for solving linearly constrained convex optimization problems. We first propose and study nonasymptotic convergence rates of a variable metric hybrid proximal extragradient framework for solving monotone inclusions. Then, the convergence rates for the former method are obtained essentially by showing that it falls within the latter framework. To the best of our knowledge, this is the first time that global pointwise (resp. pointwise and ergodic) convergence rates are obtained for the variable metric proximal alternating direction method of multipliers (resp. variable metric hybrid proximal extragradient framework).     

Relaxed inertial proximal Peaceman-Rachford splitting method for separable convex programming

The strictly contractive Peaceman-Rachford splitting method is one of effective methods for solving separable convex optimization problem, and the inertial proximal Peaceman-Rachford splitting method is one of its important variants. It is known that the convergence of the inertial proximal Peaceman-Rachford splitting method can be ensured if the relaxation factor in Lagrangian multiplier updates is underdetermined, which means that the steps for the Lagrangian multiplier updates are shrunk conservatively. Although small steps play an important role in ensuring convergence, they should be strongly avoided in practice. In this article, we propose a relaxed inertial proximal Peaceman-Rachford splitting method, which has a larger feasible set for the relaxation factor. Thus, our method provides the possibility to admit larger steps in the Lagrangian multiplier updates. We establish the global convergence of the proposed algorithm under the same conditions as the inertial proximal Peaceman-Rachford splitting method. Numerical experimental results on a sparse signal recovery problem in compressive sensing and a total variation based image denoising problem demonstrate the effectiveness of our method.     

A HOMOTOPY-BASED ALTERNATING DIRECTION METHOD OF MULTIPLIERS FOR STRUCTURED CONVEX OPTIMIZATION

The alternating direction method of multipliers(ADMM for short) is efficient for linearly constrained convex optimization problem. The practical computational cost of ADMM depends on the sub-problem solvers. The proximal point algorithm is a common sub-problem-solver. However, the proximal parameter is sensitive in the proximal ADMM. In this paper, we propose a homotopy-based proximal linearized ADMM, in which a homotopy method is used to solve the sub-problems at each iteration. Under some suitable conditions, the global convergence and the convergence rate of O(1/k) in the worst case of the proposed method are proven. Some preliminary numerical results indicate the validity of the proposed method.     

Symmetric Alternating Direction Method with Indefinite Proximal Regularization for Linearly Constrained Convex Optimization

The proximal alternating direction method of multipliers is a popular and useful method for linearly constrained, separable convex problems, especially for the linearized case. In the literature, convergence of the proximal alternating direction method has been established under the assumption that the proximal regularization matrix is positive semi-definite. Recently, it was shown that the regularizing proximal term in the proximal alternating direction method of multipliers does not necessarily have to be positive semi-definite, without any additional assumptions. However, it remains unknown as to whether the indefinite setting is valid for the proximal version of the symmetric alternating direction method of multipliers. In this paper, we confirm that the symmetric alternating direction method of multipliers can also be regularized with an indefinite proximal term. We theoretically prove the global convergence of the indefinite method and establish its worst-case convergence rate in an ergodic sense. In addition, the generalized alternating direction method of multipliers proposed by Eckstein and Bertsekas is a special case in our discussion. Finally, we demonstrate the performance improvements achieved when using the indefinite proximal term through experimental results.     

A preconditioned general modulus-based matrix splitting iteration method for linear complementarity problems of <Emphasis Type="Italic">H</Emphasis>-matrices

In this paper, we propose a preconditioned general modulus-based matrix splitting iteration method for solving modulus equations arising from linear complementarity problems. Its convergence theory is proved when the system matrix is an H₊-matrix, from which some new convergence conditions can be derived for the (general) modulus-based matrix splitting iteration methods. Numerical results further show that the proposed methods are superior to the existing methods.     

An extension of proximal methods for quasiconvex minimization on the nonnegative orthant

In this paper we propose an extension of proximal methods to solve minimization problems with quasiconvex objective functions on the nonnegative orthant. Assuming that the function is bounded from below and lower semicontinuous and using a general proximal distance, it is proved that the iterations given by our algorithm are well defined and stay in the positive orthant. If the objective function is quasiconvex we obtain the convergence of the iterates to a certain set which contains the set of optimal solutions and convergence to a KKT point if the function is continuously differentiable and the proximal parameters are bounded. Furthermore, we introduce a sufficient condition on the proximal distance such that the sequence converges to an optimal solution of the problem.     

A new inexact alternating directions method for monotone variational inequalities

The alternating directions method (ADM) is an effective method for solving a class of variational inequalities (VI) when the proximal and penalty parameters in sub-VI problems are properly selected. In this paper, we propose a new ADM method which needs to solve two strongly monotone sub-VI problems in each iteration approximately and allows the parameters to vary from iteration to iteration. The convergence of the proposed ADM method is proved under quite mild assumptions and flexible parameter conditions. Received: January 4, 2000 / Accepted: October 2001?Published online February 14, 2002     

On finite convergence of proximal point algorithms for variational inequalities

In this paper, we first characterize finite convergence of an arbitrary iterative algorithm for solving the variational inequality problem (VIP), where the finite convergence means that the algorithm can find an exact solution of the problem in a finite number of iterations. By using this result, we obtain that the well-known proximal point algorithm possesses finite convergence if the solution set of VIP is weakly sharp. As an extension, we show finite convergence of the inertial proximal method for solving the general variational inequality problem under the condition of weak g-sharpness.     

Gradient trust region algorithm with limited memory BFGS update for nonsmooth convex minimization

By means of a gradient strategy, the Moreau-Yosida regularization, limited memory BFGS update, and proximal method, we propose a trust-region method for nonsmooth convex minimization. The search direction is the combination of the gradient direction and the trust-region direction. The global convergence of this method is established under suitable conditions. Numerical results show that this method is competitive to other two methods.     

An Alternating Direction Method for Nash Equilibrium of Two-Person Games with Alternating Offers

In this paper, we propose a method for finding a Nash equilibrium of two-person games with alternating offers. The proposed method is referred to as the inexact proximal alternating direction method. In this method, the idea of alternating direction method simulates alternating offers in the game, while the inexact solutions of subproblems can be matched to the assumptions of incomplete information and bounded individual rationality in practice. The convergence of the proposed method is proved under some suitable conditions. Numerical tests show that the proposed method is competitive to the state-of-the-art algorithms.     

A Direct Splitting Method for Nonsmooth Variational Inequalities

We propose a direct splitting method for solving a nonsmooth variational inequality in Hilbert spaces. The weak convergence is established when the operator is the sum of two point-to-set and monotone operators. The proposed method is a natural extension of the incremental subgradient method for nondifferentiable optimization, which strongly explores the structure of the operator using projected subgradient-like techniques. The advantage of our method is that any nontrivial subproblem must be solved, like the evaluation of the resolvent operator. The necessity to compute proximal iterations is the main difficulty of other schemes for solving this kind of problem.     

松弛算子分裂法线性收敛性分析的新框架

提出了一种新的分析框架来研究松弛算子分裂法的线性收敛性,可以将这种框架看成是经典的Krasnosel''-Mann迭代和Banach-Picard收缩的扩展形式.随后,将提出的这个框架应用于分析广义邻近点算法和松弛向前向后分裂算法的线性收敛性,其过程十分简洁和直接.     

A UNIFIED FRAMEWORK FOR SOME INEXACT PROXIMAL POINT ALGORITHMS*

We present a unified framework for the design and convergence analysis of a class of algorithms based on approximate solution of proximal point subproblems. Our development further enhances the constructive approximation approach of the recently proposed hybrid projection–proximal and extragradient–proximal methods. Specifically, we introduce an even more flexible error tolerance criterion, as well as provide a unified view of these two algorithms. Our general method possesses global convergence and local (super)linear rate of convergence under standard assumptions, while using a constructive approximation criterion suitable for a number of specific implementations. For example, we show that close to a regular solution of a monotone system of semismooth equations, two Newton iterations are sufficient to solve the proximal subproblem within the required error tolerance. Such systems of equations arise naturally when reformulating the nonlinear complementarity problem.

     

Solving policy design problems: Alternating direction method of multipliers-based methods for structured inverse variational inequalities

Inverse variational inequalities have broad applications in various disciplines, and some of them have very appealing structures. There are several algorithms (e.g., proximal point algorithms and projection-type algorithms) for solving the inverse variational inequalities in general settings, while few of them have fully exploited the special structures. In this paper, we consider a class of inverse variational inequalities that has a separable structure and linear constraints, which has its root in spatial economic equilibrium problems. To design an efficient algorithm, we develop an alternating direction method of multipliers (ADMM) based method by utilizing the separable structure. Under some mild assumptions, we prove its global convergence. We propose an improved variant that makes the subproblems much easier and derive the convergence result under the same conditions. Finally, we present the preliminary numerical results to show the capability and efficiency of the proposed methods.     

非正则线性增广拉格朗日乘子法的收敛速率研究CSCD

Recently,an indefinite linearized augmented Lagrangian method(IL-ALM)was proposed for the convex programming problems with linear constraints.The IL-ALM differs from the linearized augmented Lagrangian method in that the augmented Lagrangian is linearized by adding an indefinite quadratic proximal term.But,it preserves the algorithmic feature of the linearized ALM and usually has the advantage to improve the performance.The IL-ALM is proved to be convergent from contraction perspective,but its convergence rate is still missing.This is mainly because that the indefinite setting destroys the structures when we directly employ the contraction frameworks.In this paper,we derive the convergence rate for this algorithm by using a different analysis.We prove that a worst-case O(1/t)convergence rate is still hold for this algorithm,where t is the number of iterations.Additionally we show that the customized proximal point algorithm can employ larger step sizes by proving its equivalence to the linearized ALM.     

Proximal point method for minimizing quasiconvex locally Lipschitz functions on Hadamard manifolds

In this paper we propose an extension of the proximal point method to solve minimization problems with quasiconvex locally Lipschitz objective functions on Hadamard manifolds. To reach this goal, we use the concept of Clarke subdifferential on Hadamard manifolds and assuming that the function is bounded from below, we prove the global convergence of the sequence generated by the method to a critical point of the function.     

ON THE CONVERGENCE OF AN APPROXIMATE PROXIMAL METHOD FOR DC FUNCTIONS

In this paper we prove the convergence of the approximate proximal method for DC functions proposed by Sun et al [6]. Our analysis also permits to treat the exact method. We then propose an interesting result in the case where the second component of the DC function is differentiable and provide some computational experiences which proved the efficiency of our method.     

Descentwise inexact proximal algorithms for smooth optimization

The proximal method is a standard regularization approach in optimization. Practical implementations of this algorithm require (i)?an algorithm to compute the proximal point, (ii)?a rule to stop this algorithm, (iii)?an update formula for the proximal parameter. In this work we focus on?(ii), when smoothness is present??so that Newton-like methods can be used for?(i): we aim at giving adequate stopping rules to reach overall efficiency of the method. Roughly speaking, usual rules consist in stopping inner iterations when the current iterate is close to the proximal point. By contrast, we use the standard paradigm of numerical optimization: the basis for our stopping test is a ??sufficient?? decrease of the objective function, namely a fraction of the ideal decrease. We establish convergence of the algorithm thus obtained and we illustrate it on some ill-conditioned problems. The experiments show that combining the proposed inexact proximal scheme with a standard smooth optimization algorithm improves the numerical behaviour of the latter for those ill-conditioned problems.