Self-adaptive operator splitting methods for monotone variational inequalities

Summary. Solving a variational inequality problem VI(Ω,F) is equivalent to finding a solution of a system of nonsmooth equations (a hard problem). The Peaceman-Rachford and /or Douglas-Rachford operator splitting methods are advantageous when they are applied to solve variational inequality problems, because they solve the original problem via solving a series of systems of nonlinear smooth equations (a series of easy problems). Although the solution of VI(Ω,F) is invariant under multiplying F by some positive scalar β, yet the numerical experiment has shown that the number of iterations depends significantly on the positive parameter β which is a constant in the original operator splitting methods. In general, it is difficult to choose a proper parameter β for individual problems. In this paper, we present a modified operator splitting method which adjusts the scalar parameter automatically per iteration based on the message of the iterates. Exact and inexact forms of the modified method with self-adaptive variable parameter are suggested and proved to be convergent under mild assumptions. Finally, preliminary numerical tests show that the self-adaptive adjustment rule is proper and necessary in practice.     

解一类结构变分不等式问题的非精确并行交替方向法

带线性约束的具有两分块结构的单调变分不等式问题, 出现在许多现代应用中, 如交通和经济问题等. 基于该问题良好的可分结构, 分裂型算法被广泛研究用于其求解. 提出新的带回代的非精确并行交替方向法解该类问题, 在每一步迭代中,首先以并行模式通过投影得到预测点, 然后对其校正得到下一步的迭代点. 在压缩型算法的理论框架下, 在适当条件下证明了所提算法的全局收敛性. 数值结果表明了算法的有效性. 此外, 该算法可推广到求解具有多分块结构的问题.     

一种求解合作博弈最公平核心的非精确平行分裂算法

针对合作博弈核心和Shapley值的特点, 将最公平核心问题转化为带有两个变 量的可分离凸优化问题, 引入结构变分不等式的算子分裂方法框架, 提出了求解最公平核心的一种非精确平行分裂算法. 而且, 该算法充分利用了所求解问题的可行域的简单闭凸性, 子问题的非精确求解是容易的. 最后, 简单算例的数值实验表明了算法的收敛性和有效性.     

An inexact parallel splitting augmented Lagrangian method for monotone variational inequalities with separable structures

Splitting methods have been extensively studied in the context of convex programming and variational inequalities with separable structures. Recently, a parallel splitting method based on the augmented Lagrangian method (abbreviated as PSALM) was proposed in He (Comput. Optim. Appl. 42:195?C212, 2009) for solving variational inequalities with separable structures. In this paper, we propose the inexact version of the PSALM approach, which solves the resulting subproblems of PSALM approximately by an inexact proximal point method. For the inexact PSALM, the resulting proximal subproblems have closed-form solutions when the proximal parameters and inexact terms are chosen appropriately. We show the efficiency of the inexact PSALM numerically by some preliminary numerical experiments.     

Inexact implicit methods for monotone general variational inequalities

Solving a variational inequality problem is equivalent to finding a solution of a system of nonsmooth equations. Recently, we proposed an implicit method, which solves monotone variational inequality problem via solving a series of systems of nonlinear smooth (whenever the operator is smooth) equations. It can exploit the facilities of the classical Newton–like methods for smooth equations. In this paper, we extend the method to solve a class of general variational inequality problems Moreover, we improve the implicit method to allow inexact solutions of the systems of nonlinear equations at each iteration. The method is shown to preserve the same convergence properties as the original implicit method. Received July 31, 1995 / Revised version received January 15, 1999? Published online May 28, 1999     

Extended Projection Methods for Monotone Variational Inequalities

In this paper, we prove that each monotone variational inequality is equivalent to a two-mapping variational inequality problem. On the basis of this fact, a new class of iterative methods for the solution of nonlinear monotone variational inequality problems is presented. The global convergence of the proposed methods is established under the monotonicity assumption. The conditions concerning the implementability of the algorithms are also discussed. The proposed methods have a close relationship to the Douglas–Rachford operator splitting method for monotone variational inequalities.     

Inexact Operator Splitting Methods with Selfadaptive Strategy for Variational Inequality Problems

The Peaceman-Rachford and Douglas-Rachford operator splitting methods are advantageous for solving variational inequality problems, since they attack the original problems via solving a sequence of systems of smooth equations, which are much easier to solve than the variational inequalities. However, solving the subproblems exactly may be prohibitively difficult or even impossible. In this paper, we propose an inexact operator splitting method, where the subproblems are solved approximately with some relative error tolerance. Another contribution is that we adjust the scalar parameter automatically at each iteration and the adjustment parameter can be a positive constant, which makes the methods more practical and efficient. We prove the convergence of the method and present some preliminary computational results, showing that the proposed method is promising. This work was supported by the NSFC grant 10501024.     

一种求解单调变分不等式的部分并行分裂LQP交替方向法

LQP交替方向法是求解可分离结构型单调变分不等式问题的一种非常有效的方法.它不仅可以充分地利用目标函数的可分结构,将原问题分解为多个更易求解的子问题,还更适合求解大规模问题.对于带有三个可分离算子的单调变分不等式问题,结合增广拉格朗日算法和LQP交替方向法提出了一种部分并行分裂LQP交替方向法,构造了新算法的两个下降方向,结合这两个下降方向得到了一个新的下降方向,沿着这个新的下降方向给出了最优步长.并在较弱的假设条件下,证明了新算法的全局收敛性.     

A modified inexact operator splitting method for monotone variational inequalities

The Douglas–Peaceman–Rachford–Varga operator splitting methods (DPRV methods) are attractive methods for monotone variational inequalities. He et al. [Numer. Math. 94, 715–737 (2003)] proposed an inexact self-adaptive operator splitting method based on DPRV. This paper relaxes the inexactness restriction further. And numerical experiments indicate the improvement of this relaxation.      

Error bounds for proximal point subproblems and associated inexact proximal point algorithms

We study various error measures for approximate solution of proximal point regularizations of the variational inequality problem, and of the closely related problem of finding a zero of a maximal monotone operator. A new merit function is proposed for proximal point subproblems associated with the latter. This merit function is based on Burachik-Iusem-Svaiter’s concept of ε-enlargement of a maximal monotone operator. For variational inequalities, we establish a precise relationship between the regularized gap function, which is a natural error measure in this context, and our new merit function. Some error bounds are derived using both merit functions for the corresponding formulations of the proximal subproblem. We further use the regularized gap function to devise a new inexact proximal point algorithm for solving monotone variational inequalities. This inexact proximal point method preserves all the desirable global and local convergence properties of the classical exact/inexact method, while providing a constructive error tolerance criterion, suitable for further practical applications. The use of other tolerance rules is also discussed. Received: April 28, 1999 / Accepted: March 24, 2000?Published online July 20, 2000     

可分离凸优化问题的非精确平行分裂算法

针对一类可分离凸优化问题提出了一种非精确平行分裂算法.该算法充分利用了所求解问题的可分离结构,并对子问题进行非精确求解.在适当的条件下,证明了所提出的非精确平行分裂算法的全局收敛性,初步的数值实验说明了算法有效性.     

A distributed Douglas-Rachford splitting method for multi-block convex minimization problems

A customized Douglas-Rachford splitting method (DRSM) was recently proposed to solve two-block separable convex optimization problems with linear constraints and simple abstract constraints. The algorithm has advantage over the well-known alternating direction method of multipliers (ADMM), the dual application of DRSM to the two-block convex minimization problem, in the sense that the subproblems can have larger opportunity of possessing closed-form solutions since they are unconstrained. In this paper, we further study along this way by considering the primal application of DRSM for the general case m≥3, i.e., we consider the multi-block separable convex minimization problem with linear constraints where the objective function is separable into m individual convex functions without coupled variables. The resulting method fully exploits the separable structure and enjoys decoupled subproblems which can be solved simultaneously. Both the exact and inexact versions of the new method are presented in a unified framework. Under mild conditions, we manage to prove the global convergence of the algorithm. Preliminary numerical experiments for extracting the background from corrupted surveillance video verify the encouraging efficiency of the new algorithm.     

Convergence rate analysis of iteractive algorithms for solving variational inequality problems

 We present a unified convergence rate analysis of iterative methods for solving the variational inequality problem. Our results are based on certain error bounds; they subsume and extend the linear and sublinear rates of convergence established in several previous studies. We also derive a new error bound for $\gamma$-strictly monotone variational inequalities. The class of algorithms covered by our analysis in fairly broad. It includes some classical methods for variational inequalities, e.g., the extragradient, matrix splitting, and proximal point methods. For these methods, our analysis gives estimates not only for linear convergence (which had been studied extensively), but also sublinear, depending on the properties of the solution. In addition, our framework includes a number of algorithms to which previous studies are not applicable, such as the infeasible projection methods, a separation-projection method, (inexact) hybrid proximal point methods, and some splitting techniques. Finally, our analysis covers certain feasible descent methods of optimization, for which similar convergence rate estimates have been recently obtained by Luo [14]. Received: April 17, 2001 / Accepted: December 10, 2002 Published online: April 10, 2003 RID="⋆" ID="⋆" Research of the author is partially supported by CNPq Grant 200734/95–6, by PRONEX-Optimization, and by FAPERJ. Key Words. Variational inequality – error bound – rate of convergence Mathematics Subject Classification (2000): 90C30, 90C33, 65K05     

A Direct Splitting Method for Nonsmooth Variational Inequalities

We propose a direct splitting method for solving a nonsmooth variational inequality in Hilbert spaces. The weak convergence is established when the operator is the sum of two point-to-set and monotone operators. The proposed method is a natural extension of the incremental subgradient method for nondifferentiable optimization, which strongly explores the structure of the operator using projected subgradient-like techniques. The advantage of our method is that any nontrivial subproblem must be solved, like the evaluation of the resolvent operator. The necessity to compute proximal iterations is the main difficulty of other schemes for solving this kind of problem.     

Further applications of a splitting algorithm to decomposition in variational inequalities and convex programming

A classical method for solving the variational inequality problem is the projection algorithm. We show that existing convergence results for this algorithm follow from one given by Gabay for a splitting algorithm for finding a zero of the sum of two maximal monotone operators. Moreover, we extend the projection algorithm to solveany monotone affine variational inequality problem. When applied to linear complementarity problems, we obtain a matrix splitting algorithm that is simple and, for linear/quadratic programs, massively parallelizable. Unlike existing matrix splitting algorithms, this algorithm converges under no additional assumption on the problem. When applied to generalized linear/quadratic programs, we obtain a decomposition method that, unlike existing decomposition methods, can simultaneously dualize the linear constraints and diagonalize the cost function. This method gives rise to highly parallelizable algorithms for solving a problem of deterministic control in discrete time and for computing the orthogonal projection onto the intersection of convex sets.This research is partially supported by the U.S. Army Research Office, contract DAAL03-86-K-0171 (Center for Intelligent Control Systems), and by the National Science Foundation under grant NSF-ECS-8519058.Thanks are due to Professor J.-S. Pang for his helpful comments.     

Approximate multiplication in adaptive wavelet methods

Cohen, Dahmen and DeVore designed in [Adaptive wavelet methods for elliptic operator equations: convergence rates, Math. Comp., 2001, 70(233), 27–75] and [Adaptive wavelet methods II¶beyond the elliptic case, Found. Comput. Math., 2002, 2(3), 203–245] a general concept for solving operator equations. Its essential steps are: transformation of the variational formulation into the well-conditioned infinite-dimensional l ₂-problem, finding the convergent iteration process for the l ₂-problem and finally using its finite dimensional approximation which works with an inexact right-hand side and approximate matrix-vector multiplication. In our contribution, we pay attention to approximate matrix-vector multiplication which is enabled by an off-diagonal decay of entries of the wavelet stiffness matrices. We propose a more efficient technique which better utilizes actual decay of matrix and vector entries and we also prove that this multiplication algorithm is asymptotically optimal in the sense that storage and number of floating point operations, needed to resolve the problem with desired accuracy, remain proportional to the problem size when the resolution of the discretization is refined.     

Two-level pressure projection finite element methods for Navier-Stokes equations with nonlinear slip boundary conditions

The two-level pressure projection stabilized finite element methods for Navier-Stokes equations with nonlinear slip boundary conditions are investigated in this paper, whose variational formulation is the Navier-Stokes type variational inequality problem of the second kind. Based on the P₁-P₁ triangular element and using the pressure projection stabilized finite element method, we solve a small Navier-Stokes type variational inequality problem on the coarse mesh with mesh size H and solve a large Stokes type variational inequality problem for simple iteration or a large Oseen type variational inequality problem for Oseen iteration on the fine mesh with mesh size h. The error analysis obtained in this paper shows that if h=O(H²), the two-level stabilized methods have the same convergence orders as the usual one-level stabilized finite element methods, which is only solving a large Navier-Stokes type variational inequality problem on the fine mesh. Finally, numerical results are given to verify the theoretical analysis.     

Smale's α-theory for inexact Newton methods under the γ-condition

The present paper is concerned with the convergence problem of inexact Newton methods. Assuming that the nonlinear operator satisfies the γ-condition, a convergence criterion for inexact Newton methods is established which includes Smale's type convergence criterion. The concept of an approximate zero for inexact Newton methods is proposed in this paper and the criterion for judging an initial point being an approximate zero is established. Consequently, Smale's α-theory is generalized to inexact Newton methods. Furthermore, a numerical example is presented to illustrate the applicability of our main results.     

一类单调变分不等式的非精确交替方向法

交替方向法适合于求解大规模问题.该文对于一类变分不等式提出了一种新的交替方向法.在每步迭代计算中,新方法提出了易于计算的子问题,该子问题由强单调的线性变分不等式和良态的非线性方程系统构成.基于子问题的精确求解,该文证明了算法的收敛性.进一步,又提出了一类非精确交替方向法,每步迭代计算只需非精确求解子问题.在一定的非精确条件下,算法的收敛性得以证明.     

An Improved Extra-Gradient Method for Minimizing a Sum of p-norms—A Variational Inequality Approach

This paper presents a variational inequality (VI) approach to the problem of minimizing a sum of p-norms. First the original problem is reformulated as an equivalent linear VI. Then an improved extra-gradient method is presented to solve the linear VI. Applications to the problem of p-norm Steiner Minimum Trees (SMT) shows that the proposed method is effective. Comparison with the general extra-gradient method is also provided to show the improvements of the new method.