曲率障碍下四阶变分不等式的交替方向乘子法北大核心CSCD

对于重调和算子和曲率障碍表示的变分不等式,提出了自适应交替方向乘子数值解法(SADMM).对问题引入一个辅助变量表示曲率函数的增广Lagrange函数,导出一个约束极小值问题,并且该问题等价于一个鞍点问题.然后采用交替方向乘子法(ADMM)求解这个鞍点问题.通过采用平衡原理和迭代函数,得到了自动调整罚参数的自适应法则,从而提高了计算效率.证明了该方法的收敛性,并给出了利用迭代函数近似罚参数的具体方法.最后,用数值计算结果验证了该方法的有效性.     

Alternating direction method of multiplier for the unilateral contact problem with an automatic penalty parameter selection

We propose an alternating direction method of multiplier (ADMM) for the unilateral (frictionless) contact problem with an optimal parameter selection. We first introduce an auxiliary unknown to seprate the linear elasticity subproblem from the unilateral contact condition. Then an alternating direction is applied to the corresponding augmented Lagrangian. By eliminating the primal and auxiliary unknowns, at the discrete level, we derive a pure dual algorithm, starting point for the convergence analysis and the optimal parameter approximation. Numerical experiments are proposed to illustrate the efficiency of the proposed (optimal) penalty parameter selection method.     

A computation study on an integrated alternating direction method of multipliers for large scale optimization

The alternating direction method of multipliers (ADMM) has recently received a lot of attention especially due to its capability to harness the power of the new parallel and distributed computing environments. However, ADMM could be notoriously slow especially if the penalty parameter, assigned to the augmented term in the objective function, is not properly chosen. This paper aims to accelerate ADMM by integrating that with the Barzilai–Borwein gradient method and an acceleration technique known as line search. Line search accelerates an iterative method by performing a one-dimensional search along the line segment connecting two successive iterations. We pay a special attention to the large-scale nonnegative least squares problems, and our experiments using real datasets indicate that the integration not only accelerate ADMM but also robustifies that against the penalty parameter.     

广义乘子交替方向法求解线性约束不可分非凸优化问题的收敛性

In this paper, we consider the convergence of the generalized alternating direction method of multipliers(GADMM) for solving linearly constrained nonconvex minimization model whose objective contains coupled functions. Under the assumption that the augmented Lagrangian function satisfies the Kurdyka-Lojasiewicz inequality, we prove that the sequence generated by the GADMM converges to a critical point of the augmented Lagrangian function when the penalty parameter in the augmented Lagrangian function is sufficiently large. Moreover, we also present some sufficient conditions guaranteeing the sublinear and linear rate of convergence of the algorithm.     

An Alternating Direction Method of Multipliers for MCP-penalized Regression with High-dimensional Data

The minimax concave penalty (MCP) has been demonstrated theoretically and practically to be effective in nonconvex penalization for variable selection and parameter estimation. In this paper, we develop an efficient alternating direction method of multipliers (ADMM) with continuation algorithm for solving the MCP-penalized least squares problem in high dimensions. Under some mild conditions, we study the convergence properties and the Karush–Kuhn–Tucker (KKT) optimality conditions of the proposed method. A high-dimensional BIC is developed to select the optimal tuning parameters. Simulations and a real data example are presented to illustrate the efficiency and accuracy of the proposed method.     

A nonconvex formulation for low rank subspace clustering: algorithms and convergence analysis

We consider the problem of subspace clustering with data that is potentially corrupted by both dense noise and sparse gross errors. In particular, we study a recently proposed low rank subspace clustering approach based on a nonconvex modeling formulation. This formulation includes a nonconvex spectral function in the objective function that makes the optimization task challenging, e.g., it is unknown whether the alternating direction method of multipliers (ADMM) framework proposed to solve the nonconvex model formulation is provably convergent. In this paper, we establish that the spectral function is differentiable and give a formula for computing the derivative. Moreover, we show that the derivative of the spectral function is Lipschitz continuous and provide an explicit value for the Lipschitz constant. These facts are then used to provide a lower bound for how the penalty parameter in the ADMM method should be chosen. As long as the penalty parameter is chosen according to this bound, we show that the ADMM algorithm computes iterates that have a limit point satisfying first-order optimality conditions. We also present a second strategy for solving the nonconvex problem that is based on proximal gradient calculations. The convergence and performance of the algorithms is verified through experiments on real data from face and digit clustering and motion segmentation.     

Low-rank and sparse matrix recovery from noisy observations via 3-block ADMM algorithm

Recovering low-rank and sparse matrix from a given matrix arises in many applications, such as image processing, video background substraction, and so on. The 3-block alternating direction method of multipliers (ADMM) has been applied successfully to solve convex problems with 3-block variables. However, the existing sufficient conditions to guarantee the convergence of the 3-block ADMM usually require the penalty parameter $\gamma$ to satisfy a certain bound, which may affect the performance of solving the large scale problem in practice. In this paper, we propose the 3-block ADMM to recover low-rank and sparse matrix from noisy observations. In theory, we prove that the 3-block ADMM is convergent when the penalty parameters satisfy a certain condition and the objective function value sequences generated by 3-block ADMM converge to the optimal value. Numerical experiments verify that proposed method can achieve higher performance than existing methods in terms of both efficiency and accuracy.     

惩罚框架下求解广义Nash均衡问题的分解算法

广义Nash均衡问题(GNEP),是非合作博弈论中一类重要的问题,它在经济学、管理科学和交通规划等领域有着广泛的应用.本文主要提出一种新的惩罚算法来求解一般的广义Nash均衡问题,并根据罚函数的特殊结构,采用交替方向法求解子问题.在一定的条件下,本文证明新算法的全局收敛性.多个数值例子的试验结果表明算法是可行的,并且是有效的.     

Exact penalty function algorithm with simple updating of the penalty parameter

A new globally convergent algorithm for minimizing an objective function subject to equality and inequality constraints is presented. The algorithm determines a search direction by solving a quadratic programming subproblem, which always has an optimal solution, and uses an exact penalty function to compute the steplength along this direction through an Armijo-type scheme. The special structure of the quadratic subproblem is exploited to construct a new and simple method for updating the penalty parameter. This method may increase or reduce the value of the penalty parameter depending on some easily performed tests. A new method for updating the Hessian of the Lagrangian is presented, and a Q-superlinear rate of convergence is established.This work was supported in part by the British Council and the Conselho Nacional de Desenvolvimento Cientifico & Tecnologico/CNPq, Rio de Janeiro, Brazil.The authors are very grateful to Mr. Lam Yeung for his invaluable assistance in computing the results and to a reviewer for constructive advice.     

A parameter-free self-adapting boundary genetic search for pipe network optimization

Commercial application of genetic algorithms (GAs) to engineering design problems, including optimal design of pipe networks, could be facilitated by the development of algorithms that require the least number of parameter tuning. This paper presents an attempt to eliminate the need for defining a priori the proper penalty parameter in GA search for pipe networks optimal designs. The method is based on the assumption that the optimal solution of a pipe network design problem lies somewhere on, or near, the boundary of the feasible region. The proposed method uses the ratio of the best feasible and infeasible designs at each generation to guide the direction of the search towards the boundary of the feasible domain by automatically adjusting the value of the penalty parameter. The value of the ratio greater than unity is interpreted as the search being performed in the feasible region and vice versa. The new adapted value of the penalty parameter at each generation is therefore calculated as the product of its current value and the aforementioned ratio. The genetic search so constructed is shown to converge to the boundary of the feasible region irrespective of the starting value of the constraint violation penalty parameter. The proposed method is described here in the context of pipe network optimisation problems but is equally applicable to any other constrained optimisation problem. The effectiveness of the method is illustrated with a benchmark pipe network optimization example from the literature.     

High-order total variation-based Poissonian image deconvolution with spatially adapted regularization parameter

Images captured by image acquisition systems using photon-counting devices such as astronomical imaging, positron emission tomography and confocal microscopy imaging, are often contaminated by Poisson noise. Total variation (TV) regularization, which is a classic regularization technique in image restoration, is well-known for recovering sharp edges of an image. Since the regularization parameter is important for a good recovery, Chen and Cheng (2012) proposed an effective TV-based Poissonian image deblurring model with a spatially adapted regularization parameter. However, it has drawbacks since the TV regularization produces staircase artifacts. In this paper, in order to remedy the shortcoming of TV of their model, we introduce an extra high-order total variation (HTV) regularization term. Furthermore, to balance the trade-off between edges and the smooth regions in the images, we also incorporate a weighting parameter to discriminate the TV and the HTV penalty. The proposed model is solved by an iterative algorithm under the framework of the well-known alternating direction method of multipliers. Our numerical results demonstrate the effectiveness and efficiency of the proposed method, in terms of signal-to-noise ratio (SNR) and relative error (RelRrr).     

Linear Convergence Rates for Variants of the Alternating Direction Method of Multipliers in Smooth Cases

In the present paper, we propose a novel convergence analysis of the alternating direction method of multipliers, based on its equivalence with the overrelaxed primal–dual hybrid gradient algorithm. We consider the smooth case, where the objective function can be decomposed into one differentiable with Lipschitz continuous gradient part and one strongly convex part. Under these hypotheses, a convergence proof with an optimal parameter choice is given for the primal–dual method, which leads to convergence results for the alternating direction method of multipliers. An accelerated variant of the latter, based on a parameter relaxation, is also proposed, which is shown to converge linearly with same asymptotic rate as the primal–dual algorithm.     

Automatic Response Category Combination in Multinomial Logistic Regression

We propose a penalized likelihood method that simultaneously fits the multinomial logistic regression model and combines subsets of the response categories. The penalty is nondifferentiable when pairs of columns in the optimization variable are equal. This encourages pairwise equality of these columns in the estimator, which corresponds to response category combination. We use an alternating direction method of multipliers algorithm to compute the estimator and we discuss the algorithm’s convergence. Prediction and model selection are also addressed. Supplemental materials for this article are available online.     

A decomposition heuristic for mixed-integer supply chain problems

Mixed-integer supply chain models typically are very large but are also very sparse and can be decomposed into loosely coupled blocks. In this paper, we use general-purpose techniques to obtain a block decomposition of supply chain instances and apply a tailored penalty alternating direction method, which exploits the structural properties of the decomposed instances. We further describe problem-specific enhancements of the algorithm and present numerical results on real-world instances that illustrate the applicability of the approach.     

Positive shrinkage, improved pretest and absolute penalty estimators in partially linear models

Shrinkage estimators of a partially linear regression parameter vector are constructed by shrinking estimators in the direction of the estimate which is appropriate when the regression parameters are restricted to a linear subspace. We investigate the asymptotic properties of positive Stein-type and improved pretest semiparametric estimators under quadratic loss. Under an asymptotic distributional quadratic risk criterion, their relative dominance picture is explored analytically. It is shown that positive Stein-type semiparametric estimators perform better than the usual Stein-type and least square semiparametric estimators and that an improved pretest semiparametric estimator is superior to the usual pretest semiparametric estimator. We also consider an absolute penalty type estimator for partially linear models and give a Monte Carlo simulation comparisons of positive shrinkage, improved pretest and the absolute penalty type estimators. The comparison shows that the shrinkage method performs better than the absolute penalty type estimation method when the dimension of the parameter space is much larger than that of the linear subspace.     

A new inexact alternating directions method for monotone variational inequalities

The alternating directions method (ADM) is an effective method for solving a class of variational inequalities (VI) when the proximal and penalty parameters in sub-VI problems are properly selected. In this paper, we propose a new ADM method which needs to solve two strongly monotone sub-VI problems in each iteration approximately and allows the parameters to vary from iteration to iteration. The convergence of the proposed ADM method is proved under quite mild assumptions and flexible parameter conditions. Received: January 4, 2000 / Accepted: October 2001?Published online February 14, 2002     

Error estimates for integral constraint regularization of state-constrained elliptic control problems

The alternating direction method of multipliers (ADMM) is a benchmark for solving a two-block linearly constrained convex minimization model whose objective function is the sum of two functions without coupled variables. Meanwhile, it is known that the convergence is not guaranteed if the ADMM is directly extended to a multiple-block convex minimization model whose objective function has more than two functions. Recently, some authors have actively studied the strong convexity condition on the objective function to sufficiently ensure the convergence of the direct extension of ADMM or the resulting convergence when the original scheme is appropriately twisted. We focus on the three-block case of such a model whose objective function is the sum of three functions, and discuss the convergence of the direct extension of ADMM. We show that when one function in the objective is strongly convex, the penalty parameter and the operators in the linear equality constraint are appropriately restricted, it is sufficient to guarantee the convergence of the direct extension of ADMM. We further estimate the worst-case convergence rate measured by the iteration complexity in both the ergodic and nonergodic senses, and derive the globally linear convergence in asymptotical sense under some additional conditions.     

NONCONFORMING FINITE ELEMENT PENALTY METHOD FOR STOKES EQUATION

a special penalty method is presented to improve the accuracy of the standard penaltymethod (or solving Stokes equation with nonconforming finite element, It is shown that thismethod with a larger penalty parameter can achieve the same accuracy as the staodaxd methodwith a smaller penalty parameter. The convergence rate of the standard method is just hall order of this penalty method when using the same penalty parameter, while the extrapolationmethod proposed by Faik et al can not yield so high accuracy of convergence. At last, we alsoget the super-convergence estimates for total flux.     

Composite penalty method of a low order anisotropic nonconforming finite element for the Stokes problem

Composite penalty method of a low order anisotropic nonconforming quadrilateral finite element for the Stokes problem is presented. This method with a large penalty parameter can achieve the same accuracy as the stand method with a small penalty parameter and the convergence rate of this method is two times as that of the standard method under the condition of the same order penalty parameter. The superconvergence for velocity is established as well. The results of this paper are also valid to the most of the known nonconforming finite element methods.