非凸多分块优化的Bregman ADMM的收敛率研究

Wang等提出了求解带线性约束的多块可分非凸优化问题的带Bregman距离的交替方向乘子法(Bregman ADMM),并证明了其收敛性.该文将进一步研究求解带线性约束的多块可分非凸优化问题的Bregman ADMM的收敛率,以及算法产生的迭代点列有界的充分条件.在效益函数的Kurdyka-Lojasiewicz (KL)性质下,该文建立了值和迭代的收敛速率,证明了与目标函数相关的各种KL指数值可获得Bregman ADMM的三种不同收敛速度.更确切地说,该文证明了如下结果:如果效益函数的KL指数θ=0,那么由Bregman ADMM生成的序列经过有限次迭代后收敛;如果θ∈(0,1/2],那么Bregman ADMM是线性收敛的;如果θ∈(1/2,1),那么Bregman ADMM是次线性收敛的.     

非凸优化带松弛步的对称ADMM局部线性收敛率分析

基于对称交替方向乘子法(ADMM),结合松弛步技巧,该文提出一种带松弛步的对称ADMM用于求解两分块线性约束非凸优化问题.同时,新算法乘子更新步采用不同的松弛因子.常规假设下,给出新算法子序列的收敛性证明.误差界条件下,分析并获得由新算法产生的迭代点列以线性收敛的速率局部趋于问题稳定点,相应增广拉格朗日函数序列亦线性收敛.最后,初步试验结果表明新算法是有效的.     

随机需求下直销多商品流供应链网络均衡模型

建立了随机需求下直销多商品流供应链网络均衡模型,导出了供应链网络达到均衡的条件,它等价于一个有限维变分不等式.通过构建修改拟牛顿法,获得了随机需求下直销多商品流供应链网络均衡模型的解(变分不等式的解),并给出了1个算例,验证了模型的正确性和算法的可行性.     

非凸非光滑不可分离优化的线性对称邻近ADMM收敛性分析

交替方向乘子法(ADMM)是一种求解可分离优化问题的简单有效的方法,相关研究已经较为完善.然而,当目标函数存在耦合项时,对ADMM算法收敛性的研究还处于初期.文章针对非凸非光滑不可分离优化问题,基于对称交替方向乘子法(SADMM),结合线性化技术,提出了一种新的线性对称邻近ADMM.在一定的假设条件下,证明了算法生成的序列有界并收敛至增广拉格朗日函数的稳定点.其次,当辅助函数满足Kurdyka-Lojasiewicz性质时,证明了算法的强收敛性.最后,数值实验的结果表明了算法的有效性.     

一种推广的求解可分离凸优化问题的黄金比率邻近ADMM算法

ADMM算法是求解可分离凸优化问题的经典算法之一,但其无法保证原始迭代序列的收敛性且其子问题计算量很大.为了保证该算法所有迭代点列的全局收敛性及提高计算效率,采用凸组合技术的黄金比率邻近ADMM算法被提出,其中凸组合因子Ψ是关键参数.本文在黄金比率邻近ADMM算法的基础上,扩大了凸组合因子Ψ的取值范围,提出了收敛步长范围更广的推广黄金比率邻近ADMM算法.并在一定的假设下,证明了算法的全局收敛性及函数值残差和约束违反度在遍历意义下的O(1/N)次线性收敛速度.以及,当目标函数中任意一个函数强凸时,证明了算法在遍历意义下的O(1/N²)收敛率.最后,本文通过数值试验表明推广算法的有效性.     

非对称线性互补问题的并行二级多分裂迭代法

提出了求解非对称线性互补问题的并行二级多分裂迭代算法,并证明了该算法的收敛性,最后通过数值实验验证了算法的有效性和可行性.     

我和乘子交替方向法20年

1997 年, 交通网络分析方面的问题把我引进乘子交替方向法(ADMM)的研究领域. 近10 年来, 原本用来求解变分不等式的ADMM在优化计算中被广泛采用, 影响越来越大. 这里总结了20 年来我们在ADMM 方面的工作, 特别是近10 年 ADMM 在凸优化分裂收缩算法方面的进展. 梳理主要结果, 说清来龙去脉. 文章利用变分不等式的形式研究凸优化的ADMM 类算法, 论及的所有方法都能纳入一个简单的预测-校正统一框架. 在统一框架下证明算法的收缩性质特别简单. 通读, 有利于了解ADMM类算法的概貌. 仔细阅读, 也许就掌握了根据实际问题需要构造分裂算法的基本技巧. 也要清醒地看到, ADMM类算法源自增广拉格朗日乘子法 (ALM) 和邻近点 (PPA)算法, 它只是便于利用问题的可分离结构, 并没有消除 ALM和PPA等一阶算法固有的缺点.     

线性约束两分块非凸优化的ADMM-SQP算法

基于乘子交替方向法(ADMM)和序列二次规划(SQP)方法思想, 致力于研究线 性约束两分块非凸优化的新型高效算法. 首先, 以SQP思想为主线, 在其二次规划(QP)子问题的求解中引入ADMM思想, 将QP分解为两个相互独立的小规模QP求解. 其次, 借助增广拉格朗日函数和Armijo线搜索产生原始变量新迭代点. 最后, 以显式解析式更新对偶变量. 因此, 构建了一个新型ADMM-SQP算法. 在较弱条件下, 分析了算法通常意义下的全局收敛性, 并对算法进行了初步的数值试验.     

多目标最优控制的小波逼近解

基于小波多尺度逼近特性,提出了一种求解线性时变系统中多目标最优控制的新方法.该法避免求解带附加积分约束的R iccati微分方程而只需求解一个代数二次约束规划问题,适合于计算机求解.数值研究表明,所提算法是精确可行的.     

随机ADMM算法及其在电力系统凸经济调度问题中的应用

针对电力系统中的一类凸经济调度问题,提出了随机ADMM算法,设计了周期循环更新规则和随机选择更新规则,证明了随机ADMM算法在周期循环更新规则下的收敛性,以及得出了在随机选择更新规则下按期望收敛的结论.数值实验结果表明该方法可以有效解决电力系统中的凸经济调度问题.     

稀疏线性规划研究

稀疏线性规划在金融计算、工业生产、装配调度等领域应用十分广泛.本文首先给出稀疏线性规划问题的一般模型并证明问题是NP困难问题;其次采用交替方向乘子法(ADMM)求解该问题;最后证明了算法在近似问题上的收敛性.数值实验表明,算法在大规模数值算例上的表现优于已有的混合遗传算法;同时通过对金融实例的计算验证了算法及模型在稀疏投资组合问题上的有效性.     

A proximal point algorithm revisit on the alternating direction method of multipliers

The alternating direction method of multipliers(ADMM)is a benchmark for solving convex programming problems with separable objective functions and linear constraints.In the literature it has been illustrated as an application of the proximal point algorithm(PPA)to the dual problem of the model under consideration.This paper shows that ADMM can also be regarded as an application of PPA to the primal model with a customized choice of the proximal parameter.This primal illustration of ADMM is thus complemental to its dual illustration in the literature.This PPA revisit on ADMM from the primal perspective also enables us to recover the generalized ADMM proposed by Eckstein and Bertsekas easily.A worst-case O(1/t)convergence rate in ergodic sense is established for a slight extension of Eckstein and Bertsekas’s generalized ADMM.     

An efficient alternating direction method of multipliers for optimal control problems constrained by random Helmholtz equations

Based on the alternating direction method of multipliers (ADMM), we develop three numerical algorithms incrementally for solving the optimal control problems constrained by random Helmholtz equations. First, we apply the standard Monte Carlo technique and finite element method for the random and spatial discretization, respectively, and then ADMM is used to solve the resulting system. Next, combining the multi-modes expansion, Monte Carlo technique, finite element method, and ADMM, we propose the second algorithm. In the third algorithm, we preprocess certain quantities before the ADMM iteration, so that nearly no random variable is in the inner iteration. This algorithm is the most efficient one and is easy to implement. The error estimates of these three algorithms are established. The numerical experiments verify the efficiency of our algorithms.     

Primal-dual algorithms for total variation based image restoration under Poisson noise Dedicated to Professor Lin Qun on the Occasion of his 80th Birthday

We consider the problem of restoring images corrupted by Poisson noise. Under the framework of maximum a posteriori estimator, the problem can be converted into a minimization problem where the objective function is composed of a Kullback-Leibler(KL)-divergence term for the Poisson noise and a total variation(TV) regularization term. Due to the logarithm function in the KL-divergence term, the non-differentiability of TV term and the positivity constraint on the images, it is not easy to design stable and efficiency algorithm for the problem. Recently, many researchers proposed to solve the problem by alternating direction method of multipliers(ADMM). Since the approach introduces some auxiliary variables and requires the solution of some linear systems, the iterative procedure can be complicated. Here we formulate the problem as two new constrained minimax problems and solve them by Chambolle-Pock's first order primal-dual approach. The convergence of our approach is guaranteed by their theory. Comparing with ADMM approaches, our approach requires about half of the auxiliary variables and is matrix-inversion free. Numerical results show that our proposed algorithms are efficient and outperform the ADMM approach.     

A gaussian upper bound for gaussian multi-stage stochastic linear programs

This paper deals with two-stage and multi-stage stochastic programs in which the right-hand sides of the constraints are Gaussian random variables. Such problems are of interest since the use of Gaussian estimators of random variables is widespread. We introduce algorithms to find upper bounds on the optimal value of two-stage and multi-stage stochastic (minimization) programs with Gaussian right-hand sides. The upper bounds are obtained by solving deterministic mathematical programming problems with dimensions that do not depend on the sample space size. The algorithm for the two-stage problem involves the solution of a deterministic linear program and a simple semidefinite program. The algorithm for the multi-stage problem invovles the solution of a quadratically constrained convex programming problem.     

An Augmented ADMM Algorithm With Application to the Generalized Lasso Problem

In this article, we present a fast and stable algorithm for solving a class of optimization problems that arise in many statistical estimation procedures, such as sparse fused lasso over a graph, convex clustering, and trend filtering, among others. We propose a so-called augmented alternating direction methods of multipliers (ADMM) algorithm to solve this class of problems. Compared to a standard ADMM algorithm, our proposal significantly reduces the computational cost at each iteration while maintaining roughly the same overall convergence speed. We also consider a new varying penalty scheme for the ADMM algorithm, which could further accelerate the convergence, especially when solving a sequence of problems with tuning parameters of different scales. Extensive numerical experiments on the sparse fused lasso problem show that the proposed algorithm is more efficient than the standard ADMM and two other existing state-of-the-art specialized algorithms. Finally, we discuss a possible extension and some interesting connections to two well-known algorithms. Supplementary materials for the article are available online.     

Interior-Point Methods with Decomposition for Solving Large-Scale Linear Programs

This paper deals with an algorithm incorporating the interior-point method into the Dantzig–Wolfe decomposition technique for solving large-scale linear programming problems. The algorithm decomposes a linear program into a main problem and a subproblem. The subproblem is solved approximately. Hence, inexact Newton directions are used in solving the main problem. We show that the algorithm is globally linearly convergent and has polynomial-time complexity.     

求解两块可分离凸极小化问题的惯性交替方向乘子法

The alternating direction method of multipliers(ADMM)is a widely used method for solving many convex minimization models arising in signal and image processing.In this paper,we propose an inertial ADMM for solving a two-block separable convex minimization problem with linear equality constraints.This algorithm is obtained by making use of the inertial Douglas-Rachford splitting algorithm to the corresponding dual of the primal problem.We study the convergence analysis of the proposed algorithm in infinite-dimensional Hilbert spaces.Furthermore,we apply the proposed algorithm on the robust principal component analysis problem and also compare it with other state-of-the-art algorithms.Numerical results demonstrate the advantage of the proposed algorithm.     

A Finite Algorithm for Almost Linear Complementarity Problems

We present an algorithm for solving a class of nonlinear complementarity problems called the almost linear complementarity problem (ALCP), which can be used to simulate free boundary problems. The algorithm makes use of a procedure for identifying an active index subset of an ALCP by bounding its solution with an interval vector. It is shown that an acceptable solution of the given ALCP can be obtained by solving at most n systems of equations. Numerical results are reported to illustrate the efficiency of the algorithm for large-scale problems.