求解稀疏逻辑回归问题的嵌套BB算法的分裂增广拉格朗日算法

逻辑回归是经典的分类方法,广泛应用于数据挖掘、机器学习和计算机视觉.现研究带有程。模约束的逻辑回归问题.这类问题广泛用于分类问题中的特征提取,且一般是NP-难的.为了求解这类问题,提出了嵌套BB(Barzilai and Borwein)算法的分裂增广拉格朗日算法(SALM-BB).该算法在迭代中交替地求解一个无约束凸优化问题和一个带程。模约束的二次优化问题.然后借助BB算法求解无约束凸优化问题.通过简单的等价变形直接得到带程。模约束二次优化问题的精确解,并且给出了算法的收敛性定理.最后通过数值实验来测试SALM-BB算法对稀疏逻辑回归问题的计算精确性.数据来源包括真实的UCI数据和模拟数据.数值实验表明,相对于一阶算法SLEP,SALM-BB能够得到更低的平均逻辑损失和错分率.     

求解带线性约束的凸优化的一类自适应不定线性化增广拉格朗日方法

增广拉格朗日方法是求解带线性约束的凸优化问题的有效算法.线性化增广拉格朗日方法通过线性化增广拉格朗日函数的二次罚项并加上一个临近正则项,使得子问题容易求解,其中正则项系数的恰当选取对算法的收敛性和收敛速度至关重要.较大的系数可保证算法收敛性,但容易导致小步长.较小的系数允许迭代步长增大,但容易导致算法不收敛.本文考虑求解带线性等式或不等式约束的凸优化问题.我们利用自适应技术设计了一类不定线性化增广拉格朗日方法,即利用当前迭代点的信息自适应选取合适的正则项系数,在保证收敛性的前提下尽量使得子问题步长选择范围更大,从而提高算法收敛速度.我们从理论上证明了算法的全局收敛性,并利用数值实验说明了算法的有效性.     

张量鲁棒主成分分析的增广拉格朗日交替方向方法

张量的鲁棒主成分分析是将未知的一个低秩张量与一个稀疏张量从已知的它们的和中分离出来.因为在计算机视觉与模式识别中有着广阔的应用前景,该问题在近期成为学者们的研究热点.本文提出了一种针对张量鲁棒主成分分析的新的模型,并给出交替方向极小化的求解算法,在求解过程中给出了两种秩的调整策略.针对低秩分量本文对其全部各阶展开矩阵进行低秩矩阵分解,针对稀疏分量采用软阈值收缩的策略.无论目标低秩张量为精确低秩或近似低秩,本文所提方法均可适用.本文对算法给出了一定程度上的收敛性分析,即算法迭代过程中产生的任意收敛点均满足KKT条件.如果目标低秩张量为精确低秩,当迭代终止时可对输出结果进行基于高阶奇异值分解的修正.针对人工数据和真实视频数据的数值实验表明,与同类型算法相比,本文所提方法可以得到更好的结果.     

关于一类潜在空间问题的数值计算

潜在空间模型是网络数据统计建模和可视化的有效工具.随着网络规模的不断扩大,潜在空间模型的计算也面临着巨大挑战.在本文我们应用对偶半邻近交替方向乘子法(dual semiproximal Alternating Direction Method of Multipliers,简称dsADMM)求解大型网络的通用潜在空间模型拟合问题.并在一些温和的条件下分析了该算法的全局收敛性.数值试验验证了该算法的有效性.     

关于高斯低通滤波器的超分辨分析

对于经过高斯低通滤波的信号,通过求解一类凸优化模型稳定地恢复该信号的高频信息.当信号满足一定的分离条件时,给出了误差估计的界,从理论上证明了求解凸优化方法的稳定性.理论的证明依赖于压缩感知中的对偶理论.一个显著的差异在于高斯低通滤波器并不满足压缩感知中对于测量矩阵的要求,例如相关性,约束等距性质等.     

对等式约束非线性规划问题的Hestenes-Powell增广拉格朗日函数的进一步研究

本文对用无约束极小化方法求解等式约束非线性规划问题的Hestenes-Powell 增广拉格朗日函数作了进一步研究．在适当的条件下,我们建立了Hestenes-Powell增广拉格朗日函数在原问题变量空间上的无约束极小与原约束问题的解之间的关系,并且也给出了Hestenes-Powell增广拉格朗日函数在原问题变量和乘子变量的积空间上的无约束极小与原约束问题的解之间的一个关系．因此,从理论的观点来看,原约束问题的解和对应的拉格朗日乘子值不仅可以用众所周知的乘子法求得,而且可以通过对Hestenes-Powell 增广拉格朗日函数在原问题变量和乘子变量的积空间上执行一个单一的无约束极小化来获得．     

基于非凸优化模型的块稀疏信号恢复条件

压缩感知(compressed sensing,CS)是一种全新的信息采集与处理理论，它表明稀疏信号能够在远低于Shannon-Nyquist采样率的条件下被精确重构.现从压缩感知理论出发，对块稀疏信号重构算法进行研究，通过混合l₂/l_q(0相似文献   

基于基底合并的分片稀疏恢复

如我们所知,诸如视频和图像等信号可以在某些框架下被表示为稀疏信号,因此稀疏恢复(或稀疏表示)是信号处理、图像处理、计算机视觉、机器学习等领域中被广泛研究的问题之一.通常大多数在稀疏恢复中的有效快速算法都是基于求解$l^0$或者$l^1$优化问题.但是,对于求解$l^0$或者$l^1$优化问题以及相关算法所得到的理论充分性条件对信号的稀疏性要求过严.考虑到在很多实际应用中,信号是具有一定结构的,也即,信号的非零元素具有一定的分布特点.在本文中,我们研究分片稀疏恢复的唯一性条件和可行性条件.分片稀疏性是指一个稀疏信号由多个稀疏的子信号合并所得.相应的采样矩阵是由多个基底合并组成.考虑到采样矩阵的分块结构,我们引入了子矩阵的互相干性,由此可以得到相应$l^0$或者$l^1$优化问题可精确恢复解的稀疏度的新上界.本文结果表明.通过引入采样矩阵的分块结构信息.可以改进分片稀疏恢复的充分性条件.以及相应$l^0$或者$l^1$优化问题整体稀疏解的可靠性条件.     

去除稀疏和结构化噪声的交替方向增强拉格朗日算法

图像和视频去噪是数字图像处理的必要环节之一.为了去除图像和视频中广泛存在的稀疏噪声和结构化噪声,提出了一种分离低秩矩阵、稀疏矩阵和结构化矩阵的优化模型一主成分离群点追求.在交替方向最小化思想的基础上,利用增强拉格朗日乘子法求解主成分离群点追求模型,设计了求解模型的交替方向增强拉格朗日(ADAL)算法,加入了一种连续技术以提高算法的收敛速率.仿真实验结果表明,提出的模型和算法能够有效去除不同尺寸矩阵的不同比例的稀疏噪声和结构化噪声.     

固定时间梯度流在?_1-?_2范数中的稀疏重构

压缩感知(compressed sensing,CS)是一种全新的信号采样技术,对于稀疏信号,它能够以远小于传统的Nyquist采样定理的采样点来重构信号.在压缩感知中,采用动态连续系统,对?_1-?_2范数的稀疏信号重构问题进行了研究.提出了一种基于固定时间梯度流的稀疏信号重构算法,证明了该算法在Lyapunov意义上的稳定性并且收敛于问题的最优解.最后通过与现有的投影神经网络算法的对比,体现了该算法的可行性以及在收敛速度上的优势.     

A gradient projection method for the sparse signal reconstruction in compressive sensing

Many problems arising from machine learning, compressive sensing, linear inverse problem, and statistical inference involve finding sparse solutions to under-determined or ill-conditioned equations. In this paper, a gradient projection method is proposed to recover sparse signal in compressive sensing by solving the nonlinear convex constrained equations. The global convergence is established with the backtracking line search. Preliminary numerical experiments coping with the sparse signal reconstruction in compressive sensing are performed, which show that the proposed method is very effective and stable.     

Analysis on a superlinearly convergent augmented Lagrangian method

The augmented Lagrangian method is a classical method for solving constrained optimization.Recently,the augmented Lagrangian method attracts much attention due to its applications to sparse optimization in compressive sensing and low rank matrix optimization problems.However,most Lagrangian methods use first order information to update the Lagrange multipliers,which lead to only linear convergence.In this paper,we study an update technique based on second order information and prove that superlinear convergence can be obtained.Theoretical properties of the update formula are given and some implementation issues regarding the new update are also discussed.     

本刊英文版Vol.30(2014),No.1论文摘要

<正>Analysis on a Superlinearly Convergent Augmented Lagrangian Method Ya Xiang YUAN Abstract The augmented Lagrangian method is a classical method for solving constrained optimization.Recently,the augmented Lagrangian method attracts much attention due to its applications to sparse optimization in compressive sensing and low rank matrix optimization problems.However,most Lagrangian methods use first order information to update the Lagrange multipliers,which lead to only linear convergence.In this paper,we study an update technique     

Relaxed inertial proximal Peaceman-Rachford splitting method for separable convex programming

The strictly contractive Peaceman-Rachford splitting method is one of effective methods for solving separable convex optimization problem, and the inertial proximal Peaceman-Rachford splitting method is one of its important variants. It is known that the convergence of the inertial proximal Peaceman-Rachford splitting method can be ensured if the relaxation factor in Lagrangian multiplier updates is underdetermined, which means that the steps for the Lagrangian multiplier updates are shrunk conservatively. Although small steps play an important role in ensuring convergence, they should be strongly avoided in practice. In this article, we propose a relaxed inertial proximal Peaceman-Rachford splitting method, which has a larger feasible set for the relaxation factor. Thus, our method provides the possibility to admit larger steps in the Lagrangian multiplier updates. We establish the global convergence of the proposed algorithm under the same conditions as the inertial proximal Peaceman-Rachford splitting method. Numerical experimental results on a sparse signal recovery problem in compressive sensing and a total variation based image denoising problem demonstrate the effectiveness of our method.     

A proximal augmented Lagrangian method for equilibrium problems

Considering a recently proposed proximal point method for equilibrium problems, we construct an augmented Lagrangian method for solving the same problem in reflexive Banach spaces with cone constraints generating a strongly convergent sequence to a certain solution of the problem. This is an inexact hybrid method meaning that at a certain iterate, a solution of an unconstrained equilibrium problem is found, allowing a proper error bound, followed by a Bregman projection of the initial iterate onto the intersection of two appropriate halfspaces. Assuming a set of reasonable hypotheses, we provide a full convergence analysis.     

Quantization of compressive samples with stable and robust recovery

In this paper we study the quantization stage that is implicit in any compressed sensing signal acquisition paradigm. We propose using Sigma–Delta (ΣΔ) quantization and a subsequent reconstruction scheme based on convex optimization. We prove that the reconstruction error due to quantization decays polynomially in the number of measurements. Our results apply to arbitrary signals, including compressible ones, and account for measurement noise. Additionally, they hold for sub-Gaussian (including Gaussian and Bernoulli) random compressed sensing measurements, as well as for both high bit-depth and coarse quantizers, and they extend to 1-bit quantization. In the noise-free case, when the signal is strictly sparse we prove that by optimizing the order of the quantization scheme one can obtain root-exponential decay in the reconstruction error due to quantization.     

修正HS共轭梯度法在大规模信号重构问题中的应用

本义研究了压缩感知在大规模信号恢复问题中应用的问题.利用修正HS共轭梯度法及光滑化方法,获得了具有较好重构效果的算法.数值实验表明用修正HS共轭梯度法解决大规模信号恢复问题是可行的.     

A linearly convergent algorithm for sparse signal reconstruction

For the sparse signal reconstruction problem in compressive sensing, we propose a projection-type algorithm without any backtracking line search based on a new formulation of the problem. Under suitable conditions, global convergence and its linear convergence of the designed algorithm are established. The efficiency of the algorithm is illustrated through some numerical experiments on some sparse signal reconstruction problem.     

Image decoding optimization based on compressive sensing

Transform-based image codec follows the basic principle: the reconstructed quality is decided by the quantization level. Compressive sensing (CS) breaks the limit and states that sparse signals can be perfectly recovered from incomplete or even corrupted information by solving convex optimization. Under the same acquisition of images, if images are represented sparsely enough, they can be reconstructed more accurately by CS recovery than inverse transform. So, in this paper, we utilize a modified TV operator to enhance image sparse representation and reconstruction accuracy, and we acquire image information from transform coefficients corrupted by quantization noise. We can reconstruct the images by CS recovery instead of inverse transform. A CS-based JPEG decoding scheme is obtained and experimental results demonstrate that the proposed methods significantly improve the PSNR and visual quality of reconstructed images compared with original JPEG decoder.     

Image decoding optimization based on compressive sensing

Transform-based image codec follows the basic principle: the reconstructed quality is decided by the quantization level. Compressive sensing (CS) breaks the limit and states that sparse signals can be perfectly recovered from incomplete or even corrupted information by solving convex optimization. Under the same acquisition of images, if images are represented sparsely enough, they can be reconstructed more accurately by CS recovery than inverse transform. So, in this paper, we utilize a modified TV operator to enhance image sparse representation and reconstruction accuracy, and we acquire image information from transform coefficients corrupted by quantization noise. We can reconstruct the images by CS recovery instead of inverse transform. A CS-based JPEG decoding scheme is obtained and experimental results demonstrate that the proposed methods significantly improve the PSNR and visual quality of reconstructed images compared with original JPEG decoder.