Tensor train rank minimization with hybrid smoothness regularization for visual data recovery

Recently, the tensor train (TT) rank has received much attention for tensor completion, due to its ability to explore the global low-rankness of tensors. However, existing methods still leave room for improvement, since the low-rankness itself is generally not sufficient to recover the underlying data. Inspired by this, we consider a novel tensor completion model by simultaneously exploiting the global low-rankness and local smoothness of visual data. In particular, we use low-rank matrix factorization to characterize the global TT low-rankness, and framelet and total variation regularization to enhance the local smoothness. We develop an efficient proximal alternating minimization algorithm to solve the proposed new model with guaranteed convergence. Extensive experiments on various data demonstrated that the proposed method outperforms compared methods in terms of visual and quantitative measures.     

Construction of Multivariate Tight Framelet Packets Associated with Dilation Matrix

In this paper, we present a method for constructing multivariate tight framelet packets associated with an arbitrary dilation matrix using unitary extension principles.We also prove how to construct various tight frames for L2(Rd) by replac-ing some mother framelets.     

松弛型并行多分裂方法解非线性方程组的安全界

本文对某些非线性方程组F(x)=0,导出了一个算法,用它可以迭代建立F(x)=0的解的紧致上、下界。算法基于某些矩阵的多分裂,因此具有自然的并行性。我们证明了趋向于解的界之收敛原则,给出了参数的收敛性区域并考察了方法的收敛速度。     

A framelet algorithm for de-blurring images corrupted by multiplicative noise

This paper considers a variational model for restoring images from blurry and speckled observations. This model utilizes the favorable properties of framelet regularization (e.g., the sparsity and multiresolution properties of the framelet) that are well suited for speckle noise reduction. For solving the model, we first propose an approximation model that is motivated by the well-known variable-splitting and penalty techniques in optimization. We then develop an alternating minimization algorithm to solve the approximation model. We also show that the sequence generated by the algorithm converges to the solution of the proposed model. The numerical results on simulated data and real utrasound images demonstrate that our approach outperforms several state-of-the-art algorithms.     

On the existence of multiresolution analysis for framelets

We show that a compactly supported tight framelet comes from an MRA if the intersection of all dyadic dilations of the space of negative dilates, which is defined as the shift-invariant space generated by the negative scales of a framelet, is trivial. We also construct examples of (non-tight) framelets, which are arbitrarily close to tight frame framelets, such that the corresponding space of negative dilates is equal to the entire space L ²ℝ.The first author was partially supported by the NSF grant DMS–0441817     

Simultaneously inpainting in image and transformed domains

In this paper, we focus on the restoration of images that have incomplete data in either the image domain or the transformed domain or in both. The transform used can be any orthonormal or tight frame transforms such as orthonormal wavelets, tight framelets, the discrete Fourier transform, the Gabor transform, the discrete cosine transform, and the discrete local cosine transform. We propose an iterative algorithm that can restore the incomplete data in both domains simultaneously. We prove the convergence of the algorithm and derive the optimal properties of its limit. The algorithm generalizes, unifies, and simplifies the inpainting algorithm in image domains given in Cai et al. (Appl Comput Harmon Anal 24:131–149, 2008) and the inpainting algorithms in the transformed domains given in Cai et al. (SIAM J Sci Comput 30(3):1205–1227, 2008), Chan et al. (SIAM J Sci Comput 24:1408–1432, 2003; Appl Comput Harmon Anal 17:91–115, 2004). Finally, applications of the new algorithm to super-resolution image reconstruction with different zooms are presented. R. H. Chan’s research was supported in part by HKRGC Grant 400505 and CUHK DAG 2060257. L. Shen’s research was supported by the US National Science Foundation under grant DMS-0712827. Z. Shen’s research was supported in part by Grant R-146-000-060-112 at the National University of Singapore.     

A Component-Wise EM Algorithm for Mixtures

Maximum likelihood estimation in finite mixture distributions is typically approached as an incomplete data problem to allow application of the expectation-maximization (EM) algorithm. In its general formulation, the EM algorithm involves the notion of a complete data space, in which the observed measurements and incomplete data are embedded. An advantage is that many difficult estimation problems are facilitated when viewed in this way. One drawback is that the simultaneous update used by standard EM requires overly informative complete data spaces, which leads to slow convergence in some situations. In the incomplete data context, it has been shown that the use of less informative complete data spaces, or equivalently smaller missing data spaces, can lead to faster convergence without sacrifying simplicity. However, in the mixture case, little progress has been made in speeding up EM. In this article we propose a component-wise EM for mixtures. It uses, at each iteration, the smallest admissible missing data space by intrinsically decoupling the parameter updates. Monotonicity is maintained, although the estimated proportions may not sum to one during the course of the iteration. However, we prove that the mixing proportions will satisfy this constraint upon convergence. Our proof of convergence relies on the interpretation of our procedure as a proximal point algorithm. For performance comparison, we consider standard EM as well as two other algorithms based on missing data space reduction, namely the SAGE and AECME algorithms. We provide adaptations of these general procedures to the mixture case. We also consider the ECME algorithm, which is not a data augmentation scheme but still aims at accelerating EM. Our numerical experiments illustrate the advantages of the component-wise EM algorithm relative to these other methods.     

Acceleration of the EM algorithm using the vector epsilon algorithm

The expectation–maximization (EM) algorithm is a very general and popular iterative computational algorithm to find maximum likelihood estimates from incomplete data and broadly used to statistical analysis with missing data, because of its stability, flexibility and simplicity. However, it is often criticized that the convergence of the EM algorithm is slow. The various algorithms to accelerate the convergence of the EM algorithm have been proposed. The vector ε algorithm of Wynn (Math Comp 16:301–322, 1962) is used to accelerate the convergence of the EM algorithm in Kuroda and Sakakihara (Comput Stat Data Anal 51:1549–1561, 2006). In this paper, we provide the theoretical evaluation of the convergence of the ε-accelerated EM algorithm. The ε-accelerated EM algorithm does not use the information matrix but only uses the sequence of estimates obtained from iterations of the EM algorithm, and thus it keeps the flexibility and simplicity of the EM algorithm.     

Data-Driven Tight Frame Construction for Impulsive Noise Removal

The method of data-driven tight frame has been shown very useful in image restoration problems.We consider in this paper extending this important technique,by incorporating L₁ data fidelity into the original data-driven model,for removing impulsive noise which is a very common and basic type of noise in image data.The model contains three variables and can be solved through an efficient iterative alternating minimization algorithm in patch implementation,where the tight frame is dynamically updated.It constructs a tight frame system from the input corrupted image adaptively,and then removes impulsive noise by the derived system.We also show that the sequence generated by our algorithm converges globally to a stationary point of the optimization model.Numerical experiments and comparisons demonstrate that our approach performs well for various kinds of images.This benefits from its data-driven nature and the learned tight frames from input images capture richer image structures adaptively.     

基于CT图像重建的多重集合分裂可行性问题应用分析

为了较好地应用CQ算法解决稀疏角度CT 图像重建的问题,提出了一种新的实时的分块逐次混合算法．首先将稀疏角度CT 图像重建的重建问题转化成分裂可行性问题．其次,通过分析非空闭凸集C和Q的不同的定义,在N维实空间中分别针对不同的CQ算法给出了7种不同的实现方案．通过试验,分别对不同算法及其方案的重建精度和收敛速度进行了对比分析,并对多重集合分裂可行性问题算法中约束权因子的选取及其对输出的影响进行了研究,从而给出了CQ算法在稀疏角度CT图像重建问题中应用的最佳凸集定义方案．以此为基础,给出了所提出算法的最佳实现方案．试验结果表明,该算法收敛速度快,重建精度高,为多重集合分裂可行性问题及其改进算法在该重建问题上的应用提供了参考．     

On the Convergence of the Factorization Update Algorithm

In this paper, we make a Kantorovich-type analysis for the spares Johnson and Austria's algorithm given in [2], which is called factorization update algorithm. When the mapping is linear, it is shown that a modification of that algorithm leads to global and Q-superlinear convergence. Finally, we point out the modification is also of local and Q-superlinear convergence for nonlinear systems of equations and give its corresponding Kantorovich-type convergence result.     

基于字典学习方法的CT不完全投影图像重建算法

对于不完全投影角度的重建研究是CT图像重建中一个重要的问题.将压缩感知中字典学习的方法与CT重建算法ART迭代算法相结合.字典学习方法中字典更新采用K-SVD(K-奇异值分解)算法,稀疏编码采用OMP(正交匹配追踪)算法.最后通过对标准Head头部模型进行仿真实验,验证了字典学习方法在CT图像重建中对于提高图像的重建质量和提高信噪比的可行性与有效性.另外还研究了字典学习中图像块大小和滑动距离对重建图像的影响     

A modified fixed-point iterative algorithm for image restoration using fourth-order PDE model

In this paper, we propose a modified fixed point iterative algorithm to solve the fourth-order PDE model for image restoration problem. Compared with the standard fixed point algorithm, the proposed algorithm needn?t to compute inverse matrices so that it can speed up the convergence and reduce the roundoff error. Furthermore, we prove the convergence of the proposed algorithm and give some experimental results to illustrate its effectiveness by comparing with the standard fixed point algorithm, the time marching algorithm and the split Bregman algorithm.     

A class of tight framelet packets

This paper obtains a class of tight framelet packets on L ²(ℝ ^d ) from the extension principles and constructs the relationships between the basic framelet packets and the associated filters.     

Updating incomplete factorization preconditioners for model order reduction

When solving a sequence of related linear systems by iterative methods, it is common to reuse the preconditioner for several systems, and then to recompute the preconditioner when the matrix has changed significantly. Rather than recomputing the preconditioner from scratch, it is potentially more efficient to update the previous preconditioner. Unfortunately, it is not always known how to update a preconditioner, for example, when the preconditioner is an incomplete factorization. A recently proposed iterative algorithm for computing incomplete factorizations, however, is able to exploit an initial guess, unlike existing algorithms for incomplete factorizations. By treating a previous factorization as an initial guess to this algorithm, an incomplete factorization may thus be updated. We use a sequence of problems from model order reduction. Experimental results using an optimized GPU implementation show that updating a previous factorization can be inexpensive and effective, making solving sequences of linear systems a potential niche problem for the iterative incomplete factorization algorithm.     

Two-Subspace Projection Method for Coherent Overdetermined Systems

We present a Projection onto Convex Sets (POCS) type algorithm for solving systems of linear equations. POCS methods have found many applications ranging from computer tomography to digital signal and image processing. The Kaczmarz method is one of the most popular solvers for overdetermined systems of linear equations due to its speed and simplicity. Here we introduce and analyze an extension of the Kaczmarz method that iteratively projects the estimate onto a solution space given by two randomly selected rows. We show that this projection algorithm provides exponential convergence to the solution in expectation. The convergence rate improves upon that of the standard randomized Kaczmarz method when the system has correlated rows. Experimental results confirm that in this case our method significantly outperforms the randomized Kaczmarz method.     

Minimum residual methods for augmented systems

For large systems of linear equations, iterative methods provide attractive solution techniques. We describe the applicability and convergence of iterative methods of Krylov subspace type for an important class of symmetric and indefinite matrix problems, namely augmented (or KKT) systems. Specifically, we consider preconditioned minimum residual methods and discuss indefinite versus positive definite preconditioning. For a natural choice of starting vector we prove that when the definite and indenfinite preconditioners are related in the obvious way, MINRES (which is applicable in the case of positive definite preconditioning) and full GMRES (which is applicable in the case of indefinite preconditioning) give residual vectors with identical Euclidean norm at each iteration. Moreover, we show that the convergence of both methods is related to a system of normal equations for which the LSQR algorithm can be employed. As a side result, we give a rare example of a non-trivial normal(1) matrix where the corresponding inner product is explicitly known: a conjugate gradient method therefore exists and can be employed in this case. This work was supported by British Council/German Academic Exchange Service Research Collaboration Project 465 and NATO Collaborative Research Grant CRG 960782     

Multi-grid methods for stokes and navier-stokes equations

Summary In the present paper we introduce transforming iterations, an approach to construct smoothers for indefinite systems. This turns out to be a convenient tool to classify several well-known smoothing iterations for Stokes and Navier-Stokes equations and to predict their convergence behaviour, epecially in the case of high Reynolds-numbers. Using this approach, we are able to construct a new smoother for the Navier-Stokes equations, based on incomplete LU-decompositions, yielding a highly effective and robust multi-grid method. Besides some qualitative theoretical convergence results, we give large numerical comparisons and tests for the Stokes as well as for the Navier-Stokes equations. For a general convergence theory we refer to [29].This work was supported in part by Deutsche Forschungsgemeinschaft     

An Errata to: Convergence of a SOR-Weiszfeld Type Algorithm for Incomplete Data Sets

We discovered an error in our proof of convergence of a generalization of the Weiszfeld algorithm to incomplete data sets, and we provide here a partial correction, with slightly weaker claims.